|
In the present day, we’re asserting the final availability of Amazon Elastic Compute Cloud (Amazon EC2) G7e cases that ship cost-effective efficiency for generative AI inference workloads and the very best efficiency for graphics workloads.
G7e cases are accelerated by the NVIDIA RTX PRO 6000 Blackwell Server Version GPUs and are nicely fitted to a broad vary of GPU-enabled workloads together with spatial computing and scientific computing workloads. G7e cases ship as much as 2.3 occasions inference efficiency in comparison with G6e cases.
Enhancements made in comparison with predecessors:
- NVIDIA RTX PRO 6000 Blackwell GPUs — NVIDIA RTX PRO 6000 Blackwell Server Version GPUs provide two occasions the GPU reminiscence and 1.85 occasions the GPU reminiscence bandwidth in comparison with G6e cases. Through the use of the upper GPU reminiscence provided by G7e cases, you’ll be able to run medium-sized fashions of as much as 70B parameters with FP8 precision on a single GPU.
- NVIDIA GPUDirect P2P — For fashions which can be too massive to suit into the reminiscence of a single GPU, you’ll be able to cut up the mannequin or computations throughout a number of GPUs. G7e cases scale back the latency of your multi-GPU workloads with assist for NVIDIA GPUDirect P2P, which allows direct communication between GPUs over PCIe interconnect. These cases provide the bottom peer to see latency for GPUs on the identical PCIe swap. Moreover, G7e cases provide as much as 4 occasions the inter-GPU bandwidth in comparison with L40s GPUs featured in G6e cases, boosting the efficiency of multi-GPU workloads. These enhancements imply you’ll be able to run inference for bigger fashions throughout a number of GPUs providing as much as 768 GB of GPU reminiscence in a single node.
- Networking — G7e cases provide 4 occasions the networking bandwidth in comparison with G6e cases, which implies you should use the occasion for small-scale multi-node workloads. Moreover, multi-GPU G7e cases assist NVIDIA GPUDirect Distant Direct Reminiscence Entry (RDMA) with Elastic Cloth Adapter (EFA), which reduces the latency of distant GPU-to-GPU communication for multi-node workloads. These occasion sizes additionally assist NVIDIA GPUDirectStorage with Amazon FSx for Lustre, which will increase throughput by as much as 1.2 Tbps to the cases in comparison with G6e cases, which implies you’ll be able to shortly load your fashions.
EC2 G7e specs
G7e cases characteristic as much as 8 NVIDIA RTX PRO 6000 Blackwell Server Version GPUs with as much as 768 GB of whole GPU reminiscence (96 GB of reminiscence per GPU) and Intel Emerald Rapids processors. Additionally they assist as much as 192 vCPUs, as much as 1,600 Gbps of community bandwidth, as much as 2,048 GiB of system reminiscence, and as much as 15.2 TB of native NVMe SSD storage.
Listed below are the specs:
| Occasion title |
GPUs | GPU reminiscence (GB) | vCPUs | Reminiscence (GiB) | Storage (TB) | EBS bandwidth (Gbps) | Community bandwidth (Gbps) |
| g7e.2xlarge | 1 | 96 | 8 | 64 | 1.9 x 1 | As much as 5 | 50 |
| g7e.4xlarge | 1 | 96 | 16 | 128 | 1.9 x 1 | 8 | 50 |
| g7e.8xlarge | 1 | 96 | 32 | 256 | 1.9 x 1 | 16 | 100 |
| g7e.12xlarge | 2 | 192 | 48 | 512 | 3.8 x 1 | 25 | 400 |
| g7e.24xlarge | 4 | 384 | 96 | 1024 | 3.8 x 2 | 50 | 800 |
| g7e.48xlarge | 8 | 768 | 192 | 2048 | 3.8 x 4 | 100 | 1600 |
To get began with G7e cases, you should use the AWS Deep Studying AMIs (DLAMI) on your machine studying (ML) workloads. To run cases, you should use AWS Administration Console, AWS Command Line Interface (AWS CLI) or AWS SDKs. For a managed expertise, you should use G7e cases with Amazon Elastic Container Service (Amazon ECS), Amazon Elastic Kubernetes Service (Amazon EKS). Help for Amazon SageMaker AI can be coming quickly.
Now accessible
Amazon EC2 G7e cases can be found in the present day within the US East (N. Virginia) and US East (Ohio) AWS Areas. For Regional availability and a future roadmap, search the occasion kind within the CloudFormation assets tab of AWS Capabilities by Area.
The cases might be bought as On-Demand Situations, Financial savings Plan, and Spot Situations. G7e cases are additionally accessible in Devoted Situations and Devoted Hosts. To study extra, go to the Amazon EC2 Pricing web page.
Give G7e cases a attempt within the Amazon EC2 console. To study extra, go to the Amazon EC2 G7e cases web page and ship suggestions to AWS re:Submit for EC2 or via your standard AWS Help contacts.
— Channy
