AMD has unveiled its next-generation accelerators for synthetic intelligence (AI) inference and coaching, amongst different workloads — claiming that the brand new Intuition MI350 Sequence delivers as much as 4 occasions the uncooked compute efficiency of its last-generation elements and a whopping 35 occasions extra inference efficiency — whereas present Intuition customers may be profit from efficiency positive aspects by upgrading to ROCm 7.0.
“The world of AI isn’t slowing down — and neither are we,” claims AMD’s Vamsi Boppana, senior vice chairman of the corporate’s synthetic intelligence group. “At AMD, we’re not simply holding tempo, we’re setting the bar. Our prospects are demanding actual, deployable options that scale, and that is precisely what we’re delivering with the AMD Intuition MI350 Sequence. With cutting-edge efficiency, large reminiscence bandwidth, and versatile, open infrastructure, we’re empowering innovators throughout industries to go sooner, scale smarter, and construct what’s subsequent.”
AMD has unveiled its Intuition MI350 Sequence of AI accelerators, power-hungry petaflop-scale elements for AI coaching and inference. (📷: AMD)
AMD has confirmed two fashions at launch, the Intuition MI350X and Intuition MI355X. The previous options 288GB of Excessive Bandwidth Reminiscence 3E (HBM3E) with 8TB/s of bandwidth and delivers 72 tera-floating-point operations per second (TFLOPS) at FP64 precision rising to 18.45 peta-floating-point operations per second (PFLOPS) at FP6 or FP4 precision with structured sparsity; the latter consists of the identical reminiscence specs, which AMD says can run massive language fashions (LLMs) and different AI fashions with as much as 520 billion parameters on-device, however will increase the efficiency to 78.6 TFLOPS at FP64 and 20.1 PFLOPS at FP6/FP4 with structured sparsity.
The corporate is not anticipating customers to purchase only a single card, although: with the ever-growing energy calls for of each coaching and operating next-generation fashions, AMD additionally presents the Intuition MI350X Platform and Intuition MI355X Platform — bundles of eight playing cards, providing a mixed complete of two.3TB of HBM3E reminiscence and peak efficiency of 147.6 PFLOPS and 161 PFLOPS respectively at FP6/FP4 with structured sparsity. Anybody seeking to run such a system will want loads of energy and cooling at hand, although: every MI350X has a thermal design profile of an eyebrow-raising 1kW, with the MI355X upping that to 1.4kW.
Each new and present Intuition customers will even have the ability to profit from AMD’s upcoming ROCm 7.0 launch, the corporate says, which is able to ship a claimed tripling of coaching efficiency and greater than 3.5 occasions the inference efficiency of ROCm 7.0. “This achievement stems from advances in usability, efficiency, and assist for decrease precision information varieties like FP4 and FP6,” says AMD’s Anush Elangovan of the upcoming launch. “Additional enhancements in communication stacks have [also] optimized GPU utilization and information motion.”
The corporate additionally guarantees spectacular efficiency positive aspects for present Intuition customers, by means of the brand new ROCm 7.0 and its assist for decrease precision. (📷: AMD)
Lastly, probably the most affected person and heavily-funded AI experimenters might wish to cling fireplace till the brand new yr, with Boppana teasing the next-next-generation Intuition MI400 Sequence due for launch in 2026. “The AMD Intuition MI400 Sequence will characterize a dramatic generational leap in efficiency enabling full rack stage options for giant scale coaching and distributed inference,” he says, revealing fashions providing as much as 432GB of HBM4 reminiscence with 19.6TB/s of bandwidth and efficiency as much as 40 FLOPS at FP4 with structured sparsity. These will likely be supplied, he says, in an “AI Rack” system dubbed “Helios,” which mixes the playing cards with AMD EPYC “Venice” CPUs and Pesando “Vulcano” AI community playing cards to kind an all-in-one platform for coaching and inference workloads.
Intuition MI350 Sequence playing cards, in the meantime, will likely be made out there on-demand by means of cloud suppliers and for on-premises use from authentic gear producers together with Dell, HPE, and Supermicro. Extra data is out there on the AMD web site.
