Microsoft has deepened its partnership with AMD in a deal that will implant AMD’s Helios rack-scale AI platform directly into Azure data centers, starting in the second half of 2026. The move creates a new Azure virtual machine family—ND MI455X v7—engineered specifically for production-scale AI inference, and it marks the first time Microsoft will adopt a fully integrated AMD system spanning accelerators, CPUs, networking, and software.

What’s Coming: Helios Hardware, Venice CPUs, and Three New VM Families

The centerpiece of the announcement is the Helios rack-scale solution. Each rack packs 72 AMD Instinct MI455X accelerators, providing 31 TB of HBM4 memory, up to 2.9 exaflops of FP4 compute, and 1.4 exaflops at FP8. Those are theoretical platform-level numbers, not per-application guarantees, but they signal the kind of memory density and raw throughput Microsoft is targeting for next-generation AI services.

The accelerators are paired with undisclosed sixth-generation EPYC processors (codenamed “Venice”) and AMD’s Pensando Vulcano networking components. The whole system leans on the ROCm software stack for libraries, runtimes, and development tools.

AMD says the rack can push 260 TB per second of scale-up bandwidth among its 72 GPUs, with an additional 43 TB per second of scale-out bandwidth via Ethernet-based Pensando technology. In practice, that split matters: scale-up keeps a tight group of GPUs coherent for a single large model, while scale-out connects racks into larger clusters and to storage.

Beyond the Helios-based VMs, Microsoft also named two other Azure families built around Venice CPUs: Azure HDv2 (targeting AI data systems, agentic workloads, and heavy data pipelines) and Azure HXv2 (aimed at silicon design, engineering simulation, and technical computing). All three represent a deliberate segmentation of infrastructure around workload behavior—inference, data prep and orchestration, and specialized high-performance computing.

What It Means for You: More Choice, New Complexity

For Azure customers already running AI inference at scale, the ND MI455X v7 instances promise a genuine alternative to Nvidia-dominated GPU options. Microsoft positioning the service for “production-scale inference” means it’s meant for continuous, revenue-generating workloads—chatbots, search, code assistants, and agentic systems that may execute thousands of queries per second.

For enterprise architects and IT decision-makers: The benefit is flexibility. If your organization faces GPU shortages, long reservation queues, or pricing pressure on existing Nvidia instances, AMD-based capacity could expand your options. The large memory pool (31 TB per rack) may also let you serve bigger models or longer context windows without splitting work excessively. But it’s not a drop-in replacement. Most production AI pipelines are deeply tuned for CUDA, and even if your framework supports ROCm, you’ll need to test the full stack—inference servers, attention kernels, quantization tools, container images, and monitoring agents.

For developers: The migration lift is real. Start by inventorying CUDA-specific libraries and custom kernels in your code. AMD’s HIP tools can translate many CUDA calls, but performance tuning for a new accelerator topology can be its own project. Microsoft is likely to ease this with pre-optimized VM images, managed inference services, and model catalogs, but details haven’t emerged. Independent benchmarks using real inference traffic will be crucial before you commit resources.

For Windows and Microsoft 365 users: You won’t see a Helios rack on your desk. This is cloud infrastructure. But faster, cheaper inference could eventually mean better AI features in the apps you already use—longer Copilot answers, more responsive agents in Edge, higher limits in Microsoft 365’s AI tools. The effect is indirect but real: cloud economics shape software subscriptions.

For budget-conscious teams: The total cost per million tokens matters more than theoretical exaflops. AMD’s lower-precision emphasis (FP4/FP8) can cut memory and compute cost, but only if your models tolerate the precision loss. You’ll need to measure token generation rate, time-to-first-token, and latency under load, then map those to Azure’s hourly pricing—pricing that Microsoft hasn’t yet disclosed.

How We Got Here: From EPYC VMs to a Full-Stack Rack

Microsoft and AMD have been partners for decades, but the relationship until now lived mostly in CPUs. Azure already offers EPYC-powered VMs for general-purpose, memory-intensive, and confidential computing, and the ND MI300X v5 series introduced AMD Instinct GPUs into Azure’s catalog. That MI300X deployment let Microsoft and customers kick the tires on AMD’s accelerator software (ROCm) at cloud scale, and it likely informed the decision to go bigger with Helios.

Helios itself reflects a shift in how hyperscale AI gets built. Single GPUs can’t hold frontier models, so the performance bottleneck often isn’t arithmetic—it’s communication. Rack-scale design attacks that by treating dozens of accelerators, the host CPUs, the switches, and the memory hierarchy as one engineered system. Nvidia has pushed the same idea with its own rack-scale designs, and Microsoft’s embrace of Helios signals that AMD can now credibly compete at that level, not just with individual chips.

Microsoft’s multi-supplier strategy is deliberate. The company continues to invest heavily in Nvidia GPUs and in its own custom AI accelerators. Adding AMD doesn’t signal a switch; it’s a hedge. When one silicon supplier’s capacity is tight or a workload runs better on a different architecture, Azure can route jobs appropriately. That kind of portfolio thinking also strengthens Microsoft’s hand in procurement and pricing.

What to Do Now: Evaluate, Test, and Plan

The Helios VMs aren’t available yet, but the announcement sets a timer. If your team relies on Azure for AI inference, now is the time to act.

For infrastructure and DevOps teams:
1. Audit your AI workloads. Identify which are bound by GPU memory capacity, which are sensitive to memory bandwidth, and which could run at FP4/FP8 without quality loss. Those will be the best candidates for Helios.
2. Start a ROCm proof-of-concept. Even on existing MI300X instances, you can test your stack’s portability. Check framework support, library availability, and performance baselines. Document any missing features or unexpected behavior.
3. Watch Azure region and pricing announcements. Early capacity may be limited to a few data centers and large enterprise customers. Talk to your Microsoft account team about access programs.

For AI/ML engineers:
- Familiarize yourself with AMD’s HIP porting guide. The conceptual leap from CUDA isn’t huge, but kernel-level tuning for a different memory hierarchy and interconnect requires effort.
- Begin profiling your inference pipeline. Understanding where time is spent—GPU compute, host-to-device transfers, collective communication, disk I/O—will help you estimate how much a Helios instance might help.

For procurement and finance:
- Model scenarios. How much would a 20% reduction in inference cost change your project’s ROI? Could you afford to serve longer contexts or higher throughput if memory were more abundant? Don’t rely on theoretical peak specs; build conservative estimates and wait for real-world benchmarking data.

Outlook: The Real Test Begins in 2026

This announcement is a statement of intent, not a product launch. AMD expects to begin shipping Helios systems to Microsoft in the second half of 2026, and actual availability of ND MI455X v7 instances could lag by months. The path to success is littered with integration challenges: Microsoft must align AMD’s reference architecture with Azure’s existing networking, security, monitoring, and management stack, while ensuring that ROCm can handle the messy reality of enterprise production workloads.

Key milestones to watch:
- First public benchmarks – Independent tests with models like Llama, GPT, or Mixtral on Helios hardware will reveal actual inference throughput and latency.
- Azure region expansion – Availability in multiple geographies will signal supply chain maturity.
- Microsoft’s own use – If Azure AI services like OpenAI API endpoints or Microsoft Copilot start running on Helios, that’s a strong vote of confidence.
- ROCm ecosystem growth – Watch for official Docker images, optimized model implementations in Hugging Face, and guidance from inference server vendors.

For now, the expanded AMD partnership gives Azure customers something they’ve been asking for: a credible second source for high-end AI inference. Whether it delivers on its promise will come down to execution—hardware, software, and the operational muscle Microsoft brings to its cloud. The clock starts now.