Amazon Web Services and Nvidia drew a bold line under their 15-year collaboration at GTC 2026 earlier this month, announcing not just new GPU instance types but a unified AI control plane that aims to tame the complexity of production AI at scale. The new platform, which integrates Nvidia’s full enterprise AI software stack with AWS’s cloud-native services, marks a strategic pivot from selling raw compute to delivering a governed, end-to-end lifecycle for AI workloads—from development and training to inference and monitoring.

“This isn’t just about adding more GPUs to the cloud,” AWS CEO Matt Garman said during his keynote. “We’re giving enterprises a single pane of glass to manage their entire AI fleet, with built-in governance, cost controls, and security—everything IT needs to run AI in production responsibly.” Nvidia founder Jensen Huang echoed the sentiment, calling the control plane “the operating system for the AI factory.”

A Deeper Hardware Refresh

At the hardware level, AWS introduced the general availability of EC2 P8 instances, powered by Nvidia’s next-generation Vera Rubin GPUs. Each instance packages eight Rubin GPUs with NVLink 6 interconnect and a staggering 1.8 TB/s of GPU-to-GPU bandwidth, tied together by an AWS Nitro-accelerated Nvidia Spectrum-X800 Ethernet fabric. This configuration eradicates the bandwidth bottlenecks that have historically throttled distributed training of trillion-parameter foundation models.

Early benchmarks released by AWS show the P8 instances delivering up to 2.5x the training throughput of the previous P7 instances based on Blackwell architecture, while inference latency drops by half for large language models. To manage the immense power and thermal demands, AWS has deployed a new liquid-cooled rack design that supports up to 256 P8 instances in a single cluster, offering exaFLOPs of AI compute on demand.

Alongside the P8, AWS also announced the G6g instances, which combine Nvidia L4 GPUs with AWS Graviton4 processors for cost-efficient inference and fine-tuning of modestly sized models—a nod to the growing demand for AI not just among hyperscalers but also mid-market enterprises.

Software: From Ingredient to Managed Service

On the software side, Nvidia’s AI Enterprise suite—including NIM inference microservices, NeMo for model customization, and RAPIDS for accelerated data science—will be available as first-class managed services inside AWS. This means enterprises can deploy optimized AI models with a few clicks, while AWS handles the underlying infrastructure scaling, security patch management, and compliance.

NIM, in particular, has been deeply integrated with AWS auto-scaling groups and Amazon Elastic Kubernetes Service (EKS) to dynamically provision GPU resources based on real-time inference demand. For model builders, NeMo now includes one-click deployment to Amazon SageMaker, with automatic hyperparameter tuning powered by the SageMaker Experiments platform.

“We’re taking the complexity out of AI Ops,” noted Nvidia VP of Enterprise Computing Manuvir Das. “By turning our software into AWS managed services, data scientists can focus on models, not on Kubernetes configurations.”

The Control Plane: Taming Production AI

The marquee announcement, however, is the new AWS AI Control Plane with Nvidia, a policy-driven orchestration layer that unifies management across all Nvidia-accelerated resources on AWS. IT administrators can define security policies, quota limits, and cost controls that automatically apply across training clusters, inference endpoints, and GPU-accelerated data pipelines. The control plane integrates with AWS Identity and Access Management (IAM), CloudWatch, and AWS Organizations, giving enterprises a single pane of glass for all their Nvidia-powered AI operations.

Key capabilities include:
- Unified cluster provisioning: Spin up ephemeral training clusters or persistent inference fleets with consistent tooling, regardless of whether you use P8, P7, or G6g instances.
- Fine-grained cost governance: Set budgets per project or department, with automated shutoff of idle GPU resources and recommendations for savings plans.
- Security posture management: Enforce encryption, network isolation, and model access controls via AWS security services, extended to the Nvidia stack.
- Observability and compliance: Real-time dashboards for GPU utilization, throughput, and latency, along with audit trails that meet SOC 2 and HIPAA requirements.

This shift addresses one of the biggest headaches for enterprises scaling AI: the operational fragmentation of managing disparate GPU fleets across development, testing, and production. “Previously, you’d have a Terraform script for training, a separate monitoring stack for inference, and a manual approval process for model updates,” explained an AWS senior product manager. “Now it’s all codified in the control plane, cutting time-to-production by at least 50%.”

Tighter Integration with AWS AI Services

The control plane plugs directly into Amazon Bedrock and SageMaker, allowing data scientists to use Nvidia’s optimized foundation models without leaving their familiar development environments. Bedrock now supports Nvidia NeMo-customized large language models as serverless endpoints, while SageMaker incorporates Nvidia’s TensorRT-LLM for automatic compilation and deployment to P8 instances—slashing inference latency by up to 60% compared to stock PyTorch Serving.

Moreover, AWS has introduced Nvidia-accelerated embeddings and retrieval-augmented generation (RAG) pipelines that run natively inside Amazon OpenSearch Service. This means enterprises can build private, context-aware AI chatbots that combine their proprietary data with the power of Nvidia GPUs, all managed through the control plane’s access policies.

What It Means for Windows-Centric Enterprises

While the hyperscale cloud world often feels Linux-dominated, Nvidia’s AI Enterprise platform has supported Windows Server 2025 and later editions since the Blackwell generation, and AWS’s new control plane is designed to manage GPU instances running any operating system, including Windows. For Microsoft shops running SQL Server ML Services or building Windows-based AI applications with .NET and ML.NET, this is a major unlock: they can now leverage the same high-performance Nvidia GPUs and unified governance that Linux users enjoy, all within their existing AWS accounts.

AWS also announced a Windows AI Toolkit bundle available on Amazon WorkSpaces workstations powered by Nvidia GPUs. The toolkit, which includes pre-configured Visual Studio 2027 extensions, CUDA for Windows, and a local inference runtime for NIM, slashes the setup time for Windows AI developers from days to minutes. “We’ve heard from countless enterprise customers that their Windows developer base wants to build AI without moving to Linux containers,” said an AWS VP of Windows and Enterprise Applications. “Now they can stay in Windows, and the control plane manages the entire workflow.”

Furthermore, Windows Server instances on AWS can now join the same control plane policy groups as Linux nodes, making hybrid Windows–Linux AI environments a first-class citizen. This is critical for industries like finance that often run legacy Windows-based data pipelines alongside new AI workloads.

Hybrid and Edge: Extending the Reach

For enterprises with on-premises or edge requirements due to latency, data sovereignty, or air-gapped environments, the control plane extends to AWS Outposts racks equipped with Nvidia GPUs, and to Nvidia’s own Project Ceiba supercomputer, which was co-developed with AWS and now acts as a reference architecture for hybrid AI. Administrators can manage Cloud and on-prem clusters from the same console, pushing updated models and policies across environments with a single click.

This hybrid capability is especially appealing for manufacturing, healthcare, and defense sectors. An early adopter, a global automaker, reported that it reduced its model deployment cycle from four weeks to three days by using the control plane to manage Nvidia-powered inference on Outposts in its factories, while keeping training on the P8 cloud clusters.

Competitive Landscape: Azure and Google on Notice

The move puts pressure on Microsoft Azure, which has its own deep partnership with Nvidia and houses OpenAI’s largest training clusters, but lacks a dedicated AI control plane that spans across its services to the depth AWS now offers. Azure’s AI Studio and Machine Learning platform provide model management, but governance is often siloed by resource type. Google Cloud’s AI Hypercomputer also integrates tightly with Nvidia GPUs but hasn’t matched the enterprise policy framework Amazon Web Services now touts.

Industry analysts view the AWS–Nvidia control plane as a direct shot at locking in enterprise AI buyers. “By solving the operational headaches, AWS makes it extremely costly for customers to switch to another cloud for AI, because they’d have to rebuild their governance and orchestration layers from scratch,” said an analyst at Forrester Research. “It’s a classic platform play.”

Looking Ahead

AWS and Nvidia have already outlined a roadmap for the control plane that includes automated model evaluation, drift detection, and an AI marketplace where third-party ISVs can offer validated AI applications that plug directly into the control plane’s policy framework. Meanwhile, the two companies are jointly developing a confidential AI computing module that encrypts data in use—a direct response to enterprise demand for secure, private AI.

With AI spending projected to exceed $300 billion by 2027, the battle for enterprise wallets is shifting from raw performance to operational simplicity. AWS and Nvidia are betting that the next phase of AI adoption will be defined not by who has the fastest chip, but by who can offer the most seamless, secure, and cost-effective operational experience. With the AI control plane, they’ve laid the groundwork to become the default destination for production AI workloads, potentially locking in enterprise customers for the next decade.

For Windows-focused IT teams, this means more options and better integration. The days of treating AI as a Linux-only experiment are over. The control plane brings Windows squarely into the AI factory, where it can be managed with the same rigor as any other production workload.