What Happened
On October 27, 2026, Amazon ECS added GPU sharing through NVIDIA Multi-Process Service (MPS). Multiple ECS tasks can now run concurrently on one NVIDIA GPU. Teams specify each container’s GPU memory requirement and can optionally limit its compute share; ECS places tasks where GPU capacity is available. The feature works with ECS Managed Instances and ECS on EC2 in all AWS commercial and AWS GovCloud (US) Regions. No application code changes are required to enable sharing. [1]
Why It Matters to Businesses
Small-model inference, embedding generation and multi-model serving can leave dedicated GPUs underused. Sharing offers a way to place more work on existing capacity and potentially lower GPU costs. The benefit depends on measured utilization and latency, not simply on how many tasks fit on a device. [1]
Kimbodo Engineering Perspective
GPU memory requirements should be treated as a placement control, not a substitute for workload testing. Co-located tasks may compete for compute and memory bandwidth. We would benchmark each workload alone and alongside its intended neighbors before raising task density, particularly where latency targets matter.
How We Would Implement It
- Identify GPU-backed ECS services with low utilization and record baseline throughput, latency, GPU memory use and cost per request.
- Enable MPS for a limited service group, set per-container GPU memory requirements and use compute-share limits where contention affects performance. [1]
- Load-test realistic task combinations, then increase density only while latency and error rates remain within target.
- Monitor placement, task failures and GPU utilization. For ECS Managed Instances, include recovery testing: ECS replaces instances with unhealthy MPS daemons and moves tasks to healthy capacity. [1]
Risks, Costs and Security
Sharing increases the impact of a busy or failing task on other work using the same GPU. Keep spare capacity for rescheduling, alert on sustained contention and compare savings against any latency degradation. For sensitive workloads, assess whether co-location meets isolation requirements before sharing a device across tenants or trust boundaries.
Where Kimbodo Comes In
Kimbodo builds and operates this in production for businesses — see our AI Application Development practice, or Estimate My AI Application.
