When Google introduced Kubernetes twelve years ago, the Pod became the default unit of compute for the cloud. The design assumed a straightforward lifecycle: a program boots, binds to a port, serves traffic, and runs until you stop it.
AI agents work fundamentally differently.
An AI agent that’s running on a machine executes untrusted code, commands, and tool calls, and must be isolated from its host and its network resources. At the same time, these agents, whether they’re an interactive coding assistant, an SRE troubleshooter, or a multi-turn evaluation loop, do not run continuously. They execute a step, call a language model, and wait. Most of the time, the agent does nothing at all while it waits for a model to stream tokens or a human to review a pull request. In the same way that Kubernetes was created to address the needs of containerized applications, we introduced Agent Substrate as a new orchestrator for the emerging agentic applications. Secure by default, highly efficient and built for multi-million agent scale.
Agent Substrate was designed from the ground up for AI agent security and density and brings the best parts of Kubernetes together with a new control and data plane for fully-isolated, ultra-low-latency sandboxed compute and storage. The growing Agent Substrate community has been rapidly building core capabilities and stabilizing APIs, culminating in the latest 1.0 release.
This week at KubeCon North America in Salt Lake City, Google Cloud is donating Agent Substrate and gVisor to the Cloud Native Computing Foundation (CNCF) in Sandbox status, alongside OpenRL.
This post explains why traditional approaches struggle with stateful agents, how Agent Substrate solves the problem, how it connects to enterprise governance on Google Kubernetes Engine (GKE), and where to see it in action at KubeCon NA 2026.
Challenges of running AI agents at scale
AI agents introduce two infrastructure bottlenecks that standard Kubernetes was never built to handle: idle resource waste and control plane overload.
1. The idle resource problem
During a typical interactive session, an agent can spend 90% to 95% of its time waiting for model inference, external APIs, or human reviews.
During that wait, the agent consumes zero CPU. However, you cannot simply kill the container. The agent holds critical state in memory and on disk: active language runtimes, local Git repositories, uncommitted file edits, and tool sessions.
With standard Kubernetes, platform engineers face two poor options:
- Keep a dedicated Pod running all day: The agent responds quickly because it is already warm, but you pay for idle compute around the clock. At ten thousand active users, cluster costs become unsustainable.
- Scale to zero with serverless containers: You save compute costs while the agent sits idle, but resuming from cold storage forces the container to clone repositories, reinstall packages, and rebuild memory. Wakeup latency climbs past 20 seconds, breaking the interactive experience.
Platform teams need agents that wake up in under 500 milliseconds with full memory and disk state intact—without paying for running Pods while the agent sits idle.
2. Control plane overload
The opposite issue occurs when training or evaluating agents. Reinforcement learning loops and benchmark suites need to run untrusted agent code across thousands of tasks at the exact same time.
If every sandbox requires a full Kubernetes Pod, the control plane quickly grinds to a halt. Scheduling, creating, and destroying thousands of short-lived Pods every minute overwhelms etcd, saturates kube-apiserver, and exhausts subnet IP addresses. Kubernetes was built to manage stable application topologies, not to act as a microsecond sandbox allocator.
How Agent Substrate solves the problem
Agent Substrate adds an actor-based control and data plane built on top of Kubernetes. It solves the idle resource problem by decoupling Actors from Workers:
- Actors hold the state: An Actor represents an individual agent session, complete with its own filesystem, environment variables, memory state, and network rules.
- Workers provide the compute: A Worker is a pre-warmed Kubernetes Pod running in a shared pool (WorkerPool), equipped with container runtimes and sandbox boundaries.
The lifecycle of an agent turn
- Request parking: An incoming call hits the atenet-router L7 proxy. If the target Actor is suspended, the proxy holds the request in memory and alerts the Substrate controller.
- Fast placement: The controller assigns the Actor to an available slot in the warm Worker pool.
- Sub-second restore: The Actor’s memory and disk state restore from tiered storage, the process wakes up, and the proxy delivers the request. The agent resumes in under 500 milliseconds right where it paused.
- Suspension on idle: When the turn finishes, Substrate snapshots the Actor’s state to tiered storage and frees the Worker slot for other workloads.
Because Substrate manages Actor state directly, thousands of agents can activate, suspend, and resume every second without creating new Pod objects or stressing etcd.
Safe execution with gVisor and microVMs
Agents frequently execute model-generated, untrusted code. Standard Linux containers share the host kernel, which presents an unacceptable security risk on multi-tenant nodes.
Agent Substrate pairs with gVisor (runsc), Google's open source application kernel written in Go. gVisor intercepts system calls in user space, isolating untrusted code from the host kernel. Because gVisor virtualizes its own memory and process tables, Substrate can freeze a running environment, write memory pages to storage, and restore them on another node in under 500 milliseconds.
For workloads requiring hardware virtualization or custom kernel modules, Substrate also supports Cloud Hypervisor microVMs under the same unified API.
Why Google donated Agent Substrate to the CNCF
Over the past two years, engineering teams across the industry have run into the same agent scaling boundaries. Without a shared open standard, every organization ends up building custom schedulers on top of VMs or working around Kubernetes.
Donating Agent Substrate and gVisor to the CNCF gives the cloud-native community a neutral, vendor-independent foundation that works across any conformant Kubernetes cluster.
Together with community projects, they form an open stack for agentic infrastructure:
- Layer 1: Secure Isolation (gVisor): Sandboxes untrusted code execution with user-space kernel isolation and fast memory checkpointing.
- Layer 2: Agent Runtime (Agent Substrate): Multiplexes stateful actors onto warm worker pools with sub-second wakeups.
- Layer 3: Training Orchestration (OpenRL): Coordinates Kubernetes-native reinforcement learning loops and evaluation sandboxes.
Where to find us at KubeCon NA 2026
If you are heading to Salt Lake City for KubeCon + CloudNativeCon North America from November 9 to 12, visit our sessions, workshops, and booth demos:
Get started
Agent Substrate is open source and ready for testing today:
- Documentation: Read the guides and architecture specs at ate.dev.
- Code: Explore the runtime on GitHub at github.com/agent-substrate/substrate.
- Community: Join #substrate-dev #substrate-user on the CNCF Slack.
See you in Salt Lake City!
