Skip to the yard
Waxsealai

Google’s OpenRL Brings Reinforcement Learning Workloads to Kubernetes

Shipped
Shipper
Jonathan Anthony
Readers
793

OpenRL separates reinforcement learning experiments from Kubernetes infrastructure, while Agent Substrate provides isolated sandboxes for agent tasks.

Google’s OpenRL Brings Reinforcement Learning Workloads to Kubernetes

Reinforcement learning can put several kinds of work into one training loop: model updates, inference, agent actions and reward evaluation. Google’s OpenRL aims to make that workload easier to run on Kubernetes by giving researchers a training API while leaving scaling and infrastructure management to the platform. The project’s practical focus is the boundary between flexible experiments and the systems needed to run them reliably.

Google donated OpenRL to the Cloud Native Computing Foundation. The work grew out of GKE Labs, Google’s experimental project group. Senior software engineer Sunil Arora said the team first built an AI agent for Kubernetes tasks and then created a benchmark to assess models on those tasks.

Separating experiments from infrastructure

OpenRL debuted as a research preview in June 2026. It offers a self-hosted training API that runs on a company’s own Kubernetes cluster. Its central design choice is to separate the reinforcement learning loop from the compute plumbing: infrastructure engineers can manage scaling, while researchers and AI engineers retain control over their experiments.

That division matters because reinforcement learning brings different activities together. A system may need to train a model, generate samples, run an agent in a sandbox and assess what the agent did. As Arora put it, the workload combines training, sampling, agent activity and evaluation. That variety can put pressure on infrastructure designed around more uniform jobs.

OpenRL’s API is intended to give teams core infrastructure primitives without requiring them to build the underlying orchestration around each experiment. Researchers can focus on how the loop works, while platform teams handle the Kubernetes resources that support it. The available details don’t specify performance results, supported model sizes or deployment costs, so the preview’s value will depend on how well its abstractions fit the needs of individual teams.

Sandboxes contain agent actions

Agent environments are another part of the infrastructure problem. Reward calculations can require running millions of isolated commands, according to Arora. Each command or agent task needs a controlled place to run, and creating too much isolation overhead can make it harder to use available compute efficiently.

Google’s Agent Substrate is designed to pack more sandboxes onto limited compute, Arora said. It addresses a different layer of the work from OpenRL: OpenRL structures the training API, while Agent Substrate supports isolated environments where agents can act. Together, they reflect a broader engineering need in reinforcement learning systems, where model training and software execution must operate within the same workflow.

Developer Justin Santa Barbara described that need through kubectl-ai, an agent for Kubernetes tasks. An agent with access to operational commands could make a damaging change, such as formatting a disk. Running it in a sandbox gives the agent a bounded environment in which an unintended action affects the sandbox instead of the host system. This example shows why isolation is part of the execution design, rather than an optional add-on.

What teams should watch

For teams considering a self-hosted reinforcement learning stack, the separation of concerns is the key idea to evaluate. Researchers need room to change experiments, while platform engineers need predictable ways to allocate and scale resources. An API can reduce the amount of infrastructure code researchers must manage, but teams still need to understand how it fits their Kubernetes setup and how sandbox capacity affects throughput.

The project’s early status also sets limits on what can be concluded. OpenRL was introduced as a research preview, and the available information provides no benchmark figures for training speed, resource use or model quality. Google’s benchmark for Kubernetes agents is part of the team’s work, but no scores or comparison results are given here. Those measurements will be important for judging how the approach performs against existing workflows.

OpenRL’s donation gives the project a path to development beyond Google’s own engineering group, while its self-hosted design places the workload on a team’s own cluster. The next useful evidence will be concrete: how easily researchers can adapt their loops, how well sandboxed tasks share compute, and what operational effort the API removes. For now, the proposal is clear: treat reinforcement learning as a Kubernetes workload with distinct training and sandboxing needs, and give each part an infrastructure interface suited to its role.

Source: Jonathan Anthony · siliconangle.com

Same vesselRelated cargo