> ## Documentation Index
> Fetch the complete documentation index at: https://docs.baseten.co/llms.txt
> Use this file to discover all available pages before exploring further.

# RL Rollouts

> Run your own RL trainer and use Baseten to serve rollouts, with weight updates from checkpoints in your bucket.

With RL Rollouts, you run your own reinforcement learning (RL) trainer and use Baseten to serve rollouts on dedicated inference deployments. Your trainer publishes checkpoints to your Amazon S3 or Google Cloud Storage (GCS) bucket, and the deployment loads new weights without redeploying.

Start with [Deploy RL Rollouts](https://docs.baseten.co/rl-rollouts/deploy.md) to connect a model to your checkpoints. To use Baseten-managed trainers, see [Loops](https://docs.baseten.co/loops/overview.md).

## Capabilities

* Deltas contain compressed changes between checkpoints, reducing the data you upload after each training step. In [Baseten's GLM-5.3 measurements](https://www.baseten.co/blog/optimizing-delta-weight-syncs-for-managed-rollouts/), weight updates reached independent clusters in under 40 seconds, including about 6 seconds of request pause.
* The [Baseten Delivery Network (BDN)](https://docs.baseten.co/development/model/bdn.md) distributes checkpoints to rollout clusters and reduces repeated downloads from your bucket.
* [Multi-cloud Capacity Management (MCM)](https://docs.baseten.co/concepts/howbasetenworks.md#multi-cloud-capacity-management-mcm) scales your rollout fleet across Baseten's global compute.
* The [Baseten Inference Stack (BIS)](https://docs.baseten.co/engines/bis-llm/overview.md) serves models with [KV-aware routing](https://docs.baseten.co/engines/bis-llm/advanced-features.md#kv-aware-routing) to help multi-turn episodes reuse their cache.
* Responses can return token IDs, top-k log probabilities, mixture-of-experts (MoE) routes for router replay, and the top-p sampling mask. They can also report the policy version that produced them, so asynchronous RL can sample across policy versions.
* Baseten can work with you to align the model's numerical behavior with your trainer.

## How it works

1. **Publish an anchor.** Upload a full checkpoint to your bucket. This checkpoint is the starting point for later updates.
2. **Publish deltas.** After each training step, upload the changes from the previous snapshot.
3. **Load the snapshot.** Send a hot-load request naming the snapshot. Each replica prepares the update while serving, then pauses briefly to swap weights.
4. **Track the policy.** Request `engine_data` in each rollout to record the policy version and replica. Replicas update independently, so your trainer can briefly receive samples from different policies.

Each deployment serves an OpenAI-compatible chat completions endpoint.

## Supported models

| Model | Accelerator | Folder in the repository |
| - | - | - |
| GLM 5.3 Flash | B200:8 | `glm-5.3-flash/b200-8` |
| GLM 5.2 | B300:8 | `glm-5.2/b300-8` |
| Kimi K3 | GB300:4, 2 nodes | `kimi-k3/gb300-8` |
| DeepSeek V4.1 Flash | B200:8 | `deepseek-v4.1-flash-mxfp4/b200-8` |
| Qwen 3.5 9B | H100:1 | `qwen3.5-9b/h100-1` |

Each folder contains a configuration that Baseten has validated for the listed model and accelerator. Set its `BUCKET` value to your bucket and prefix. Check the repository's [checkpoint requirements](https://github.com/basetenlabs/rl-rollouts#best-practices) before publishing weights for your model.

## The RL Rollouts repository

Use the [RL Rollouts repository](https://github.com/basetenlabs/rl-rollouts) for model configurations, checkpoint requirements, and API details:

| File | What it's for |
| - | - |
| `<model>/<accelerator>/config.yaml` | The configuration you deploy. |
| `README.md` | Deployment instructions, checkpoint requirements, bucket layout, hot-load API, and training framework integrations. |
| `rl-api.md` | Request and response fields for token IDs, log probabilities, expert routes, and policy versions. |
| `publish_checkpoint.py` | Publishes anchors and deltas to your bucket in the required layout. |
| `CHANGELOG.md` | What changed in each configuration. |

> **[Deploy RL Rollouts](https://docs.baseten.co/rl-rollouts/deploy.md)**
>
> Clone the repository, set your bucket, and push your first deployment.