Deploy RL Rollouts

Deploy GLM 5.3 Flash on 8 B200 GPUs, then update its weights from checkpoints in your bucket. To deploy another model, choose its configuration from Supported models.

Get the code

Download the model configurations and checkpoint publisher from GitHub.

Prerequisites

Contact Baseten to enable Baseten Inference Stack (BIS) and your model’s accelerator on your workspace.
Prepare a full Hugging Face checkpoint that matches your model’s checkpoint requirements.
Create an Amazon S3 bucket or a Google Cloud Storage (GCS) bucket. Choose a prefix for this training run, ideally in a bucket near your deployment region.

Install and authenticate the CLI

Install the Baseten CLI:

macOS or Linux
Windows

Terminal

brew tap basetenlabs/baseten
brew trust --formula basetenlabs/baseten/baseten
brew install baseten

Download and extract the Windows archive for the latest Baseten CLI release:

Terminal

$release = Invoke-RestMethod https://api.github.com/repos/basetenlabs/baseten-cli/releases/latest
$asset = $release.assets | Where-Object { $_.name -like "*_windows_amd64.zip" }
Invoke-WebRequest $asset.browser_download_url -OutFile baseten.zip
Expand-Archive -Force baseten.zip .

Move baseten.exe to a directory on your PATH.

For other platforms or a specific version, see the Baseten CLI install reference.Create a Baseten API key, then set it in your terminal:

export BASETEN_API_KEY="<BASETEN_API_KEY>"

The CLI and the rollout requests below use this key. See Use API keys with CLIs for other authentication options.

Clone the repository

Download the configurations and publisher, then install the publisher’s Python dependencies:

git clone https://github.com/basetenlabs/rl-rollouts.git
cd rl-rollouts
python -m pip install boto3 numpy safetensors zstandard

Configure publisher access

The checkpoint publisher runs on your trainer’s side, and you decide how your training loop calls it; publish_checkpoint.py is a reference helper. Give it credentials to read and upload objects in your run’s prefix. Choose your cloud provider below and replace the placeholders with your credentials and bucket URI.

Amazon S3
Google Cloud Storage

Grant the publisher s3:ListBucket on the bucket and s3:GetObject, s3:PutObject, and s3:AbortMultipartUpload on objects in the prefix. See S3 permissions for policy setup.To use access keys, set these environment variables:

export AWS_ACCESS_KEY_ID="<PUBLISHER_ACCESS_KEY_ID>"
export AWS_SECRET_ACCESS_KEY="<PUBLISHER_SECRET_ACCESS_KEY>"
export AWS_DEFAULT_REGION="<BUCKET_REGION>"
export ROLLOUT_BUCKET="s3://your-bucket/your-run"

For profiles, roles, or temporary credentials, follow the Boto3 credential setup.

Create a service account with the Storage Object User role (roles/storage.objectUser) on the bucket, then create an HMAC key for it.Set the HMAC access ID and secret in the variables below. The publisher uses GCS’s S3-compatible API, which requires these AWS variable names:

export AWS_ACCESS_KEY_ID="<PUBLISHER_HMAC_ACCESS_ID>"
export AWS_SECRET_ACCESS_KEY="<PUBLISHER_HMAC_SECRET>"
unset AWS_SESSION_TOKEN
export ROLLOUT_BUCKET="gs://your-bucket/your-run"

Clearing AWS_SESSION_TOKEN prevents an existing AWS session token from accompanying your GCS credentials.

Create deployment secrets

The deployment uses the rl_checkpoint_reader workspace secret to list and read checkpoints. Create separate reader credentials and save their values in a local file named reader.json.

Amazon S3
Google Cloud Storage

Create an IAM access key for a user with s3:ListBucket on the bucket and s3:GetObject on objects in your prefix. The deployment requires static keys because temporary session credentials expire.Save the reader’s access key ID, secret access key, and bucket region:

reader.json

{
  "aws_access_key_id": "<READER_ACCESS_KEY_ID>",
  "aws_secret_access_key": "<READER_SECRET_ACCESS_KEY>",
  "aws_region": "<BUCKET_REGION>"
}

Create a separate service account with the Storage Object Viewer role (roles/storage.objectViewer) on the bucket, then create an HMAC key for it.Save the reader’s HMAC access ID and secret using the field names below. Keep aws_region set to us-east-1 for GCS:

reader.json

{
  "aws_access_key_id": "<READER_HMAC_ACCESS_ID>",
  "aws_secret_access_key": "<READER_HMAC_SECRET>",
  "aws_region": "us-east-1"
}

Upload the reader credentials and create rl_bdn_partition, a random value that isolates your checkpoint cache in the Baseten Delivery Network (BDN):

baseten org secret set --name rl_checkpoint_reader < reader.json
openssl rand -hex 32 | baseten org secret set --name rl_bdn_partition

See baseten org secret set for details. Keep reader.json out of version control.If deployment later prompts for a Hugging Face token, save a read token in an hf_access_token secret:

baseten org secret set --name hf_access_token

Set your bucket

Open glm-5.3-flash/b200-8/config.yaml. On the line marked EDIT, set BUCKET to the same bucket and prefix you used for ROLLOUT_BUCKET:

Amazon S3
Google Cloud Storage

BUCKET: s3://your-bucket/your-run

BUCKET: gs://your-bucket/your-run

Edit only BUCKET to keep the configuration Baseten has validated for your model and accelerator.

Publish your starting checkpoint

Upload your full checkpoint as the anchor step000001, then write STARTUP_SNAPSHOT so new replicas know which snapshot to load. Replace <HF_CHECKPOINT_DIR> with your checkpoint’s local directory:

python publish_checkpoint.py full --dir "<HF_CHECKPOINT_DIR>" \
  --bucket "$ROLLOUT_BUCKET" --identity step000001 --policy-version 1
python publish_checkpoint.py startup \
  --bucket "$ROLLOUT_BUCKET" --identity step000001

The startup command confirms the snapshot selection:

STARTUP_SNAPSHOT names step000001

Push the deployment

Push the configuration file to create a model named my-rollouts:

baseten model push --file glm-5.3-flash/b200-8/config.yaml \
  --override-name my-rollouts --wait

This uploads the configuration file on its own. The command waits until the deployment is active. Copy the model ID and deployment ID from its output to use in the requests below.See baseten model push for deployment options.

Confirm the starting snapshot

Check which snapshot each replica serves. Replace <MODEL_ID> and <DEPLOYMENT_ID> with the IDs from the push:

curl -sS "https://model-<MODEL_ID>.api.baseten.co/deployment/<DEPLOYMENT_ID>/sync/hot_load/v1/models/hot_load" \
  -H "Authorization: Bearer $BASETEN_API_KEY"

Repeat the request until every entry in replicas reports current_snapshot_identity: "step000001" and loading_state.stage: "done". If a replica reports error, check its reason and follow the hot-load recovery instructions.Send a rollout request that includes the policy version in its response:

curl -sS "https://model-<MODEL_ID>.api.baseten.co/deployment/<DEPLOYMENT_ID>/sync/v1/chat/completions" \
  -H "Authorization: Bearer $BASETEN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "glm-5.3-flash-fp8", "messages": [{"role": "user", "content": "Hello"}], "max_tokens": 32, "nvext": {"extra_fields": ["engine_data"]}}'

Check that nvext.engine_data.policy_version is 1, the policy version you assigned to the anchor. For another model, use the served_model_name from its configuration in your request.

Load a new snapshot

After your trainer publishes a delta as step000002, request it to update the running deployment:

curl -sS -X POST "https://model-<MODEL_ID>.api.baseten.co/deployment/<DEPLOYMENT_ID>/sync/hot_load/v1/models/hot_load" \
  -H "Authorization: Bearer $BASETEN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"identity": "step000002", "incremental_snapshot_metadata": {"compression_format": "arc_v2", "checksum_format": "alder32"}}'

HTTP 200 confirms that the service accepted the update. Repeat the status request from the previous step until every replica reports current_snapshot_identity: "step000002" and loading_state.stage: "done". See the hot-load API reference for errors and retries.

Clean up

Deactivate the deployment when you finish to stop its replicas and GPU charges:

baseten model deployment deactivate --model-id <MODEL_ID> --deployment-id <DEPLOYMENT_ID> --yes

Upgrade a deployment

To upgrade the runtime or model configuration, pull the latest files from the repository, restore your BUCKET value, and push a new deployment. Check the configuration changelog before upgrading. To update weights during training, continue publishing snapshots to the running deployment.

Next steps

Use the RL request and response fields to collect token IDs, log probabilities, expert routes, and policy versions.
Connect your training loop with the Miles and Slime integration guide.

Get started

Model APIs

Inference

Frontier Gateway

Training

Sandboxes

Development

Deployment

Engines

Model Labs

Organization

Hosting options

Observability

Troubleshooting

Deploy RL Rollouts

Get the code

Prerequisites

Clean up

Upgrade a deployment

Next steps

Get the code

​Prerequisites

​Clean up

​Upgrade a deployment

​Next steps

Prerequisites

Clean up

Upgrade a deployment

Next steps