> ## Documentation Index > Fetch the complete documentation index at: https://docs.baseten.co/llms.txt > Use this file to discover all available pages before exploring further. # Deploy RL Rollouts > Publish a checkpoint to Amazon S3 or Google Cloud Storage, deploy a rollout model, and load new weights without redeploying. Deploy GLM 5.3 Flash on 8 B200 GPUs, then update its weights from checkpoints in your bucket. To deploy another model, choose its configuration from [Supported models](https://docs.baseten.co/rl-rollouts/overview.md#supported-models). > **[Get the code](https://github.com/basetenlabs/rl-rollouts)** > > Download the model configurations and checkpoint publisher from GitHub. ## Prerequisites * [Contact Baseten](mailto:support@baseten.co) to enable [Baseten Inference Stack (BIS)](https://docs.baseten.co/engines/bis-llm/overview.md) and your model's accelerator on your workspace. * Prepare a full Hugging Face checkpoint that matches your model's [checkpoint requirements](https://github.com/basetenlabs/rl-rollouts#best-practices). * Create an [Amazon S3 bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/create-bucket-overview.html) or a [Google Cloud Storage (GCS) bucket](https://cloud.google.com/storage/docs/creating-buckets). Choose a prefix for this training run, ideally in a bucket near your deployment region. **1. Install and authenticate the CLI** Install the Baseten CLI: **macOS or Linux** ```bash Terminal theme={"system"} brew tap basetenlabs/baseten brew trust --formula basetenlabs/baseten/baseten brew install baseten ``` **Windows** Download and extract the Windows archive for the latest Baseten CLI release: ```powershell Terminal theme={"system"} $release = Invoke-RestMethod https://api.github.com/repos/basetenlabs/baseten-cli/releases/latest $asset = $release.assets | Where-Object { $_.name -like "*_windows_amd64.zip" } Invoke-WebRequest $asset.browser_download_url -OutFile baseten.zip Expand-Archive -Force baseten.zip . ``` Move `baseten.exe` to a directory on your `PATH`. For other platforms or a specific version, see the [Baseten CLI install reference](https://docs.baseten.co/reference/cli/baseten/overview.md#install). [Create a Baseten API key](https://docs.baseten.co/organization/api-keys.md#create-an-api-key), then set it in your terminal: ```sh theme={"system"} export BASETEN_API_KEY="" ``` The CLI and the rollout requests below use this key. See [Use API keys with CLIs](https://docs.baseten.co/organization/api-keys.md#use-api-keys-with-clis) for other authentication options. **2. Clone the repository** Download the configurations and publisher, then install the publisher's Python dependencies: ```sh theme={"system"} git clone https://github.com/basetenlabs/rl-rollouts.git cd rl-rollouts python -m pip install boto3 numpy safetensors zstandard ``` **3. Configure publisher access** The checkpoint publisher runs on your trainer's side, and you decide how your training loop calls it; `publish_checkpoint.py` is a reference helper. Give it credentials to read and upload objects in your run's prefix. Choose your cloud provider below and replace the placeholders with your credentials and bucket URI. **Amazon S3** Grant the publisher `s3:ListBucket` on the bucket and `s3:GetObject`, `s3:PutObject`, and `s3:AbortMultipartUpload` on objects in the prefix. See [S3 permissions](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-with-s3-policy-actions.html) for policy setup. To use access keys, set these environment variables: ```sh theme={"system"} export AWS_ACCESS_KEY_ID="" export AWS_SECRET_ACCESS_KEY="" export AWS_DEFAULT_REGION="" export ROLLOUT_BUCKET="s3://your-bucket/your-run" ``` For profiles, roles, or temporary credentials, follow the [Boto3 credential setup](https://docs.aws.amazon.com/boto3/latest/guide/credentials.html). **Google Cloud Storage** Create a service account with the [Storage Object User role](https://cloud.google.com/storage/docs/access-control/iam-roles) (`roles/storage.objectUser`) on the bucket, then [create an HMAC key](https://cloud.google.com/storage/docs/authentication/managing-hmackeys) for it. Set the HMAC access ID and secret in the variables below. The publisher uses GCS's S3-compatible API, which requires these AWS variable names: ```sh theme={"system"} export AWS_ACCESS_KEY_ID="" export AWS_SECRET_ACCESS_KEY="" unset AWS_SESSION_TOKEN export ROLLOUT_BUCKET="gs://your-bucket/your-run" ``` Clearing `AWS_SESSION_TOKEN` prevents an existing AWS session token from accompanying your GCS credentials. **4. Create deployment secrets** The deployment uses the `rl_checkpoint_reader` [workspace secret](https://docs.baseten.co/organization/secrets.md) to list and read checkpoints. Create separate reader credentials and save their values in a local file named `reader.json`. **Amazon S3** [Create an IAM access key](https://docs.aws.amazon.com/IAM/latest/UserGuide/access-key-self-managed.html) for a user with `s3:ListBucket` on the bucket and `s3:GetObject` on objects in your prefix. The deployment requires static keys because temporary session credentials expire. Save the reader's access key ID, secret access key, and bucket region: ```json reader.json theme={"system"} { "aws_access_key_id": "", "aws_secret_access_key": "", "aws_region": "" } ``` **Google Cloud Storage** Create a separate service account with the [Storage Object Viewer role](https://cloud.google.com/storage/docs/access-control/iam-roles) (`roles/storage.objectViewer`) on the bucket, then [create an HMAC key](https://cloud.google.com/storage/docs/authentication/managing-hmackeys) for it. Save the reader's HMAC access ID and secret using the field names below. Keep `aws_region` set to `us-east-1` for GCS: ```json reader.json theme={"system"} { "aws_access_key_id": "", "aws_secret_access_key": "", "aws_region": "us-east-1" } ``` Upload the reader credentials and create `rl_bdn_partition`, a random value that isolates your checkpoint cache in the Baseten Delivery Network (BDN): ```sh theme={"system"} baseten org secret set --name rl_checkpoint_reader < reader.json openssl rand -hex 32 | baseten org secret set --name rl_bdn_partition ``` See [`baseten org secret set`](https://docs.baseten.co/reference/cli/baseten/org-secret#set) for details. Keep `reader.json` out of version control. If deployment later prompts for a Hugging Face token, save a [read token](https://huggingface.co/docs/hub/security-tokens) in an `hf_access_token` secret: ```sh theme={"system"} baseten org secret set --name hf_access_token ``` **5. Set your bucket** Open `glm-5.3-flash/b200-8/config.yaml`. On the line marked `EDIT`, set `BUCKET` to the same bucket and prefix you used for `ROLLOUT_BUCKET`: **Amazon S3** ```yaml theme={"system"} BUCKET: s3://your-bucket/your-run ``` **Google Cloud Storage** ```yaml theme={"system"} BUCKET: gs://your-bucket/your-run ``` Edit only `BUCKET` to keep the configuration Baseten has validated for your model and accelerator. **6. Publish your starting checkpoint** Upload your full checkpoint as the anchor `step000001`, then write `STARTUP_SNAPSHOT` so new replicas know which snapshot to load. Replace `` with your checkpoint's local directory: ```sh theme={"system"} python publish_checkpoint.py full --dir "" \ --bucket "$ROLLOUT_BUCKET" --identity step000001 --policy-version 1 python publish_checkpoint.py startup \ --bucket "$ROLLOUT_BUCKET" --identity step000001 ``` The startup command confirms the snapshot selection: ```text theme={"system"} STARTUP_SNAPSHOT names step000001 ``` **7. Push the deployment** Push the configuration file to create a model named `my-rollouts`: ```sh theme={"system"} baseten model push --file glm-5.3-flash/b200-8/config.yaml \ --override-name my-rollouts --wait ``` This uploads the configuration file on its own. The command waits until the deployment is active. Copy the model ID and deployment ID from its output to use in the requests below. See [`baseten model push`](https://docs.baseten.co/reference/cli/baseten/model#push) for deployment options. **8. Confirm the starting snapshot** Check which snapshot each replica serves. Replace `` and `` with the IDs from the push: ```sh theme={"system"} curl -sS "https://model-.api.baseten.co/deployment//sync/hot_load/v1/models/hot_load" \ -H "Authorization: Bearer $BASETEN_API_KEY" ``` Repeat the request until every entry in `replicas` reports `current_snapshot_identity: "step000001"` and `loading_state.stage: "done"`. If a replica reports `error`, check its `reason` and follow the [hot-load recovery instructions](https://github.com/basetenlabs/rl-rollouts#hot-load-api). Send a rollout request that includes the policy version in its response: ```sh theme={"system"} curl -sS "https://model-.api.baseten.co/deployment//sync/v1/chat/completions" \ -H "Authorization: Bearer $BASETEN_API_KEY" \ -H "Content-Type: application/json" \ -d '{"model": "glm-5.3-flash-fp8", "messages": [{"role": "user", "content": "Hello"}], "max_tokens": 32, "nvext": {"extra_fields": ["engine_data"]}}' ``` Check that `nvext.engine_data.policy_version` is `1`, the policy version you assigned to the anchor. For another model, use the `served_model_name` from its configuration in your request. **9. Load a new snapshot** After your trainer [publishes a delta](https://github.com/basetenlabs/rl-rollouts#delta-format-arc_v2) as `step000002`, request it to update the running deployment: ```sh theme={"system"} curl -sS -X POST "https://model-.api.baseten.co/deployment//sync/hot_load/v1/models/hot_load" \ -H "Authorization: Bearer $BASETEN_API_KEY" \ -H "Content-Type: application/json" \ -d '{"identity": "step000002", "incremental_snapshot_metadata": {"compression_format": "arc_v2", "checksum_format": "alder32"}}' ``` HTTP `200` confirms that the service accepted the update. Repeat the status request from the previous step until every replica reports `current_snapshot_identity: "step000002"` and `loading_state.stage: "done"`. See the [hot-load API reference](https://github.com/basetenlabs/rl-rollouts#hot-load-api) for errors and retries. ## Clean up Deactivate the deployment when you finish to stop its replicas and GPU charges: ```sh theme={"system"} baseten model deployment deactivate --model-id --deployment-id --yes ``` ## Upgrade a deployment To upgrade the runtime or model configuration, pull the latest files from the repository, restore your `BUCKET` value, and push a new deployment. Check the [configuration changelog](https://github.com/basetenlabs/rl-rollouts/blob/main/CHANGELOG.md) before upgrading. To update weights during training, continue publishing snapshots to the running deployment. ## Next steps * Use the [RL request and response fields](https://github.com/basetenlabs/rl-rollouts/blob/main/rl-api.md) to collect token IDs, log probabilities, expert routes, and policy versions. * Connect your training loop with the [Miles and Slime integration guide](https://github.com/basetenlabs/rl-rollouts#using-the-publisher-in-an-rl-loop-miles-and-slime).