Rolling deployments

Rolling deployments replace replicas incrementally when promoting a deployment to an environment. Instead of swapping all traffic at once, rolling deployments scale up the candidate deployment, shift traffic proportionally, and scale down the previous deployment in controlled steps. Autoscaling continues throughout the rollout for environments where min_replica < max_replica, so both deployments scale up to meet traffic demand as it shifts between them. Use rolling deployments when you need zero-downtime updates with the ability to pause, cancel, or force-complete the deployment at any point.

Rolling deployments are not supported for Chains. This feature is available for individual model deployments only.

How rolling deployments work

A rolling deployment follows a repeating three-step cycle:

Scale up candidate deployment replicas by the configured percentage.
Shift traffic proportionally to match the new replica ratio.
Scale down the previous deployment replicas by the same percentage.

This cycle repeats until all traffic and replicas run on the candidate deployment, at which point it becomes the active deployment in the environment. The diagram below shows this cycle in action. The tab strip mirrors the promotion lifecycle: a promotion enters RELEASING when it starts, sits in RAMPING_UP while replicas scale and traffic shifts, can pause as PAUSED, and lands at SUCCEEDED once the candidate serves all traffic. Click any status to freeze the simulation on that stage, then click it again to resume.

Configure rolling_deploy_config

Adjust the values and click Apply to restart the simulation with your configuration.

Provisioning modes

Rolling deployments support two mutually exclusive provisioning modes. You must configure exactly one:

max_surge_percent: Scales up candidate replicas before scaling down previous replicas.
max_unavailable_percent: Scales down previous replicas before scaling up candidate replicas.

Both can’t be non-zero at the same time, and both can’t be zero at the same time.

Enabling rolling deployments

Enable rolling deployments on any environment by updating the environment’s promotion settings. Rolling deployments are disabled by default.

cURL
Python

curl -X PATCH \
  https://api.baseten.co/v1/models/{model_id}/environments/production \
  -H "Authorization: Api-Key $BASETEN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "promotion_settings": {
      "rolling_deploy": true,
      "rolling_deploy_config": {
        "max_surge_percent": 10,
        "max_unavailable_percent": 0,
        "stabilization_time_seconds": 60,
        "replica_overhead_percent": 0
      }
    }
  }'

import requests
import os

API_KEY = os.environ.get("BASETEN_API_KEY")

response = requests.patch(
    "https://api.baseten.co/v1/models/{model_id}/environments/production",
    headers={"Authorization": f"Api-Key {API_KEY}"},
    json={
        "promotion_settings": {
            "rolling_deploy": True,
            "rolling_deploy_config": {
                "max_surge_percent": 10,
                "max_unavailable_percent": 0,
                "stabilization_time_seconds": 60,
                "replica_overhead_percent": 0,
            },
        }
    },
)

print(response.json())

Once rolling deployments are enabled, any subsequent promotion to the environment uses the rolling deployment workflow.

Configuration reference

Configure rolling deployments through the rolling_deploy_config object in the environment’s promotion_settings.

max_surge_percent

integer

default:"10"

Percentage of additional replicas to provision during each step. Set to 0 to use max unavailable mode instead.Range: 0–50

max_unavailable_percent

integer

default:"0"

Percentage of replicas that can be unavailable during each step. Set to 0 to use max surge mode instead.Range: 0–50

stabilization_time_seconds

integer

default:"0"

Seconds to wait after each traffic shift before proceeding to the next step. Use this to monitor metrics between steps.Range: 0–3600

replica_overhead_percent

integer

default:"0"

Percentage of additional replicas to pre-provision on the current deployment before the rolling deployment starts. Useful for environments without autoscaling (min_replica == max_replica) or as a buffer for anticipated traffic spikes during the rollout.Range: 0–500

Additional promotion settings configured at the promotion_settings level:

rolling_deploy

boolean

default:"false"

Enables rolling deployments for the environment.

Deployment statuses

The in_progress_promotion field on the environment detail endpoint tracks the current state of a rolling deployment.

Status	Description
`RELEASING`	Candidate deployment is building and initializing replicas.
`RAMPING_UP`	Scaling up candidate replicas and shifting traffic.
`PAUSED`	Rolling deployment is paused at its current traffic split. No further promotion steps run, but in-flight replica changes and autoscaling continue.
`RAMPING_DOWN`	Graceful cancel in progress. Traffic is shifting back to the previous deployment.
`SUCCEEDED`	Rolling deployment completed. The candidate is now the active deployment.
`FAILED`	Rolling deployment failed. Traffic remains on the previous deployment.
`CANCELED`	Rolling deployment was canceled. Traffic returned to the previous deployment.

The in_progress_promotion object also includes percent_traffic_to_new_version, which reports the current percentage of traffic routed to the candidate deployment.

Deployment control actions

Pause, resume, and force roll forward act on the rolling deployment between steps, not immediately. Replica changes already in progress finish before the action takes effect, so the rolling deployment can keep scaling for a short time after you trigger the action. For example, if the candidate deployment is at 20% traffic and has just been told to scale from 2 to 4 replicas, clicking pause lets the candidate finish scaling to 4 replicas. The traffic split stays pinned at 20% until you resume.

Pause

Pauses the rolling deployment. Use this to inspect metrics or logs before proceeding.

cURL
Python

curl -X POST \
  https://api.baseten.co/v1/models/{model_id}/environments/production/pause_promotion \
  -H "Authorization: Api-Key $BASETEN_API_KEY"

response = requests.post(
    "https://api.baseten.co/v1/models/{model_id}/environments/production/pause_promotion",
    headers={"Authorization": f"Api-Key {API_KEY}"},
)

print(response.json())

Resume

Resumes a paused rolling deployment from where it left off.

cURL
Python

curl -X POST \
  https://api.baseten.co/v1/models/{model_id}/environments/production/resume_promotion \
  -H "Authorization: Api-Key $BASETEN_API_KEY"

response = requests.post(
    "https://api.baseten.co/v1/models/{model_id}/environments/production/resume_promotion",
    headers={"Authorization": f"Api-Key {API_KEY}"},
)

print(response.json())

Cancel

Gracefully cancels the rolling deployment. Traffic ramps back to the previous deployment and candidate replicas scale down.

cURL
Python

curl -X POST \
  https://api.baseten.co/v1/models/{model_id}/environments/production/cancel_promotion \
  -H "Authorization: Api-Key $BASETEN_API_KEY"

response = requests.post(
    "https://api.baseten.co/v1/models/{model_id}/environments/production/cancel_promotion",
    headers={"Authorization": f"Api-Key {API_KEY}"},
)

print(response.json())

Returns a status of CANCELED (instant cancel for non-rolling deployments) or RAMPING_DOWN (graceful rollback for rolling deployments).

Force cancel

Immediately cancels the rolling deployment and returns all traffic to the previous deployment. Use this when you need to roll back without waiting for the graceful ramp-down.

Force canceling may cause brief service disruption if the previous deployment is under-provisioned.

cURL
Python

curl -X POST \
  https://api.baseten.co/v1/models/{model_id}/environments/production/force_cancel_promotion \
  -H "Authorization: Api-Key $BASETEN_API_KEY"

response = requests.post(
    "https://api.baseten.co/v1/models/{model_id}/environments/production/force_cancel_promotion",
    headers={"Authorization": f"Api-Key {API_KEY}"},
)

print(response.json())

Force roll forward

Immediately completes the rolling deployment, shifting all traffic to the candidate deployment. This works even if the deployment is in the process of rolling back.

Force rolling forward may promote an under-provisioned deployment if the candidate has not finished scaling up.

cURL
Python

curl -X POST \
  https://api.baseten.co/v1/models/{model_id}/environments/production/force_roll_forward_promotion \
  -H "Authorization: Api-Key $BASETEN_API_KEY"

response = requests.post(
    "https://api.baseten.co/v1/models/{model_id}/environments/production/force_roll_forward_promotion",
    headers={"Authorization": f"Api-Key {API_KEY}"},
)

print(response.json())

Autoscaling during rolling deployments

For environments configured with autoscaling (min_replica < max_replica), Baseten continues to scale your deployment during a rolling deployment to meet traffic demand. Both the previous and candidate deployments scale up based on combined demand, and new capacity is distributed proportionally to the current traffic split. For example, with traffic split 60/40 between the previous and candidate deployments, an additional 10 replicas of demand provisions 6 replicas to the previous deployment and 4 to the candidate. A few constraints apply during the rolling deployment:

Autoscaling only adds replicas during the rollout. Replicas are not removed in response to falling demand until the rolling deployment completes.
Capacity management continues during a PAUSED rolling deployment. Pausing stops the traffic shift, not capacity management. If traffic increases while paused, both deployments still scale up.
The combined replica count is capped at the environment’s max_replica.

Environments without autoscaling

Environments where min_replica == max_replica have no autoscaling configured, so replica counts stay pinned during the rolling deployment. To pre-provision additional headroom for traffic spikes, set replica_overhead_percent to add replicas to the previous deployment before any traffic shifts. Use stabilization_time_seconds to wait between steps and monitor metrics before the next traffic shift.

Deployment cleanup

After a rolling deployment completes, the promotion_cleanup_strategy setting controls what happens to the previous deployment.

SCALE_TO_ZERO: Scales the previous deployment to zero replicas. It remains available for reactivation. This is the default.
KEEP: Leaves the previous deployment running at its current replica count.
DEACTIVATE: Deactivates the previous deployment. It stops serving traffic and releases all resources.

Set it alongside your other promotion settings:

cURL
Python

curl -X PATCH \
  https://api.baseten.co/v1/models/{model_id}/environments/production \
  -H "Authorization: Api-Key $BASETEN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "promotion_settings": {
      "promotion_cleanup_strategy": "DEACTIVATE"
    }
  }'

response = requests.patch(
    "https://api.baseten.co/v1/models/{model_id}/environments/production",
    headers={"Authorization": f"Api-Key {API_KEY}"},
    json={
        "promotion_settings": {
            "promotion_cleanup_strategy": "DEACTIVATE"
        }
    },
)

print(response.json())

Get started

About Baseten

Model APIs

Inference

Development

Deployment

Engines

Frontier Gateway

Training

Organization

Observability

Troubleshooting

Rolling deployments

How rolling deployments work

Provisioning modes

Enabling rolling deployments

Configuration reference

Deployment statuses

Deployment control actions

Pause

Resume

Cancel

Force cancel

Force roll forward

Autoscaling during rolling deployments

Environments without autoscaling

Deployment cleanup

Get started

About Baseten

Model APIs

Inference

Development

Deployment

Engines

Frontier Gateway

Training

Organization

Observability

Troubleshooting

Documentation Index

​How rolling deployments work

​Provisioning modes

​Enabling rolling deployments

​Configuration reference

​Deployment statuses

​Deployment control actions

​Pause

​Resume

​Cancel

​Force cancel

​Force roll forward

​Autoscaling during rolling deployments

​Environments without autoscaling

​Deployment cleanup

How rolling deployments work

Provisioning modes

Enabling rolling deployments

Configuration reference

Deployment statuses

Deployment control actions

Pause

Resume

Cancel

Force cancel

Force roll forward

Autoscaling during rolling deployments

Environments without autoscaling

Deployment cleanup