Rolling deployments

Rolling deployments replace replicas incrementally when promoting a deployment to an environment. Instead of swapping all traffic at once, rolling deployments scale up the candidate deployment, shift traffic proportionally, and scale down the previous deployment in controlled steps. Autoscaling continues throughout the rollout for environments where min_replica < max_replica, so both deployments scale up to meet traffic demand as it shifts between them. Use rolling deployments when you need zero-downtime updates with the ability to pause, cancel, or force-complete the deployment at any point.

Rolling deployments are not supported for Chains. This feature is available for individual model deployments only.

How rolling deployments work

A rolling deployment follows a repeating three-step cycle:

Scale up candidate deployment replicas by the configured percentage.
Shift traffic proportionally to match the new replica ratio.
Scale down the previous deployment replicas by the same percentage.

This cycle repeats until all traffic and replicas run on the candidate deployment, at which point it becomes the active deployment in the environment. The following diagram shows this cycle in action. The tab strip mirrors the promotion lifecycle: a promotion enters RELEASING when it starts, sits in RAMPING_UP while replicas scale and traffic shifts, can pause as PAUSED, and lands at SUCCEEDED once the candidate serves all traffic. Select any status to freeze the simulation on that stage, then select it again to resume.

Configure rolling_deploy_config

Adjust the values and choose Apply to restart the simulation with your configuration.

Provisioning modes

Rolling deployments support two mutually exclusive provisioning modes:

max_surge_percent: Scales up candidate replicas before scaling down previous replicas.
max_unavailable_percent: Scales down previous replicas before scaling up candidate replicas.

Set exactly one mode to a non-zero value and the other to 0.

Enable rolling deployments

Rolling deployments are disabled by default. Enable them per environment in its promotion settings:

UI
REST API

To enable rolling deployments:

Sign in to your workspace at app.baseten.co and choose Dedicated Inference in the sidebar.
Select your model.
On the model’s overview, open the environment card’s ⋯ menu.
Choose Configure promotion.
Turn on Rolling deploys.
Choose Update to apply the changes.

To enable rolling deployments:

curl -X PATCH "https://api.baseten.co/v1/models/{model_id}/environments/production" \
  -H "Authorization: Bearer $BASETEN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "promotion_settings": {
      "rolling_deploy": true,
      "rolling_deploy_config": {
        "max_surge_percent": 10,
        "max_unavailable_percent": 0,
        "stabilization_time_seconds": 60,
        "replica_overhead_percent": 0
      }
    }
  }'

{
  "status": "ACCEPTED",
  "message": "Your request to update environment settings is complete"
}

Once rolling deployments are enabled, any subsequent promotion to the environment uses the rolling deployment workflow.

Configuration reference

Configure rolling deployments through the rolling_deploy_config object in the environment’s promotion_settings.

max_surge_percent

integer

default:"10"

Percentage of additional replicas to provision during each step. Set to 0 to use max unavailable mode instead.Range: 0-50

max_unavailable_percent

integer

default:"0"

Percentage of replicas that can be unavailable during each step. Set to 0 to use max surge mode instead.Range: 0-50

stabilization_time_seconds

integer

default:"0"

Seconds to wait after each traffic shift before proceeding to the next step. Use this to monitor metrics between steps.Range: 0-3600

replica_overhead_percent

integer

default:"0"

Percentage of additional replicas to pre-provision on the current deployment before the rolling deployment starts. Useful for environments without autoscaling (min_replica == max_replica) or as a buffer for anticipated traffic spikes during the rollout.Range: 0-500

Additional promotion settings configured at the promotion_settings level:

rolling_deploy

boolean

default:"false"

Enables rolling deployments for the environment.

Deployment statuses

The in_progress_promotion field on the environment detail endpoint tracks the current state of a rolling deployment. It’s separate from each deployment’s own status.

Status	Description
`RELEASING`	Candidate deployment is building and initializing replicas.
`RAMPING_UP`	Scaling up candidate replicas and shifting traffic.
`PAUSED`	Rolling deployment is paused at its current traffic split. No further promotion steps run, but in-flight replica changes and autoscaling continue.
`RAMPING_DOWN`	Graceful cancel in progress. Traffic is shifting back to the previous deployment.
`SUCCEEDED`	Rolling deployment completed. The candidate is now the active deployment.
`FAILED`	Rolling deployment failed. Traffic remains on the previous deployment.
`CANCELED`	Rolling deployment was canceled. Traffic returned to the previous deployment.

The in_progress_promotion object also includes percent_traffic_to_new_version, which reports the current percentage of traffic routed to the candidate deployment. To watch metrics through a rollout, use the environment metrics endpoint (beta): it aggregates both deployments, and in series mode it splits the window at each promotion so every point reflects the deployments serving at that time.

Deployment control actions

Pause, resume, and force roll forward act on the rolling deployment between steps, not immediately. Replica changes already in progress finish before the action takes effect, so the rolling deployment can keep scaling for a short time after you trigger the action. For example, if the candidate deployment is at 20% traffic and has just been told to scale from 2 to 4 replicas, choosing Pause lets the candidate finish scaling to 4 replicas. The traffic split stays pinned at 20% until you resume.

Pause

Pause the rolling deployment to inspect metrics or logs before proceeding:

UI
REST API

To pause a rolling deployment:

Sign in to your workspace at app.baseten.co and choose Dedicated Inference in the sidebar.
Select your model.
On the model’s overview, find the environment’s promotion banner.
Choose Pause.

To pause a rolling deployment:

curl -X POST "https://api.baseten.co/v1/models/{model_id}/environments/production/pause_promotion" \
  -H "Authorization: Bearer $BASETEN_API_KEY"

{"success": true}

Resume

Resume a paused rolling deployment from where it left off:

UI
REST API

To resume a rolling deployment:

Sign in to your workspace at app.baseten.co and choose Dedicated Inference in the sidebar.
Select your model.
On the model’s overview, find the environment’s promotion banner.
Choose Resume.

To resume a rolling deployment:

curl -X POST "https://api.baseten.co/v1/models/{model_id}/environments/production/resume_promotion" \
  -H "Authorization: Bearer $BASETEN_API_KEY"

{"success": true}

Cancel

Gracefully cancel the rolling deployment. Traffic ramps back to the previous deployment and candidate replicas scale down:

UI
REST API

To cancel a rolling deployment:

Sign in to your workspace at app.baseten.co and choose Dedicated Inference in the sidebar.
Select your model.
On the model’s overview, find the environment’s promotion banner.
Choose Cancel.

To cancel a rolling deployment:

curl -X POST "https://api.baseten.co/v1/models/{model_id}/environments/production/cancel_promotion" \
  -H "Authorization: Bearer $BASETEN_API_KEY"

{"status": "RAMPING_DOWN", "message": "..."}

Returns a status of CANCELED (instant cancel for non-rolling deployments) or RAMPING_DOWN (graceful rollback for rolling deployments).

Force cancel

Immediately cancel the rolling deployment and return all traffic to the previous deployment when you can’t wait for the graceful ramp-down.

Force canceling may cause brief service disruption if the previous deployment is under-provisioned.

To force cancel a rolling deployment:

curl -X POST "https://api.baseten.co/v1/models/{model_id}/environments/production/force_cancel_promotion" \
  -H "Authorization: Bearer $BASETEN_API_KEY"

{"success": true}

Force roll forward

Immediately complete the rolling deployment, shifting all traffic to the candidate deployment. This works even while the deployment is rolling back:

Force rolling forward may promote an under-provisioned deployment if the candidate has not finished scaling up.

UI
REST API

To force roll forward:

Sign in to your workspace at app.baseten.co and choose Dedicated Inference in the sidebar.
Select your model.
On the model’s overview, find the environment’s promotion banner.
Choose Force promote.

To force roll forward:

curl -X POST "https://api.baseten.co/v1/models/{model_id}/environments/production/force_roll_forward_promotion" \
  -H "Authorization: Bearer $BASETEN_API_KEY"

{"success": true}

Autoscaling during rolling deployments

For environments configured with autoscaling (min_replica < max_replica), Baseten continues to scale your deployment during a rolling deployment to meet traffic demand. Both the previous and candidate deployments scale up based on combined demand, and new capacity is distributed proportionally to the current traffic split. When demand drops, Baseten scales down both deployments the same way, removing replicas split by the current traffic ratio without changing the traffic split itself. For example, with traffic split 60/40 between the previous and candidate deployments, an additional 10 replicas of demand provisions 6 replicas to the previous deployment and 4 to the candidate. A drop of 10 replicas removes 6 from the previous deployment and 4 from the candidate the same way. A few constraints apply during the rolling deployment:

Autoscaling adds and removes replicas throughout the rollout to track combined demand. Each deployment that is still part of the rollout keeps at least one replica, and the combined replica count stays within the environment’s min_replica and max_replica.
Capacity management continues during a PAUSED rolling deployment. Pausing stops the traffic shift, not capacity management. If demand changes while paused, both deployments still scale up or down.

Dynamic replica admission

Rolling deployments adapt to candidate replicas as they become ready. Rather than assuming a full batch of replicas will be available immediately, Baseten adjusts the rollout based on live capacity. For example, with 100 previous replicas and max_unavailable_percent set to 25, Baseten requests 25 new replicas. If only 5 become ready, Baseten only removes 5 previous replicas to stay within your unavailable limit:

Max unavailable 25%
Requested:       Previous 75 replicas   Candidate 25 requested
Actually ready:  Previous 75 replicas   Candidate 5 ready
Next adjustment: Previous 70 replicas   Candidate 5 ready

The same adaptive behavior applies to max_surge_percent. With 100 previous replicas and a 25% surge limit, if only 5 of the 25 requested candidate replicas become ready, Baseten scales down 5 previous replicas before requesting the next batch. This ensures the rollout progresses based on actual ready capacity.

Max surge 25%
Requested:       Previous 100 replicas  Candidate 25 requested
Actually ready:  Previous 100 replicas  Candidate 5 ready
Next adjustment: Previous 95 replicas   Candidate 5 ready

In both modes, rollouts continue from live, ready capacity to ensure your environment remains stable throughout the transition.

Environments without autoscaling

Environments where min_replica == max_replica have no autoscaling configured, so replica counts stay pinned during the rolling deployment. To pre-provision additional headroom for traffic spikes, set replica_overhead_percent to add replicas to the previous deployment before any traffic shifts. Use stabilization_time_seconds to wait between steps and monitor metrics before the next traffic shift.

Deployment cleanup

After a rolling deployment completes, the promotion_cleanup_strategy setting controls what happens to the previous deployment.

SCALE_TO_ZERO: Scales the previous deployment to zero replicas. It remains available for reactivation. This is the default.
KEEP: Leaves the previous deployment running at its current replica count.
DEACTIVATE: Deactivates the previous deployment. It stops serving traffic and releases all resources.

UI
REST API

To set the cleanup strategy:

Sign in to your workspace at app.baseten.co and choose Dedicated Inference in the sidebar.
Select your model.
On the model’s overview, open the environment card’s ⋯ menu.
Choose Configure promotion.
For After promotion, select the action taken on the previous deployment.
Choose Update to apply the changes.

To set the cleanup strategy:

curl -X PATCH "https://api.baseten.co/v1/models/{model_id}/environments/production" \
  -H "Authorization: Bearer $BASETEN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "promotion_settings": {
      "promotion_cleanup_strategy": "DEACTIVATE"
    }
  }'

{
  "status": "ACCEPTED",
  "message": "Your request to update environment settings is complete"
}

Next steps

A rolling deployment starts from an ordinary promotion, so the same lifecycle operations drive it.

Manage the deployment lifecycle to run the promotion that starts a rolling deployment.
Environments for the concepts behind promotion settings.
Scale a deployment for the autoscaling settings a rollout respects.

Overview

Get started

Model APIs

Inference

Development

Deployment

Engines

Frontier Gateway

Training

Organization

Observability

Troubleshooting

Rolling deployments

How rolling deployments work

Provisioning modes

Enable rolling deployments

Configuration reference

Deployment statuses

Deployment control actions

Pause

Resume

Cancel

Force cancel

Force roll forward

Autoscaling during rolling deployments

Dynamic replica admission

Environments without autoscaling

Deployment cleanup

Next steps

​How rolling deployments work

​Provisioning modes

​Enable rolling deployments

​Configuration reference

​Deployment statuses

​Deployment control actions

​Pause

​Resume

​Cancel

​Force cancel

​Force roll forward

​Autoscaling during rolling deployments

​Dynamic replica admission

​Environments without autoscaling

​Deployment cleanup

​Next steps

How rolling deployments work

Provisioning modes

Enable rolling deployments

Configuration reference

Deployment statuses

Deployment control actions

Pause

Resume

Cancel

Force cancel

Force roll forward

Autoscaling during rolling deployments

Dynamic replica admission

Environments without autoscaling

Deployment cleanup

Next steps