Rolling deployments aren’t supported for Chains. This feature is available for individual model deployments only.
How rolling deployments work
A rolling deployment replaces replicas within the environment’s configured provisioning limits:- Baseten scales up candidate replicas as capacity becomes available.
- Traffic shifts toward the candidate as its replicas become ready.
- Baseten scales down previous replicas as the provisioning limits allow.
Provisioning modes
Choose a provisioning mode based on the extra capacity you can use and the temporary reduction in serving capacity you can accept. Both limits are percentages of a baseline replica count: the capacity needed for current demand, within the environment’s minimum and maximum replica settings.- Max surge: Set
max_surge_percentabove0andmax_unavailable_percentto0. Baseten adds candidate replicas before retiring previous replicas, using extra capacity for replacements. - Max unavailable: Set
max_surge_percentto0andmax_unavailable_percentabove0. Baseten can scale down previous replicas before replacements are ready, temporarily reducing serving capacity. - Mixed: Set both
max_surge_percentandmax_unavailable_percentabove0. Baseten allows extra replicas and a temporary reduction in serving capacity during the same rollout.
Simulate a rollout
The simulation illustrates max surge or max unavailable mode. It doesn’t simulate mixed mode or changes in demand. Select a status to freeze the simulation on that stage, then select it again to resume. See Deployment statuses for the meaning of each status.Configure the simulation
Configure the simulation
Adjust the values and choose Apply to restart the simulation with your configuration.
The simulator accepts
replica_overhead_percent values up to 60% so the replica rows remain readable. The API accepts values from 0% through 500%.Configure rolling deployments
Set the environment’s promotion settings before promoting a deployment. These settings apply to future promotions into the environment. Set both limits explicitly when changing modes. Updates preserve settings you omit: if max surge is25, setting max unavailable to 10 enables mixed mode. Set the budget you want to turn off to 0. At least one budget must be greater than 0.
For all settings and their ranges, see Configuration reference. You can also choose a cleanup strategy before promoting.
- Baseten CLI
- UI
- Management API
Replace To configure max unavailable mode:Run the following command with max surge set to To configure mixed mode:Run the following command with both limits above For more information, see
<MODEL_ID> with your model’s ID.To configure max surge mode:Run the following command:0:0:model environment update-promotion.Start a rolling deployment
To start the rollout, promote a deployment to the configured environment. The promotion uses that environment’s saved settings. After submitting the promotion, monitor its progress until the environment points to the candidate deployment.Monitor a rollout
Inspect the target environment to track the promotion’s status and traffic split:- Baseten CLI
- UI
- Management API
To inspect a rolling deployment:Replace The example shows a promotion with 40% of traffic routed to the candidate. Run the command again to check its progress.
<MODEL_ID> with your model’s ID, then run the following command:current_deployment.id matches the promoted deployment ID. Confirm completion using the promotion verification steps, then send a request to the environment’s stable endpoint with your model’s prediction input.
Deployment statuses
Thein_progress_promotion field on the environment detail endpoint tracks the current state of a rolling deployment. It’s separate from each deployment’s own status.
percent_traffic_to_new_version reports the current percentage of traffic routed to the candidate deployment.
To watch metrics through a rollout, use the
environment metrics endpoint,
which is in beta. It aggregates both deployments, and in series mode it
splits the window at each promotion so every point reflects the deployments
serving at that time.
Manage a rollout
Pause, resume, and force roll forward requests can return before the promotion reaches the requested state. Replica changes already requested can continue after an action takes effect. Pausing stops traffic shifts; autoscaling continues. For example, a promotion paused at 20% traffic can finish starting replicas already requested for the candidate deployment. The traffic split stays at 20% until you resume. For UI actions, open your model’s overview and find the target environment’s promotion banner. For API actions, setBASETEN_API_KEY to your Baseten API key and replace {model_id} with your model’s ID.
After each action, check the promotion’s status and traffic split to confirm the result.
Pause
Pause the rolling deployment to inspect metrics or logs before proceeding:- UI
- Management API
To pause a rolling deployment:
- Choose Pause.
- Confirm the promotion reaches
PAUSED. Replicas can continue starting or terminating while it is paused.
Resume
Resume a paused rolling deployment from where it left off:- UI
- Management API
To resume a rolling deployment:
- Choose Resume.
- Confirm the promotion leaves
PAUSEDand continues or completes.
Cancel
Gracefully cancel the rolling deployment. Traffic ramps back to the previous deployment and candidate replicas scale down:- UI
- Management API
To cancel a rolling deployment:
- Choose Cancel.
- Confirm that traffic returns to the previous deployment as the promotion rolls back.
RAMPING_DOWN while traffic returns to the previous deployment. See Deployment statuses to track the rollback.
Force cancel
Immediately cancel the rolling deployment and return all traffic to the previous deployment when you can’t wait for the graceful ramp-down. To force cancel a rolling deployment through the Management API:Force roll forward
Complete the rolling deployment, shifting all traffic to the candidate deployment. This works even while the deployment is rolling back:- UI
- Management API
To force roll forward:
- Choose Force promote.
- Confirm the environment’s Current deployment changes to the candidate after the promotion completes.
Deployment cleanup
When a promotion completes, thepromotion_cleanup_strategy setting controls what happens to the deployment it replaced. The strategy applies to every promotion into the environment, not only to rolling ones.
SCALE_TO_ZERO: Sets the previous deployment’s minimum replicas to 0 and leaves its maximum unchanged, so the autoscaler drains it over the scale-down delay. The deployment staysACTIVEand scales back up on demand. This is the default.KEEP: Leaves the previous deployment running at its current replica count.DEACTIVATE: Deactivates the previous deployment, so it stops serving traffic and releases its replicas.
SCALE_TO_ZERO and DEACTIVATE set the previous deployment’s minimum replicas to 0, and that change persists. A deployment pinned to two minimum replicas keeps a minimum of 0 after the promotion, so set its autoscaling settings again before you promote it back into service.
Only DEACTIVATE releases the previous deployment’s slot against your workspace’s
deployment limit.
Under SCALE_TO_ZERO and KEEP, every promotion leaves one more deployment
holding a slot, which blocks new pushes once you reach the limit.
- Baseten CLI
- UI
- Management API
To set the cleanup strategy:Replace For the full flag set, see
<MODEL_ID> with your model’s ID, then run the following command:model environment update-promotion.Disable rolling deployments
Turn off rolling deployments for future promotions into the environment. Use Cancel or Force cancel to stop a rollout already in progress.- Baseten CLI
- UI
- Management API
To disable rolling deployments:Replace Confirm that the saved value is
<MODEL_ID> with your model’s ID, then run the following command:false.Capacity during a rollout
The baseline replica count changes as demand changes, within the environment’s minimum and maximum replica settings. The surge and unavailable budgets apply across the previous and candidate deployments. The environment’s minimum and maximum constrain the baseline, rather than the combined replica counts during a rollout. Surge can temporarily raise the combined replica count above the environment’s maximum. The unavailable budget can allow serving capacity to fall below the environment’s minimum. Baseten multiplies the baseline by each non-zero percentage and rounds the result to the nearest whole replica, with a minimum of 1 replica. A rollout with a single replica starts a replacement before retiring that replica, even in max unavailable mode. If the candidate needs a different GPU type or more GPUs per replica, scaling down previous replicas might not free suitable capacity.Scaling with demand
For environments configured with autoscaling (min_replica < max_replica), Baseten continues to adjust capacity during a rolling deployment based on combined demand. The autoscaler can add or remove replicas from either deployment as traffic changes and the rollout advances. Replica allocation accounts for rollout state as well as traffic, so the replica split doesn’t necessarily match the traffic split.
During the rollout:
- Autoscaling adds and removes replicas throughout the rollout to track combined demand. Each deployment that’s still part of the rollout keeps at least one replica.
- Capacity management continues during a
PAUSEDrolling deployment. Pausing stops the traffic shift, not capacity management. If demand changes while paused, both deployments still scale up or down.
Environments without autoscaling
Environments wheremin_replica == max_replica use a fixed baseline replica count during the rollout. To pre-provision additional headroom for traffic spikes, set replica_overhead_percent to add replicas to the previous deployment before any traffic shifts. Use stabilization_time_seconds to wait between steps and monitor metrics before the next traffic shift.
When candidate replicas become ready
Rolling deployments adapt to candidate replicas as they become ready. Rather than assuming a full batch of replicas is available immediately, Baseten adjusts the rollout based on live capacity. For example, with 100 previous replicas andmax_unavailable_percent set to 25, Baseten requests 25 new replicas. If only 5 become ready, Baseten only removes 5 previous replicas to stay within your unavailable limit:
max_surge_percent. With 100 previous replicas and a 25% surge limit, if only 5 of the 25 requested candidate replicas become ready, Baseten scales down 5 previous replicas before requesting the next batch. This ensures the rollout progresses based on actual ready capacity.
Configuration reference
Set the following fields in therolling_deploy_config object within the environment’s promotion_settings. At least one of max_surge_percent and max_unavailable_percent must be greater than 0.
Maximum percentage of the baseline replica count to provision as extra replicas across both deployments, including replicas still starting. Set to
0 to use max unavailable mode without a surge budget. In mixed mode, both limits can be greater than 0.Range: 0-100Maximum percentage of the baseline replica count that can be unavailable while replacing replicas. Set to
0 to wait for candidate replicas to become ready before retiring previous replicas. In mixed mode, both limits can be greater than 0.Range: 0-50Seconds to wait after each traffic shift before proceeding to the next step. Use this to monitor metrics between steps.Range: 0-3600
Percentage of additional replicas to pre-provision on the current deployment before the rolling deployment starts. Useful for environments without autoscaling (
min_replica == max_replica) or as a buffer for anticipated traffic spikes during the rollout.Range: 0-500promotion_settings level:
Enables rolling deployments for the environment. For new model environments,
the default is
true. Existing environments keep their value.Next steps
- Environments explains how promotion settings apply to an environment.
- Scale a deployment covers the autoscaling settings that determine the rollout’s baseline capacity.
- Requests during a promotion explains request draining and its limits.