Skip to main content
A deployment in Baseten is a containerized instance of a model that serves inference requests through an API endpoint. Deployments exist independently but can be promoted to an environment for structured access and scaling. Baseten automatically wraps every deployment in a REST API. Once deployed, query your model with an HTTP request:
predict.py
For the full request and response format, see running inference on your deployment.

​
Development deployment

A development deployment is a mutable instance designed for rapid iteration. Create one with truss push --watch (for models) or truss chains push --watch (for Chains). It stays in the development state until promoted and can’t be renamed. Key characteristics:
  • Live reload enables direct updates without redeployment.
  • Single replica, scales to zero when idle to conserve compute resources.
  • No autoscaling or zero-downtime updates.
  • Can be promoted to create a persistent deployment.
Once promoted, the development deployment transitions to a deployment and can optionally be promoted to an environment.

​
Environments and promotion

Environments provide logical isolation for managing deployments but aren’t required for a deployment to function. You can run a deployment independently or promote it to an environment for controlled traffic allocation and scaling.
  • The production environment exists by default.
  • Custom environments (for example, staging) can be created for specific workflows.
  • Promoting a deployment doesn’t modify its behavior, only its routing and lifecycle management.

​
Rolling deployments

Rolling deployments replace replicas incrementally when promoting a deployment to an environment. Instead of swapping all traffic at once, rolling deployments scale up the candidate, shift traffic proportionally, and scale down the previous deployment in controlled steps. You can pause, resume, cancel, or force-complete a rolling deployment at any point. Rolling deployments are disabled by default; enable them per environment. For more information, see Rolling deployments.

​
Canary deployments (deprecated)

Canary deployments are deprecated. Use rolling deployments for incremental traffic shifting with finer control over replica provisioning and rollback.
Canary deployments support incremental traffic shifting to a new deployment in 10 evenly distributed stages over a configurable time window. Enable or cancel canary rollouts from the UI or REST API.

​
Manage deployments

To scale, promote, deactivate, delete, or inspect deployments, see Manage deployments.

​
Name deployments

By default, deployments of a model are named deployment-1, deployment-2, and so forth sequentially. To name a deployment: Name it at deploy time with truss push --deployment-name:
Rename it later in the console:
  1. Sign in to your workspace at app.baseten.co and choose Dedicated Inference in the sidebar.
  2. Select your model.
  3. Select the deployment under Deployments.
  4. Choose Rename deployment.
  5. Type the new name, then choose Rename deployment to confirm.
Names are purely cosmetic and don’t affect API paths, which work by model and deployment IDs.

​
Label deployments

Labels are JSON key-value metadata you attach to a deployment to organize and track it, for example by team, environment, or the pipeline that created it. Set them at deploy time with the --labels flag on truss push (truss push --labels '{"team": "ml-platform", "env": "staging"}'), the labels argument to truss.push(), or the labels input on the deploy GitHub Action. To attach labels automatically in a CI pipeline, see Deploy with labels.

​
Deactivate a deployment

Deactivate a deployment to suspend inference while preserving its configuration; requests fail with a 400 error until you activate it again. To deactivate or activate, see Manage the deployment lifecycle. For demand-driven deployments, consider scale to zero instead.

​
Delete deployments

Deletion is permanent: compute is released, requests return a 404 error, and the deployment leaves the dashboard (usage logs are retained). A deployment that’s associated with an environment, or is the only deployment of a model, can’t be deleted; promote a replacement first. To delete, see Manage the deployment lifecycle.