Deployments
A deployment runs one version of your model on a selected instance type. Eachtruss push creates a deployment. Multiple deployments of the same model can run at once, allowing you to test a new version without changing production traffic. Deactivate a deployment to stop serving and billing, or delete it when you no longer need it.
For rapid iteration, use truss push --watch to create a development deployment, a mutable instance that live-reloads as you edit your model code. Development deployments can’t be promoted to an environment.
Closed-source models
Some models on Baseten are distributed by model labs that keep their weights and implementation private. You can deploy, scale, promote, and monitor one like any other deployment. The model lab controls operations that expose or change the model. For details, see Customer availability and artifact access. These deployments also run without outbound network access. The deployment uses the instance type published with the listing, and the customer cannot change it after adoption.Environments
Environments provide stable endpoints that persist as you release new deployments. A typical setup uses one environment for testing and another for production traffic. Each environment has its own endpoint, autoscaling settings, and metrics. Promoting a deployment moves the environment’s traffic to that version without changing the URL called by your application.
Resources
Every deployment runs on an instance type that defines its GPU, CPU, and memory. Set the instance type inconfig.yaml, or change it for a published deployment in the Baseten dashboard. Select an instance based on the model’s memory requirements, latency target, and expected traffic.
Autoscaling
Autoscaling adjusts the number of replicas as request load changes. Configure the minimum and maximum replicas, concurrency target, and scale-down delay. A deployment can scale to zero when idle, but the next request must wait for a replica to start. See Cold starts for ways to reduce that delay.
For the mechanics of how the autoscaler tracks in-flight requests and adjusts replicas, see How Baseten works. For engine-specific autoscaling settings (BEI and Engine-Builder-LLM), see Autoscaling engines.