Environments

An environment is a named release target for a model, such as production or staging. It has a stable endpoint and its own resource, autoscaling, monitoring, and promotion settings. Promote a new deployment to update the version that serves the environment without changing the endpoint your client calls.

Baseten dashboard showing model environments and their deployments

Use a separate environment to validate a deployment without changing production traffic.

Deployment management

Use environments to validate and release deployments with:

Automated tests and evaluations.
Manual testing in pre-production.
Performance tests with copies of live requests through shadow traffic.
Gradual traffic shifts with rolling deployments.

Baseten enables rolling deployments by default for new model environments. Existing environments keep their promotion settings.

To change this setting, enable rolling deployments or turn off rolling deployments for the environment. When you promote a deployment, Baseten applies the environment’s scaling and monitoring settings:

Dedicated API endpoint. For more information, see Predict endpoints.
Autoscaling controls: Scale behavior is managed per environment.
Monitoring and metrics: scope logs and metrics to the environment in the dashboard, or export environment metrics to your own observability stack.

The production environment operates like any other environment but has restrictions:

It can’t be deleted unless the entire model is removed.
You can’t create additional environments named “production.”

Shadow traffic

Shadow traffic copies a share of an environment’s live HTTP requests to another environment or deployment in the same model, so you can test a deployment on real traffic before you promote it. Clients continue to receive responses from the source deployment, while Baseten discards responses from the target deployment. Shadowing is best effort. Depending on platform and target conditions, Baseten might not copy every sampled request, and copies can fail. Skipped and failed copies don’t affect the client’s response, but the target can receive a smaller share of requests than the percentage you set. The target must meet the source’s regional and compliance constraints. A target environment receives the copies on whichever deployment it’s serving.

Shadow traffic is pre-release. The dashboard shows the Traffic routing section only for workspaces with traffic routing rules enabled, and the baseten model environment traffic-rule commands may change their arguments, flags, and output.

UI
Baseten CLI
Management API

To start shadow traffic:

Sign in to your workspace at app.baseten.co and choose Dedicated Inference in the sidebar, then select your model.
Under Environments, select the source environment.
In the Traffic routing section, choose Add rule.
For Target, select the environment or deployment to receive the copies.
For Traffic to shadow, set the percentage of requests to copy. Shadowing is best effort, so the target can receive a smaller share than configured.
For Expiration, choose how long the rule runs, up to 7 days.
For Sampling, choose Random to pick each request independently, or Session-aware to keep a session’s requests together.
To cancel a copy when the client cancels the original request, select Propagate client cancellation to the target.
Choose Start shadowing.

The Traffic routing section shows the running rule.To change a shadow rule:

In the source environment’s Traffic routing section, choose Edit.
Change the percentage, expiration, sampling, or cancellation, then choose Save.

To stop shadow traffic:

In the source environment’s Traffic routing section, choose Stop.
Choose Stop rule to confirm.

To start shadow traffic:

Create a rule on the source environment. Replace <expires-at> with an ISO 8601 time, such as 2026-11-10T18:00:00Z, at most 7 days out. Copy 25% of production’s requests to the deployment serving staging, picking each request independently:

baseten model environment traffic-rule create \
  --model-id abc123 \
  --environment production \
  --type shadow \
  --target-environment staging \
  --percent-traffic 25 \
  --expires-at <expires-at> \
  --sampling-mode random \
  --propagate-client-cancellation true

Copy 10% of production’s requests to one deployment, keeping each session’s requests together:

baseten model environment traffic-rule create \
  --model-id abc123 \
  --environment production \
  --type shadow \
  --target-deployment-id def456 \
  --percent-traffic 10 \
  --expires-at <expires-at> \
  --sampling-mode user-session \
  --propagate-client-cancellation false

List the environment’s rules to confirm the rule and get its ID:

baseten model environment traffic-rule list --model-id abc123 --environment production

To change a shadow rule:Run the following command with the rule’s ID. Only the flags you pass change:

baseten model environment traffic-rule update \
  --model-id abc123 \
  --environment production \
  --rule-id <rule-id> \
  --percent-traffic 50

To stop shadow traffic:Run the following command with the rule’s ID. The command asks for confirmation; pass --yes to skip it when scripting:

baseten model environment traffic-rule delete --model-id abc123 --environment production --rule-id <rule-id>

To start shadow traffic:Add a rule to the source environment’s traffic_routing_settings. Replace <expires-at> with an ISO 8601 time with a timezone, at most 7 days out:

curl -X PATCH https://api.baseten.co/v1/models/{model_id}/environments/production \
  -H "Authorization: Bearer $BASETEN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "traffic_routing_settings": {
      "rules": [
        {
          "type": "SHADOW",
          "targets": [{"environment": "staging"}],
          "percent_traffic": 25,
          "expires_at": "<expires-at>",
          "sampling_mode": "RANDOM",
          "propagate_client_cancellation": true
        }
      ]
    }
  }'

To target one deployment, set targets to [{"deployment_id": "def456"}]. To keep each session’s requests together, set sampling_mode to USER_SESSION. The response’s environment.traffic_routing.rules lists the running rule with its id.To change a shadow rule:Send the complete rule with its id. The API replaces the rule, so include every field; rules you omit stay unchanged. To change the target, delete the rule and create a new one.

curl -X PATCH https://api.baseten.co/v1/models/{model_id}/environments/production \
  -H "Authorization: Bearer $BASETEN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "traffic_routing_settings": {
      "rules": [
        {
          "id": "<rule-id>",
          "type": "SHADOW",
          "targets": [{"environment": "staging"}],
          "percent_traffic": 50,
          "expires_at": "<expires-at>",
          "sampling_mode": "RANDOM",
          "propagate_client_cancellation": true
        }
      ]
    }
  }'

To stop shadow traffic:Pass the rule’s ID in delete_rules:

curl -X PATCH https://api.baseten.co/v1/models/{model_id}/environments/production \
  -H "Authorization: Bearer $BASETEN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"traffic_routing_settings": {"delete_rules": ["<rule-id>"]}}'

Session-aware sampling groups requests by the session key the client sends, such as a consistent x-session-affinity header value, so Baseten copies every request in a sampled session. Compare the source’s and target’s logs and metrics over the same time range. Target metrics include any traffic sent directly to the target, and the target’s replica-hours bill separately. A rule also ends when it expires, when a promotion goes into its source or target environment or promotes its target deployment, or when you deactivate or delete its source or target. A source environment has one active shadow rule at a time, so creating another fails until the rule expires or you delete it. Stopping a rule leaves both deployments running. For more information, see baseten model environment traffic-rule and Update an environment’s settings.

Manage environments from the Baseten CLI

The baseten model environment commands work on the same environments as the dashboard and the Management API:

baseten model environment list --model-id <model-id>: list a model’s environments.
baseten model environment describe --model-id <model-id> --environment production: inspect an environment’s current deployment, autoscaling, and promotion settings.
baseten model environment update-autoscaling and update-promotion: change an environment’s settings; see schedule autoscaling and rolling deployments.
baseten model deployment promote --model-id <model-id> --deployment-id <deployment-id>: promote a deployment into an environment; see Manage deployments.

Custom environments

Each workspace has a configured per-model environment limit. In addition to the standard production environment, you can create custom environments until the model reaches that limit. Create them from the model management page in the Baseten dashboard or through the create environment endpoint in the Management API.

Deployment promotion

When you promote a deployment to an environment, Baseten associates the deployment with that environment and applies the environment’s autoscaling settings. If Baseten can reuse the deployment directly, promotion doesn’t create a new deployment ID. Otherwise, Baseten creates a deployment with a unique ID, initializes its resources, and replaces the existing deployment in that environment. A new deployment is created when:

You promote a development deployment.
The deployment is already associated with another environment.
The environment has a different instance type or resource profile.
The deployment moves to a different region.
The applicable compliance policy differs from the deployment’s current policy.
Re-deploy on promotion is enabled.

When Baseten creates a new deployment, it applies the environment’s autoscaling settings to that deployment. After promotion completes, the environment’s promotion cleanup strategy controls the deployment it replaced. If Baseten creates a new deployment from a published source, the new deployment reuses the source image. Promoting a development deployment creates a new published deployment from its current state and triggers an image build.

Published deployment promotion

If Baseten can reuse a published deployment, it updates that deployment’s autoscaling settings to match the environment. If promotion creates a new deployment, the source published deployment keeps its existing settings.

Direct deployment to an environment

Run the following command from your Truss project to deploy to an existing environment. Replace <environment-name> with its name:

Terminal

baseten model push --environment <environment-name> --wait

To create a new custom environment during the push, run the following command:

Terminal

baseten model push --environment staging --create-environment-if-missing --wait

Without --create-environment-if-missing, a push to a missing custom environment fails. Production already exists for every model. --wait confirms deployment readiness. Before sending application traffic, verify that the environment points to the promoted deployment and send a request to its stable endpoint. The Truss CLI equivalent for an existing environment is truss push --environment <environment-name>.

Only one active promotion per environment is allowed at a time.

Environment access in code

Python Truss models receive the environment through the environment keyword argument. Its value is None when the deployment isn’t associated with an environment, so normalize it before reading its name:

model/model.py

def __init__(self, **kwargs):
    self._environment = kwargs.get("environment") or {}

You can use the environment in your load() method to configure per-environment behavior:

model/model.py

def load(self):
    if self._environment.get("name") == "production":
        self.setup_sentry()
        self.model = self.load_production_weights()
    else:
        self.model = self.load_default_weights()

Validate an unassociated deployment first: it should load the default weights. After promotion to production, confirm that initialization uses the production weights through your model’s prediction result or deployment logs. If you use environment-specific configuration in load(), you’ll need to enable re-deploy on promotion to ensure the environment is correctly initialized after each promotion. See Re-deploy on promotion for details.

The environment keyword argument is only available to Python Truss models. Custom servers read the environment name from the filesystem instead. See Environment name.

Re-deploy on promotion

By default, promoting a deployment reuses the existing deployment when possible. This is the fastest promotion path, but it means load() doesn’t re-run. Any environment-specific configuration set during the original load() call persists, even if the deployment moves to a different environment. You can configure an environment to create a fresh deployment every time you promote to it. For a published source deployment, the new deployment reuses the source image and reruns load() with the target environment’s context, so environment-specific configuration takes effect. Enable this if your load() method uses kwargs["environment"] to configure per-environment behavior, or if you promote the same source deployment to multiple environments and want each to get a fresh deployment. To enable re-deploy on promotion:

Sign in to your workspace at app.baseten.co and choose Dedicated Inference in the sidebar, then select your model.
Open the environment’s settings.
Toggle Re-deploy when promoting.

You can also set it through the update environment settings endpoint.

If you promote a deployment that’s already associated with an environment, Baseten creates a new deployment regardless of this setting.

Regional environments

Regional environments constrain replicas to a designated geographic region and provide a dedicated endpoint for region-specific inference routing. Baseten configures the regional restriction for an environment name within a team. See Regional environments for setup, required-endpoint behavior, endpoint formats, and request restrictions. To select a region for an individual deployment instead, see regional deployments.

Environment deletion

You can delete environments, except for production. To remove a production deployment, first promote another deployment to production or delete the entire model.

Deleted environments are removed from the overview but remain in billing history.
They don’t consume resources after deletion.
API requests to a deleted environment return a 404 error.

Get started

Model APIs

Inference

Training

Baseten Grid

Frontier Gateway

Sandboxes

Development

Deployment

Engines

Model Labs

Organization

Hosting options

Observability

Troubleshooting

Deployment management

Shadow traffic

Manage environments from the Baseten CLI

Custom environments

Deployment promotion

Published deployment promotion

Direct deployment to an environment

Environment access in code

Re-deploy on promotion

Regional environments

Environment deletion