🆕 Environments

​

What is an environment?

Environments enable you to manage your model’s release cycles. They provide a way to ensure the quality, stability, and scalability of your model before it reaches end users. Let’s say you’ve made some changes to a model and you want to better understand the efficacy of its outputs within your app. You want to do this without changing any behavior in your user-facing app. This paradigm pops up quite frequently as you iterate on the model over time. By providing a stable URL and autoscaling settings, environments allow you to create repeatable release processes for your model.

To take advantage of environments, you can create an environment with a custom name, ie: staging, and promote a candidate deployment to that environment. This deployment now becomes the recipient of any requests you make to the “staging” environment. Now you can verify the quality of your changes before promoting the deployment to production. Some common methods of verifying the quality of the deployment:

• Tests/Evals
• Manual testing in pre-production environment
• Canarying/Gradual rollout
• Shadow serving traffic

​

Environments API

Each environment comes with its own

• Predict endpoint
• Async inference endpoint
• Set of management endpoints for
  • Promoting deployments
  • Activating deployments
  • Deactivating deployments
  • Updating settings

​

What about production?

• The production environment will exist by default for every model.
• Note: You must have a production deployment before you can deploy to a user-created environment

​

Promotion

• First, ensure that you have a production deployment. If you don’t, you can promote your development deployment to production.
• Once that condition is met, any deployment can be promoted to any environment, whether it is a development deployment, a published deployment, or a deployment that’s already in an environment.
• Deployments can be promoted from the UI or via the REST API
• If a deployment in the “staging” environment is promoted to “production”, baseten will create of a copy of the staging deployment. Once created, this copy will be deployed in the production environment.

​

Get access

To get access to environments in beta, email us at hello@baseten.co with the subject line “Environments”.