Skip to main content
Develop and deploy your own models on dedicated infrastructure. Truss is the model packaging framework. Its config.yaml file uses YAML syntax to define the model container, dependencies, and runtime configuration. Use the Baseten CLI to deploy and manage the package on Baseten with autoscaling and observability. To call a hosted model without deploying it, use Model APIs.

​
How you develop a model

Configure your model package, add inference code if needed, and deploy it with the Baseten CLI:
  • Configuration only. Use config.yaml to declare the GPU, dependencies, base image, and weights. Many open-source LLMs require no custom serving code. See Configuration and Dependencies.
  • Custom Python. Add model/model.py when you need preprocessing, postprocessing, or model logic that configuration cannot express. The Python Model class defines load and predict. See The Model class.
  • Baseten CLI. Use baseten model push --watch to create a development deployment, baseten model watch to sync changes, and baseten model deployment promote to ship to production. See The development loop.
If you already have a server based on vLLM, SGLang, Triton, or another container image, deploy it as a custom Docker server.

​
Pick a starting point

​
The development cycle

Whichever approach you choose, the development cycle is the same: create a development deployment, sync local changes, then publish an immutable deployment for production traffic.
  1. Push to development. Run baseten model push --watch to create a single-replica development deployment with live reload.
  2. Iterate with live reload. Run baseten model watch to sync local changes without rebuilding the container. Edit a file, save it, and review the result in the deployment logs.
  3. Publish to production. Run baseten model push to create an immutable, production-ready deployment with full autoscaling. Promote it to an environment for a stable endpoint URL that doesn’t change between versions.
Development deployments are limited to one replica and optimized for iteration rather than production traffic. See The development loop for the full workflow.

​
Build multi-model systems

Use Chains when a workflow contains multiple models or steps that need different hardware. For example, a RAG pipeline can run retrieval and generation on separate instances with independent scaling rules.