Skip to main content
Develop and deploy your own models on dedicated infrastructure. Truss defines the model container, dependencies, and runtime configuration. Baseten deploys it with autoscaling and observability. To call a hosted model without deploying it, use Model APIs.

​
How you develop a model

Truss combines runtime configuration, optional Python code, and CLI commands:
  • Configuration only. Use config.yaml to declare the GPU, dependencies, base image, and weights. Many open-source LLMs require no custom serving code. See Configuration and Dependencies.
  • Custom Python. Add model/model.py when you need preprocessing, postprocessing, or model logic that configuration cannot express. The Python Model class defines load and predict. See The Model class.
  • Development CLI. Use truss push --watch to create a development deployment, truss watch to sync changes, and truss push --promote to ship to production. See The development loop.
If you already have a server based on vLLM, SGLang, Triton, or another container image, deploy it as a custom Docker server.

​
Pick a starting point

​
The development cycle

Whichever approach you choose, the development cycle is the same: create a development deployment, sync local changes, then publish an immutable deployment for production traffic.
  1. Push to development. Run truss push --watch to create a single-replica development deployment with live reload.
  2. Iterate with live reload. Run truss watch to sync local changes without rebuilding the container. Edit a file, save it, and review the result in the deployment logs.
  3. Publish to production. Run truss push to create an immutable, production-ready deployment with full autoscaling. Promote it to an environment for a stable endpoint URL that doesn’t change between versions.
Development deployments are limited to one replica and optimized for iteration rather than production traffic. See The development loop for the full workflow.

​
Build multi-model systems

Use Chains when a workflow contains multiple models or steps that need different hardware. For example, a RAG pipeline can run retrieval and generation on separate instances with independent scaling rules.