Truss is an open-source model packaging framework developed by Baseten. You can use Truss to package and deploy any ML model to Baseten, from open-source models on Hugging Face and GitHub to your own proprietary models.

​
What is Truss used for?

Truss powers a huge portion of Baseten’s developer experience:

  • Package and test model code, weights, and dependencies with a model server that behaves the same in development and production.
  • Truss enables a fast developer loop with a live reload workflow where your local changes are patched onto a development deployment of your model.
  • From transformers and diffusers to PyTorch and Tensorflow to XGBoost and sklearn, Truss supports models created with any framework, even entirely custom models.

​
What is the difference between Truss and Baseten?

Every model deployed on Baseten is packaged as a Truss. Truss is to Baseten somewhat like Git is to GitHub or NextJS is to Vercel.

With Truss, you can:

  • Package ML models with their weights and requirements for deployment.
  • Configure your model server with resources, secrets, caches, and more.
  • Deploy and test models with a live reload dev loop using Baseten as a remote host.

With Baseten, you can:

  • Call your deployed models from API endpoints.
  • Configure model resources and autoscaling model infrastructure.
  • View and manage model metrics, logs, deployments, and lifecycle.

​
How do I use Truss?

Truss is its own project and has a set of docs that compliment the Baseten docs. Get started with:

Truss homepage

Get an overview of the Truss framework and its docs.

Truss quickstart

Deploy and call your first model with Truss in 5 minutes.

Truss guides

Learn popular features and common design patterns like secrets, caching, and alternative model servers like TGI and vLLM.

Truss examples

Adapt real Trusses for your use case. Every Truss example is heavily commented and tested with a nightly CI job.

Welcome to Baseten!Deploy from model library