Welcome to Baseten!
Baseten provides all the infrastructure you need to deploy and serve ML models performantly, scalably, and cost-efficiently. With Baseten, you can:
- Deploy your own ML models with Truss, our open-source model packaging framework.
- Deploy open source ML models from our model library.
- Handle variable traffic with autoscaling model deployments.
- Save on infra costs with scale to zero and lighting fast cold starts.
- Manage your models’ deployments, health status, metrics, and spend in your Baseten workspace.
Getting started
Every new workspace comes with free model resource credits to kick off your exploration. To use your credits:
- Sign up for or sign in to your Baseten account.
- Generate an API key.
- Install Truss, our open-source model packaging framework.
pip install --upgrade truss
Example models
Chat LLM: Mistral 7B
Image generation: SDXL
Transcription: Whisper
Audio generation: MusicGen
Mistral 7B is an open source large language model released by Mistral AI under the Apache 2.0 license, meaning it’s fully licensed for commercial use. It beats Llama 2 7B on most benchmarks.
Package Mistral with Truss
We did this part for you! Grab a Truss for Mistral Instruct 7B from the model library.
git clone https://github.com/basetenlabs/truss-examples
cd model-library/mistral-instruct-7b
If you want to learn about model packaging, get started with this Truss tutorial.
Deploy Mistral to dev
Use truss push to deploy your model. Then, paste your Baseten API key when prompted.
truss push
You can see deployment information, model logs, and model metrics from your Baseten workspace.
Call Mistral from terminal
Use truss predict to call your model once it has finished deploying.
truss predict -d '{
"prompt": "What is the difference between a wizard and a sorcerer?",
"stream": true
}'
Your model is served behind an API endpoint and will scale to zero when not in use to save you money.
Next, you can iterate on your model with Truss’ live reload dev loop or promote it to production.
Model deployment
Explore open-source models
Choose from dozens of popular open-source foundation models and deploy in just two clicks from Baseten’s model library.
Deploy your model with Truss
Package any public or private model with Truss, our open-source model packaging framework, and deploy it to Baseten with the Truss CLI.
Model performance
Instance sizing
Pick the right GPU (if needed) plus CPU and RAM to run your ML model inference.
Autoscaling
Configure autoscaling infrastructure to handle variable traffic and high throughput.