Baseten provides all the infrastructure you need to deploy and serve ML models performantly, scalably, and cost-efficiently. With Baseten, you can:

Getting started

Every new workspace comes with free model resource credits to kick off your exploration. To use your credits:

pip install --upgrade truss

Example models

  • Chat LLM: Mistral 7B

  • Image generation: SDXL

  • Transcription: Whisper

  • Audio generation: MusicGen

Mistral 7B is an open source large language model released by Mistral AI under the Apache 2.0 license, meaning it's fully licensed for commercial use. It beats Llama 2 7B on most benchmarks.

1

Package Mistral with Truss

We did this part for you! Grab a Truss for Mistral Instruct 7B from the model library.

git clone https://github.com/basetenlabs/truss-examples
cd model-library/mistral-instruct-7b

If you want to learn about model packaging, get started with this Truss tutorial.

2

Deploy Mistral to dev

Use truss push to deploy your model. Then, paste your Baseten API key when prompted.

truss push

You can see deployment information, model logs, and model metrics from your Baseten workspace.

3

Call Mistral from terminal

Use truss predict to call your model once it has finished deploying.

truss predict -d '{
    "prompt": "What is the difference between a wizard and a sorcerer?",
    "stream": true
  }'

Your model is served behind an API endpoint and will scale to zero when not in use to save you money.

Next, you can iterate on your model with Truss' live reload dev loop or promote it to production.

Model deployment

Model performance

Model observability