Baseten overview

Baseten is a platform for model inference and training. You can call hosted models through an OpenAI-compatible API, deploy your own models on dedicated infrastructure, or train models and serve the resulting checkpoints. Baseten manages model containers, GPU capacity across clouds and regions, scaling, and observability. Its inference engines optimize supported model architectures for production traffic.

​

Choose how to start

To point a coding agent at Model APIs, see Coding agents. To let an agent operate your workspace, see Connect your coding agent.

​

Model APIs

Model APIs expose hosted endpoints for a curated set of models. They support the OpenAI Chat Completions API and the Anthropic Messages API in beta, so you can use familiar client SDKs. Choose Model APIs when a supported model fits your application and you want to start with an API call. Baseten manages the model, engine, hardware, and scaling.

​

Dedicated inference

Deploy an open-source model, private checkpoint, or custom model on dedicated infrastructure with Truss. Truss packages the model and its dependencies, then Baseten builds and deploys the container. For supported architectures, a config.yaml can define the model, hardware, and inference engine without custom serving code. Use a Python model class or a custom Docker server when you need custom preprocessing, postprocessing, dependencies, or server behavior. Dedicated deployments give you control over the model, hardware, autoscaling settings, and release lifecycle. They also support stable environments for development, staging, and production.

​

Inference engines

Baseten maintains inference engines for common model architectures:

• Engine-Builder-LLM compiles dense text-generation models with TensorRT-LLM.
• BIS-LLM serves large mixture-of-experts models with distributed inference and KV-aware routing.
• BEI serves embedding, reranking, and classification models.

You can select an engine in config.yaml, let Baseten select one for a supported architecture, or run a server such as vLLM or SGLang in a custom Docker container.

​

Training

Baseten has two ways to train models:

• Loops provides dedicated trainers and samplers for supervised fine-tuning and reinforcement learning on supported base models.
• Training Jobs runs your framework and training code in your own container on dedicated GPU clusters.

Both paths store checkpoints that you can deploy to the Baseten inference stack. See Training on Baseten to compare them.

​

Production operations

Every dedicated deployment includes logs, metrics, request traces, autoscaling, and environment promotion. Export observability data to Datadog, Prometheus, Grafana, or New Relic, and store secrets in the workspace instead of in source control. Multi-cloud Capacity Management provisions GPUs across cloud providers and regions. Baseten also supports multiple hosting options for teams that need private networking, workload isolation, infrastructure ownership, or specific data regions. Baseten is SOC 2 Type II and HIPAA compliant, and supports regional environments for data-residency requirements.

​

Baseten for Model Labs

Baseten for Model Labs is how you serve and distribute models to your customers. It is enabled per workspace. To get access, talk to us.

​

Next steps