Skip to main content
Baseten exposes two types of HTTPS inference endpoints. Model APIs provide hosted LLMs through a shared OpenAI-compatible endpoint. Deployed model endpoints serve models and Chains on dedicated subdomains.

​
Model APIs

Model APIs run supported models such as DeepSeek, GLM, and Kimi on infrastructure managed by Baseten. You choose a model slug and pay per million tokens. All models support tool calling, structured outputs, and JSON mode; some also support reasoning, vision, or audio. Model APIs implement the OpenAI Chat Completions format. To update an OpenAI client, change its base URL, API key, and model slug. Requests use this endpoint:
The Chat Completions reference documents request and response schemas. See the Model APIs guide for supported models, features, and examples.

​
Deployed model endpoints

Baseten assigns each deployed model or Chain a dedicated subdomain. Use a deployed endpoint for models outside the Model APIs catalog, fine-tuned weights, custom serving logic, or multi-step Chains. You control the hardware, autoscaling settings, and serving engine. Each endpoint URL includes a deployment target: an environment name like production, the development deployment, or a specific deployment ID. For models:
For chains:
  • model_id: the model’s alphanumeric ID, found in your model dashboard.
  • chain_id: the chain’s alphanumeric ID, found in your chain dashboard.
  • deployment_type_or_id: either development, production, or a specific deployment’s alphanumeric ID.
  • endpoint: the API action, such as predict.
For regional environments, the environment name is embedded in the hostname instead of the URL path:
For long-running tasks, the inference API supports asynchronous inference with priority queuing.
Use the inference API OpenAPI specification with code generators, SDK tools, and API clients.

​
Predict endpoints

All predict endpoints accept a JSON request body that is forwarded directly to the model’s predict function (for models) or chain entrypoint (for chains).

​
Status endpoints

​
Wake endpoints

​
Timeouts

Each request to a deployed model or chain has a server-side timeout. Requests that exceed it return a 504. Timeouts aren’t user-configurable. Set client timeouts based on your model’s expected response time. For more information, see Configure HTTP clients. For how to interpret and respond to a 504, see Inference errors.

​
Request size

The ingress proxy limits each request body to 100 MB. Larger requests return 413 Request Entity Too Large before reaching the model or Chain. The limit includes the JSON envelope and any base64-encoded media and cannot be changed. For large audio, video, or batched media, upload the file to S3, GCS, Azure Blob, or other object storage. Send a presigned URL in the request instead of embedding the file. The model can fetch the file directly from storage. For how to interpret and respond to a 413, including the separate async payload limit, see Inference errors.