Model APIs

Call hosted LLMs through endpoints compatible with the OpenAI Chat Completions API and the Anthropic Messages API (beta). Point an existing OpenAI or Anthropic SDK at Baseten’s inference endpoint. You do not need to deploy a model. Model APIs run a supported set of models on infrastructure managed by Baseten. Choose a dedicated deployment when you need another model, dedicated GPUs, or control over hardware and scaling.

Supported models

Choose a model below, then run inference with its slug.

Context and output limits can differ from a model’s advertised maximum. Use the limits in this table when configuring requests. The /v1/models catalog can report different values from the limits enforced by the endpoint.

Fast variants use the same weights as their base models with dedicated capacity for higher sustained throughput. They use separate slugs, pricing, and rate limits. When Fast capacity is unavailable, Baseten uses base-model capacity.

Feature support

All Model APIs support tool calling. All except Muse Spark Open support structured outputs and JSON mode. The table lists model-specific support for reasoning, vision, and sampling parameters. Muse Spark Open also supports audio input. Web search is available in early access on enabled workspaces and models.

Defaults and limitations

Muse Spark Open accepts logprobs and top_logprobs but ignores them and returns no log probabilities. It supports one response per prompt; n greater than 1 returns HTTP 400. Neither seed nor temperature: 0 guarantees reproducible output.

Regions and compliance

Model APIs use a shared endpoint, and the regions serving a model can change. If your workloads must stay in specific regions, contact support to confirm model availability and enable a Model APIs compliance policy. The policy restricts which deployments can serve your requests. Availability depends on the model having capacity in an allowed region. See Secure model inference for data privacy and compliance policies.

Run inference

Choose the OpenAI Chat Completions API or the Anthropic Messages API, then set the base URL, API key, and model slug.

Use the OpenAI SDK

The OpenAI Chat Completions API is available at https://inference.baseten.co/v1/chat/completions.

Python
JavaScript
cURL
Baseten CLI

To call a model with the Chat Completions API:

chat_completions.py

from openai import OpenAI
import os

client = OpenAI(
    base_url="https://inference.baseten.co/v1",
    api_key=os.environ["BASETEN_API_KEY"],
)

response = client.chat.completions.create(
    model="zai-org/GLM-5.3",
    extra_headers={"x-session-affinity": "YOUR_SESSION_ID"},
    messages=[
        {"role": "system", "content": "You are a concise technical writer."},
        {"role": "user", "content": "What is gradient descent?"},
        {"role": "assistant", "content": "An optimization algorithm that iteratively adjusts model parameters by moving in the direction of steepest decrease in the loss function."},
        {"role": "user", "content": "How does the learning rate affect it?"}
    ],
)

print(response.choices[0].message.content)

To call a model with the Chat Completions API:

chat_completions.js

import OpenAI from "openai";

const client = new OpenAI({
    baseURL: "https://inference.baseten.co/v1",
    apiKey: process.env.BASETEN_API_KEY,
});

const response = await client.chat.completions.create({
    model: "zai-org/GLM-5.3",
    messages: [
        { role: "system", content: "You are a concise technical writer." },
        { role: "user", content: "What is gradient descent?" },
        { role: "assistant", content: "An optimization algorithm that iteratively adjusts model parameters by moving in the direction of steepest decrease in the loss function." },
        { role: "user", content: "How does the learning rate affect it?" }
    ],
});

console.log(response.choices[0].message.content);

To call a model with the Chat Completions API:

Request

curl https://inference.baseten.co/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $BASETEN_API_KEY" \
  -d '{
    "model": "zai-org/GLM-5.3",
    "messages": [
      {"role": "system", "content": "You are a concise technical writer."},
      {"role": "user", "content": "What is gradient descent?"},
      {"role": "assistant", "content": "An optimization algorithm that iteratively adjusts model parameters by moving in the direction of steepest decrease in the loss function."},
      {"role": "user", "content": "How does the learning rate affect it?"}
    ]
  }'

To call a model with the Chat Completions API:With the Baseten CLI installed and signed in, --content sends a single user message and prints the assistant’s reply:

baseten model-api predict --model zai-org/GLM-5.3 --content "What is gradient descent?"

For multi-message conversations like the examples above, send the full body with --data. See model-api predict for the full flag set.

Replace the example slug with any model from the supported models table. Reuse the same x-session-affinity value for related requests to improve cached input tokens reuse. We recommend at most ~60 requests/min sent via the same session.

Use the Anthropic SDK

The Anthropic Messages API is available at https://inference.baseten.co/v1/messages.

Anthropic Messages API support is in beta. Behavior may change before general availability. For production workloads, use the OpenAI Chat Completions API.

The Anthropic SDK sends the API key as x-api-key by default. Baseten reads Authorization, so override default_headers as shown.

Python
JavaScript
cURL

To call a model with the Messages API:

messages_api.py

import anthropic
import os

API_KEY = os.environ["BASETEN_API_KEY"]

client = anthropic.Anthropic(
    base_url="https://inference.baseten.co",
    api_key=API_KEY,
    default_headers={"Authorization": f"Bearer {API_KEY}"},
)

response = client.messages.create(
    model="zai-org/GLM-5.3",
    max_tokens=4096,
    system="You are a concise technical writer.",
    messages=[
        {"role": "user", "content": "What is gradient descent?"},
        {"role": "assistant", "content": "An optimization algorithm that iteratively adjusts model parameters by moving in the direction of steepest decrease in the loss function."},
        {"role": "user", "content": "How does the learning rate affect it?"}
    ],
)

for block in response.content:
    if block.type == "text":
        print(block.text)

To call a model with the Messages API:

messages_api.js

import Anthropic from "@anthropic-ai/sdk";

const apiKey = process.env.BASETEN_API_KEY;

const client = new Anthropic({
    baseURL: "https://inference.baseten.co",
    apiKey: apiKey,
    defaultHeaders: { Authorization: `Bearer ${apiKey}` },
});

const response = await client.messages.create({
    model: "zai-org/GLM-5.3",
    max_tokens: 4096,
    system: "You are a concise technical writer.",
    messages: [
        { role: "user", content: "What is gradient descent?" },
        { role: "assistant", content: "An optimization algorithm that iteratively adjusts model parameters by moving in the direction of steepest decrease in the loss function." },
        { role: "user", content: "How does the learning rate affect it?" }
    ],
});

for (const block of response.content) {
    if (block.type === "text") console.log(block.text);
}

To call a model with the Messages API:

Request

curl https://inference.baseten.co/v1/messages \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $BASETEN_API_KEY" \
  -d '{
    "model": "zai-org/GLM-5.3",
    "max_tokens": 4096,
    "system": "You are a concise technical writer.",
    "messages": [
      {"role": "user", "content": "What is gradient descent?"},
      {"role": "assistant", "content": "An optimization algorithm that iteratively adjusts model parameters by moving in the direction of steepest decrease in the loss function."},
      {"role": "user", "content": "How does the learning rate affect it?"}
    ]
  }'

List available models

List the current model catalog and metadata, including pricing, context windows, and supported features, with baseten model-api list or the /v1/models REST endpoint:

baseten model-api list

curl https://inference.baseten.co/v1/models \
  -H "Authorization: Bearer $BASETEN_API_KEY"

Describe a single model’s details with baseten model-api describe:

baseten model-api describe --model <slug>

Migrate

To migrate to Baseten, change the base URL, API key, and model name.

OpenAI SDK
Anthropic SDK

To migrate from the OpenAI SDK:

Replace your OpenAI API key with a Baseten API key.
Change the base URL to https://inference.baseten.co/v1.
Update the model name to a Baseten model slug.

migrate.py

from openai import OpenAI
import os

client = OpenAI(
    base_url="https://inference.baseten.co/v1",  
    api_key=os.environ["BASETEN_API_KEY"]  
)

response = client.chat.completions.create(
    model="zai-org/GLM-5.3",  
    messages=[{"role": "user", "content": "Hello"}]
)

To migrate from the Anthropic SDK:

Replace your Anthropic API key with a Baseten API key.
Change the base URL to https://inference.baseten.co.
Override default_headers so the SDK sends Authorization instead of x-api-key.
Update the model name to a supported Baseten model slug.

migrate.py

import anthropic
import os

API_KEY = os.environ["BASETEN_API_KEY"]

client = anthropic.Anthropic(
    base_url="https://inference.baseten.co",  
    api_key=API_KEY,  
    default_headers={"Authorization": f"Bearer {API_KEY}"},  
)

response = client.messages.create(
    model="zai-org/GLM-5.3",  
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello"}]
)

Handle errors

Model API requests can return the following HTTP error codes:

Code	Meaning
400	Invalid request (check your parameters)
401	Invalid or missing API key
402	Payment required
404	Model not found
429	Rate limit exceeded
500	Internal server error
529	Overloaded; retry with backoff

HTTP error responses include a JSON body with details about the failure. Once streaming starts, a request can fail without a new HTTP status or an error object. See streaming completion markers and recovery. Use bounded retries with backoff and jitter for transient failures. See Inference errors for causes and recommended responses by status code.

Next steps

Coding agents

Call Model APIs from Pi, Droid, or OpenCode, or connect Claude Code and Codex CLI with Baseten Switch.

Reasoning

Control extended thinking for complex tasks.

Vision

Send images and videos alongside text.

Web search

Search the web during a Model API response.

Pricing and limits

Review pricing, rate limits, budgets, and usage tracking.

API reference

Review Chat Completions parameters.

Get started

Inference

Training

Sandboxes

Development

Deployment

Engines

Model Labs

Organization

Hosting options

Observability

Troubleshooting

Model APIs

Supported models

Feature support

Defaults and limitations

Regions and compliance

Run inference

Use the OpenAI SDK

Use the Anthropic SDK

List available models

Migrate

Handle errors

Next steps

Coding agents

Reasoning

Vision

Web search

Pricing and limits

API reference

​Supported models

​Feature support

​Defaults and limitations

​Regions and compliance

​Run inference

​Use the OpenAI SDK

​Use the Anthropic SDK

​List available models

​Migrate

​Handle errors

​Next steps

Coding agents

Reasoning

Vision

Web search

Pricing and limits

API reference

Supported models

Feature support

Defaults and limitations

Regions and compliance

Run inference

Use the OpenAI SDK

Use the Anthropic SDK

List available models

Migrate

Handle errors

Next steps