Model APIs

Call hosted LLMs through endpoints compatible with the OpenAI Chat Completions API and the Anthropic Messages API (beta). Point an existing OpenAI or Anthropic SDK at Baseten’s inference endpoint. You do not need to deploy a model. Model APIs run a supported set of models on infrastructure managed by Baseten. Choose a dedicated deployment when you need another model, dedicated GPUs, or control over hardware and scaling.

Supported models

Choose a model below, then run inference with its slug.

Context and output limits reflect Baseten’s live serving configuration and can differ from a model’s advertised maximum. Baseten may extend a limit after validating its performance. This table and /v1/models reflect the current serving configuration.

Some models also have a Fast tier for latency-sensitive workloads. Fast models use the same weights as their base models but run on dedicated capacity for higher sustained per-user throughput. Each Fast model has its own slug, pricing, and rate limits. For example, use zai-org/GLM-5.2-Fast instead of zai-org/GLM-5.2. If Fast capacity is saturated, Baseten serves the request on base-model capacity instead of failing it.

Feature support

All models support tool calling (also known as function calling), structured outputs, and JSON mode. The table shows which models also support reasoning, vision, and audio. See Reasoning, Vision, and Audio for feature-specific configuration.

GLM 4.7 and Nemotron Ultra also support top_p and top_k sampling parameters. GLM 5.2 and GLM 5.2 Fast support top_p.

Run inference

Choose the OpenAI Chat Completions API or the Anthropic Messages API, then set the base URL, API key, and model slug.

Use the OpenAI SDK

The OpenAI Chat Completions API is available at https://inference.baseten.co/v1/chat/completions.

Python
JavaScript
cURL

To call a model with the Chat Completions API:

chat_completions.py

from openai import OpenAI
import os

client = OpenAI(
    base_url="https://inference.baseten.co/v1",
    api_key=os.environ["BASETEN_API_KEY"],
)

response = client.chat.completions.create(
    model="zai-org/GLM-5.2",
    messages=[
        {"role": "system", "content": "You are a concise technical writer."},
        {"role": "user", "content": "What is gradient descent?"},
        {"role": "assistant", "content": "An optimization algorithm that iteratively adjusts model parameters by moving in the direction of steepest decrease in the loss function."},
        {"role": "user", "content": "How does the learning rate affect it?"}
    ],
)

print(response.choices[0].message.content)

To call a model with the Chat Completions API:

chat_completions.js

import OpenAI from "openai";

const client = new OpenAI({
    baseURL: "https://inference.baseten.co/v1",
    apiKey: process.env.BASETEN_API_KEY,
});

const response = await client.chat.completions.create({
    model: "zai-org/GLM-5.2",
    messages: [
        { role: "system", content: "You are a concise technical writer." },
        { role: "user", content: "What is gradient descent?" },
        { role: "assistant", content: "An optimization algorithm that iteratively adjusts model parameters by moving in the direction of steepest decrease in the loss function." },
        { role: "user", content: "How does the learning rate affect it?" }
    ],
});

console.log(response.choices[0].message.content);

To call a model with the Chat Completions API:

Request

curl https://inference.baseten.co/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $BASETEN_API_KEY" \
  -d '{
    "model": "zai-org/GLM-5.2",
    "messages": [
      {"role": "system", "content": "You are a concise technical writer."},
      {"role": "user", "content": "What is gradient descent?"},
      {"role": "assistant", "content": "An optimization algorithm that iteratively adjusts model parameters by moving in the direction of steepest decrease in the loss function."},
      {"role": "user", "content": "How does the learning rate affect it?"}
    ]
  }'

Replace the example slug with any model from the supported models table.

Use the Anthropic SDK

The Anthropic Messages API is available at https://inference.baseten.co/v1/messages.

Anthropic Messages API support is in beta. Behavior may change before general availability. For production workloads, use the OpenAI Chat Completions API.

Python
JavaScript
cURL

To call a model with the Messages API:

messages_api.py

import anthropic
import os

API_KEY = os.environ["BASETEN_API_KEY"]

client = anthropic.Anthropic(
    base_url="https://inference.baseten.co",
    api_key=API_KEY,
    default_headers={"Authorization": f"Bearer {API_KEY}"},
)

response = client.messages.create(
    model="zai-org/GLM-5.2",
    max_tokens=4096,
    system="You are a concise technical writer.",
    messages=[
        {"role": "user", "content": "What is gradient descent?"},
        {"role": "assistant", "content": "An optimization algorithm that iteratively adjusts model parameters by moving in the direction of steepest decrease in the loss function."},
        {"role": "user", "content": "How does the learning rate affect it?"}
    ],
)

for block in response.content:
    if block.type == "text":
        print(block.text)

To call a model with the Messages API:

messages_api.js

import Anthropic from "@anthropic-ai/sdk";

const apiKey = process.env.BASETEN_API_KEY;

const client = new Anthropic({
    baseURL: "https://inference.baseten.co",
    apiKey: apiKey,
    defaultHeaders: { Authorization: `Bearer ${apiKey}` },
});

const response = await client.messages.create({
    model: "zai-org/GLM-5.2",
    max_tokens: 4096,
    system: "You are a concise technical writer.",
    messages: [
        { role: "user", content: "What is gradient descent?" },
        { role: "assistant", content: "An optimization algorithm that iteratively adjusts model parameters by moving in the direction of steepest decrease in the loss function." },
        { role: "user", content: "How does the learning rate affect it?" }
    ],
});

for (const block of response.content) {
    if (block.type === "text") console.log(block.text);
}

To call a model with the Messages API:

Request

curl https://inference.baseten.co/v1/messages \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $BASETEN_API_KEY" \
  -d '{
    "model": "zai-org/GLM-5.2",
    "max_tokens": 4096,
    "system": "You are a concise technical writer.",
    "messages": [
      {"role": "user", "content": "What is gradient descent?"},
      {"role": "assistant", "content": "An optimization algorithm that iteratively adjusts model parameters by moving in the direction of steepest decrease in the loss function."},
      {"role": "user", "content": "How does the learning rate affect it?"}
    ]
  }'

The Anthropic SDK sends the API key as x-api-key by default. Baseten reads Authorization, so override default_headers as shown.

List available models

Query /v1/models for the current model list and metadata, including pricing, context windows, and supported features:

Request

curl https://inference.baseten.co/v1/models \
  -H "Authorization: Bearer $BASETEN_API_KEY"

Migrate

To migrate to Baseten, change the base URL, API key, and model name.

OpenAI SDK
Anthropic SDK

To migrate from the OpenAI SDK:

Replace your OpenAI API key with a Baseten API key.
Change the base URL to https://inference.baseten.co/v1.
Update the model name to a Baseten model slug.

migrate.py

from openai import OpenAI
import os

client = OpenAI(
    base_url="https://inference.baseten.co/v1",  
    api_key=os.environ["BASETEN_API_KEY"]  
)

response = client.chat.completions.create(
    model="zai-org/GLM-5.2",  
    messages=[{"role": "user", "content": "Hello"}]
)

To migrate from the Anthropic SDK:

Replace your Anthropic API key with a Baseten API key.
Change the base URL to https://inference.baseten.co.
Override default_headers so the SDK sends Authorization instead of x-api-key.
Update the model name to a supported Baseten model slug.

migrate.py

import anthropic
import os

API_KEY = os.environ["BASETEN_API_KEY"]

client = anthropic.Anthropic(
    base_url="https://inference.baseten.co",  
    api_key=API_KEY,  
    default_headers={"Authorization": f"Bearer {API_KEY}"},  
)

response = client.messages.create(
    model="zai-org/GLM-5.2",  
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello"}]
)

Handle errors

Model APIs return standard HTTP error codes:

Code	Meaning
400	Invalid request (check your parameters)
401	Invalid or missing API key
402	Payment required
404	Model not found
429	Rate limit exceeded
500	Internal server error
529	Overloaded; retry with backoff

Each error response includes a JSON body with details about the failure. See Inference errors for causes and recommended responses.

Next steps

Coding agents

Call Model APIs from Pi, Droid, or OpenCode, or connect Claude Code and Codex CLI with Baseten Switch.

Reasoning

Control extended thinking for complex tasks.

Vision

Send images and videos alongside text.

Audio

Send audio alongside text.

Pricing and limits

Review pricing, rate limits, budgets, and usage tracking.

API reference

Review Chat Completions parameters.

Get started

Inference

Training

Development

Deployment

Engines

Model Labs

Organization

Hosting options

Observability

Troubleshooting

Model APIs

Supported models

Feature support

Run inference

Use the OpenAI SDK

Use the Anthropic SDK

List available models

Migrate

Handle errors

Next steps

Coding agents

Reasoning

Vision

Audio

Pricing and limits

API reference

​Supported models

​Feature support

​Run inference

​Use the OpenAI SDK

​Use the Anthropic SDK

​List available models

​Migrate

​Handle errors

​Next steps

Coding agents

Reasoning

Vision

Audio

Pricing and limits

API reference

Supported models

Feature support

Run inference

Use the OpenAI SDK

Use the Anthropic SDK

List available models

Migrate

Handle errors

Next steps