Model APIs

Call hosted language models and decision models without deploying a model. Baseten manages the serving infrastructure; you send requests with a Baseten API key. Language models support:

Decision models use the System One API to classify, route, and score inputs.

Choose how to start

Your goal	Start here
Make your first API call	Follow the quickstart to create a key and send a request.
Connect an existing application	Migrate your client and choose the matching API protocol.
Use models in a coding agent	Configure your coding agent to use Model APIs.
Choose a language model	Compare supported models and feature support.
Classify, route, or score inputs	Send structured questions to decision models.

Choose a dedicated deployment when you need another model, dedicated GPUs, or control over hardware and scaling.

Supported models

A model slug identifies the hosted model, such as zai-org/GLM-5.3. Set the request’s model field to the exact slug from the table, then run inference.

Context and output limits can differ from a model’s advertised maximum. Use the limits in this table when configuring requests. The /v1/models catalog can report different values from the limits enforced by the endpoint.

Fast variants use the same weights as their base models with dedicated capacity for higher sustained throughput. They use separate slugs, pricing, and rate limits. When Fast capacity is unavailable, Baseten uses base-model capacity.

Feature support

The language models in this table support tool calling. All except Muse Spark Open support structured outputs and JSON mode. The table lists model-specific support for reasoning, vision, and sampling parameters. Muse Spark Open also supports audio input. Web search is available on supported models.

Defaults and limitations

Muse Spark Open accepts logprobs and top_logprobs but ignores them and returns no log probabilities. It supports one response per prompt; n greater than 1 returns HTTP 400. Neither seed nor temperature: 0 guarantees reproducible output.

Regions and compliance

Model APIs use a shared endpoint, and the regions serving a model can change. If your workloads must stay in specific regions, contact support to confirm model availability and enable a Model APIs compliance policy. The policy restricts which deployments can serve your requests. Availability depends on the model having capacity in an allowed region. For synchronous Model API requests, Baseten doesn’t store inputs or outputs under its zero data retention policy. For queued execution, use async inference with a dedicated deployment and review its input-retention behavior.

Run inference

Set BASETEN_API_KEY to your API key. If you need to create a key, follow the quickstart. Requests use the model’s token pricing. Generated text can vary. The cURL responses are excerpts; ... marks omitted fields. To use another model, replace zai-org/GLM-5.3 with its slug.

Use the OpenAI SDK

Set the OpenAI client’s base URL to https://inference.baseten.co/v1 and pass your Baseten API key. Choose the API format your application uses, then select a language. The CLI sends Chat Completions requests.

Chat Completions
Responses
Baseten CLI

Python
JavaScript
cURL

These examples require the openai package (pip install openai).To call a model with the Chat Completions API:Save the following code as chat.py, then run python chat.py:

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://inference.baseten.co/v1",
    api_key=os.environ["BASETEN_API_KEY"],
)

response = client.chat.completions.create(
    model="zai-org/GLM-5.3",
    messages=[{"role": "user", "content": "Reply with exactly: Hello from Baseten."}],
)

print(response.choices[0].message.content)

Hello from Baseten.

These examples require the openai package (npm install openai).To call a model with the Chat Completions API:Save the following code as chat.mjs, then run node chat.mjs:

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://inference.baseten.co/v1",
  apiKey: process.env.BASETEN_API_KEY,
});

const response = await client.chat.completions.create({
  model: "zai-org/GLM-5.3",
  messages: [{ role: "user", content: "Reply with exactly: Hello from Baseten." }],
});

console.log(response.choices[0].message.content);

Hello from Baseten.

To call a model with the Chat Completions API:Run the following command:

curl https://inference.baseten.co/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $BASETEN_API_KEY" \
  -d '{
    "model": "zai-org/GLM-5.3",
    "messages": [
      {
        "role": "user",
        "content": "Reply with exactly: Hello from Baseten."
      }
    ]
  }'

{
  "choices": [
    {
      "index": 0,
      "message": {
        "content": "Hello from Baseten.",
        "role": "assistant",
        ...
      },
      "finish_reason": "stop"
    }
  ],
  ...
}

Read the answer from choices[0].message.content in the JSON response.For more information, see the Chat Completions API reference.

Python
JavaScript
cURL

These examples require the openai package (pip install openai).To call a model with the Responses API:Save the following code as responses.py, then run python responses.py:

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://inference.baseten.co/v1",
    api_key=os.environ["BASETEN_API_KEY"],
)

response = client.responses.create(
    model="zai-org/GLM-5.3",
    input="Reply with exactly: Hello from Baseten.",
)

print(response.output_text)

Hello from Baseten.

These examples require the openai package (npm install openai).To call a model with the Responses API:Save the following code as responses.mjs, then run node responses.mjs:

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://inference.baseten.co/v1",
  apiKey: process.env.BASETEN_API_KEY,
});

const response = await client.responses.create({
  model: "zai-org/GLM-5.3",
  input: "Reply with exactly: Hello from Baseten.",
});

console.log(response.output_text);

Hello from Baseten.

To call a model with the Responses API:Run the following command:

curl https://inference.baseten.co/v1/responses \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $BASETEN_API_KEY" \
  -d '{
    "model": "zai-org/GLM-5.3",
    "input": "Reply with exactly: Hello from Baseten."
  }'

{
  "output": [
    ...,
    {
      "type": "message",
      "content": [
        {
          "type": "output_text",
          "annotations": [],
          "logprobs": [],
          "text": "Hello from Baseten."
        }
      ],
      "id": "<message-id>",
      "role": "assistant",
      "status": "completed"
    }
  ],
  ...
}

The SDK’s output_text property collects the answer text. In the JSON response, find it in output items with type: "message", inside content blocks with type: "output_text".Responses requests are stateless. Send the full conversation as input on every call.For more information, see conversation history and Agents SDK setup.

These examples require the Baseten CLI.To call a model from the terminal:Run the following command. --content sends a single user message and prints the assistant’s reply:

baseten model-api predict --model zai-org/GLM-5.3 --content "Reply with exactly: Hello from Baseten."

Hello from Baseten.

For more information, see baseten model-api predict.

Use the Anthropic SDK

Anthropic Messages API support is in beta. Behavior may change before general availability. For production workloads, use the OpenAI Chat Completions API.

The Messages endpoint is https://inference.baseten.co/v1/messages. Set the Anthropic SDK’s base URL to https://inference.baseten.co. The SDK sends x-api-key by default; override its default headers to send your Baseten API key in Authorization. Messages requests require max_tokens. These examples allow up to 4,096 output tokens, including reasoning and the final answer.

Python
JavaScript
cURL

These examples require the anthropic package (pip install anthropic).To call a model with the Messages API:Save the following code as messages.py, then run python messages.py:

import os
import anthropic

api_key = os.environ["BASETEN_API_KEY"]
client = anthropic.Anthropic(
    base_url="https://inference.baseten.co",
    api_key=api_key,
    default_headers={"Authorization": f"Bearer {api_key}"},
)

response = client.messages.create(
    model="zai-org/GLM-5.3",
    max_tokens=4096,
    messages=[{"role": "user", "content": "Reply with exactly: Hello from Baseten."}],
)

for block in response.content:
    if block.type == "text":
        print(block.text)

Hello from Baseten.

These examples require the @anthropic-ai/sdk package (npm install @anthropic-ai/sdk).To call a model with the Messages API:Save the following code as messages.mjs, then run node messages.mjs:

import Anthropic from "@anthropic-ai/sdk";

const apiKey = process.env.BASETEN_API_KEY;
const client = new Anthropic({
  baseURL: "https://inference.baseten.co",
  apiKey,
  defaultHeaders: { Authorization: `Bearer ${apiKey}` },
});

const response = await client.messages.create({
  model: "zai-org/GLM-5.3",
  max_tokens: 4096,
  messages: [{ role: "user", content: "Reply with exactly: Hello from Baseten." }],
});

for (const block of response.content) {
  if (block.type === "text") console.log(block.text);
}

Hello from Baseten.

To call a model with the Messages API:Run the following command:

curl https://inference.baseten.co/v1/messages \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $BASETEN_API_KEY" \
  -d '{
    "model": "zai-org/GLM-5.3",
    "max_tokens": 4096,
    "messages": [
      {
        "role": "user",
        "content": "Reply with exactly: Hello from Baseten."
      }
    ]
  }'

{
  "content": [
    ...,
    {
      "type": "text",
      "text": "Hello from Baseten."
    }
  ],
  ...
}

Read the answer from content blocks with type: "text". For related requests, use a stable session affinity ID to improve cache reuse. Session affinity controls routing; continue to send the conversation history with each request. For more information, see the Messages API reference.

List available models

List the current model catalog and metadata, including pricing, context windows, and supported features, with baseten model-api list or the /v1/models REST endpoint:

baseten model-api list

curl https://inference.baseten.co/v1/models \
  -H "Authorization: Bearer $BASETEN_API_KEY"

To inspect a model’s details with baseten model-api describe, run the following command. Replace zai-org/GLM-5.3 with the model’s slug:

baseten model-api describe --model zai-org/GLM-5.3

Migrate

To connect an existing application to Model APIs:

Replace your provider’s key with a Baseten API key.

Configure your client’s base URL and authentication for its API format:

Client	Base URL	Authentication
OpenAI SDK, Chat Completions or Responses	`https://inference.baseten.co/v1`	Pass the Baseten key as `api_key` in Python or `apiKey` in JavaScript.
Anthropic SDK, Messages (beta)	`https://inference.baseten.co`	Pass the Baseten key and override the default headers to send `Authorization: Bearer <BASETEN_API_KEY>`.

Set model to a supported slug and check its feature support and reasoning controls.
Send a test request using the matching Chat Completions, Responses, or Messages example. Check the answer and token usage before switching your application’s traffic.

The API format defines the request shape. The model determines which features and parameter values work. Check the model’s limits and behavior when changing models, even if you keep the same client.

Handle errors

Model API requests can return the following HTTP error codes:

Code	Meaning
400	Invalid request (check your parameters)
401	Invalid or missing API key
402	Payment required
404	Model not found
429	Rate limit exceeded
500	Internal server error
529	Overloaded; retry with backoff

HTTP error responses include a JSON body with details about the failure. Once streaming starts, a request can fail without a new HTTP status or an error object. See streaming completion markers and recovery. Use bounded retries with backoff and jitter for transient failures. See Inference errors for causes and recommended responses by status code.

Next steps

Use models in coding agents

Call Model APIs from Pi, Droid, or OpenCode, or connect Claude Code and Codex CLI with Baseten Switch.

Reasoning

Control extended thinking for complex tasks.

Vision

Send images and videos alongside text.

Web search

Search the web during a Model API response.

Pricing and limits

Review pricing, rate limits, budgets, and usage tracking.

API reference

Review Chat Completions parameters.

Get started

Inference

Frontier Gateway

Training

Sandboxes

Development

Deployment

Engines

Model Labs

Organization

Hosting options

Observability

Troubleshooting

Model APIs

Choose how to start

Supported models

Feature support

Defaults and limitations

Regions and compliance

Run inference

Use the OpenAI SDK

Use the Anthropic SDK

List available models

Migrate

Handle errors

Next steps

Use models in coding agents

Reasoning

Vision

Web search

Pricing and limits

API reference

​Choose how to start

​Supported models

​Feature support

​Defaults and limitations

​Regions and compliance

​Run inference

​Use the OpenAI SDK

​Use the Anthropic SDK

​List available models

​Migrate

​Handle errors

​Next steps

Use models in coding agents

Reasoning

Vision

Web search

Pricing and limits

API reference

Choose how to start

Supported models

Feature support

Defaults and limitations

Regions and compliance

Run inference

Use the OpenAI SDK

Use the Anthropic SDK

List available models

Migrate

Handle errors

Next steps