Quickstart

Model APIs let you call hosted open-source LLMs without deploying or managing infrastructure. Use models such as DeepSeek V4 Flash 0731, GLM 5.3, and Kimi K3 through APIs compatible with the OpenAI and Anthropic SDKs. See the full list of supported models. In this quickstart, you will set an API key and base URL, then send a request to a hosted LLM.

Set up your API key and SDK

Create a Baseten account and a personal API key, then install a client SDK.

Export your API key

export BASETEN_API_KEY="paste-your-api-key-here"

setx BASETEN_API_KEY "paste-your-api-key-here"

Install a client SDK

uv pip install openai

npm install openai

Run inference

All Model APIs work with the OpenAI SDK. Anthropic SDK support is in beta. Models may also support tool calling, structured outputs, reasoning, and other features. Call a model using the OpenAI SDK. This example uses zai-org/GLM-5.3, but you can swap in any supported model.

Python
JavaScript
cURL
Baseten CLI

To create a chat completion:

chat.py

from openai import OpenAI
import os

client = OpenAI(
    base_url="https://inference.baseten.co/v1",
    api_key=os.environ["BASETEN_API_KEY"],
)

response = client.chat.completions.create(
    model="zai-org/GLM-5.3",
    messages=[
        {"role": "user", "content": "What is inference in machine learning?"}
    ],
)

print(response.choices[0].message.content)

To create a chat completion:

chat.mjs

import OpenAI from "openai";

const client = new OpenAI({
    baseURL: "https://inference.baseten.co/v1",
    apiKey: process.env.BASETEN_API_KEY,
});

const response = await client.chat.completions.create({
    model: "zai-org/GLM-5.3",
    messages: [
        { role: "user", content: "What is inference in machine learning?" }
    ],
});

console.log(response.choices[0].message.content);

To create a chat completion:

curl https://inference.baseten.co/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $BASETEN_API_KEY" \
  -d '{
    "model": "zai-org/GLM-5.3",
    "messages": [
      {"role": "user", "content": "What is inference in machine learning?"}
    ]
  }'

To create a chat completion:With the Baseten CLI installed and signed in, --content sends a single user message and prints the assistant’s reply:

baseten model-api predict --model zai-org/GLM-5.3 --content "What is inference in machine learning?"

For multi-message or Anthropic-shaped requests, send the full body with --data. See model-api predict for the full flag set.

Success looks like this:

Inference in machine learning refers to the process of using a trained model
to make predictions or generate outputs from new input data...

Stream the response

Streaming returns tokens as the model generates them instead of waiting for the full response. Use it for chat interfaces and other interactive applications.

Python
JavaScript

Set stream=True to receive tokens as the model generates them:

stream.py

stream = client.chat.completions.create(
    model="zai-org/GLM-5.3",
    messages=[
        {"role": "user", "content": "Write a haiku about machine learning."}
    ],
    stream=True,
)

for chunk in stream:
    if not chunk.choices:
        continue
    content = chunk.choices[0].delta.content
    if content:
        print(content, end="")

Set stream: true to receive tokens as the model generates them:

stream.mjs

const stream = await client.chat.completions.create({
    model: "zai-org/GLM-5.3",
    messages: [
        { role: "user", content: "Write a haiku about machine learning." }
    ],
    stream: true,
});

for await (const chunk of stream) {
    const content = chunk.choices[0]?.delta?.content;
    if (content) process.stdout.write(content);
}

Explore Model API features

Structured outputs

Generate JSON that conforms to a schema you define.

Tool calling

Let the model invoke functions and use the results in its response.

Reasoning

Enable extended thinking for multi-step problem solving.

Next steps

Platform overview

Explore hosted models, dedicated deployments, training, and production operations.

Deploy your first model

Deploy a Hugging Face model on dedicated GPUs with config.yaml and the Baseten CLI.

Get started

Model APIs

Inference

Training

Sandboxes

Development

Deployment

Engines

Model Labs

Organization

Hosting options

Observability

Troubleshooting

Set up your API key and SDK

Run inference

Stream the response

Explore Model API features

Structured outputs

Tool calling

Reasoning

Next steps

Platform overview

Deploy your first model

​Set up your API key and SDK

​Run inference

​Stream the response

​Explore Model API features

Structured outputs

Tool calling

Reasoning

​Next steps

Platform overview

Deploy your first model

Set up your API key and SDK

Run inference

Stream the response

Explore Model API features

Next steps