Call your model

Send requests to a model deployed in your Baseten workspace. For hosted models that don’t require a deployment, use Model APIs.

Before you begin

You need an active deployment, its model ID, and a Baseten API key in BASETEN_API_KEY. The deployed model defines its input format; copy the request schema from its configuration or example. The model ID (and the deployment ID, when you need to target a specific deployment) comes from the model’s page URL in your workspace:

Anatomy of the deployment page URL. In app.baseten.co/models/abc123/deployments/def456, abc123 is the model ID and def456 is the deployment ID.

Call your model from server-side code to avoid exposing your Baseten API key. Dedicated deployment endpoints don’t currently include CORS response headers, so browsers may block direct calls.

Choose an interface

OpenAI-compatible model: Use the OpenAI SDK with the routes supported by your engine or custom server.
Custom Truss model: Send your model’s JSON input to /predict.
Custom HTTP server: Call a route exposed by the server through /sync.

OpenAI SDK

Use this interface for a deployment that exposes OpenAI-compatible chat, such as vLLM. Install the OpenAI SDK with pip install openai or npm install openai. Set BASETEN_MODEL_ID to your deployed model’s ID and BASETEN_SERVED_MODEL to the name accepted by its server. For vLLM, this is the value of --served-model-name in start_command.

Python
JavaScript

To get a chat response from your deployment:Run the following script:

import os
from openai import OpenAI

client = OpenAI(
    base_url=f"https://model-{os.environ['BASETEN_MODEL_ID']}.api.baseten.co/environments/production/sync/v1",
    api_key=os.environ["BASETEN_API_KEY"],
)
response = client.chat.completions.create(
    model=os.environ["BASETEN_SERVED_MODEL"],
    messages=[{"role": "user", "content": "Reply with exactly: Paris."}],
)
print(response.choices[0].message.content)

Paris.

To get a chat response from your deployment:Run the following script:

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: `https://model-${process.env.BASETEN_MODEL_ID}.api.baseten.co/environments/production/sync/v1`,
  apiKey: process.env.BASETEN_API_KEY,
});
const response = await client.chat.completions.create({
  model: process.env.BASETEN_SERVED_MODEL,
  messages: [{ role: "user", content: "Reply with exactly: Paris." }],
});
console.log(response.choices[0].message.content);

Paris.

The base URL selects the deployment; the model field must satisfy the server’s naming rules. Validation rules depend on the engine or server. Only call endpoints that your deployed server supports. For more information, see Streaming and the vLLM example.

Predict API endpoints

For a custom Truss model, send a JSON body that matches its predict() method. The following request uses an identity Truss, which returns its input unchanged. Install requests with pip install requests for the Python example, and set BASETEN_MODEL_ID to your model ID.

Python
cURL

To send JSON to your model:Run the following script, replacing the input with your model’s expected payload:

import os
import requests

response = requests.post(
    f"https://model-{os.environ['BASETEN_MODEL_ID']}.api.baseten.co/environments/production/predict",
    headers={"Authorization": f"Bearer {os.environ['BASETEN_API_KEY']}"},
    json={"prompt": "Hello, world!"},
    timeout=60,
)
response.raise_for_status()
print(response.json())

{'prompt': 'Hello, world!'}

To send JSON to your model:Run the following command, replacing the input with your model’s expected payload:

curl "https://model-${BASETEN_MODEL_ID}.api.baseten.co/environments/production/predict" \
  -H "Authorization: Bearer $BASETEN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"prompt": "Hello, world!"}'

{"prompt": "Hello, world!"}

Use /predict for a complete synchronous response. To queue work and receive its output later, follow Async inference.

Target an environment or deployment

The endpoint selects which version of your model receives a request:

Anatomy of the model API endpoint. In https://model-abc123.api.baseten.co/environments/production/predict, abc123 is the model ID and production is the environment that serves the request.

Production: /environments/production/predict.
Named environment: /environments/ENVIRONMENT_NAME/predict.
Development deployment: /development/predict.
Specific deployment: /deployment/DEPLOYMENT_ID/predict.

For more information, see Inference endpoint URL formats.

Sync API endpoints

For a custom HTTP server, /sync forwards the remaining path to a route exposed by that server. For example, /environments/production/sync/health forwards to /health, while /environments/production/sync/v1/chat/completions forwards to /v1/chat/completions. To call a custom server route: Use the route, HTTP method, and payload defined by your server. For a server with a GET /health route that returns HTTP 200 when ready, run the following command to print its status code:

curl -s -o /dev/null -w '%{http_code}\n' \
  "https://model-${BASETEN_MODEL_ID}.api.baseten.co/environments/production/sync/health" \
  -H "Authorization: Bearer $BASETEN_API_KEY"

The response body and success status come from your server’s implementation. For more information, see Custom servers.

Authentication

Use Authorization: Bearer <api_key> for dedicated deployment requests. Baseten also accepts the legacy Authorization: Api-Key <api_key> scheme, so existing scripts continue to work.

External LLM gateways

Configure the gateway with these values:

Base URL: https://model-YOUR_MODEL_ID.api.baseten.co/environments/production/sync/v1.
Model name: The name accepted by the deployed server. For vLLM, use --served-model-name from start_command.
API key: A Baseten API key with access to the deployment.

The gateway sends requests to {base_url}/chat/completions with model set to the served name. For more information, see Integrations.

Alternative invocation methods

Baseten CLI: Send a prediction request from your terminal.
Deployment playground: Test your model in your workspace.

Get started

Model APIs

Inference

Training

Baseten Grid

Frontier Gateway

Sandboxes

Development

Deployment

Engines

Model Labs

Organization

Hosting options

Observability

Troubleshooting

Call your model

Before you begin

Choose an interface

OpenAI SDK

Predict API endpoints

Target an environment or deployment

Sync API endpoints

Authentication

External LLM gateways

Alternative invocation methods

​Before you begin

​Choose an interface

​OpenAI SDK

​Predict API endpoints

​Target an environment or deployment

​Sync API endpoints

​Authentication

​External LLM gateways

​Alternative invocation methods

Before you begin

Choose an interface

OpenAI SDK

Predict API endpoints

Target an environment or deployment

Sync API endpoints

Authentication

External LLM gateways

Alternative invocation methods