Call a model through Frontier Gateway

The organization that configures Gateway resources and issues federated keys is the Gateway operator. If you have a federated key, use this guide to call the models that key grants access to through Baseten Frontier Gateway. Choose the credentials and URL for the call path:

Frontier Gateway calls use a federated API key with the gateway URL.
Shared Model API calls use a Baseten workspace API key with https://inference.baseten.co/v1.
Dedicated calls use a Baseten workspace API key with the model-specific deployment URL.

This guide covers the Frontier Gateway path. The gateway is OpenAI-compatible, so any OpenAI SDK or HTTP client works with two changes: the base URL and the auth header.

The gateway accepts the OpenAI Chat Completions API. Configure your client with the Gateway base URL and federated API key.

Base URL

The Gateway operator provides the base URL. Unless it gives you a custom domain, use:

https://inference.baseten.co/v1

Use a custom domain only when the Gateway operator provides one; the request shape is the same.

Authentication

Pass your federated API key in the Authorization header using the Api-Key scheme, not Bearer:

Authorization: Api-Key YOUR_API_KEY

If your client defaults to Authorization: Bearer ..., override it. Federated keys sent as Bearer tokens are rejected. The organization that issued your key manages its rotation and limits through Baseten’s federated key management. Treat the key like any other API secret: store it in an environment variable or secret manager, never in source control.

OpenAI SDK example

Make a chat completion request with the federated key the Gateway operator gave you. Replace YOUR_API_KEY with that key, and your-org/your-model with the model slug the Gateway operator gave you.

Python
JavaScript

Install the OpenAI SDK:

pip install openai

Make a chat completion request:

chat.py

from openai import OpenAI

client = OpenAI(
    base_url="https://inference.baseten.co/v1",
    api_key="YOUR_API_KEY",
    default_headers={"Authorization": "Api-Key YOUR_API_KEY"},
)

response = client.chat.completions.create(
    model="your-org/your-model",
    messages=[{"role": "user", "content": "Hello, world!"}],
)

print(response.choices[0].message.content)

Install the OpenAI SDK:

npm install openai

Make a chat completion request:

chat.js

import OpenAI from "openai";

const client = new OpenAI({
    baseURL: "https://inference.baseten.co/v1",
    apiKey: "YOUR_API_KEY",
    defaultHeaders: { Authorization: "Api-Key YOUR_API_KEY" },
});

const response = await client.chat.completions.create({
    model: "your-org/your-model",
    messages: [{ role: "user", content: "Hello, world!" }],
});

console.log(response.choices[0].message.content);

The response follows the standard OpenAI Chat Completions schema:

Output

{
  "id": "chatcmpl-...",
  "object": "chat.completion",
  "model": "your-org/your-model",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello! How can I help you today?"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 10,
    "completion_tokens": 9,
    "total_tokens": 19
  }
}

curl example

For raw HTTP usage:

curl --request POST \
  --url https://inference.baseten.co/v1/chat/completions \
  --header "Content-Type: application/json" \
  --header "Authorization: Api-Key YOUR_API_KEY" \
  --data '{
    "model": "your-org/your-model",
    "messages": [
      {"role": "user", "content": "Hello, world!"}
    ]
  }'

{
  "id": "chatcmpl-...",
  "object": "chat.completion",
  "model": "your-org/your-model",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello! How can I help you today?"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 10,
    "completion_tokens": 9,
    "total_tokens": 19
  }
}

Model slug format

Use the your-org/your-model format (for example, acme/llama-3-70b). Pass the slug as the model parameter on every request. The Gateway operator tells you which slug or slugs your key has access to; a single key can be authorized for one or more models.

Streaming, structured outputs, and tool calling

Frontier Gateway forwards OpenAI-compatible parameters supported by the target. Ask the Gateway operator which features each slug supports. For Baseten-hosted targets, use these guides:

For more information on streaming responses, see Streaming.
For more information on JSON-schema and structured generation, see Structured outputs.
For more information on tool calling and function definitions, see Function calling.

For third-party or custom targets, consult the relevant upstream provider or server documentation for supported features.

Rate limits

The Gateway operator sets rate and usage limits for your federated key. If a request exceeds a limit, the gateway returns 429 Too Many Requests. For more information on the limit shape, daily reset behavior, and 429 handling, see Rate and usage limits.

Manage access for callers

If you’re the Gateway operator rather than a developer consuming a key, the Frontier Gateway overview covers group and key management, rate limits, and billing webhooks.

Get started

Model APIs

Inference

Training

Development

Deployment

Engines

Model Labs

Organization

Hosting options

Observability

Troubleshooting

Call a model through Frontier Gateway

Base URL

Authentication

OpenAI SDK example

curl example

Model slug format

Streaming, structured outputs, and tool calling

Rate limits

Manage access for callers

​Base URL

​Authentication

​OpenAI SDK example

​curl example

​Model slug format

​Streaming, structured outputs, and tool calling

​Rate limits

​Manage access for callers

Base URL

Authentication

OpenAI SDK example

curl example

Model slug format

Streaming, structured outputs, and tool calling

Rate limits

Manage access for callers