Pricing and limits

Model APIs bill by token and enforce request and token rate limits for each model. You can set a workspace budget, inspect token usage, and attribute costs by API key, user, model, or service tier. Choose the task you need:

Check your current rate limits.
Confirm account verification or request higher limits.
Understand token charges.
Check whether caching is working.
Monitor usage, inspect costs, or set a budget.

Pricing

Model APIs bill per million input and output tokens. Rates vary by model. See the Model APIs pricing page for current prices.

Cached input tokens

Baseten serves prompt tokens from the KV cache at a discounted rate when a request reuses a cached prefix. Caching is automatic and requires no request flags. Cached and uncached input tokens count equally toward token rate limits. Caching generated output for the next request depends on the model’s cache configuration and whether the prefix remains in the cache. Check usage.prompt_tokens_details.cached_tokens to measure how much of each prompt came from the cache.

Session affinity

Session affinity identifies requests that belong to the same conversation or agent task. Baseten routes requests with the same session ID to the same replica, where the shared prompt prefix is more likely to remain in the KV cache. Coding agents like Claude Code, Codex, and OpenCode provide session IDs that Baseten recognizes, so you don’t need to configure session affinity. For other clients, send x-session-affinity with every related request:

curl https://inference.baseten.co/v1/chat/completions \
  --header "Authorization: Bearer $BASETEN_API_KEY" \
  --header "Content-Type: application/json" \
  --header "x-session-affinity: 2f1c79e8-4901-4f42-b01d-b8372d2b1b6c" \
  --data '{
    "model": "zai-org/GLM-5.3",
    "messages": [{"role": "user", "content": "What is gradient descent?"}]
  }'

{
  "id": "chatcmpl-01",
  "object": "chat.completion",
  "model": "zai-org/GLM-5.3",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Gradient descent adjusts model parameters to reduce a loss function."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 8,
    "completion_tokens": 11,
    "total_tokens": 19,
    "prompt_tokens_details": {
      "cached_tokens": 0
    }
  }
}

To connect Claude Code, Codex CLI, or another coding agent to Baseten, see Set up coding agents. Keep the value stable for one conversation or agent task, including its subagents. Use a different value for unrelated work so Baseten can distribute requests across replicas. Use an opaque value, such as a UUID, or hash an internal conversation ID before sending it. Baseten recommends at most about 60 requests per minute for one session. Session affinity controls routing, not conversation state. Continue to include the conversation messages in each request.

Check cache reuse

To measure cache reuse, send requests with the same long prompt prefix and session affinity ID, then inspect usage.prompt_tokens_details.cached_tokens. The count is part of usage.prompt_tokens; don’t add it to the input total. Set BASETEN_API_KEY to your API key and install uv to run the example. It sends two billable requests with a shared policy document. Save it as check_cache.py:

check_cache.py

import os
import uuid
from openai import OpenAI

client = OpenAI(
    base_url="https://inference.baseten.co/v1",
    api_key=os.environ["BASETEN_API_KEY"],
)
session_id = str(uuid.uuid4())
policy = "\n".join(
    f"Policy {number}: Customers can return unused products within 30 days."
    for number in range(200)
)

for attempt in range(1, 3):
    response = client.chat.completions.create(
        model="zai-org/GLM-5.3",
        extra_headers={"x-session-affinity": session_id},
        messages=[
            {"role": "system", "content": policy},
            {"role": "user", "content": "What is the return window?"},
        ],
        max_tokens=64,
    )
    usage = response.usage
    cached = usage.prompt_tokens_details.cached_tokens
    print(f"Request {attempt}: input={usage.prompt_tokens}, cached={cached}")

Run it with the OpenAI SDK:

uv run --with openai check_cache.py

Example output:

Request 1: input=2824, cached=0
Request 2: input=2824, cached=2816

In the second request, 2,816 of the 2,824 input tokens came from the cache. The remaining 8 input tokens use the uncached rate. Exact counts vary by model, prompt, and cache state. A positive cached count confirms reuse for that request. A zero count means that request reported no cached input. When later requests still report zero:

Keep the shared prefix identical, including system instructions and tool definitions.
Reuse the same session affinity ID for related requests.
Check that you use the same model and that the prompt has a substantial shared prefix. A short greeting isn’t a useful cache test.

Session affinity improves the chance of reuse but doesn’t reserve a replica or guarantee a cache hit. The prefix must still be present in the model’s cache.

Rate limits

Baseten enforces the following limits for each Model API:

Limit	Counts
Requests per minute (RPM)	Each request.
Tokens per minute (TPM)	Input and output tokens. Cached and uncached input tokens count equally.

Check your current limits

To retrieve the limits for a model in your workspace, set BASETEN_API_KEY to your API key, then call Get a Model API by name:

curl "https://api.baseten.co/v1/model_apis/zai-org%2FGLM-5.3" \
  -H "Authorization: Bearer $BASETEN_API_KEY"

{
  "name": "zai-org/GLM-5.3",
  "rate_limits": [
    {"type": "REQUEST", "unit": "MINUTE", "threshold": 120},
    {"type": "TOKEN", "unit": "MINUTE", "threshold": 500000}
  ]
}

Replace zai-org%2FGLM-5.3 with your model’s URL-encoded slug (/ becomes %2F). Read threshold for the REQUEST and TOKEN entries with unit: "MINUTE" to find RPM and TPM. Your returned limits can differ from this example and the account defaults below. You can also inspect the rate limit response headers on inference responses to see your limits and the remaining allowance.

Default limits by account tier

Account tier	RPM	TPM
Basic (unverified)	15	100,000
Basic (verified)	120	500,000
Pro	120	1,000,000
Enterprise	Custom	Custom

These are account defaults. A model, service tier, or organization-specific configuration can set different limits. Check your current limits for the model you call.

Account verification and higher limits

To confirm whether your Basic account is verified and which verification steps apply, contact support. Include your workspace name, sign-in email, the model slug, and the limits returned by the model details request. Keep the full API key out of your message. To request more capacity for your workload, submit a rate limit increase request. Include the model, expected requests and tokens per minute, and whether traffic arrives steadily or in bursts. Confirm the effective limits again after your account or model configuration changes. Changing a request’s max_tokens controls its output length; it doesn’t raise the workspace’s TPM limit. Fast variants have separate slugs and rate limits, so check the limits for the exact variant you call.

Rate limit response headers

Header	Value
`x-ratelimit-limit-requests`	Maximum requests per minute.
`x-ratelimit-remaining-requests`	Requests currently available.
`x-ratelimit-limit-tokens`	Maximum tokens per minute.
`x-ratelimit-remaining-tokens`	Tokens currently available.

Rate limits replenish continuously instead of resetting at fixed intervals. Send requests at a steady rate, and use exponential backoff after a 429 Too Many Requests response. A 529 Overloaded response means that the Model API has no available serving capacity. This response can occur even when your organization remains within its rate limits. See 529: overloaded for retry guidance.

Budgets

Set a monthly budget to receive email notifications as workspace spend approaches the configured amount. If you enforce the budget, Baseten rejects Model API requests after the workspace reaches it. Budget enforcement does not affect Dedicated Inference or Training.

Usage

For Chat Completions, read request token totals from usage.prompt_tokens and usage.completion_tokens. The accepted_prediction_tokens and rejected_prediction_tokens fields in completion_tokens_details report speculative-decoding statistics. They aren’t separate billing counters, so don’t add them to completion usage or use them to calculate charges. Rejected predictions can exceed the completion token count. Choose a usage interface based on how you plan to use the data:

Interface	Use it for
Baseten CLI	Inspect usage or export data from a terminal.
REST API	Build scheduled reports or usage integrations.
Prometheus	Monitor usage and create alerts.

The CLI and REST API return input, cached input, uncached input, and output token counts alongside request counts. Select 1m, 1h, or 1d buckets, then group or filter results by API key, user, or model. Baseten retains Model API usage data for 92 days. Queries for older buckets return no results.

Baseten CLI
REST API

Show one day of usage by model:

baseten model-api usage \
  --start 2026-09-01T00:00:00Z \
  --end 2026-09-02T00:00:00Z

DATE        MODEL              REQUESTS   INPUT   CACHED   OUTPUT
2026-09-01  zai-org/GLM-5.3          24  14,820     9,200    3,200
ALL                                  24  14,820     9,200    3,200

Use baseten model-api usage to change the time range, bucket width, grouping, filters, or output format.

Retrieve one day of hourly usage grouped by model:

curl --get "https://api.baseten.co/v1/model_apis/usage" \
  --header "Authorization: Bearer $BASETEN_API_KEY" \
  --data-urlencode "start_time=2026-09-01T00:00:00Z" \
  --data-urlencode "end_time=2026-09-02T00:00:00Z" \
  --data-urlencode "bucket_width=1h" \
  --data-urlencode "group_by=model"

{
  "items": [
    {
      "start_time": "2026-09-01T00:00:00Z",
      "end_time": "2026-09-01T01:00:00Z",
      "results": [
        {
          "model": "zai-org/GLM-5.3",
          "input_tokens": 14820,
          "cached_input_tokens": 9200,
          "uncached_input_tokens": 5620,
          "output_tokens": 3200,
          "request_count": 24
        }
      ]
    }
  ],
  "pagination": {
    "has_more": false,
    "cursor": null
  }
}

See GET /v1/model_apis/usage for filters, pagination, and response fields.

When you group usage by user, Baseten attributes personal API keys and OAuth credentials to their owner. Credentials without user scope return a null user ID. For continuous monitoring, export Model API metrics to Prometheus:

baseten_model_api_tokens_total separates uncached input, cached input, and output tokens.
baseten_model_api_inference_requests_total counts requests by status code.

See Model API metrics for metric and label definitions.

Costs

baseten org billing usage reports total Model API cost as part of the workspace billing summary. The GET /v1/billing/model_apis endpoint returns daily Model API costs and can group them by API key, user, model, or service tier. Both calculate costs from recorded token usage and the model price that applied when each request ran.

Baseten CLI
REST API

Show the total Model API cost for the last seven days:

baseten org billing usage --jq '.model_apis_usage.total'

"171.15"

The CLI returns the Model API total from the workspace billing summary. See baseten org billing usage for date-range and output options.

Group daily Model API costs by model and service tier:

curl --get "https://api.baseten.co/v1/billing/model_apis" \
  --header "Authorization: Bearer $BASETEN_API_KEY" \
  --data-urlencode "start_date=2026-09-01" \
  --data-urlencode "end_date=2026-09-08" \
  --data-urlencode "group_by=model" \
  --data-urlencode "group_by=service_tier"

{
  "items": [
    {
      "date": "2026-09-01",
      "results": [
        {
          "model": "zai-org/GLM-5.3",
          "service_tier": "default",
          "subtotal": "3.842710"
        }
      ]
    }
  ],
  "pagination": {
    "has_more": false,
    "cursor": null
  }
}

Cost breakdowns

Omit group_by to return one workspace subtotal per day. Repeat it to break down costs by API key, user, model, or service tier. Filter results with api_key_prefixes, user_ids, models, and service_tiers. Pass API key prefixes, not full API keys. Each result represents one observed combination of the dimensions you request. Baseten omits combinations without usage, so the number of results can differ between days. When you group by user, Baseten attributes personal API keys and OAuth credentials to their owner. Credentials without user scope return a null user ID.

Cost response fields

The API returns one bucket for each UTC day, from midnight inclusive to the next midnight exclusive. It orders buckets from oldest to newest without gaps. A day without matching usage has an empty results array. When you group by API key, api_key_prefixes contains the attributed prefix in a one-item array. Fields for dimensions you don’t request return null. Requested API key, user, and service-tier fields can also return null when attribution is unavailable. subtotal is an exact decimal string in USD, such as "0.00000123". Parse it with a decimal or fixed-point type to preserve fractional-cent amounts.

Cost history and pagination

Cost data begins on August 5, 2026, at 20:45 UTC. The August 5 bucket includes only usage recorded after that time. The API rejects an earlier start_date.

The endpoint includes start_date and excludes end_date. start_date defaults to the previous UTC date, and end_date defaults to the day after the current UTC date. If you omit both parameters, the response includes yesterday and the current day. Baseten can add costs to the current-day bucket as it records usage. A date range can cover up to 90 days. limit controls the number of daily buckets in a page. It defaults to 7 and has a maximum of 31. When pagination.has_more is true, pass pagination.cursor in the next request. The cursor retains the original date range, grouping, filters, and limit. When you pass a cursor, the endpoint ignores other query parameters. Cost subtotals exclude credits, adjustments, taxes, and other invoice terms. Use the billing usage summary endpoint to review combined Dedicated Inference, Training, and Model API costs. Your invoice is the final record of billed charges. See the Model API costs reference for filters, pagination, attribution rules, and response fields.

Get started

Model APIs

Inference

Frontier Gateway

Training

Sandboxes

Development

Deployment

Engines

Model Labs

Organization

Hosting options

Observability

Troubleshooting

Pricing and limits

Pricing

Cached input tokens

Session affinity

Check cache reuse

Rate limits

Check your current limits

Default limits by account tier

Account verification and higher limits

Rate limit response headers

Budgets

Usage

Costs

Cost breakdowns

Cost response fields

​Pricing

​Cached input tokens

​Session affinity

​Check cache reuse

​Rate limits

​Check your current limits

​Default limits by account tier

​Account verification and higher limits

​Rate limit response headers

​Budgets

​Usage

​Costs

​Cost breakdowns

​Cost response fields

​Cost history and pagination

Pricing

Cached input tokens

Session affinity

Check cache reuse

Rate limits

Check your current limits

Default limits by account tier

Account verification and higher limits

Rate limit response headers

Budgets

Usage

Costs

Cost breakdowns

Cost response fields

Cost history and pagination