Pricing and limits

Baseten enforces rate limits on Model APIs to ensure system stability:

Request rate limits: Maximum API requests per minute.
Token rate limits: Maximum tokens processed per minute (input + output combined). Cached input tokens count at full weight: a token served from the KV cache costs less but counts the same toward the token rate limit as an uncached one.

To monitor token and request consumption by API key, model, or service tier, see Usage. Default limits vary by account status.

Account	RPM	TPM
Basic (unverified)	15	100,000
Basic (verified)	120	500,000
Pro	120	1,000,000
Enterprise	Custom	Custom

To raise your limits, contact us to request email verification for higher Basic limits, or to move to Pro or Enterprise.

If you exceed these limits, the API returns a 429 Too Many Requests error. See Inference errors for how to respond.

Pricing

Model APIs bill per million tokens. For current per-model rates, see the Model APIs pricing page. Fast tier variants have their own per-model pricing and rate limits; see Supported models for details.

Cached input tokens

Cached input tokens are prompt tokens served from the KV cache, billed at a discounted rate. Every request participates in caching automatically, with no flags or opt-in steps. To raise cache hits across related requests, send the x-session-affinity header with a consistent value to pin them to the same region or replica. Scope one ID to a single conversation or full agentic task, including all subagents; reusing it across unrelated agents concentrates load on one replica and defeats the load-balancing benefit. A good ID is stable per task but uncorrelated across tasks. For example, use a SHA-256 hash of your internal conversation or session ID, truncated to 32 hex characters:

x-session.py

import hashlib

conversation_id = "conv-7f3a2b1c"  # your existing conversation or session identifier
affinity = hashlib.sha256(conversation_id.encode()).hexdigest()[:32]

Send the resulting value in the x-session-affinity header on every request in the conversation:

curl -X POST https://inference.baseten.co/v1/chat/completions \
  -H "Authorization: Bearer $BASETEN_API_KEY" \
  -H "Content-Type: application/json" \
  -H "x-session-affinity: 8f9ff769f61b13e167fb40ef2bae9f3b" \
  -d '{
    "model": "zai-org/GLM-5.2",
    "messages": [
      {"role": "system", "content": "You are a concise technical writer."},
      {"role": "user", "content": "What is gradient descent?"},
      {"role": "assistant", "content": "An optimization algorithm that iteratively adjusts model parameters by moving in the direction of steepest decrease in the loss function."},
      {"role": "user", "content": "How does the learning rate affect it?"}
    ]
  }'

{
  "id": "chatcmpl-4a7e03f0b97e453fa5b8eeeac78b3d13",
  "object": "chat.completion",
  "model": "zai-org/GLM-5.2",
  "choices": [
    {
      "index": 0,
      "message": {
        "content": "The learning rate determines the step size of each update. Too high, and the model overshoots the minimum or diverges; too low, and convergence becomes prohibitively slow or risks getting stuck in local minima.",
        "role": "assistant"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 56,
    "completion_tokens": 44,
    "total_tokens": 100,
    "prompt_tokens_details": {
      "cached_tokens": 32
    }
  }
}

Budgets

You can set a monthly budget to receive email notifications as your workspace’s total spend approaches a configured amount. If you choose to enforce a budget, Baseten rejects Model API requests once your workspace reaches the budget. Budget enforcement doesn’t affect dedicated inference or training.

Usage

Track Model APIs usage with GET /v1/model_apis/usage. The endpoint returns input, cached input, uncached input, and output token counts, plus request counts, in 1m, 1h, or 1d buckets. Group results by API key, model, and service tier, or filter the response to specific values.

curl --get "https://api.baseten.co/v1/model_apis/usage" \
  --header "Authorization: Bearer $BASETEN_API_KEY" \
  --data-urlencode "start_time=2026-08-06T12:00:00Z" \
  --data-urlencode "end_time=2026-08-06T13:00:00Z" \
  --data-urlencode "bucket_width=1h" \
  --data-urlencode "group_by=api_key" \
  --data-urlencode "group_by=model"

{
  "items": [
    {
      "start_time": "2026-08-06T12:00:00Z",
      "end_time": "2026-08-06T13:00:00Z",
      "results": [
        {
          "api_key_prefix": "<API_KEY_PREFIX>",
          "model": "zai-org/GLM-5.2",
          "service_tier": null,
          "input_tokens": 14820,
          "cached_input_tokens": 9200,
          "uncached_input_tokens": 5620,
          "output_tokens": 3200,
          "request_count": 24
        }
      ]
    }
  ],
  "pagination": {
    "has_more": false,
    "cursor": null
  }
}

Usage data is available from August 5, 2026 at 20:45 UTC (2026-08-05T20:45:00Z). Baseten did not backfill earlier Model APIs usage.

Next steps

Inference errors

Handle 429 Too Many Requests and other status codes

Model APIs overview

Supported models and feature support

Get started

Model APIs

Inference

Training

Development

Deployment

Engines

Model Labs

Organization

Hosting options

Observability

Troubleshooting

Pricing and limits

Pricing

Cached input tokens

Budgets

Usage

Next steps

Inference errors

Model APIs overview

​Pricing

​Cached input tokens

​Budgets

​Usage

​Next steps

Inference errors

Model APIs overview

Pricing

Cached input tokens

Budgets

Usage

Next steps