Pricing and limits

Model APIs bill by token and enforce request and token rate limits for each model. You can set a workspace budget, inspect token usage, and attribute costs by API key, user, model, or service tier.

Pricing

Model APIs bill per million input and output tokens. Rates vary by model. See the Model APIs pricing page for current prices.

Cached input tokens

Baseten serves prompt tokens from the KV cache at a discounted rate when a request reuses a cached prefix. Caching is automatic and requires no request flags. Cached and uncached input tokens count equally toward token rate limits.

Session affinity

Session affinity identifies requests that belong to the same conversation or agent task. Baseten routes requests with the same session ID to the same replica, where the shared prompt prefix is more likely to remain in the KV cache. Coding agents like Claude Code, Codex, and OpenCode provide session IDs that Baseten recognizes, so you don’t need to configure session affinity. For other clients, send x-session-affinity with every related request:

curl https://inference.baseten.co/v1/chat/completions \
  --header "Authorization: Bearer $BASETEN_API_KEY" \
  --header "Content-Type: application/json" \
  --header "x-session-affinity: 2f1c79e8-4901-4f42-b01d-b8372d2b1b6c" \
  --data '{
    "model": "zai-org/GLM-5.2",
    "messages": [{"role": "user", "content": "What is gradient descent?"}]
  }'

{
  "id": "chatcmpl-01",
  "object": "chat.completion",
  "model": "zai-org/GLM-5.2",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Gradient descent adjusts model parameters to reduce a loss function."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 8,
    "completion_tokens": 11,
    "total_tokens": 19,
    "prompt_tokens_details": {
      "cached_tokens": 0
    }
  }
}

To connect Claude Code, Codex CLI, or another coding agent to Baseten, see Set up coding agents. Keep the value stable for one conversation or agent task, including its subagents. Use a different value for unrelated work so Baseten can distribute requests across replicas. Use an opaque value, such as a UUID, or hash an internal conversation ID before sending it. Session affinity controls routing, not conversation state. Continue to include the conversation messages in each request.

Rate limits

Baseten enforces the following limits for each Model API:

Limit	Counts
Requests per minute (RPM)	Each request.
Tokens per minute (TPM)	Input and output tokens. Cached and uncached input tokens count equally.

Default limits by account tier

Account tier	RPM	TPM
Basic (unverified)	15	100,000
Basic (verified)	120	500,000
Pro	120	1,000,000
Enterprise	Custom	Custom

These are account defaults. A model, service tier, or organization-specific configuration can set different limits. Check the following headers on successful responses and 429 Too Many Requests errors, or call GET /v1/model_apis/{model_api_name} to retrieve your effective limits for a model. To raise your limits, request a rate limit increase.

Header	Value
`x-ratelimit-limit-requests`	Maximum requests per minute.
`x-ratelimit-remaining-requests`	Requests currently available.
`x-ratelimit-limit-tokens`	Maximum tokens per minute.
`x-ratelimit-remaining-tokens`	Tokens currently available.

Rate limits replenish continuously instead of resetting at fixed intervals. Send requests at a steady rate, and use exponential backoff after a 429 Too Many Requests response. A 529 Overloaded response means that the Model API has no available serving capacity. This response can occur even when your organization remains within its rate limits. See 529: overloaded for retry guidance.

Budgets

Set a monthly budget to receive email notifications as workspace spend approaches the configured amount. If you enforce the budget, Baseten rejects Model API requests after the workspace reaches it. Budget enforcement does not affect Dedicated Inference or Training.

Usage

Choose a usage interface based on how you plan to use the data:

Interface	Use it for
Baseten CLI	Inspect usage or export data from a terminal.
REST API	Build scheduled reports or usage integrations.
Prometheus	Monitor usage and create alerts.

The CLI and REST API return input, cached input, uncached input, and output token counts alongside request counts. Select 1m, 1h, or 1d buckets, then group or filter results by API key, user, or model. Baseten retains Model API usage data for 92 days. Queries for older buckets return no results.

Baseten CLI
REST API

Show one day of usage by model:

baseten model-api usage \
  --start 2026-09-01T00:00:00Z \
  --end 2026-09-02T00:00:00Z

DATE        MODEL              REQUESTS   INPUT   CACHED   OUTPUT
2026-09-01  zai-org/GLM-5.2          24  14,820     9,200    3,200
ALL                                  24  14,820     9,200    3,200

Use baseten model-api usage to change the time range, bucket width, grouping, filters, or output format.

Retrieve one day of hourly usage grouped by model:

curl --get "https://api.baseten.co/v1/model_apis/usage" \
  --header "Authorization: Bearer $BASETEN_API_KEY" \
  --data-urlencode "start_time=2026-09-01T00:00:00Z" \
  --data-urlencode "end_time=2026-09-02T00:00:00Z" \
  --data-urlencode "bucket_width=1h" \
  --data-urlencode "group_by=model"

{
  "items": [
    {
      "start_time": "2026-09-01T00:00:00Z",
      "end_time": "2026-09-01T01:00:00Z",
      "results": [
        {
          "model": "zai-org/GLM-5.2",
          "input_tokens": 14820,
          "cached_input_tokens": 9200,
          "uncached_input_tokens": 5620,
          "output_tokens": 3200,
          "request_count": 24
        }
      ]
    }
  ],
  "pagination": {
    "has_more": false,
    "cursor": null
  }
}

See GET /v1/model_apis/usage for filters, pagination, and response fields.

When you group usage by user, Baseten attributes personal API keys and OAuth credentials to their owner. Credentials without user scope return a null user ID. For continuous monitoring, export Model API metrics to Prometheus:

baseten_model_api_tokens_total separates uncached input, cached input, and output tokens.
baseten_model_api_inference_requests_total counts requests by status code.

See Model API metrics for metric and label definitions.

Costs

baseten org billing usage reports total Model API cost as part of the workspace billing summary. The GET /v1/billing/model_apis endpoint returns daily Model API costs and can group them by API key, user, model, or service tier. Both calculate costs from recorded token usage and the model price that applied when each request ran.

Baseten CLI
REST API

Show the total Model API cost for the last seven days:

baseten org billing usage --jq '.model_apis_usage.total'

"171.15"

The CLI returns the Model API total from the workspace billing summary. See baseten org billing usage for date-range and output options.

Group daily Model API costs by model and service tier:

curl --get "https://api.baseten.co/v1/billing/model_apis" \
  --header "Authorization: Bearer $BASETEN_API_KEY" \
  --data-urlencode "start_date=2026-09-01" \
  --data-urlencode "end_date=2026-09-08" \
  --data-urlencode "group_by=model" \
  --data-urlencode "group_by=service_tier"

{
  "items": [
    {
      "date": "2026-09-01",
      "results": [
        {
          "model": "zai-org/GLM-5.2",
          "service_tier": "default",
          "subtotal": "3.842710"
        }
      ]
    }
  ],
  "pagination": {
    "has_more": false,
    "cursor": null
  }
}

Cost breakdowns

Omit group_by to return one workspace subtotal per day. Repeat it to break down costs by API key, user, model, or service tier. Filter results with api_key_prefixes, user_ids, models, and service_tiers. Pass API key prefixes, not full API keys. Each result represents one observed combination of the dimensions you request. Baseten omits combinations without usage, so the number of results can differ between days. When you group by user, Baseten attributes personal API keys and OAuth credentials to their owner. Credentials without user scope return a null user ID.

Cost response fields

The API returns one bucket for each UTC day, from midnight inclusive to the next midnight exclusive. It orders buckets from oldest to newest without gaps. A day without matching usage has an empty results array. When you group by API key, api_key_prefixes contains the attributed prefix in a one-item array. Fields for dimensions you don’t request return null. Requested API key, user, and service-tier fields can also return null when attribution is unavailable. subtotal is an exact decimal string in USD, such as "0.00000123". Parse it with a decimal or fixed-point type to preserve fractional-cent amounts.

Cost history and pagination

Cost data begins on August 5, 2026, at 20:45 UTC. The August 5 bucket includes only usage recorded after that time. The API rejects an earlier start_date.

The endpoint includes start_date and excludes end_date. start_date defaults to the previous UTC date, and end_date defaults to the day after the current UTC date. If you omit both parameters, the response includes yesterday and the current day. Baseten can add costs to the current-day bucket as it records usage. A date range can cover up to 90 days. limit controls the number of daily buckets in a page. It defaults to 7 and has a maximum of 31. When pagination.has_more is true, pass pagination.cursor in the next request. The cursor retains the original date range, grouping, filters, and limit. When you pass a cursor, the endpoint ignores other query parameters. Cost subtotals exclude credits, adjustments, taxes, and other invoice terms. Use the billing usage summary endpoint to review combined Dedicated Inference, Training, and Model API costs. Your invoice is the final record of billed charges. See the Model API costs reference for filters, pagination, attribution rules, and response fields.

Get started

Model APIs

Inference

Training

Development

Deployment

Engines

Model Labs

Organization

Hosting options

Observability

Troubleshooting

Pricing and limits

Pricing

Cached input tokens

Session affinity

Rate limits

Default limits by account tier

Budgets

Usage

Costs

Cost breakdowns

Cost response fields

​Pricing

​Cached input tokens

​Session affinity

​Rate limits

​Default limits by account tier

​Budgets

​Usage

​Costs

​Cost breakdowns

​Cost response fields

​Cost history and pagination

Pricing

Cached input tokens

Session affinity

Rate limits

Default limits by account tier

Budgets

Usage

Costs

Cost breakdowns

Cost response fields

Cost history and pagination