> ## Documentation Index > Fetch the complete documentation index at: https://docs.baseten.co/llms.txt > Use this file to discover all available pages before exploring further. # Pricing and limits > Understand Model API pricing and rate limits, then track usage and costs Model APIs bill by token and enforce request and token rate limits for each model. You can set a workspace budget, inspect token usage, and attribute costs by API key, user, model, or service tier. ## Pricing Model APIs bill per million input and output tokens. Rates vary by model. See the [Model APIs pricing page](https://www.baseten.co/pricing) for current prices. ### Cached input tokens Baseten serves prompt tokens from the KV cache at a discounted rate when a request reuses a cached prefix. Caching is automatic and requires no request flags. Cached and uncached input tokens count equally toward token rate limits. ### Session affinity Session affinity identifies requests that belong to the same conversation or agent task. Baseten routes requests with the same session ID to the same replica, where the shared prompt prefix is more likely to remain in the KV cache. Coding agents like Claude Code, Codex, and OpenCode provide session IDs that Baseten recognizes, so you don't need to configure session affinity. For other clients, send `x-session-affinity` with every related request: ```bash Request theme={"system"} curl https://inference.baseten.co/v1/chat/completions \ --header "Authorization: Bearer $BASETEN_API_KEY" \ --header "Content-Type: application/json" \ --header "x-session-affinity: 2f1c79e8-4901-4f42-b01d-b8372d2b1b6c" \ --data '{ "model": "zai-org/GLM-5.2", "messages": [{"role": "user", "content": "What is gradient descent?"}] }' ``` ```json Response theme={"system"} { "id": "chatcmpl-01", "object": "chat.completion", "model": "zai-org/GLM-5.2", "choices": [ { "index": 0, "message": { "role": "assistant", "content": "Gradient descent adjusts model parameters to reduce a loss function." }, "finish_reason": "stop" } ], "usage": { "prompt_tokens": 8, "completion_tokens": 11, "total_tokens": 19, "prompt_tokens_details": { "cached_tokens": 0 } } } ``` To connect Claude Code, Codex CLI, or another coding agent to Baseten, see [Set up coding agents](/inference/model-apis/coding-agents). Keep the value stable for one conversation or agent task, including its subagents. Use a different value for unrelated work so Baseten can distribute requests across replicas. Use an opaque value, such as a UUID, or hash an internal conversation ID before sending it. Session affinity controls routing, not conversation state. Continue to include the conversation messages in each request. ## Rate limits Baseten enforces the following limits for each Model API: | Limit | Counts | | ------------------------- | ------------------------------------------------------------------------ | | Requests per minute (RPM) | Each request. | | Tokens per minute (TPM) | Input and output tokens. Cached and uncached input tokens count equally. | ### Default limits by account tier | Account tier | RPM | TPM | | ------------------ | -----: | --------: | | Basic (unverified) | 15 | 100,000 | | Basic (verified) | 120 | 500,000 | | Pro | 120 | 1,000,000 | | Enterprise | Custom | Custom | These are account defaults. A model, service tier, or organization-specific configuration can set different limits. Check the following headers on successful responses and `429 Too Many Requests` errors, or call [`GET /v1/model_apis/{model_api_name}`](/reference/management-api/model-apis/gets-a-model-api-by-name) to retrieve your effective limits for a model. To raise your limits, [request a rate limit increase](https://www.baseten.co/talk-to-us/increase-rate-limits/). | Header | Value | | -------------------------------- | ----------------------------- | | `x-ratelimit-limit-requests` | Maximum requests per minute. | | `x-ratelimit-remaining-requests` | Requests currently available. | | `x-ratelimit-limit-tokens` | Maximum tokens per minute. | | `x-ratelimit-remaining-tokens` | Tokens currently available. | Rate limits replenish continuously instead of resetting at fixed intervals. Send requests at a steady rate, and use exponential backoff after a [`429 Too Many Requests`](/inference/errors#429-too-many-requests) response. A `529 Overloaded` response means that the Model API has no available serving capacity. This response can occur even when your organization remains within its rate limits. See [`529: overloaded`](/inference/errors#529-overloaded) for retry guidance. ## Budgets Set a [monthly budget](/organization/billing#monthly-budget) to receive email notifications as workspace spend approaches the configured amount. If you enforce the budget, Baseten rejects Model API requests after the workspace reaches it. Budget enforcement does not affect Dedicated Inference or Training. ## Usage Choose a usage interface based on how you plan to use the data: | Interface | Use it for | | ----------- | ---------------------------------------------- | | Baseten CLI | Inspect usage or export data from a terminal. | | REST API | Build scheduled reports or usage integrations. | | Prometheus | Monitor usage and create alerts. | The CLI and REST API return input, cached input, uncached input, and output token counts alongside request counts. Select `1m`, `1h`, or `1d` buckets, then group or filter results by API key, user, or model. Baseten retains Model API usage data for 92 days. Queries for older buckets return no results. Show one day of usage by model: ```bash Command theme={"system"} baseten model-api usage \ --start 2026-09-01T00:00:00Z \ --end 2026-09-02T00:00:00Z ``` ```txt Output theme={"system"} DATE MODEL REQUESTS INPUT CACHED OUTPUT 2026-09-01 zai-org/GLM-5.2 24 14,820 9,200 3,200 ALL 24 14,820 9,200 3,200 ``` Use [`baseten model-api usage`](/reference/cli/baseten/model-api#usage) to change the time range, bucket width, grouping, filters, or output format. Retrieve one day of hourly usage grouped by model: ```bash Request theme={"system"} curl --get "https://api.baseten.co/v1/model_apis/usage" \ --header "Authorization: Bearer $BASETEN_API_KEY" \ --data-urlencode "start_time=2026-09-01T00:00:00Z" \ --data-urlencode "end_time=2026-09-02T00:00:00Z" \ --data-urlencode "bucket_width=1h" \ --data-urlencode "group_by=model" ``` ```json Response theme={"system"} { "items": [ { "start_time": "2026-09-01T00:00:00Z", "end_time": "2026-09-01T01:00:00Z", "results": [ { "model": "zai-org/GLM-5.2", "input_tokens": 14820, "cached_input_tokens": 9200, "uncached_input_tokens": 5620, "output_tokens": 3200, "request_count": 24 } ] } ], "pagination": { "has_more": false, "cursor": null } } ``` See [`GET /v1/model_apis/usage`](/reference/management-api/model-apis/gets-model-apis-token-usage) for filters, pagination, and response fields. When you group usage by user, Baseten attributes personal API keys and OAuth credentials to their owner. Credentials without user scope return a `null` user ID. For continuous monitoring, export Model API metrics to Prometheus: * `baseten_model_api_tokens_total` separates uncached input, cached input, and output tokens. * `baseten_model_api_inference_requests_total` counts requests by status code. See [Model API metrics](/observability/export-metrics/supported-metrics#model-api-metrics) for metric and label definitions. ## Costs [`baseten org billing usage`](/reference/cli/baseten/org-billing) reports total Model API cost as part of the workspace billing summary. The [`GET /v1/billing/model_apis`](/reference/management-api/billing/gets-model-apis-costs) endpoint returns daily Model API costs and can group them by API key, user, model, or service tier. Both calculate costs from recorded token usage and the model price that applied when each request ran. Show the total Model API cost for the last seven days: ```bash Command theme={"system"} baseten org billing usage --jq '.model_apis_usage.total' ``` ```json Output theme={"system"} "171.15" ``` The CLI returns the Model API total from the workspace billing summary. See [`baseten org billing usage`](/reference/cli/baseten/org-billing) for date-range and output options. Group daily Model API costs by model and service tier: ```bash Request theme={"system"} curl --get "https://api.baseten.co/v1/billing/model_apis" \ --header "Authorization: Bearer $BASETEN_API_KEY" \ --data-urlencode "start_date=2026-09-01" \ --data-urlencode "end_date=2026-09-08" \ --data-urlencode "group_by=model" \ --data-urlencode "group_by=service_tier" ``` ```json Response theme={"system"} { "items": [ { "date": "2026-09-01", "results": [ { "model": "zai-org/GLM-5.2", "service_tier": "default", "subtotal": "3.842710" } ] } ], "pagination": { "has_more": false, "cursor": null } } ``` ### Cost breakdowns Omit `group_by` to return one workspace subtotal per day. Repeat it to break down costs by API key, user, model, or service tier. Filter results with `api_key_prefixes`, `user_ids`, `models`, and `service_tiers`. Pass API key prefixes, not full API keys. Each result represents one observed combination of the dimensions you request. Baseten omits combinations without usage, so the number of results can differ between days. When you group by user, Baseten attributes personal API keys and OAuth credentials to their owner. Credentials without user scope return a `null` user ID. ### Cost response fields The API returns one bucket for each UTC day, from midnight inclusive to the next midnight exclusive. It orders buckets from oldest to newest without gaps. A day without matching usage has an empty `results` array. When you group by API key, `api_key_prefixes` contains the attributed prefix in a one-item array. Fields for dimensions you don't request return `null`. Requested API key, user, and service-tier fields can also return `null` when attribution is unavailable. `subtotal` is an exact decimal string in USD, such as `"0.00000123"`. Parse it with a decimal or fixed-point type to preserve fractional-cent amounts. ### Cost history and pagination Cost data begins on August 5, 2026, at 20:45 UTC. The August 5 bucket includes only usage recorded after that time. The API rejects an earlier `start_date`. The endpoint includes `start_date` and excludes `end_date`. `start_date` defaults to the previous UTC date, and `end_date` defaults to the day after the current UTC date. If you omit both parameters, the response includes yesterday and the current day. Baseten can add costs to the current-day bucket as it records usage. A date range can cover up to 90 days. `limit` controls the number of daily buckets in a page. It defaults to 7 and has a maximum of 31. When `pagination.has_more` is `true`, pass `pagination.cursor` in the next request. The cursor retains the original date range, grouping, filters, and limit. When you pass a cursor, the endpoint ignores other query parameters. Cost subtotals exclude credits, adjustments, taxes, and other invoice terms. Use the [billing usage summary endpoint](/reference/management-api/billing/gets-billing-usage-summary-for-a-date-range) to review combined Dedicated Inference, Training, and Model API costs. Your invoice is the final record of billed charges. See the [Model API costs reference](/reference/management-api/billing/gets-model-apis-costs) for filters, pagination, attribution rules, and response fields.