> ## Documentation Index
> Fetch the complete documentation index at: https://docs.baseten.co/llms.txt
> Use this file to discover all available pages before exploring further.

# Server-side tool execution

> Tool selectors, request configuration, response metadata, and limits for server-side Model API tool execution

<Warning>
  **Early access.** Server-side tool execution is available as a rate limited preview for playground usage (25 RPM across all models).
  For scaled evaluations or high-volume production integration, contact [Baseten support](mailto:support@baseten.co) or use the support chat in the [Baseten app](https://app.baseten.co).
</Warning>

Use this reference to select tools that Baseten executes server-side, configure the execution loop, and parse protocol-specific responses. Server-side tool execution works with [`Messages`](/reference/inference-api/messages), [`Responses`](https://developers.openai.com/api/reference/responses/overview), and [`ChatCompletions`](/reference/inference-api/chat-completions). See [Web search](/inference/model-apis/web-search) for runnable requests and outputs.

## Request fields

### `x-baseten-server-tools`

Include this header on every request that uses server-side tool execution:

```http theme={"system"}
x-baseten-server-tools: true
```

The header routes the request through the server-side tool execution loop. Without it, the model endpoint rejects the `baseten__*` selector as an unknown tool type.

### `tools`

Select a tool for Baseten to execute server-side with an object that contains its `type`:

```json theme={"system"}
{"type": "baseten__<provider>__<tool>"}
```

Selectors use `baseten__<provider>__<tool>` in the request's `tools` array. Extra fields on a selector return `400`.

## Tool catalog

Search tools discover pages. Fetch tools retrieve content from known URLs. Give the model both when it should discover sources and then read selected pages.

| Provider | Capability      | Selector                                | Timeout    | Price                    |
| -------- | --------------- | --------------------------------------- | ---------- | ------------------------ |
| Exa      | Search          | `baseten__exa__web_search_exa`          | 15 seconds | \~\$0.007 per search     |
| Exa      | Advanced search | `baseten__exa__web_search_advanced_exa` | 30 seconds | \~\$0.007 per search     |
| Exa      | Fetch           | `baseten__exa__web_fetch_exa`           | 15 seconds | \~\$0.001 per URL        |
| Keenable | Search          | `baseten__keenable__search_web_pages`   | 15 seconds | \$0.004 per search       |
| Keenable | Fetch           | `baseten__keenable__fetch_page_content` | 15 seconds | \$0.001 per fetch        |
| Parallel | Search          | `baseten__parallel__web_search`         | 30 seconds | \~\$0.001 per search     |
| Parallel | Fetch           | `baseten__parallel__web_fetch`          | 30 seconds | \~\$0.001 per extraction |
| You.com  | Search          | `baseten__youcom__you-search`           | 30 seconds | \$0.005 per search       |
| You.com  | Fetch           | `baseten__youcom__you-contents`         | 30 seconds | \$0.001 per page         |

Baseten passes these provider costs straight through with no markup. Exa and Parallel report their charge for each call at runtime, so calls with this provider can deviate from the above pricing.

The advanced Exa search schema adds domain filters, subpage crawling, and a summary option to the basic search.

<Note>
  Server-side tools run on third-party search providers. By adding their tools, you agree to their respective terms:
  [Exa](https://exa.ai/terms-of-service), [Keenable](https://keenable.ai/terms), [Parallel](https://parallel.ai/customer-terms), [You.com](https://you.com/terms).
</Note>

The provider's tool schema defines the arguments, and the model supplies their values.

## Request configuration

Set request-level loop limits under `baseten.tool_settings`:

```json theme={"system"}
{
  "baseten": {
    "tool_settings": {
      "max_react_iterations": 6,
      "max_tool_calls_per_iteration": 4
    }
  }
}
```

| Field                          | Default | Accepted values  | Behavior                                                                                                                                                                                                                                                 |
| ------------------------------ | ------- | ---------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `max_react_iterations`         | `12`    | `2` through `20` | Limits model iterations. Baseten reserves the final iteration for an answer, so `N` iterations allow up to `N - 1` tool-call rounds. Baseten accepts `1` only when the request cannot call a server tool, for example when `tool_choice` disables tools. |
| `max_tool_calls_per_iteration` | `10`    | `1` through `10` | Limits server-side tool calls in one model iteration. Calls over the limit become error results that the model can handle.                                                                                                                               |

On `Responses`, the native `max_tool_calls` field sets a whole-request budget of server-side tool calls. When the loop spends the budget, Baseten steers the model to answer from the results it already has. `Messages` and `ChatCompletions` have no equivalent field.

<Accordion title="tool_choice">
  Most requests can omit `tool_choice` and let the model decide whether to use a tool. Set it when the first model iteration must call a tool, call a particular tool, or avoid tools.

  `tool_choice` applies only to the first model iteration. Later iterations use automatic selection so the model can stop calling tools and answer.

  <Warning>
    Changing `tool_choice` between requests invalidates the KV cache. In a multi-turn conversation, keep `tool_choice` stable, or change it infrequently.
  </Warning>

  | Intent               | `Messages`                                             | `Responses`                                                | `ChatCompletions`                                                       |
  | -------------------- | ------------------------------------------------------ | ---------------------------------------------------------- | ----------------------------------------------------------------------- |
  | Let the model decide | Omit or `{"type":"auto"}`                              | Omit or `"auto"`                                           | Omit or `"auto"`                                                        |
  | Require any tool     | `{"type":"any"}`                                       | `"required"`                                               | `"required"`                                                            |
  | Require one tool     | `{"type":"tool","name":"baseten__<provider>__<tool>"}` | `{"type":"function","name":"baseten__<provider>__<tool>"}` | `{"type":"function","function":{"name":"baseten__<provider>__<tool>"}}` |
  | Disable tools        | `{"type":"none"}`                                      | `"none"`                                                   | `"none"`                                                                |

  If a request includes both server-side and application-side tool execution, the `require any tool` forms allow the model to choose either type. Name the server-side tool when the first iteration must search.
</Accordion>

## Response shapes

Each protocol returns the final answer through its normal response field and represents completed server-side calls differently. A turn that ends on an open application call reports the protocol's tool-calling stop condition. `Messages` reports `stop_reason: "tool_use"`, `Responses` has no stop field and carries the open call as a `function_call` item in `output`, and `ChatCompletions` reports `finish_reason: "tool_calls"`.

| Protocol          | Final answer                                                       | Completed server-side calls                  | Open application calls                      |
| ----------------- | ------------------------------------------------------------------ | -------------------------------------------- | ------------------------------------------- |
| `Messages`        | `content` text blocks                                              | `tool_use` and `tool_result` blocks          | `tool_use` blocks without a matching result |
| `Responses`       | `output` message items, exposed as `output_text` by the OpenAI SDK | Completed `mcp_call` items                   | `function_call` items                       |
| `ChatCompletions` | `choices[0].message.content`                                       | `baseten.iterations[].continuation_messages` | `choices[0].message.tool_calls`             |

`Messages` uses generic `tool_use` and `tool_result` blocks for tools that Baseten executes server-side. These blocks round-trip through Baseten's `Messages` endpoint, but Anthropic's API does not accept `tool_result` inside an assistant message. The top-level `usage` is the billed total across all model iterations, and `usage.input_tokens` excludes cache reads.

`ChatCompletions` concatenates the text of every iteration into `choices[0].message.content`. A preamble such as "I will search now" arrives attached to the answer with no separator. Read `baseten.iterations[].continuation_messages` for the separated transcript.

`Responses` uses completed `mcp_call` items for tools that Baseten executes server-side. `server_label` identifies the provider. Application tools remain standard `function_call` items.

## `baseten` response metadata

The optional `baseten` object describes work across model iterations. Read it to display `ChatCompletions` tool execution progress, retain `ChatCompletions` search history, inspect per-iteration usage, or distinguish an iteration limit from a model token limit. A client that only displays the final answer can ignore it.

The object has two scopes. `iterations[]` holds one entry per internal model call. `request` holds fields that describe the whole HTTP request.

```json Extension example (ChatCompletions, buffered) theme={"system"}
"baseten": {
  "iterations": [
    {
      "index": 0,
      "usage": {"prompt_tokens": 599, "completion_tokens": 414, "total_tokens": 1013},
      "server_tool_calls": [
        {
          "id": "baseten__<provider>__<tool>_0_abcd1234",
          "provider": "<provider>",
          "name": "baseten__<provider>__<tool>",
          "arguments": "{\"query\": \"…\"}",
          "is_error": false
        }
      ],
      "continuation_messages": [
        {"role": "assistant", "tool_calls": ["…"]},
        {"role": "tool", "tool_call_id": "…", "content": "{\"results\": […]}"}
      ],
      "debug_msg": ["appended steering message: …"]
    }
  ],
  "request": {
    "termination_reason": "max_react_iterations_reached",
    "server_tool_calls": [
      {
        "id": "baseten__<provider>__<tool>_0_abcd1234",
        "provider": "<provider>",
        "name": "baseten__<provider>__<tool>",
        "status": "succeeded",
        "billable": true,
        "sku": "search",
        "quantity": 1.0
      }
    ]
  }
}
```

| Field                                  | Protocol          | Use it to                                                                                                                  |
| -------------------------------------- | ----------------- | -------------------------------------------------------------------------------------------------------------------------- |
| `iterations[].index`                   | All               | Merge partial metadata from the same iteration while streaming.                                                            |
| `iterations[].usage`                   | All               | Inspect token usage for one model iteration.                                                                               |
| `iterations[].server_tool_calls[]`     | `ChatCompletions` | Display a server-side call's provider, name, arguments, completion state, and error state. Merge repeated records by `id`. |
| `iterations[].continuation_messages[]` | `ChatCompletions` | Append the complete server-side tool execution transcript to a follow-up request.                                          |
| `iterations[].debug_msg[]`             | All               | Read disclosed server-side notes, such as the final-iteration steering message. Free-form strings. Do not branch on them.  |
| `request.termination_reason`           | All               | Distinguish a server-side tool execution iteration limit from the protocol's native stop condition.                        |
| `request.server_tool_calls[]`          | All               | Inspect the outcome of every server-side call in the request: `status`, `billable`, `sku`, `quantity`.                     |

The billing fields `billable`, `sku`, and `quantity` are provisional during early access. Their names and semantics can change without notice. `sku` can be `null`, and the meaning of `quantity` varies by provider.

Baseten may add fields to this object. Ignore fields that your client does not use.

## Streaming metadata

Server-side tool execution supports buffered and streaming requests. The `baseten` object appears in these locations:

| Protocol          | Buffered           | Streaming                                                                                                                              |
| ----------------- | ------------------ | -------------------------------------------------------------------------------------------------------------------------------------- |
| `Messages`        | Response body root | The in-flight events the SDK does not rebuild, such as `content_block_start` and `message_delta`. `message_start` carries no metadata. |
| `Responses`       | Response body root | Event root on events that carry the extension, and inside `response` on the terminal event                                             |
| `ChatCompletions` | Response body root | Chunks with an empty `choices` array                                                                                                   |

Streaming iteration entries are partial. Merge iterations by `index` and `ChatCompletions` server-side calls by `id`. Append each `continuation_messages` array once, in arrival order.

`ChatCompletions` metadata chunks have `choices: []`. Check for an empty array before reading `choices[0]` in a hand-written streaming client.

## Iteration limit

If the model continues to call server-side tools at the iteration limit, Baseten returns a resumable response with `max_react_iterations_reached`:

| Protocol          | Native status                     | Detailed reason                      |
| ----------------- | --------------------------------- | ------------------------------------ |
| `Messages`        | `stop_reason: "pause_turn"`       | `baseten.request.termination_reason` |
| `Responses`       | `status: "completed"` (unchanged) | `baseten.request.termination_reason` |
| `ChatCompletions` | `finish_reason: "length"`         | `baseten.request.termination_reason` |

The transcript ends with a completed server-side call and result. Send it back without a new user message to resume from that point, or retry the original request with a larger `max_react_iterations` value.

In `ChatCompletions`, `finish_reason: "length"` without `baseten.request.termination_reason` means the model reached `max_tokens`. OpenAI SDK helpers can raise `LengthFinishReasonError` in either case. The completion remains available as `error.completion`.

## Observability

* On enabled workspaces, a Model API page's Metrics tab graphs server-side tool calls: throughput, throughput per tool, errors by error code, and call latency.
* Billed tool calls appear in workspace settings under Billing → Usage, in the Tool calls view, grouped by provider.
* Per-request detail is in the [`baseten` response metadata](#baseten-response-metadata): per-iteration usage, call outcomes, and `request.termination_reason`.
* The metrics export endpoint does not include server-side tool series. The usage and costs REST APIs report token counts only, with no tool-call dimension.
* A failure after a streaming response starts arrives as an SSE `error` event, because the HTTP status is already `200`. Streaming clients must handle that event.

## Limits and errors

| Limit or condition                                  | Behavior                                                                                                                                                                                                                                                                                         |
| --------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Invalid selector, setting, or field under `baseten` | Returns `400`.                                                                                                                                                                                                                                                                                   |
| Feature not enabled for the organization            | Returns `403` with `server-side tools are not enabled for this organization`.                                                                                                                                                                                                                    |
| Feature not enabled for the model                   | Returns `400` with `server-side tools are not enabled for this endpoint`.                                                                                                                                                                                                                        |
| Request duration                                    | Stops after 285 seconds.                                                                                                                                                                                                                                                                         |
| Request body                                        | Accepts up to 10 MiB. Larger bodies return `413`.                                                                                                                                                                                                                                                |
| Tool result size                                    | Results over 1 MiB fail as an error tool result that the model can inspect.                                                                                                                                                                                                                      |
| Provider timeout                                    | Stops the tool after the timeout listed in the [tool catalog](#tool-catalog).                                                                                                                                                                                                                    |
| Server-tools rate limit                             | Returns `429` with `server-tools rate limit exceeded for this organization`. The early-access limit is 25 requests per minute, shared across all models. To raise it, contact [Baseten support](mailto:support@baseten.co) or use the support chat in the [Baseten app](https://app.baseten.co). |
| Selected tool unavailable                           | Returns `400` when the tool's provider is not connected.                                                                                                                                                                                                                                         |
| Model API rate limit                                | Returns `429` with [Model API semantics](/inference/errors#429-too-many-requests), distinct from the server-tools limit.                                                                                                                                                                         |
| Model-plane failure or overload                     | `500`, `502`, and `529` pass through from the model plane, and `503` means the tool-execution fleet is at capacity, not the search provider. Same semantics as [Model APIs](/inference/errors). Retry with exponential backoff.                                                                  |

Server-side tool execution implements a subset of each protocol. A request that uses a refused feature returns `400` instead of silently degrading.

| Protocol          | Refused features                                                                                                                                                                                                    |
| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `Responses`       | `store: true`, `previous_response_id`, `conversation`, and `background: true`. The loop keeps no server-side state. Echo the previous response's `output` as the next request's `input` to continue a conversation. |
| `ChatCompletions` | `n > 1` (multiple answer choices per request). The loop produces one choice per model turn.                                                                                                                         |
| All               | Fixed arguments on a `baseten__*` selector. Extra fields return `400`.                                                                                                                                              |
| All               | Async inference requests.                                                                                                                                                                                           |

Two output deviations from the stock protocols:

* `Messages` represents server-side calls as generic `tool_use` and `tool_result` blocks, not Anthropic's `server_tool_use` and `web_search_tool_result` types. Thinking blocks carry `signature: "erased"`. Echo these blocks back to Baseten only, not to Anthropic's API.
* `Responses` does not echo request parameters on the response envelope. Fields such as `temperature`, `tools`, and `store` are `null` on the response.

A provider failure usually becomes a tool result that the model can inspect. The model can retry, change its query, choose another offered tool, or answer without that result. Errors that prevent the loop from starting return a non-`2xx` response.

For general inference error guidance, see [Inference errors](/inference/errors).

Use a client read timeout longer than 285 seconds for buffered requests. Streaming responses send a `: keep-alive` comment line every 15 seconds while a server-side call runs.