Web search

Early access. Server-side tool execution is available as a rate limited preview for playground usage (25 RPM across all models). For scaled evaluations or high-volume production integration, contact Baseten support or use the support chat in the Baseten app.

Web search lets LLMs in Model APIs search the web, fetch pages, and use the results before responding. The integration is config-driven, with few changes required in your client code. To get an intuitive feel, try it out in our playground (example is for GLM 5.3, but almost all Model APIs are supported). Your application sends one request. Baseten runs the model and executes search tools in a server-side loop, then returns the answer in the same Model API response, including live updates as server-sent events (SSE). The request moves through this loop: The model generates each search tool’s arguments from the schema Baseten expands into the request automatically. Baseten adds each result to the model’s context, and the model either makes more tool calls or answers. The loop stops when the model answers or reaches its iteration limit. If the model calls a tool that your application runs (client-side), Baseten returns that call to your code instead. Hybrid usage works too. To enable server-side tool execution for a Model API request:

Set the x-baseten-server-tools request header to true.
Add one or more baseten__* selectors to tools. Only the type is needed. Baseten expands the schema automatically.
Add baseten.tool_settings when you want to change the loop limits.

Server-side tools run on third-party search providers. By adding their tools, you agree to their respective terms: Exa, Keenable, Parallel, You.com.

Inference with web search grounding

Set BASETEN_API_KEY. Pick a provider for the examples: Tools are available on three API protocols. For multi-turn integrations with echo-back and KV-cache stability, Baseten recommends Messages or Responses. Chat Completions works fine for single-turn but requires extra care for multi-turn.

Messages
Responses
Chat Completions

To search the web with the Messages API:

Install the Anthropic SDK:
uv pip install anthropic

Create an Anthropic client that sends x-baseten-server-tools: true. Add the server-side tool selectors and loop limits to the request. Save the code as web_search_messages.py:

web_search_messages.py

import os

import anthropic

client = anthropic.Anthropic(
    api_key=os.environ["BASETEN_API_KEY"],
    base_url="https://inference.baseten.co",
    default_headers={
        "Authorization": f"Bearer {os.environ['BASETEN_API_KEY']}",
        "x-baseten-server-tools": "true",
    },
)

messages = [
    {
        "role": "user",
        "content": "What is the latest Baseten news?",
    }
]
request = {
    "model": "moonshotai/Kimi-K3",
    "max_tokens": 1200,
    "tools": [
        {"type": "baseten__PROVIDER__SEARCH"},
        {"type": "baseten__PROVIDER__FETCH"},
    ],
    "extra_body": {
        "baseten": {
            "tool_settings": {
                "max_react_iterations": 6,
            }
        }
    },
}

response = client.messages.create(messages=messages, **request)

for block in response.content:
    if block.type == "text":
        print(block.text)
# Server-side calls and their outcomes ride the `baseten` extension:
print((response.model_extra or {}).get("baseten"))

def continue_conversation(messages, response, prompt):
    messages.append({"role": "assistant", "content": response.content})
    messages.append({"role": "user", "content": prompt})
    return client.messages.create(messages=messages, **request)

Call continue_conversation() for a follow-up. The helper appends the complete response.content array, including the search calls and results that the printed text omits.

Run the request. The example output is abbreviated because search results and wording change over time:
uv run python web_search_messages.py
Baseten has published several pieces about Moonshot AI's Kimi models. ... Sources: - https://www.baseten.co/blog/kimi-k2-thinking-at-140-tps-on-nvidia-blackwell
For streaming, use client.messages.stream(). The baseten extension rides in-flight events such as content_block_start and message_delta. The SDK does not rebuild it into the final message, so read the events as they pass. For continuation, call stream.get_final_message() inside the with block and append its content.

To search the web with the Responses API:

Install the OpenAI SDK:
uv pip install openai

Create an OpenAI client that sends x-baseten-server-tools: true. Add the server-side tool selectors and loop limits to the request. Save the code as web_search_responses.py:

web_search_responses.py

import os

from openai import OpenAI

client = OpenAI(
    api_key=os.environ["BASETEN_API_KEY"],
    base_url="https://inference.baseten.co/v1",
    default_headers={"x-baseten-server-tools": "true"},
)

input_items = [
    {
        "role": "user",
        "content": "What is the latest Baseten news?",
    }
]
request = {
    "model": "moonshotai/Kimi-K3",
    "max_output_tokens": 1200,
    "extra_body": {
        "tools": [
            {"type": "baseten__PROVIDER__SEARCH"},
            {"type": "baseten__PROVIDER__FETCH"},
        ],
        "baseten": {
            "tool_settings": {
                "max_react_iterations": 6,
            }
        },
    },
}

response = client.responses.create(input=input_items, **request)

print(response.output_text)
# Server-side calls and their outcomes ride the `baseten` extension:
print((response.model_extra or {}).get("baseten"))

def continue_conversation(input_items, response, prompt):
    input_items.extend(
        item.model_dump(exclude_none=True) for item in response.output
    )
    input_items.append({"role": "user", "content": prompt})
    return client.responses.create(input=input_items, **request)

Call continue_conversation() for a follow-up. The helper appends every response.output item to the next request’s input. This preserves the search history without server-side conversation state.

Run the request. The example output is abbreviated because search results and wording change over time:
uv run python web_search_responses.py
Baseten has published several pieces about Moonshot AI's Kimi models. ... Sources: - https://www.baseten.co/blog/kimi-k2-thinking-at-140-tps-on-nvidia-blackwell
For streaming, use client.responses.stream(). A server-side call first appears as a function_call item and resolves in place to an mcp_call item at the same output index. The baseten extension rides the event root in flight and sits inside response on the terminal event.

To search the web with the Chat Completions API:

Install the OpenAI SDK:
uv pip install openai

Create an OpenAI client that sends x-baseten-server-tools: true. Add the server-side tool selectors and loop limits to the request. Save the code as web_search_chat_completions.py:

web_search_chat_completions.py

import os

from openai import OpenAI

client = OpenAI(
    api_key=os.environ["BASETEN_API_KEY"],
    base_url="https://inference.baseten.co/v1",
    default_headers={"x-baseten-server-tools": "true"},
)

messages = [
    {
        "role": "user",
        "content": "What is the latest Baseten news?",
    }
]
request = {
    "model": "moonshotai/Kimi-K3",
    "max_tokens": 1200,
    "extra_body": {
        "tools": [
            {"type": "baseten__PROVIDER__SEARCH"},
            {"type": "baseten__PROVIDER__FETCH"},
        ],
        "baseten": {
            "tool_settings": {
                "max_react_iterations": 6,
            }
        },
    },
}

response = client.chat.completions.create(messages=messages, **request)

print(response.choices[0].message.content)
# Server-side calls and their outcomes ride the `baseten` extension:
print((response.model_extra or {}).get("baseten"))

def continue_conversation(messages, response, prompt):
    extension = response.model_extra["baseten"]
    for iteration in extension["iterations"]:
        messages.extend(iteration.get("continuation_messages", []))
    messages.append({"role": "user", "content": prompt})
    return client.chat.completions.create(messages=messages, **request)

Call continue_conversation() for a follow-up. The helper appends every continuation_messages entry in iteration order. Do not append choices[0].message too. The continuation transcript already contains the final assistant message.

Run the request. The example output is abbreviated because search results and wording change over time:
uv run python web_search_chat_completions.py
Baseten has published several pieces about Moonshot AI's Kimi models. ... Sources: - https://www.baseten.co/blog/kimi-k2-thinking-at-140-tps-on-nvidia-blackwell
For streaming, use client.chat.completions.stream(). The extension rides raw chunks with an empty choices array, so guard any choices[0] access. Each extension chunk carries one iteration’s complete continuation_messages. Collect them from the passing chunks, then continue as in the helper above.

Search provider selection

For a list of search and fetch tools, see the server-side tool execution reference. The reference also covers provider-specific selectors, loop limits, and tool_choice formats for each API protocol. Use the system prompt to control search policy. Tell the model when to search, whether to fetch primary sources, and how to cite conflicting evidence.

Mixed server-side and client-side tools

You can configure both tool types when the model needs web results and data that only your application can access, such as private company documentation. Baseten executes tools selected with baseten__* server-side. Your application executes the functions you define. Define each application tool with the schema for your API protocol:

Messages
Responses
Chat Completions

Mixed tools

{
  "tools": [
    {"type": "baseten__PROVIDER__SEARCH"},
    {
      "name": "search_company_docs",
      "description": "Search the company's private documentation.",
      "input_schema": {
        "type": "object",
        "properties": {
          "query": {"type": "string"}
        },
        "required": ["query"]
      }
    }
  ]
}

Mixed tools

{
  "tools": [
    {"type": "baseten__PROVIDER__SEARCH"},
    {
      "type": "function",
      "name": "search_company_docs",
      "description": "Search the company's private documentation.",
      "parameters": {
        "type": "object",
        "properties": {
          "query": {"type": "string"}
        },
        "required": ["query"]
      }
    }
  ]
}

Mixed tools

{
  "tools": [
    {"type": "baseten__PROVIDER__SEARCH"},
    {
      "type": "function",
      "function": {
        "name": "search_company_docs",
        "description": "Search the company's private documentation.",
        "parameters": {
          "type": "object",
          "properties": {
            "query": {"type": "string"}
          },
          "required": ["query"]
        }
      }
    }
  ]
}

If the model selects search_company_docs, Baseten returns the open call to your application. Execute the function and return its result through the protocol’s tool-calling loop. Baseten executes a server-side call as soon as the model makes it, and results keep the model’s declaration order in the response. A turn that ends on an open client call reports the protocol’s tool-calling stop condition. Messages reports stop_reason: "tool_use", Responses has no stop field and carries the open call as a function_call item in output, and Chat Completions reports finish_reason: "tool_calls". The server-side tool execution reference shows where each protocol returns open application calls. For a complete client-side execution loop, see Function calling.

Failures

A provider error usually becomes a tool result that the model can inspect. The model can retry, change its query, choose another offered tool, or answer without that result. You can offer multiple search providers and use the system prompt to tell the model which provider to prefer.

Streaming errors

An error after a streaming response starts arrives as an SSE error frame, because the HTTP status is already 200. Streaming clients must handle that frame. Each protocol uses its native error frame:

Messages: an error event with Anthropic’s error body, for example {"type":"error","error":{"type":"api_error","message":"..."}}. The Anthropic SDK raises it as anthropic.APIStatusError.
Responses: a terminal response.failed event whose response.error carries code and message.
Chat Completions: a final {"error": {...}} data frame without an event name. The OpenAI SDK raises it as openai.APIError.

Status codes

Invalid selectors, settings, or Baseten-specific fields return 400. Limits and errors lists every early rejection, including an organization or model without the feature. A 429 is either the server-tools early-access limit (25 requests per minute per organization) or the standard Model API rate limit. The error body says which. 500, 502, and 529 are model-plane failures with the same semantics as Model APIs. Retry with exponential backoff. A 503 comes from the tool-execution fleet at capacity, not from the search provider. Retry with exponential backoff. For errors outside server-side tool execution, see Inference errors.

Get started

Model APIs

Inference

Training

Development

Deployment

Engines

Model Labs

Organization

Hosting options

Observability

Troubleshooting

Inference with web search grounding

Search provider selection

Mixed server-side and client-side tools

Failures

Streaming errors

Status codes

Next steps

Server-side tool execution

Function calling

​Inference with web search grounding

​Search provider selection

​Mixed server-side and client-side tools

​Failures

​Streaming errors

​Status codes

​Next steps

Server-side tool execution

Function calling

Inference with web search grounding

Search provider selection

Mixed server-side and client-side tools

Failures

Streaming errors

Status codes

Next steps