> ## Documentation Index
> Fetch the complete documentation index at: https://docs.baseten.co/llms.txt
> Use this file to discover all available pages before exploring further.

# Baseten Realtime Voice recipes

> Example session.update bodies for common Baseten Realtime Voice setups, including a phone support agent, client tools, delegation, and large language model (LLM) switching.

> [!NOTE]
>
> Baseten Realtime Voice is in **Private Preview**. [Join the waitlist](https://baseten.co/talk-to-us/voice-beta-launch)
> to get it enabled for your organization.

Each recipe is a `session.update` body to send after the WebSocket connects, plus the events it requires. Every field is optional and an update changes only the fields it sends, so combine recipes by merging their `session` objects. Each recipe sends `"type": "realtime"`. The server doesn't need it, but the OpenAI Agents SDK checks raw events against OpenAI's schema, which does. Some recipes connect to a remote Model Context Protocol (MCP) server. Another switches the large language model (LLM) for each turn. For every field and its limits, see the [Baseten Realtime Voice API reference](https://docs.baseten.co/reference/inference-api/predict-endpoints/realtime-voice-api.md).

Replace `<BASETEN_API_KEY>` with your Baseten API key. Keys sent in a session apply to that session only and are never echoed in `session.updated`.

## Phone support agent

This recipe sets a persona with phone-style instructions, server voice activity detection (VAD) that lets the caller talk over the agent, and key terms that help speech-to-text (STT) hear product names:

```json theme={"system"}
{
  "type": "session.update",
  "session": {
    "type": "realtime",
    "instructions": "You are Riley, a support agent for Northwind Fiber, a home internet provider, on a live phone call. Plans: Fiber 300 at $49.99 a month, Fiber Gig at $69.99, Fiber Max at $99.99. No contracts or data caps. For billing changes, offer to connect the caller with a specialist.",
    "audio": {
      "input": {
        "turn_detection": {
          "type": "server_vad",
          "create_response": true,
          "interrupt_response": true
        },
        "transcription": {
          "keywords": ["Northwind Fiber", "Fiber 300", "Fiber Gig", "Fiber Max"]
        }
      }
    }
  }
}
```

Baseten Realtime Voice adds voice-call guidance after your instructions, such as short replies and one question at a time, so your instructions don't need to cover speaking style. To have the agent speak first, ask for a greeting. Per-response instructions are added to the session's, so it stays in character:

```json theme={"system"}
{
  "type": "response.create",
  "response": {
    "instructions": "Greet the caller: say you're Riley from Northwind Fiber support and ask how you can help. One short sentence."
  }
}
```

## Telephony audio

Send and receive G.711 μ-law at 8 kHz, the format that phone media streams use, with no transcoding on your side:

```json theme={"system"}
{
  "type": "session.update",
  "session": {
    "type": "realtime",
    "audio": {
      "input": { "format": { "type": "audio/pcmu" } },
      "output": { "format": { "type": "audio/pcmu" } }
    }
  }
}
```

Use `audio/pcma` for A-law. With `audio/pcmu`, each `input_audio_buffer.append` carries base64 μ-law audio, and `response.output_audio.delta` returns it. With `audio/pcma`, both carry A-law audio.

## Voice and transcription settings

Pick a voice and set the transcription language and key terms:

```json theme={"system"}
{
  "type": "session.update",
  "session": {
    "type": "realtime",
    "audio": {
      "output": { "voice": "ryan" },
      "input": {
        "transcription": {
          "language": "English",
          "keywords": ["Kubernetes", "Baseten", "Truss"]
        }
      }
    }
  }
}
```

Voice names depend on the deployment's TTS model. A Qwen3-TTS deployment speaks named voices, such as `ryan`. A Magpie deployment speaks names such as `Magpie-Multilingual.EN-US.Mia`, some with an emotion (`Magpie-Multilingual.EN-US.Mia.Calm`); which emotions a speaker has varies. Names are case-sensitive, and a name the model doesn't have produces replies with no audio.

To see the voices a deployment offers, read `session.x_providers.tts` in the `session.created` event, which arrives as soon as the WebSocket connects; every `session.updated` repeats it. `voice` is the current voice, and `voices` lists every name `audio.output.voice` accepts:

```json theme={"system"}
{
  "provider": "internal",
  "voice": "Magpie-Multilingual.EN-US.Mia",
  "voices": [
    "Magpie-Multilingual.EN-US.Aria",
    "Magpie-Multilingual.EN-US.Aria.Angry",
    "Magpie-Multilingual.EN-US.Aria.Calm",
    "..."
  ]
}
```

`voices` is `null` when the TTS model can't list its voices. `x_providers.stt.available` lists the STT routes the same way. To switch the STT route, send `"x_stt": {"provider": "<route>"}` before any caller audio. The language format depends on the STT model: `English` for Qwen3-ASR, `en` for Whisper.

## Push-to-talk

Turn off automatic turn detection so the client decides when the caller is done:

```json theme={"system"}
{
  "type": "session.update",
  "session": {
    "type": "realtime",
    "audio": { "input": { "turn_detection": null } }
  }
}
```

Stream audio with `input_audio_buffer.append` while the button is held. On release, end the turn and ask for the reply:

```json theme={"system"}
{ "type": "input_audio_buffer.commit" }
```

```json theme={"system"}
{ "type": "response.create" }
```

## Client tools

Function tools run in your application, next to your data. The agent speaks a short acknowledgment if a tool takes more than a moment:

```json theme={"system"}
{
  "type": "session.update",
  "session": {
    "type": "realtime",
    "instructions": "You are a support agent for Northwind Fiber on a live phone call. Before account-specific help, ask for the phone number on the account and look it up. Never invent account details; use the tools.",
    "tools": [
      {
        "type": "function",
        "name": "lookup_account",
        "description": "Find the caller's account by the phone number on it.",
        "parameters": {
          "type": "object",
          "properties": { "phone": { "type": "string", "description": "Phone number, digits only" } },
          "required": ["phone"]
        }
      },
      {
        "type": "function",
        "name": "check_outage",
        "description": "Check for known outages or network problems at a ZIP code.",
        "parameters": {
          "type": "object",
          "properties": { "zip_code": { "type": "string" } },
          "required": ["zip_code"]
        }
      }
    ],
    "tool_choice": "auto"
  }
}
```

When the model calls a tool, the server sends `response.function_call_arguments.done` with `name`, `call_id`, and `arguments`. Run the tool, return the result, and then ask for the next response:

```json theme={"system"}
{
  "type": "conversation.item.create",
  "item": {
    "type": "function_call_output",
    "call_id": "<call_id>",
    "output": "{\"account_id\": \"NW-1042\", \"plan\": \"Fiber Gig\", \"zip\": \"94110\"}"
  }
}
```

```json theme={"system"}
{ "type": "response.create" }
```

Set `x_auto_continue: true` to have the server start the next response when the result arrives.

## MCP server

Baseten Realtime Voice connects to a remote MCP server and runs its tools directly, without a client round trip. This example uses DeepWiki's public server, which answers questions about GitHub repositories:

```json theme={"system"}
{
  "type": "session.update",
  "session": {
    "type": "realtime",
    "instructions": "You are a developer support agent on a phone call. Use the deepwiki ask_wiki_question tool to look things up, then answer in two or three spoken sentences, without code or URLs.",
    "tools": [
      {
        "type": "mcp",
        "server_label": "deepwiki",
        "server_url": "https://mcp.deepwiki.com/mcp",
        "allowed_tools": ["ask_wiki_question"],
        "require_approval": "never",
        "x_call_timeout_secs": 60
      }
    ]
  }
}
```

For a server that needs a key, add `"authorization": "<MCP_API_KEY>"`. The server URL must use HTTPS on a public host.

An `mcp_list_tools` item and `x_providers.mcp_tools` report the server's tools. Check them after the update: a name in `allowed_tools` that the server doesn't offer registers nothing, and the agent then can't make the call it promises.

A call can outlast the response that made it. That response ends with the `mcp_call` still in progress, and the call finishes later under the same item ID: read its `output` from `conversation.item.done`, not from `response.done`. See [Tools the agent runs](https://docs.baseten.co/reference/inference-api/predict-endpoints/realtime-voice-api.md#tools-the-agent-runs).

To approve each call first, set `"require_approval": "always"`. The response then ends with an `mcp_approval_request` item. Answer it, then ask for the next response:

```json theme={"system"}
{
  "type": "conversation.item.create",
  "item": {
    "type": "mcp_approval_response",
    "approval_request_id": "<mcp_approval_request item id>",
    "approve": true
  }
}
```

```json theme={"system"}
{ "type": "response.create" }
```

## Delegation

A fast voice model keeps the conversation while a stronger model on [Model APIs](https://docs.baseten.co/inference/model-apis/overview.md) does the work: it reads the policy in your `instructions`, calls your tools, and writes what the caller hears. The session names the delegate's model and sends your Baseten API key. Baseten bills the usage to your account:

```json theme={"system"}
{
  "type": "session.update",
  "session": {
    "type": "realtime",
    "instructions": "You are a support agent for Northwind Fiber on a live phone call. Look up the account before answering any billing question. Credits over $50 need a supervisor; offer a callback instead.\n\nBilling policy: ...",
    "tools": [{
      "type": "function",
      "name": "lookup_account",
      "description": "Look up a customer's account and current bill by phone number.",
      "parameters": {
        "type": "object",
        "properties": { "phone": { "type": "string" } },
        "required": ["phone"]
      }
    }],
    "x_delegation": {
      "model": "zai-org/GLM-5.3",
      "api_key": "<BASETEN_API_KEY>",
      "reasoning_effort": "none"
    }
  }
}
```

The delegate's `lookup_account` calls arrive as `function_call` items; reply with `function_call_output`, and the delegate continues on its own. The server reports each handoff as an `mcp_call` item with `server_label: "delegation"`. To turn delegation off:

```json theme={"system"}
{ "type": "session.update", "session": { "type": "realtime", "x_delegation": false } }
```

## Switch the LLM per turn

Route everyday turns to the Chain's own LLM and harder ones to a larger model on Model APIs. Turn off `create_response` so the server detects and transcribes each turn but waits for the client to start the reply:

```json theme={"system"}
{
  "type": "session.update",
  "session": {
    "type": "realtime",
    "audio": {
      "input": {
        "turn_detection": {
          "type": "server_vad",
          "create_response": false,
          "interrupt_response": true
        }
      }
    }
  }
}
```

When `conversation.item.input_audio_transcription.completed` arrives, decide which model the turn needs. For a harder turn, switch to Model APIs with your own key, then ask for the reply:

```json theme={"system"}
{
  "type": "session.update",
  "session": {
    "type": "realtime",
    "x_llm": {
      "base_url": "https://inference.baseten.co/v1",
      "api_key": "<BASETEN_API_KEY>",
      "model": "openai/gpt-oss-120b",
      "reasoning_effort": "low"
    }
  }
}
```

```json theme={"system"}
{ "type": "response.create" }
```

The server applies events in order, so the reply runs on the new model, and the conversation carries over. To switch back to the Chain's own LLM, which needs no key:

```json theme={"system"}
{ "type": "session.update", "session": { "type": "realtime", "x_llm": { "provider": "internal" } } }
```

`session.updated` confirms each switch in `x_providers.llm`.
