> ## Documentation Index
> Fetch the complete documentation index at: https://docs.baseten.co/llms.txt
> Use this file to discover all available pages before exploring further.

# Real-time transcription API (v2)

> Stream audio to a speech-to-text model over a WebSocket using a unified event protocol across models.

Stream audio to a model deployed with the Baseten speech-to-text (STT) server and receive partial and final transcripts over a WebSocket. This reference defines the shared v2 protocol; for model-specific protocols, see [Real-time transcription API (v1)](https://docs.baseten.co/reference/inference-api/predict-endpoints/streaming-transcription-api.md).

## Deployment compatibility

The deployed server determines the protocol; the WebSocket URL doesn't contain a protocol version. The Qwen3-ASR **Deploy Streaming** option in the Model Library uses [v1](https://docs.baseten.co/reference/inference-api/predict-endpoints/streaming-transcription-api.md#qwen3-asr-streaming). It doesn't send `session.created` and can't run a v2 client.

To use v2, contact [Baseten support](mailto:support@baseten.co) for a unified STT server deployment. Confirm that it sends `session.created` before integrating. For a complete client and recording, follow [Transcribe with the unified API](https://docs.baseten.co/inference/audio/quickstart.md#transcribe-with-the-unified-api).

## Connection

```text theme={"system"}
wss://model-{model_id}.api.baseten.co/environments/production/websocket
```

This is the production environment's [WebSocket endpoint](https://docs.baseten.co/reference/inference-api/predict-endpoints/environments-websocket.md). To validate a unified deployment before promoting it, use its [deployment endpoint](https://docs.baseten.co/reference/inference-api/predict-endpoints/deployment-websocket.md):

```text theme={"system"}
wss://model-{model_id}.api.baseten.co/deployment/{deployment_id}/websocket
```

Unified STT presets use a custom Docker server, which requires a published deployment. Development deployments aren't supported for these presets. For connection behavior, see [WebSockets](https://docs.baseten.co/development/model/websockets.md).

### Parameters

**model\_id** `string` *required*

The ID of the model you want to call.

**deployment\_id** `string`

The published deployment ID. Required when using the deployment endpoint.

**Authorization** `string` *required*

Your Baseten API key: `Authorization: Bearer $BASETEN_API_KEY`. The server also accepts `Api-Key` as the scheme.

Baseten checks your API key during the WebSocket handshake. If the check fails, the handshake fails with an HTTP status and the socket never opens: `400` with no `Authorization` header, or `403` if the key is invalid or can't access the model. You don't get an `error` event or a close code. See [how a request reaches your model](https://docs.baseten.co/deployment/autoscaling/request-lifecycle.md#how-a-request-reaches-your-model).

The browser's `WebSocket` API can't set the `Authorization` header. Connect through your backend, which adds the header and keeps your API key on the server.

If the deployment has scaled to zero, Baseten holds the connection until a replica is ready, which can take several minutes. If every replica is at its [concurrency target](https://docs.baseten.co/deployment/autoscaling/overview.md#param-concurrency-target), the connection waits for a free slot or a new replica. Set your client's handshake timeout to allow for this, or set a minimum replica count greater than zero. See [what happens when no replica is available](https://docs.baseten.co/deployment/autoscaling/request-lifecycle.md#what-happens-when-no-replica-is-available).

## Session flow

1. Connect. The server sends `session.created` with the model's capabilities and default settings.
2. Optionally, send `session.update` to change the settings. The server replies with `session.updated`, listing what it granted and anything it refused.
3. Stream audio with `input_audio_buffer.append`.
4. End each turn with `input_audio_buffer.flush` (`client_flush`), or let a pause end it (`server_vad`). See [segmentation modes](#segmentation-modes).
5. Receive `transcription.done` with each turn's final transcript. If you enable partials, the server also sends `transcription.delta` while audio arrives.
6. After every turn has a final result, send `close`. The server closes the socket with code `1000`. A normal close alone doesn't confirm that all audio was transcribed.

Configure the session before sending audio. To change settings after audio starts, open a new connection.

## Client events

Every event is a JSON object with a `type` field. The server ends the session with an `invalid_event` error for any unknown type or field, wrong value type, or out-of-range value, including misspelled field names.

### `session.update`

Configures the session. If you omit this event, the session uses `defaults` from `session.created`. The server reports unsupported settings in `session.updated.refusals` and the effective settings in `session.updated.granted`.

Include every setting you want to keep in each update. Omitted fields reset to their defaults, except `context`, which retains its previous value.

**session.input\_format** `object`

The format of the audio you send. Each model accepts one format, shown in `capabilities.input_format`. Send audio in that format; resample or downmix it first if needed. You can leave this field out.

**Properties**

**session.input\_format.encoding** `string` *required*

Always `pcm_s16le` (16-bit little-endian signed PCM). Any other value is an `invalid_event` error.

**session.input\_format.sample\_rate\_hz** `integer` *required*

The model's sample rate, for example `16000`. The server refuses a different value and uses the model's value.

**session.input\_format.channels** `integer` *required*

The model's channel count, for example `1`. The server refuses a different value and uses the model's value.

**session.segmentation\_mode** `string`

What ends a turn: `client_flush` or `server_vad`. See [segmentation modes](#segmentation-modes). Defaults to the first mode in `capabilities.segmentation_modes`. The server refuses modes the model doesn't support.

**session.language** `string`

The language to transcribe, as a BCP-47 code such as `en`, or `auto` to detect it. Defaults to `capabilities.default_language`.

If there's no exact match, the server drops the region and tries again, so `en-US` gets `en`. The server refuses unsupported languages and keeps the default. It also refuses this field if the model doesn't support language selection. A value longer than 35 characters is an `invalid_event` error.

**session.partial\_interval\_s** `float`

How often, in seconds, the server sends `transcription.delta` events during a turn. Partials are off unless you set this.

The server raises shorter intervals to `capabilities.min_partial_interval_s`. A value of `0` or less is an `invalid_event` error. The server refuses this field if the model doesn't produce partials.

**session.context** `object`

Text that helps the model recognize words from your domain. If you leave it out, the session keeps the context from your last update, not the default. If `capabilities.context` is absent, the model takes no context and the field is refused. If the model takes only `text` or only `keyterms`, the other is refused.

**Properties**

**session.context.text** `string`

A short description of the audio, for example `"A cardiology consult."`.

**session.context.keyterms** `string[]`

Words and phrases to favor, most important first. The server removes duplicates.

If the model sets `capabilities.context.max_words`, keyterms are dropped from the end of the list until the context fits, and `text` is dropped if it's too long by itself. Each drop is listed as a refusal, and `granted.context.truncated` is `true` if any keyterms were dropped. If nothing fits, `granted.context` is absent.

**session.vad** `object`

Turn detector settings for `server_vad` sessions. Every field is optional. Omitted fields use the values from `defaults.vad` in `session.created`, which can differ by model or deployment. A value out of range is an `invalid_event` error.

**Properties**

**session.vad.silence\_s** `float`

Seconds of silence that end a turn. From 0.1 to 10.0. Values from 0.0 up to 0.1 are raised to 0.1 instead of rejected.

**session.vad.speech\_start\_s** `float`

Seconds of speech that start a turn. From 0.0 to 10.0.

**session.vad.threshold** `float`

How confident the detector must be that a frame is speech. From 0.0 to 1.0.

**session.vad.min\_volume** `float`

How loud a frame must be to count as speech. From 0.0 to 1.0. `0` turns the check off.

### `input_audio_buffer.append`

Sends audio for the current turn. You can send audio in real time or faster.

Under `server_vad`, use small chunks, such as 100 ms of audio, so the voice detector can observe speech and silence across messages. A large append containing an entire utterance and its trailing silence can miss the turn boundary. Under `client_flush`, your flush ends the turn, so you can send larger chunks within the [WebSocket message size limit](https://docs.baseten.co/development/model/websockets.md#maximum-message-size).

**audio** `string` *required*

Base64-encoded audio in the granted `input_format`. Invalid base64 is an `invalid_event` error.

### `input_audio_buffer.flush`

Ends the current turn. Takes no other fields: `{"type": "input_audio_buffer.flush"}`.

The server transcribes the turn's audio and replies with `transcription.done`, or `transcription.failed` if it can't. Keep receiving until that result arrives; closing the session or losing the connection can interrupt pending work. If you flush with no audio, the transcript is empty. Under `server_vad`, a flush ends the turn without waiting for a pause.

### `input_audio_buffer.clear`

Discards the current turn's audio without transcribing it. Takes no other fields: `{"type": "input_audio_buffer.clear"}`.

The server replies with `input_audio_buffer.cleared`. The discarded turn still uses up a `turn_id`, and its audio still counts toward the session's timestamps.

### `close`

Ends the session. Takes no other fields: `{"type": "close"}`.

The server drains pending transcription work for a limited time, then closes the socket with code `1000`. The default drain timeout is 5 seconds; deployments can configure it. The server can abandon work that exceeds the timeout even when the close code is `1000`.

Flush the last turn and wait for every expected `transcription.done` before sending `close`. Treat `transcription.failed`, a receive timeout, or closure before an expected result as incomplete transcription. Use a receive timeout appropriate for your workload.

Without a flush, the server transcribes the last turn only if its voice detector heard speech in it, in either segmentation mode. Audio it scored as silence is dropped.

## Server events

### `session.created`

The first message on every connection. If the replica is full, you get an `at_capacity` [error](#error) instead.

**session\_id** `string`

The session's ID. Include it in support requests.

**capabilities** `object`

The model's supported settings and limits. Unsupported optional fields are absent.

**Properties**

**capabilities.input\_format** `object`

The only audio format the model accepts, with `encoding`, `sample_rate_hz`, and `channels`.

**capabilities.segmentation\_modes** `string[]`

The modes the model supports: `client_flush`, `server_vad`, or both. The first is the default.

**capabilities.languages** `string[]`

The languages the model transcribes, as BCP-47 codes, plus `auto` if it can detect the language. Empty if the model takes no language.

**capabilities.default\_language** `string`

The language a session gets if it doesn't ask for one. Absent if `languages` is empty.

**capabilities.partials** `boolean`

Whether the model can send partial transcripts before a turn ends.

**capabilities.partial\_style** `string`

How partials relate to each other. `replace`: each partial supersedes the previous one for its turn, and its text can change as more audio arrives, so render the latest and discard the rest. Some models decode a partial from only the latest audio; see `transcription.delta`. `append`: each partial continues the previous one. `no_partials`: none are sent.

**capabilities.min\_partial\_interval\_s** `float`

The shortest partial interval the model grants.

**capabilities.max\_turn\_s** `float`

The longest a turn can be, in seconds. A turn that reaches it is cut, and the speech continues in the next turn. See [long turns](#long-turns). Absent if there's no limit.

**capabilities.max\_sessions\_per\_replica** `integer`

Maximum simultaneous sessions per replica. The replica refuses sessions beyond this limit. Baseten's routing concurrency target can limit sessions below this value.

**capabilities.context** `object`

The context the model accepts. `text` and `keyterms` (`boolean`) say which kinds it takes. `max_words` (`integer`), if present, limits their combined length in words. Absent if the model takes no context.

**capabilities.vad** `object`

The model's preferred turn detector settings, if it has any.

**defaults** `object`

The settings the session uses if you don't send `session.update`. Same shape as `granted` in `session.updated`.

### `session.updated`

The reply to every `session.update`.

**session\_id** `string`

The session's ID.

**granted** `object`

The effective session settings. Use these values to configure your client.

**Properties**

**granted.input\_format** `object`

The audio format: `encoding`, `sample_rate_hz`, and `channels`.

**granted.segmentation\_mode** `string`

`client_flush` or `server_vad`.

**granted.language** `string`

The language, as a BCP-47 code, or `auto`. Absent if the model takes no language.

**granted.partials** `boolean`

Whether the session gets `transcription.delta` events.

**granted.partial\_interval\_s** `float`

The partial interval. Absent when `partials` is `false`.

**granted.context** `object`

The context in use: `text` (`string`), `keyterms` (`string[]`), and `truncated` (`boolean`, `true` if keyterms were dropped to fit the word limit). Absent if the session has no context.

**granted.vad** `object`

The turn detector settings: `silence_s`, `speech_start_s`, `threshold`, and `min_volume`. Always present. Under `client_flush`, the detector runs but doesn't end turns.

**refusals** `array`

Settings that differ from your request. An empty array means the server granted every requested setting.

**Properties**

**refusals\[].parameter** `string`

The refused setting, for example `partial_interval_s` or `context.keyterms`.

**refusals\[].requested** `any`

The requested value.

**refusals\[].granted** `any`

The granted value. Absent if the setting wasn't applied, for example a language on a model that takes none.

**refusals\[].reason** `string`

A readable explanation, for logs or display.

### `transcription.delta`

A partial transcript of the turn in progress. Sent only when `granted.partials` is `true`, about every `partial_interval_s` seconds while new audio arrives. Use `capabilities.partial_style` to decide how to combine them.

Some models decode each partial from only the most recent audio in the turn. Qwen3-ASR uses the last 8 seconds, so on a longer turn its partials show only the end of what was said. The final always covers the whole turn. Partials can also arrive after you flush, until the turn's final arrives.

**transcript** `object`

A [transcript](#transcript-object).

### `transcription.done`

A turn's final transcript.

**transcript** `object`

A [transcript](#transcript-object).

**ended\_by** `string`

`max_turn_length` if the server cut the turn at `max_turn_s`. The speaker didn't necessarily stop, so don't treat it as the end of a sentence. Absent on every other final.

### `transcription.failed`

A turn couldn't be transcribed. The server sends this instead of the turn's `transcription.done`. The session remains open for subsequent turns.

**turn\_id** `integer`

The turn that failed.

**code** `string`

`decode_failed`: the model couldn't transcribe this turn's audio.

**message** `string`

What went wrong.

### `input_audio_buffer.cleared`

The reply to `input_audio_buffer.clear`.

**turn\_id** `integer`

The ID of the discarded turn.

### `error`

Describes the error that causes the server to close the socket. See [errors and close codes](#errors-and-close-codes).

**code** `string`

`invalid_event`, `internal_error`, or `at_capacity`.

**message** `string`

A description of the error.

## Transcript object

`transcription.delta` and `transcription.done` carry the same transcript object.

**turn\_id** `integer`

The turn this transcript belongs to, starting at `0`. Use it to match partials to their final.

**segments** `array`

The transcribed text, in order. A turn with no speech can have no segments, or segments with empty text. Join the text before deciding a turn is empty, and don't assume there's a first segment.

**Properties**

**segments\[].text** `string`

The segment's text. A segment that continues a sentence starts with its own space, so join segments without adding a separator.

**segments\[].start** `float`

Start time, in seconds since the session started.

**segments\[].end** `float`

End time, in seconds since the session started.

**language** `string`

The language the text was transcribed in, as a BCP-47 code from `capabilities.languages`. Absent when it isn't known, for example a turn with no text, or `auto` on a model that doesn't report what it detected. A partial's language can differ from its final's; the final's is the one to keep.

Timings count all the audio sent in the session, including turns discarded with `input_audio_buffer.clear`. Every segment has `start` and `end`, but their precision depends on the model.

## Segmentation modes

Each session uses one mode. `capabilities.segmentation_modes` lists the modes a model supports.

| Mode | What ends a turn | When to use it |
| - | - | - |
| `client_flush` | Your `input_audio_buffer.flush`, or the model's maximum turn length. | Your app already knows where turns end, for example push-to-talk or a voice agent with its own turn detection. |
| `server_vad` | `vad.silence_s` seconds of silence, or an earlier flush. | You want the server to find where turns end in a continuous stream. |

### Long turns

In either mode, the server cuts a turn when it reaches `capabilities.max_turn_s` seconds, so a long stretch of speech becomes several turns. Speech continues in the next turn. A cut turn's `transcription.done` has `ended_by: "max_turn_length"`. Silence the server skips doesn't count toward the limit.

## Errors and close codes

When the server closes the socket because of an error, it first sends an `error` event with the reason. Use the close code to decide what to do:

| Code | Error | Meaning | Retry |
| - | - | - | - |
| `1000` | None | Normal close after your `close` event. Pending work can still be abandoned; confirm turn results before closing. | Not applicable. |
| `1008` | `invalid_event` | The server couldn't accept an event: malformed JSON, an unknown event type or field, a wrong type, an out-of-range value, or a binary frame. | No. Fix the request. |
| `1011` | `internal_error` | The model failed in a way that affects the whole session. | Yes, on a new connection. |
| `1013` | `at_capacity` | The replica was full, and no slot freed up while the server waited. | Yes, after a short wait. |

If a connection reaches a replica before its model has loaded, the server closes with `1013` and no `error` event. Usually Baseten holds the connection until the replica is ready instead; see [connection](#connection).

A rejected API key fails the handshake before the socket opens, so there's no close code. See [connection](#connection).

A failure that affects only one turn doesn't close the session. The server sends `transcription.failed` for that turn instead.

## Transcribe an audio file

Follow [Transcribe with the unified API](https://docs.baseten.co/inference/audio/quickstart.md#transcribe-with-the-unified-api) for a complete Python client, a sample recording, and completion handling.

## Example session

The following `session.created` example is from a Qwen3-ASR unified deployment. Capabilities, limits, and defaults depend on the deployment; read them from your own connection.

The `session.updated` example answers this request. Qwen3-ASR raises the requested partial interval from 0.2 s to its 0.3 s minimum and reports the change in `refusals`:

```json session.update theme={"system"}
{
  "type": "session.update",
  "session": {
    "segmentation_mode": "server_vad",
    "language": "en",
    "partial_interval_s": 0.2
  }
}
```

```json session.created (Qwen3-ASR) theme={"system"}
{
  "type": "session.created",
  "session_id": "sess_68443fdccda44776",
  "capabilities": {
    "input_format": {"encoding": "pcm_s16le", "sample_rate_hz": 16000, "channels": 1},
    "segmentation_modes": ["client_flush", "server_vad"],
    "partials": true,
    "partial_style": "replace",
    "max_sessions_per_replica": 98,
    "vad": {"silence_s": 0.5, "speech_start_s": 0.2, "threshold": 0.7, "min_volume": 0.0},
    "min_partial_interval_s": 0.3,
    "languages": [
      "zh", "en", "yue", "ar", "de", "fr", "es", "pt", "id", "it", "ko",
      "ru", "th", "vi", "ja", "tr", "hi", "ms", "nl", "sv", "da", "fi",
      "pl", "cs", "fil", "fa", "el", "hu", "mk", "ro", "auto"
    ],
    "default_language": "auto",
    "max_turn_s": 29.0
  },
  "defaults": {
    "input_format": {"encoding": "pcm_s16le", "sample_rate_hz": 16000, "channels": 1},
    "segmentation_mode": "client_flush",
    "partials": false,
    "language": "auto",
    "vad": {"silence_s": 0.5, "speech_start_s": 0.2, "threshold": 0.7, "min_volume": 0.0}
  }
}
```

```json session.updated theme={"system"}
{
  "type": "session.updated",
  "session_id": "sess_68443fdccda44776",
  "granted": {
    "input_format": {"encoding": "pcm_s16le", "sample_rate_hz": 16000, "channels": 1},
    "segmentation_mode": "server_vad",
    "partials": true,
    "partial_interval_s": 0.3,
    "language": "en",
    "vad": {"silence_s": 0.5, "speech_start_s": 0.2, "threshold": 0.7, "min_volume": 0.0}
  },
  "refusals": [
    {
      "parameter": "partial_interval_s",
      "requested": 0.2,
      "granted": 0.3,
      "reason": "0.2s is below this backend's floor of 0.3s"
    }
  ]
}
```

```json transcription.done theme={"system"}
{
  "type": "transcription.done",
  "transcript": {
    "turn_id": 0,
    "segments": [{"text": "The quick brown fox.", "start": 0.0, "end": 2.1}],
    "language": "en"
  }
}
```