Real-time transcription API (v2)

{
  "type": "session.created",
  "session_id": "sess_68443fdccda44776",
  "capabilities": {
    "input_format": {"encoding": "pcm_s16le", "sample_rate_hz": 16000, "channels": 1},
    "segmentation_modes": ["client_flush", "server_vad"],
    "partials": true,
    "partial_style": "replace",
    "max_sessions_per_replica": 98,
    "vad": {"silence_s": 0.5, "speech_start_s": 0.2, "threshold": 0.7, "min_volume": 0.0},
    "min_partial_interval_s": 0.3,
    "languages": [
      "zh", "en", "yue", "ar", "de", "fr", "es", "pt", "id", "it", "ko",
      "ru", "th", "vi", "ja", "tr", "hi", "ms", "nl", "sv", "da", "fi",
      "pl", "cs", "fil", "fa", "el", "hu", "mk", "ro", "auto"
    ],
    "default_language": "auto",
    "max_turn_s": 29.0
  },
  "defaults": {
    "input_format": {"encoding": "pcm_s16le", "sample_rate_hz": 16000, "channels": 1},
    "segmentation_mode": "client_flush",
    "partials": false,
    "language": "auto",
    "vad": {"silence_s": 0.5, "speech_start_s": 0.2, "threshold": 0.7, "min_volume": 0.0}
  }
}

{
  "type": "session.updated",
  "session_id": "sess_68443fdccda44776",
  "granted": {
    "input_format": {"encoding": "pcm_s16le", "sample_rate_hz": 16000, "channels": 1},
    "segmentation_mode": "server_vad",
    "partials": true,
    "partial_interval_s": 0.3,
    "language": "en",
    "vad": {"silence_s": 0.5, "speech_start_s": 0.2, "threshold": 0.7, "min_volume": 0.0}
  },
  "refusals": [
    {
      "parameter": "partial_interval_s",
      "requested": 0.2,
      "granted": 0.3,
      "reason": "0.2s is below this backend's floor of 0.3s"
    }
  ]
}

{
  "type": "transcription.done",
  "transcript": {
    "turn_id": 0,
    "segments": [{"text": "The quick brown fox.", "start": 0.0, "end": 2.1}],
    "language": "en"
  }
}

Stream audio to a model deployed with the Baseten speech-to-text (STT) server and receive partial and final transcripts over a WebSocket. This reference defines the shared v2 protocol; for model-specific protocols, see Real-time transcription API (v1).

Deployment compatibility

The deployed server determines the protocol; the WebSocket URL doesn’t contain a protocol version. The Qwen3-ASR Deploy Streaming option in the Model Library uses v1. It doesn’t send session.created and can’t run a v2 client. To use v2, contact Baseten support for a unified STT server deployment. Confirm that it sends session.created before integrating. For a complete client and recording, follow Transcribe with the unified API.

Connection

wss://model-{model_id}.api.baseten.co/environments/production/websocket

This is the production environment’s WebSocket endpoint. To validate a unified deployment before promoting it, use its deployment endpoint:

wss://model-{model_id}.api.baseten.co/deployment/{deployment_id}/websocket

Unified STT presets use a custom Docker server, which requires a published deployment. Development deployments aren’t supported for these presets. For connection behavior, see WebSockets.

Parameters

string

required

The ID of the model you want to call.

string

The published deployment ID. Required when using the deployment endpoint.

string

required

Your Baseten API key: Authorization: Bearer $BASETEN_API_KEY. The server also accepts Api-Key as the scheme.

Baseten checks your API key during the WebSocket handshake. If the check fails, the handshake fails with an HTTP status and the socket never opens: 400 with no Authorization header, or 403 if the key is invalid or can’t access the model. You don’t get an error event or a close code. See how a request reaches your model. The browser’s WebSocket API can’t set the Authorization header. Connect through your backend, which adds the header and keeps your API key on the server. If the deployment has scaled to zero, Baseten holds the connection until a replica is ready, which can take several minutes. If every replica is at its concurrency target, the connection waits for a free slot or a new replica. Set your client’s handshake timeout to allow for this, or set a minimum replica count greater than zero. See what happens when no replica is available.

Session flow

Connect. The server sends session.created with the model’s capabilities and default settings.
Optionally, send session.update to change the settings. The server replies with session.updated, listing what it granted and anything it refused.
Stream audio with input_audio_buffer.append.
End each turn with input_audio_buffer.flush (client_flush), or let a pause end it (server_vad). See segmentation modes.
Receive transcription.done with each turn’s final transcript. If you enable partials, the server also sends transcription.delta while audio arrives.
After every turn has a final result, send close. The server closes the socket with code 1000. A normal close alone doesn’t confirm that all audio was transcribed.

Configure the session before sending audio. To change settings after audio starts, open a new connection.

Client events

Every event is a JSON object with a type field. The server ends the session with an invalid_event error for any unknown type or field, wrong value type, or out-of-range value, including misspelled field names.

`session.update`

Configures the session. If you omit this event, the session uses defaults from session.created. The server reports unsupported settings in session.updated.refusals and the effective settings in session.updated.granted. Include every setting you want to keep in each update. Omitted fields reset to their defaults, except context, which retains its previous value.

object

The format of the audio you send. Each model accepts one format, shown in capabilities.input_format. Send audio in that format; resample or downmix it first if needed. You can leave this field out.

Show Properties

string

required

Always pcm_s16le (16-bit little-endian signed PCM). Any other value is an invalid_event error.

integer

required

The model’s sample rate, for example 16000. The server refuses a different value and uses the model’s value.

integer

required

The model’s channel count, for example 1. The server refuses a different value and uses the model’s value.

string

What ends a turn: client_flush or server_vad. See segmentation modes. Defaults to the first mode in capabilities.segmentation_modes. The server refuses modes the model doesn’t support.

string

The language to transcribe, as a BCP-47 code such as en, or auto to detect it. Defaults to capabilities.default_language.If there’s no exact match, the server drops the region and tries again, so en-US gets en. The server refuses unsupported languages and keeps the default. It also refuses this field if the model doesn’t support language selection. A value longer than 35 characters is an invalid_event error.

float

How often, in seconds, the server sends transcription.delta events during a turn. Partials are off unless you set this.The server raises shorter intervals to capabilities.min_partial_interval_s. A value of 0 or less is an invalid_event error. The server refuses this field if the model doesn’t produce partials.

object

Text that helps the model recognize words from your domain. If you leave it out, the session keeps the context from your last update, not the default. If capabilities.context is absent, the model takes no context and the field is refused. If the model takes only text or only keyterms, the other is refused.

Show Properties

string

A short description of the audio, for example "A cardiology consult.".

string[]

Words and phrases to favor, most important first. The server removes duplicates.

If the model sets capabilities.context.max_words, keyterms are dropped from the end of the list until the context fits, and text is dropped if it’s too long by itself. Each drop is listed as a refusal, and granted.context.truncated is true if any keyterms were dropped. If nothing fits, granted.context is absent.

object

Turn detector settings for server_vad sessions. Every field is optional. Omitted fields use the values from defaults.vad in session.created, which can differ by model or deployment. A value out of range is an invalid_event error.

Show Properties

float

Seconds of silence that end a turn. From 0.1 to 10.0. Values from 0.0 up to 0.1 are raised to 0.1 instead of rejected.

float

Seconds of speech that start a turn. From 0.0 to 10.0.

float

How confident the detector must be that a frame is speech. From 0.0 to 1.0.

float

How loud a frame must be to count as speech. From 0.0 to 1.0. 0 turns the check off.

`input_audio_buffer.append`

Sends audio for the current turn. You can send audio in real time or faster. Under server_vad, use small chunks, such as 100 ms of audio, so the voice detector can observe speech and silence across messages. A large append containing an entire utterance and its trailing silence can miss the turn boundary. Under client_flush, your flush ends the turn, so you can send larger chunks within the WebSocket message size limit.

string

required

Base64-encoded audio in the granted input_format. Invalid base64 is an invalid_event error.

`input_audio_buffer.flush`

Ends the current turn. Takes no other fields: {"type": "input_audio_buffer.flush"}. The server transcribes the turn’s audio and replies with transcription.done, or transcription.failed if it can’t. Keep receiving until that result arrives; closing the session or losing the connection can interrupt pending work. If you flush with no audio, the transcript is empty. Under server_vad, a flush ends the turn without waiting for a pause.

`input_audio_buffer.clear`

Discards the current turn’s audio without transcribing it. Takes no other fields: {"type": "input_audio_buffer.clear"}. The server replies with input_audio_buffer.cleared. The discarded turn still uses up a turn_id, and its audio still counts toward the session’s timestamps.

`close`

Ends the session. Takes no other fields: {"type": "close"}. The server drains pending transcription work for a limited time, then closes the socket with code 1000. The default drain timeout is 5 seconds; deployments can configure it. The server can abandon work that exceeds the timeout even when the close code is 1000. Flush the last turn and wait for every expected transcription.done before sending close. Treat transcription.failed, a receive timeout, or closure before an expected result as incomplete transcription. Use a receive timeout appropriate for your workload. Without a flush, the server transcribes the last turn only if its voice detector heard speech in it, in either segmentation mode. Audio it scored as silence is dropped.

Server events

`session.created`

The first message on every connection. If the replica is full, you get an at_capacity error instead.

string

The session’s ID. Include it in support requests.

object

The model’s supported settings and limits. Unsupported optional fields are absent.

Show Properties

object

The only audio format the model accepts, with encoding, sample_rate_hz, and channels.

string[]

The modes the model supports: client_flush, server_vad, or both. The first is the default.

string[]

The languages the model transcribes, as BCP-47 codes, plus auto if it can detect the language. Empty if the model takes no language.

string

The language a session gets if it doesn’t ask for one. Absent if languages is empty.

boolean

Whether the model can send partial transcripts before a turn ends.

string

How partials relate to each other. replace: each partial supersedes the previous one for its turn, and its text can change as more audio arrives, so render the latest and discard the rest. Some models decode a partial from only the latest audio; see transcription.delta. append: each partial continues the previous one. no_partials: none are sent.

float

The shortest partial interval the model grants.

float

The longest a turn can be, in seconds. A turn that reaches it is cut, and the speech continues in the next turn. See long turns. Absent if there’s no limit.

integer

Maximum simultaneous sessions per replica. The replica refuses sessions beyond this limit. Baseten’s routing concurrency target can limit sessions below this value.

object

The context the model accepts. text and keyterms (boolean) say which kinds it takes. max_words (integer), if present, limits their combined length in words. Absent if the model takes no context.

object

The model’s preferred turn detector settings, if it has any.

object

The settings the session uses if you don’t send session.update. Same shape as granted in session.updated.

`session.updated`

The reply to every session.update.

string

The session’s ID.

object

The effective session settings. Use these values to configure your client.

Show Properties

object

The audio format: encoding, sample_rate_hz, and channels.

string

client_flush or server_vad.

string

The language, as a BCP-47 code, or auto. Absent if the model takes no language.

boolean

Whether the session gets transcription.delta events.

float

The partial interval. Absent when partials is false.

object

The context in use: text (string), keyterms (string[]), and truncated (boolean, true if keyterms were dropped to fit the word limit). Absent if the session has no context.

object

The turn detector settings: silence_s, speech_start_s, threshold, and min_volume. Always present. Under client_flush, the detector runs but doesn’t end turns.

array

Settings that differ from your request. An empty array means the server granted every requested setting.

Show Properties

string

The refused setting, for example partial_interval_s or context.keyterms.

any

The requested value.

any

The granted value. Absent if the setting wasn’t applied, for example a language on a model that takes none.

string

A readable explanation, for logs or display.

`transcription.delta`

A partial transcript of the turn in progress. Sent only when granted.partials is true, about every partial_interval_s seconds while new audio arrives. Use capabilities.partial_style to decide how to combine them. Some models decode each partial from only the most recent audio in the turn. Qwen3-ASR uses the last 8 seconds, so on a longer turn its partials show only the end of what was said. The final always covers the whole turn. Partials can also arrive after you flush, until the turn’s final arrives.

object

A transcript.

`transcription.done`

A turn’s final transcript.

object

A transcript.

string

max_turn_length if the server cut the turn at max_turn_s. The speaker didn’t necessarily stop, so don’t treat it as the end of a sentence. Absent on every other final.

`transcription.failed`

A turn couldn’t be transcribed. The server sends this instead of the turn’s transcription.done. The session remains open for subsequent turns.

integer

The turn that failed.

string

decode_failed: the model couldn’t transcribe this turn’s audio.

string

What went wrong.

`input_audio_buffer.cleared`

The reply to input_audio_buffer.clear.

integer

The ID of the discarded turn.

`error`

Describes the error that causes the server to close the socket. See errors and close codes.

string

invalid_event, internal_error, or at_capacity.

string

A description of the error.

Transcript object

transcription.delta and transcription.done carry the same transcript object.

integer

The turn this transcript belongs to, starting at 0. Use it to match partials to their final.

array

The transcribed text, in order. A turn with no speech can have no segments, or segments with empty text. Join the text before deciding a turn is empty, and don’t assume there’s a first segment.

Show Properties

string

The segment’s text. A segment that continues a sentence starts with its own space, so join segments without adding a separator.

float

Start time, in seconds since the session started.

float

End time, in seconds since the session started.

string

The language the text was transcribed in, as a BCP-47 code from capabilities.languages. Absent when it isn’t known, for example a turn with no text, or auto on a model that doesn’t report what it detected. A partial’s language can differ from its final’s; the final’s is the one to keep.

Timings count all the audio sent in the session, including turns discarded with input_audio_buffer.clear. Every segment has start and end, but their precision depends on the model.

Segmentation modes

Each session uses one mode. capabilities.segmentation_modes lists the modes a model supports.

Mode	What ends a turn	When to use it
`client_flush`	Your `input_audio_buffer.flush`, or the model’s maximum turn length.	Your app already knows where turns end, for example push-to-talk or a voice agent with its own turn detection.
`server_vad`	`vad.silence_s` seconds of silence, or an earlier flush.	You want the server to find where turns end in a continuous stream.

Long turns

In either mode, the server cuts a turn when it reaches capabilities.max_turn_s seconds, so a long stretch of speech becomes several turns. Speech continues in the next turn. A cut turn’s transcription.done has ended_by: "max_turn_length". Silence the server skips doesn’t count toward the limit.

Errors and close codes

When the server closes the socket because of an error, it first sends an error event with the reason. Use the close code to decide what to do:

Code	Error	Meaning	Retry
`1000`	None	Normal close after your `close` event. Pending work can still be abandoned; confirm turn results before closing.	Not applicable.
`1008`	`invalid_event`	The server couldn’t accept an event: malformed JSON, an unknown event type or field, a wrong type, an out-of-range value, or a binary frame.	No. Fix the request.
`1011`	`internal_error`	The model failed in a way that affects the whole session.	Yes, on a new connection.
`1013`	`at_capacity`	The replica was full, and no slot freed up while the server waited.	Yes, after a short wait.

If a connection reaches a replica before its model has loaded, the server closes with 1013 and no error event. Usually Baseten holds the connection until the replica is ready instead; see connection. A rejected API key fails the handshake before the socket opens, so there’s no close code. See connection. A failure that affects only one turn doesn’t close the session. The server sends transcription.failed for that turn instead.

Transcribe an audio file

Follow Transcribe with the unified API for a complete Python client, a sample recording, and completion handling.

Example session

The following session.created example is from a Qwen3-ASR unified deployment. Capabilities, limits, and defaults depend on the deployment; read them from your own connection. The session.updated example answers this request. Qwen3-ASR raises the requested partial interval from 0.2 s to its 0.3 s minimum and reports the change in refusals:

session.update

{
  "type": "session.update",
  "session": {
    "segmentation_mode": "server_vad",
    "language": "en",
    "partial_interval_s": 0.2
  }
}

{
  "type": "session.created",
  "session_id": "sess_68443fdccda44776",
  "capabilities": {
    "input_format": {"encoding": "pcm_s16le", "sample_rate_hz": 16000, "channels": 1},
    "segmentation_modes": ["client_flush", "server_vad"],
    "partials": true,
    "partial_style": "replace",
    "max_sessions_per_replica": 98,
    "vad": {"silence_s": 0.5, "speech_start_s": 0.2, "threshold": 0.7, "min_volume": 0.0},
    "min_partial_interval_s": 0.3,
    "languages": [
      "zh", "en", "yue", "ar", "de", "fr", "es", "pt", "id", "it", "ko",
      "ru", "th", "vi", "ja", "tr", "hi", "ms", "nl", "sv", "da", "fi",
      "pl", "cs", "fil", "fa", "el", "hu", "mk", "ro", "auto"
    ],
    "default_language": "auto",
    "max_turn_s": 29.0
  },
  "defaults": {
    "input_format": {"encoding": "pcm_s16le", "sample_rate_hz": 16000, "channels": 1},
    "segmentation_mode": "client_flush",
    "partials": false,
    "language": "auto",
    "vad": {"silence_s": 0.5, "speech_start_s": 0.2, "threshold": 0.7, "min_volume": 0.0}
  }
}

{
  "type": "session.updated",
  "session_id": "sess_68443fdccda44776",
  "granted": {
    "input_format": {"encoding": "pcm_s16le", "sample_rate_hz": 16000, "channels": 1},
    "segmentation_mode": "server_vad",
    "partials": true,
    "partial_interval_s": 0.3,
    "language": "en",
    "vad": {"silence_s": 0.5, "speech_start_s": 0.2, "threshold": 0.7, "min_volume": 0.0}
  },
  "refusals": [
    {
      "parameter": "partial_interval_s",
      "requested": 0.2,
      "granted": 0.3,
      "reason": "0.2s is below this backend's floor of 0.3s"
    }
  ]
}

{
  "type": "transcription.done",
  "transcript": {
    "turn_id": 0,
    "segments": [{"text": "The quick brown fox.", "start": 0.0, "end": 2.1}],
    "language": "en"
  }
}

⌘I

{
  "type": "session.created",
  "session_id": "sess_68443fdccda44776",
  "capabilities": {
    "input_format": {"encoding": "pcm_s16le", "sample_rate_hz": 16000, "channels": 1},
    "segmentation_modes": ["client_flush", "server_vad"],
    "partials": true,
    "partial_style": "replace",
    "max_sessions_per_replica": 98,
    "vad": {"silence_s": 0.5, "speech_start_s": 0.2, "threshold": 0.7, "min_volume": 0.0},
    "min_partial_interval_s": 0.3,
    "languages": [
      "zh", "en", "yue", "ar", "de", "fr", "es", "pt", "id", "it", "ko",
      "ru", "th", "vi", "ja", "tr", "hi", "ms", "nl", "sv", "da", "fi",
      "pl", "cs", "fil", "fa", "el", "hu", "mk", "ro", "auto"
    ],
    "default_language": "auto",
    "max_turn_s": 29.0
  },
  "defaults": {
    "input_format": {"encoding": "pcm_s16le", "sample_rate_hz": 16000, "channels": 1},
    "segmentation_mode": "client_flush",
    "partials": false,
    "language": "auto",
    "vad": {"silence_s": 0.5, "speech_start_s": 0.2, "threshold": 0.7, "min_volume": 0.0}
  }
}

{
  "type": "session.updated",
  "session_id": "sess_68443fdccda44776",
  "granted": {
    "input_format": {"encoding": "pcm_s16le", "sample_rate_hz": 16000, "channels": 1},
    "segmentation_mode": "server_vad",
    "partials": true,
    "partial_interval_s": 0.3,
    "language": "en",
    "vad": {"silence_s": 0.5, "speech_start_s": 0.2, "threshold": 0.7, "min_volume": 0.0}
  },
  "refusals": [
    {
      "parameter": "partial_interval_s",
      "requested": 0.2,
      "granted": 0.3,
      "reason": "0.2s is below this backend's floor of 0.3s"
    }
  ]
}

{
  "type": "transcription.done",
  "transcript": {
    "turn_id": 0,
    "segments": [{"text": "The quick brown fox.", "start": 0.0, "end": 2.1}],
    "language": "en"
  }
}

Reference

Inference API

Management API

Loops API

Training API

Frontier Gateway API

Sandbox runtime API

CLI reference

SDKs and clients

Configuration

Metrics

Integrations

Real-time transcription API (v2)

Deployment compatibility

Connection

Parameters

Session flow

Client events

`session.update`

`input_audio_buffer.append`

`input_audio_buffer.flush`

`input_audio_buffer.clear`

`close`

Server events

`session.created`

`session.updated`

`transcription.delta`

`transcription.done`

`transcription.failed`

`input_audio_buffer.cleared`

`error`

Transcript object

Segmentation modes

Long turns

Errors and close codes

Transcribe an audio file

Example session

​Deployment compatibility

​Connection

​Parameters

​Session flow

​Client events

​session.update

​input_audio_buffer.append

​input_audio_buffer.flush

​input_audio_buffer.clear

​close

​Server events

​session.created

​session.updated

​transcription.delta

​transcription.done

​transcription.failed

​input_audio_buffer.cleared

​error

​Transcript object

​Segmentation modes

​Long turns

​Errors and close codes

​Transcribe an audio file

​Example session

Deployment compatibility

Connection

Parameters

Session flow

Client events

`session.update`

`input_audio_buffer.append`

`input_audio_buffer.flush`

`input_audio_buffer.clear`

`close`

Server events

`session.created`

`session.updated`

`transcription.delta`

`transcription.done`

`transcription.failed`

`input_audio_buffer.cleared`

`error`

Transcript object

Segmentation modes

Long turns

Errors and close codes

Transcribe an audio file

Example session