VibeVoice

Speech-to-text

Microsoft’s VibeVoice-ASR speech-to-text model, returning JSON segments with speaker labels and timestamps through an OpenAI-compatible API.

Setup

Install the Baseten CLI and sign in, then install the OpenAI SDK.

Install and sign in to Baseten

macOS or Linux
Windows

Terminal

brew tap basetenlabs/baseten
brew trust --formula basetenlabs/baseten/baseten
brew install baseten

Download and extract the binary, then move baseten.exe to a directory on your PATH:

Terminal

Invoke-WebRequest `
  https://github.com/basetenlabs/baseten-cli/releases/download/v1.1.0/baseten_1.1.0_windows_amd64.zip `
  -OutFile baseten.zip; Expand-Archive -Force baseten.zip .

For other platforms or a specific version, see the Baseten CLI install reference.

baseten auth login

Install the OpenAI SDK

uv pip install openai

To install without Homebrew, see the Baseten CLI install reference. This preset serves VibeVoice-ASR on a single H100 through vLLM with an OpenAI-compatible chat completions endpoint, tuned for low-latency transcription with speaker labels and timestamps.

Hardware

H100

Engine

vLLM 0.14.1

Concurrency

Write the config

Create and move into the project directory:

mkdir vibevoice-asr-latency && cd vibevoice-asr-latency

Then create a file named config.yaml and paste the following:

config.yaml

model_name: "model:vibevoice-asr preset:latency"
python_version: py310

model_metadata:
  repo_id: microsoft/VibeVoice-ASR
  tags:
    - openai-compatible
    - audio
    - asr
    - speech-to-text
  example_model_input:
    model: vibevoice
    messages:
      - role: system
        content: You are a helpful assistant that transcribes audio input into text output in JSON format.
      - role: user
        content:
          - type: audio_url
            audio_url:
              url: https://github.com/ggerganov/whisper.cpp/raw/master/samples/jfk.wav
          - type: text
            text: Transcribe this audio.
    max_tokens: 64
    temperature: 0.0

base_image:
  image: vllm/vllm-openai:v0.14.1
  python_executable_path: /usr/bin/python3

environment_variables:
  HF_HOME: /cache/org
  HF_HUB_CACHE: /cache/org
  TRANSFORMERS_CACHE: /cache/org
  VIBEVOICE_FFMPEG_MAX_CONCURRENCY: "64"
  VLLM_MEDIA_LOADING_THREAD_COUNT: "16"
  PYTORCH_ALLOC_CONF: "expandable_segments:True"

requirements:
  - transformers==4.57.6
  - accelerate>=0.30.0
  - safetensors
  - huggingface-hub>=0.23.0
  - librosa>=0.10.0
  - soundfile
  - scipy
  - pydub
  - diffusers
  - git+https://github.com/microsoft/VibeVoice.git@main

resources:
  accelerator: H100
  cpu: "4"
  memory: 32Gi
  use_gpu: true

runtime:
  predict_concurrency: 32

system_packages:
  - ffmpeg
  - git

# Weights are pre-downloaded at build time and mounted at /models/vibevoice-asr,
# so cold starts skip the 9.2 GB HF download entirely.
weights:
  - source: "hf://microsoft/VibeVoice-ASR@d0c9efdb8d614685062c04425d91e01b6f37d944"
    mount_location: "/models/vibevoice-asr"

# Pass-through mode: no model.py, Truss just runs vllm serve and proxies
# /predict requests to /v1/chat/completions on the container's localhost.
docker_server:
  server_port: 8000
  predict_endpoint: /v1/chat/completions
  readiness_endpoint: /v1/models
  liveness_endpoint: /v1/models
  start_command: bash /app/data/start.sh

This config runs the vllm/vllm-openai:v0.14.1 image with Microsoft’s VibeVoice plugin patches applied at startup, serving weights pre-mounted at /models/vibevoice-asr so cold starts skip the 9.2 GB Hugging Face download. The server runs in eager mode with a 32k context and up to 16 concurrent sequences, exposing the model as vibevoice on the chat completions endpoint.

Deploy

Deploy the model with the Baseten CLI:

Baseten CLI

baseten model push

You should see output similar to:

Pushing model "vibevoice-asr-latency"...
Uploading model...
Uploaded model in 0s
✨ Model vibevoice-asr-latency was successfully pushed ✨

  Model:       vibevoice-asr-latency (abc1d2ef)
  Deployment:  xyz123
  Environment: production

🪵 View logs:
   deployment:   baseten model deployment logs --model-id abc1d2ef --deployment-id xyz123
   environment:  baseten model environment logs --model-id abc1d2ef --environment production  (once deployed)
   app:          https://app.baseten.co/models/abc1d2ef/logs/xyz123

🚀 Invoke your model:
   URL:  https://model-abc1d2ef.api.baseten.co/deployment/xyz123/predict
   CLI:  baseten model predict --model-id abc1d2ef

baseten model push prints your model ID (abc1d2ef in the example). The examples below use it wherever you see {model_id}, and read your API key from the BASETEN_API_KEY environment variable.

Call the model

Your deployment serves an OpenAI-compatible chat completions API at /v1/chat/completions that accepts audio inputs. Send audio as an audio_url content item on a chat message. The model returns the transcription as the assistant message content.

Python
cURL

main.py

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["BASETEN_API_KEY"],
    base_url="https://model-{model_id}.api.baseten.co/environments/production/sync/v1",
)

response = client.chat.completions.create(
    model="vibevoice",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "audio_url",
                    "audio_url": {
                        "url": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_en.wav"
                    },
                }
            ],
        }
    ],
)

print(response.choices[0].message.content)

curl -s https://model-{model_id}.api.baseten.co/environments/production/sync/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $BASETEN_API_KEY" \
  -d '{
    "model": "vibevoice",
    "messages": [
      {"role": "user", "content": [
        {"type": "audio_url", "audio_url": {"url": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_en.wav"}}
      ]}
    ]
  }'

Examples

Models

Engines

Custom Docker servers

Custom Python models

Chains

API workflows

Baseten for Model Labs

Setup

Hardware

Engine

Concurrency

Write the config

Deploy

Call the model

Next steps

Call your model

Autoscaling

​Setup

Hardware

Engine

Concurrency

​Write the config

​Deploy

​Call the model

​Next steps

Call your model

Autoscaling

Setup

Write the config

Deploy

Call the model

Next steps