Qwen3-ASR

Alibaba’s Qwen3-ASR is a compact 1.7B speech-to-text model with multilingual transcription support.

Setup

Install the Baseten CLI and sign in, then install the OpenAI SDK.

Install and sign in to Baseten

macOS or Linux
Windows

Terminal

brew tap basetenlabs/baseten
brew trust --formula basetenlabs/baseten/baseten
brew install baseten

Download and extract the binary, then move baseten.exe to a directory on your PATH:

Terminal

Invoke-WebRequest `
  https://github.com/basetenlabs/baseten-cli/releases/download/v1.0.0/baseten_1.0.0_windows_amd64.zip `
  -OutFile baseten.zip; Expand-Archive -Force baseten.zip .

For other platforms or a specific version, see the Baseten CLI install reference.

baseten auth login

Install the OpenAI SDK

uv pip install openai

To install without Homebrew, see the Baseten CLI install reference. This preset serves Qwen3-ASR on a single RTX PRO 6000 through vLLM, tuned for fast multilingual transcription.

Hardware

RTX-PRO-6000

Engine

vLLM (0.22.0-cu129 build)

Concurrency

256

Write the config

Create and move into the project directory:

mkdir qwen3-asr-1.7b-latency && cd qwen3-asr-1.7b-latency

Then create a file named config.yaml and paste the following:

config.yaml

model_name: "model:qwen3-asr-1.7b preset:latency"
model_metadata:
  repo_id: Qwen/Qwen3-ASR-1.7B
  example_model_input:
    stream: false
    messages:
      - role: user
        content:
          - type: audio_url
            audio_url:
              url: https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_en.wav
  tags:
    - openai-compatible
weights:
  - source: "hf://Qwen/Qwen3-ASR-1.7B@main"
    mount_location: "/app/checkpoint/model"
base_image:
  image: vllm/vllm-openai:v0.22.0-cu129
docker_server:
  start_command: sh -c "vllm serve $BASETEN_MODEL_PATH --tensor-parallel-size 1 --served-model-name Qwen/Qwen3-ASR-1.7B --gpu-memory-utilization 0.8 --host 0.0.0.0 --port 8000 --load-format runai_streamer"
  readiness_endpoint: /health
  liveness_endpoint: /health
  predict_endpoint: /v1/chat/completions
  server_port: 8000
resources:
  instance_type: RTX-PRO-6000
requirements:
  - vllm[audio]
  - librosa
  - torch
  - torchaudio
  - pynvml
  - ffmpeg-python
environment_variables:
  BASETEN_MODEL_PATH: "/app/checkpoint/model"
system_packages:
  # No python3.10-venv here: nothing in this truss builds a venv (vLLM is started
  # via docker_server), and pinning a Python minor version in apt ties the build to
  # whatever the base image happens to ship. That pin already broke the community-1
  # diarizer when its base moved to Debian trixie:
  #   E: Unable to locate package python3.10-venv
  - ffmpeg
  - openmpi-bin
  - libopenmpi-dev
runtime:
  predict_concurrency: 256

Flags

The start_command passes these flags to the engine. Each one controls a runtime or serving behavior:

Flag	Value	What it does
`--tensor-parallel-size`	`1`	Number of GPUs to shard the model across.
`--gpu-memory-utilization`	`0.8`	Fraction of GPU memory vLLM may use for weights and KV cache.
`--load-format`	`runai_streamer`	Weight loading backend. runai_streamer: Stream weights from object storage without materializing to disk.

Deploy

Deploy the model with the Baseten CLI:

Baseten CLI

baseten model push

You should see output similar to:

Pushing model "qwen3-asr-1.7b-latency"...
Uploading model...
Uploaded model in 0s
✨ Model qwen3-asr-1.7b-latency was successfully pushed ✨

  Model:       qwen3-asr-1.7b-latency (abc1d2ef)
  Deployment:  xyz123
  Environment: production

🪵 View logs:
   deployment:   baseten model deployment logs --model-id abc1d2ef --deployment-id xyz123
   environment:  baseten model environment logs --model-id abc1d2ef --environment production  (once deployed)
   app:          https://app.baseten.co/models/abc1d2ef/logs/xyz123

🚀 Invoke your model:
   URL:  https://model-abc1d2ef.api.baseten.co/deployment/xyz123/predict
   CLI:  baseten model predict --model-id abc1d2ef

baseten model push prints your model ID (abc1d2ef in the example). The examples below use it wherever you see {model_id}, and read your API key from the BASETEN_API_KEY environment variable.

Call the model

Your deployment serves an OpenAI-compatible chat completions API at /v1/chat/completions that accepts audio inputs. Send audio as an audio_url content item on a chat message. The model returns the transcription as the assistant message content.

Python
cURL

main.py

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["BASETEN_API_KEY"],
    base_url="https://model-{model_id}.api.baseten.co/environments/production/sync/v1",
)

response = client.chat.completions.create(
    model="Qwen/Qwen3-ASR-1.7B",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "audio_url",
                    "audio_url": {
                        "url": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_en.wav"
                    },
                }
            ],
        }
    ],
)

print(response.choices[0].message.content)

curl -s https://model-{model_id}.api.baseten.co/environments/production/sync/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $BASETEN_API_KEY" \
  -d '{
    "model": "Qwen/Qwen3-ASR-1.7B",
    "messages": [
      {"role": "user", "content": [
        {"type": "audio_url", "audio_url": {"url": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_en.wav"}}
      ]}
    ]
  }'

Examples

Models

Engines

Custom Docker servers

Custom Python models

Chains

Setup

Hardware

Engine

Concurrency

Write the config

Flags

Deploy

Call the model

Next steps

Call your model

Autoscaling

​Setup

Hardware

Engine

Concurrency

​Write the config

​Flags

​Deploy

​Call the model

​Next steps

Call your model

Autoscaling

Setup

Write the config

Flags

Deploy

Call the model

Next steps