> ## Documentation Index
> Fetch the complete documentation index at: https://docs.baseten.co/llms.txt
> Use this file to discover all available pages before exploring further.

# Qwen3-ASR

> Alibaba's Qwen3-ASR is a compact 1.7B speech-to-text model with multilingual transcription support.

<div className="capability-pills">
  <a href="/examples/models/capabilities/speech-to-text" className="capability-pill">Speech-to-text</a>
</div>

Alibaba's Qwen3-ASR is a compact 1.7B speech-to-text model with multilingual transcription support.

## Setup

Install the Baseten CLI and sign in, then install the OpenAI SDK.

<Columns cols={2}>
  <Column>
    **Install and sign in to Baseten**

    <Tabs>
      <Tab title="macOS or Linux">
        ```bash Terminal theme={"system"}
        brew tap basetenlabs/baseten
        brew install baseten
        ```
      </Tab>

      <Tab title="Windows">
        Download and extract the binary, then move `baseten.exe` to a directory on your `PATH`:

        ```powershell Terminal theme={"system"}
        Invoke-WebRequest `
          https://github.com/basetenlabs/baseten-cli/releases/download/v0.4.0/baseten_0.4.0_windows_amd64.zip `
          -OutFile baseten.zip; Expand-Archive -Force baseten.zip .
        ```
      </Tab>
    </Tabs>

    For other platforms or a specific version, see the [Baseten CLI install reference](/reference/cli/baseten/overview#install).

    ```sh theme={"system"}
    baseten auth login
    ```
  </Column>

  <Column>
    **Install the OpenAI SDK**

    ```sh theme={"system"}
    uv pip install openai
    ```
  </Column>
</Columns>

Prefer not to install? Sign in with `uvx truss login --browser` and deploy with `uvx truss push`.

This preset serves Qwen3-ASR on a single RTX PRO 6000 through vLLM, tuned for fast multilingual transcription.

<CardGroup cols={3}>
  <Card title="Hardware" icon="microchip">RTX-PRO-6000</Card>
  <Card title="Engine" icon="server">vLLM (0.22.0-cu129 build)</Card>
  <Card title="Concurrency" icon="layer-group">256</Card>
</CardGroup>

## Write the config

Create and move into the project directory:

```sh theme={"system"}
mkdir qwen3-asr-1.7b-latency && cd qwen3-asr-1.7b-latency
```

Then create a file named `config.yaml` and paste the following:

```yaml config.yaml theme={"system"}
model_name: "model:qwen3-asr-1.7b preset:latency"
model_metadata:
  repo_id: Qwen/Qwen3-ASR-1.7B
  example_model_input:
    stream: false
    messages:
      - role: user
        content:
          - type: audio_url
            audio_url:
              url: https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_en.wav
  tags:
    - openai-compatible
secrets:
  hf_access_token: null
weights:
  - source: "hf://Qwen/Qwen3-ASR-1.7B@main"
    mount_location: "/app/checkpoint/model"
    auth_secret_name: "hf_access_token"
base_image:
  image: vllm/vllm-openai:v0.22.0-cu129
docker_server:
  start_command: sh -c "vllm serve $BASETEN_MODEL_PATH --tensor-parallel-size 1 --served-model-name Qwen/Qwen3-ASR-1.7B --gpu-memory-utilization 0.8 --host 0.0.0.0 --port 8000 --load-format runai_streamer"
  readiness_endpoint: /health
  liveness_endpoint: /health
  predict_endpoint: /v1/chat/completions
  server_port: 8000
resources:
  instance_type: RTX-PRO-6000
requirements:
  - vllm[audio]
  - librosa
  - torch
  - torchaudio
  - pynvml
  - ffmpeg-python
environment_variables:
  BASETEN_MODEL_PATH: "/app/checkpoint/model"
system_packages:
  # No python3.10-venv here: nothing in this truss builds a venv (vLLM is started
  # via docker_server), and pinning a Python minor version in apt ties the build to
  # whatever the base image happens to ship. That pin already broke the community-1
  # diarizer when its base moved to Debian trixie:
  #   E: Unable to locate package python3.10-venv
  - ffmpeg
  - openmpi-bin
  - libopenmpi-dev
runtime:
  predict_concurrency: 256
```

## Flags

The `start_command` passes these flags to the engine. Each one controls a runtime or serving behavior:

| Flag                       | Value            | What it does                                                                                                   |
| -------------------------- | ---------------- | -------------------------------------------------------------------------------------------------------------- |
| `--tensor-parallel-size`   | `1`              | Number of GPUs to shard the model across.                                                                      |
| `--gpu-memory-utilization` | `0.8`            | Fraction of GPU memory vLLM may use for weights and KV cache.                                                  |
| `--load-format`            | `runai_streamer` | Weight loading backend. **runai\_streamer:** Stream weights from object storage without materializing to disk. |

## Deploy

Push the config to Baseten with the Baseten CLI, or with the Truss CLI if you prefer it:

<CodeGroup>
  ```sh Baseten CLI theme={"system"}
  baseten model push
  ```

  ```sh Truss CLI theme={"system"}
  uvx truss push
  ```
</CodeGroup>

You should see output similar to:

```output theme={"system"}
Pushing model "qwen3-asr-1.7b-latency"...
Uploading model...
Uploaded model in 0s
✨ Model qwen3-asr-1.7b-latency was successfully pushed ✨

  Model:       qwen3-asr-1.7b-latency (abc1d2ef)
  Deployment:  xyz123
  Environment: production

🪵 View logs:
   deployment:   baseten model deployment logs --model-id abc1d2ef --deployment-id xyz123
   environment:  baseten model environment logs --model-id abc1d2ef --environment production  (once deployed)
   app:          https://app.baseten.co/models/abc1d2ef/logs/xyz123

🚀 Invoke your model:
   URL:  https://model-abc1d2ef.api.baseten.co/deployment/xyz123/predict
   CLI:  baseten model predict --model-id abc1d2ef
```

`baseten model push` prints your **model ID** (`abc1d2ef` in the example). The examples below use it wherever you see `{model_id}`, and read your API key from the `BASETEN_API_KEY` environment variable.

## Call the model

Your deployment serves an OpenAI-compatible chat completions API at `/v1/chat/completions` that accepts audio inputs.

Send audio as an `audio_url` content item on a chat message. The model returns the transcription as the assistant message content.

<Tabs>
  <Tab title="Python">
    ```python main.py theme={"system"}
    import os
    from openai import OpenAI

    client = OpenAI(
        api_key=os.environ["BASETEN_API_KEY"],
        base_url="https://model-{model_id}.api.baseten.co/environments/production/sync/v1",
    )

    response = client.chat.completions.create(
        model="Qwen/Qwen3-ASR-1.7B",
        messages=[
            {
                "role": "user",
                "content": [
                    {
                        "type": "audio_url",
                        "audio_url": {
                            "url": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_en.wav"
                        },
                    }
                ],
            }
        ],
    )

    print(response.choices[0].message.content)
    ```
  </Tab>

  <Tab title="cURL">
    ```sh theme={"system"}
    curl -s https://model-{model_id}.api.baseten.co/environments/production/sync/v1/chat/completions \
      -H "Content-Type: application/json" \
      -H "Authorization: Bearer $BASETEN_API_KEY" \
      -d '{
        "model": "Qwen/Qwen3-ASR-1.7B",
        "messages": [
          {"role": "user", "content": [
            {"type": "audio_url", "audio_url": {"url": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_en.wav"}}
          ]}
        ]
      }'
    ```
  </Tab>
</Tabs>

## Next steps

<CardGroup cols={2}>
  <Card title="Call your model" icon="code" href="/inference/calling-your-model">
    Endpoint anatomy, authentication, and sync versus async inference
  </Card>

  <Card title="Autoscaling" icon="arrow-up-right-dots" href="/deployment/autoscaling/overview">
    Scale replicas with traffic, including scale to zero
  </Card>
</CardGroup>