moss-transcribe-diarize

Speech-to-text

MOSS-Transcribe-Diarize transcribes speech, identifies speakers, and adds timestamps in a single pass. It handles multilingual audio and serves the OpenAI-compatible transcription endpoint, so a client that already calls /v1/audio/transcriptions gets diarized output without changing its request shape.

Setup

Install the Baseten CLI and sign in, then install the OpenAI SDK.

Install and sign in to Baseten

macOS or Linux
Windows

Terminal

brew tap basetenlabs/baseten
brew trust --formula basetenlabs/baseten/baseten
brew install baseten

Download and extract the binary, then move baseten.exe to a directory on your PATH:

Terminal

Invoke-WebRequest `
  https://github.com/basetenlabs/baseten-cli/releases/download/v1.0.0/baseten_1.0.0_windows_amd64.zip `
  -OutFile baseten.zip; Expand-Archive -Force baseten.zip .

For other platforms or a specific version, see the Baseten CLI install reference.

baseten auth login

Install the OpenAI SDK

uv pip install openai

To install without Homebrew, see the Baseten CLI install reference. This preset serves MOSS-Transcribe-Diarize on one H100 with SGLang Omni pinned to v0.1.2, optimized for single-pass transcription that returns speaker labels and timestamps alongside the transcript.

Hardware

H100

Engine

SGLang (dev build)

Context

32K

Concurrency

128

Write the config

Create and move into the project directory:

mkdir moss-transcribe-diarize-latency && cd moss-transcribe-diarize-latency

Then create a file named config.yaml and paste the following:

config.yaml

model_name: moss-transcribe-diarize
model_metadata:
  repo_id: OpenMOSS-Team/MOSS-Transcribe-Diarize
  # No example_model_input on purpose: /v1/audio/transcriptions only accepts
  # multipart/form-data (file upload), and the CI predict smoke POSTs
  # example_model_input as JSON, which the server rejects with 422.
  tags:
    - openai-compatible
    - audio
    - asr
    - diarization
secrets:
  hf_access_token: null
weights:
  - source: "hf://OpenMOSS-Team/MOSS-Transcribe-Diarize@e8681d68e7042738ffca8ac8212bc8fcb1131ab8"
    mount_location: "/app/checkpoint/model"
    auth_secret_name: "hf_access_token"
environment_variables:
  HF_HOME: /app/models/huggingface
  TORCHINDUCTOR_CACHE_DIR: /app/models/inductor
  TRITON_CACHE_DIR: /app/models/triton
base_image:
  # Runtime prerequisites image recommended by the v0.1.2 installation guide,
  # pinned by digest because upstream publishes it under the mutable `dev` tag.
  image: "lmsysorg/sglang-omni:dev@sha256:46235435997d1fa93fc81fb1c2d5b7fd8470d77395a5c348c0176094ffddf95e"
build_commands:
  # Install the exact v0.1.2 release package; its flash-attn-4 dependency is a beta.
  - uv venv /opt/sglang-omni-v0.1.2 --python 3.12
  # Pin the resolved flash-attn-4 beta so rebuilds stay reproducible; its 4.x line
  # publishes only pre-releases, so --prerelease=allow is still required to install it.
  - uv pip install --prerelease=allow --python /opt/sglang-omni-v0.1.2/bin/python "sglang-omni==0.1.2" "flash-attn-4==4.0.0b19"
docker_server:
  start_command: >-
    /opt/sglang-omni-v0.1.2/bin/sgl-omni serve
    --model-path /app/checkpoint/model
    --port 8000
    --max-running-requests 16
    --cuda-graph-max-bs 16
    --mem-fraction-static 0.80
  readiness_endpoint: /health
  liveness_endpoint: /health
  predict_endpoint: /v1/audio/transcriptions
  server_port: 8000
resources:
  accelerator: H100
  cpu: "8"
  memory: 32Gi
  use_gpu: true
runtime:
  predict_concurrency: 128

This preset deploys the MOSS-Transcribe-Diarize checkpoint through SGLang Omni on one H100, pinned to the v0.1.2 release with a digest-locked runtime image. The deployment exposes the OpenAI-compatible transcription endpoint, so a client that already calls /v1/audio/transcriptions gets diarized output without changing its request shape.

Flags

The start_command passes these flags to the engine. Each one controls a runtime or serving behavior:

Flag	Value	What it does
`--max-running-requests`	`16`	Maximum requests the SGLang scheduler runs concurrently. Arrivals past the cap queue instead of running.
`--cuda-graph-max-bs`	`16`	Largest batch size SGLang captures CUDA graphs for. Larger values capture more graphs and use more memory.
`--mem-fraction-static`	`0.80`	Fraction of GPU memory SGLang reserves for model weights and the KV cache pool. Lower it to leave room for activations and CUDA graph buffers, at the cost of peak concurrency.

Deploy

Deploy the model with the Baseten CLI:

Baseten CLI

baseten model push

You should see output similar to:

Pushing model "moss-transcribe-diarize-latency"...
Uploading model...
Uploaded model in 0s
✨ Model moss-transcribe-diarize-latency was successfully pushed ✨

  Model:       moss-transcribe-diarize-latency (abc1d2ef)
  Deployment:  xyz123
  Environment: production

🪵 View logs:
   deployment:   baseten model deployment logs --model-id abc1d2ef --deployment-id xyz123
   environment:  baseten model environment logs --model-id abc1d2ef --environment production  (once deployed)
   app:          https://app.baseten.co/models/abc1d2ef/logs/xyz123

🚀 Invoke your model:
   URL:  https://model-abc1d2ef.api.baseten.co/deployment/xyz123/predict
   CLI:  baseten model predict --model-id abc1d2ef

baseten model push prints your model ID (abc1d2ef in the example). The examples below use it wherever you see {model_id}, and read your API key from the BASETEN_API_KEY environment variable.

Call the model

Your deployment serves the OpenAI-compatible transcription endpoint at /v1/audio/transcriptions. Upload an audio file as multipart form data. The endpoint returns the transcription in the response body.

Python
cURL

main.py

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["BASETEN_API_KEY"],
    base_url="https://model-{model_id}.api.baseten.co/environments/production/sync/v1",
)

with open("audio.wav", "rb") as audio_file:
    response = client.audio.transcriptions.create(
        model="OpenMOSS-Team/MOSS-Transcribe-Diarize",
        file=audio_file,
    )

print(response.text)

curl -s https://model-{model_id}.api.baseten.co/environments/production/sync/v1/audio/transcriptions \
  -H "Authorization: Bearer $BASETEN_API_KEY" \
  -F "file=@audio.wav" \
  -F "model=OpenMOSS-Team/MOSS-Transcribe-Diarize"

Examples

Models

Engines

Custom Docker servers

Custom Python models

Chains

API workflows

Baseten for Model Labs

moss-transcribe-diarize

Setup

Hardware

Engine

Context

Concurrency

Write the config

Flags

Deploy

Call the model

Next steps

Call your model

Autoscaling

​Setup

Hardware

Engine

Context

Concurrency

​Write the config

​Flags

​Deploy

​Call the model

​Next steps

Call your model

Autoscaling

Setup

Write the config

Flags

Deploy

Call the model

Next steps