> ## Documentation Index > Fetch the complete documentation index at: https://docs.baseten.co/llms.txt > Use this file to discover all available pages before exploring further. # moss-transcribe-diarize > MOSS-Transcribe-Diarize transcribes speech, identifies speakers, and adds timestamps in a single pass.
Speech-to-text
MOSS-Transcribe-Diarize transcribes speech, identifies speakers, and adds timestamps in a single pass. It handles multilingual audio and serves the OpenAI-compatible transcription endpoint, so a client that already calls `/v1/audio/transcriptions` gets diarized output without changing its request shape. ## Setup Sign in to Baseten with Truss, then install the OpenAI SDK. **Sign in to Baseten** ```sh theme={"system"} uvx truss login --browser ``` **Install the OpenAI SDK** ```sh theme={"system"} uv pip install openai ``` This preset serves MOSS-Transcribe-Diarize on one H100, with SGLang Omni pinned to v0.1.2, optimized for single-pass transcription that returns speaker labels and timestamps alongside the transcript. H100 SGLang (46235435997d1fa9... build) 128 ## Write the config Create and move into the project directory: ```sh theme={"system"} mkdir moss-transcribe-diarize-latency && cd moss-transcribe-diarize-latency ``` Then create a file named `config.yaml` and paste the following: ```yaml config.yaml theme={"system"} model_name: moss-transcribe-diarize model_metadata: repo_id: OpenMOSS-Team/MOSS-Transcribe-Diarize # No example_model_input on purpose: /v1/audio/transcriptions only accepts # multipart/form-data (file upload), and the CI predict smoke POSTs # example_model_input as JSON, which the server rejects with 422. tags: - openai-compatible - audio - asr - diarization secrets: hf_access_token: null weights: - source: "hf://OpenMOSS-Team/MOSS-Transcribe-Diarize@e8681d68e7042738ffca8ac8212bc8fcb1131ab8" mount_location: "/app/checkpoint/model" auth_secret_name: "hf_access_token" environment_variables: HF_HOME: /app/models/huggingface TORCHINDUCTOR_CACHE_DIR: /app/models/inductor TRITON_CACHE_DIR: /app/models/triton base_image: # Runtime prerequisites image recommended by the v0.1.2 installation guide, # pinned by digest because upstream publishes it under the mutable `dev` tag. image: "lmsysorg/sglang-omni:dev@sha256:46235435997d1fa93fc81fb1c2d5b7fd8470d77395a5c348c0176094ffddf95e" build_commands: # Install the exact v0.1.2 release package; its flash-attn-4 dependency is a beta. - uv venv /opt/sglang-omni-v0.1.2 --python 3.12 # Pin the resolved flash-attn-4 beta so rebuilds stay reproducible; its 4.x line # publishes only pre-releases, so --prerelease=allow is still required to install it. - uv pip install --prerelease=allow --python /opt/sglang-omni-v0.1.2/bin/python "sglang-omni==0.1.2" "flash-attn-4==4.0.0b19" docker_server: start_command: >- /opt/sglang-omni-v0.1.2/bin/sgl-omni serve --model-path /app/checkpoint/model --port 8000 --max-running-requests 16 --cuda-graph-max-bs 16 --mem-fraction-static 0.80 readiness_endpoint: /health liveness_endpoint: /health predict_endpoint: /v1/audio/transcriptions server_port: 8000 resources: accelerator: H100 cpu: "8" memory: 32Gi use_gpu: true runtime: predict_concurrency: 128 ``` The deployment exposes `/v1/audio/transcriptions`, the same OpenAI-compatible route a plain Whisper deployment serves, so existing clients keep their request shape. Pass `response_format=verbose_json` to get parsed speaker segments instead of a flat transcript, and raise `max_new_tokens` for long recordings. Weights are pinned to a Hugging Face revision and the runtime image to a digest, so a rebuild reproduces the same engine. ## Flags The `start_command` passes these flags to the engine. Each one controls a runtime or serving behavior: | Flag | Value | What it does | | ------------------------ | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--max-running-requests` | `16` | Maximum requests the SGLang scheduler runs concurrently. Arrivals past the cap queue instead of running. | | `--cuda-graph-max-bs` | `16` | Largest batch size SGLang captures CUDA graphs for. Larger values capture more graphs and use more memory. | | `--mem-fraction-static` | `0.80` | Fraction of GPU memory SGLang reserves for model weights and the KV cache pool. Lower it to leave room for activations and CUDA graph buffers, at the cost of peak concurrency. | ## Deploy Push the config to Baseten: ```sh theme={"system"} uvx truss push ``` You should see output similar to: ```output theme={"system"} ✨ Model moss-transcribe-diarize-latency was successfully pushed ✨ Model ID: abc1d2ef Deployment ID: xyz123 Endpoint: model-abc1d2ef.api.baseten.co Logs: https://app.baseten.co/models/abc1d2ef/logs/xyz123 ``` `truss push` prints your **model ID** (`abc1d2ef` in the example). The examples below use it wherever you see `{model_id}`, and read your API key from the `BASETEN_API_KEY` environment variable. ## Call the model Your deployment serves an OpenAI-compatible chat completions API at `/v1/audio/transcriptions` that accepts audio inputs. Send audio as an `audio_url` content item on a chat message. The model returns the transcription as the assistant message content. ```python main.py theme={"system"} import os from openai import OpenAI client = OpenAI( api_key=os.environ["BASETEN_API_KEY"], base_url="https://model-{model_id}.api.baseten.co/environments/production/sync/v1", ) response = client.chat.completions.create( model="OpenMOSS-Team/MOSS-Transcribe-Diarize", messages=[ { "role": "user", "content": [ { "type": "audio_url", "audio_url": { "url": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_en.wav" }, } ], } ], ) print(response.choices[0].message.content) ``` ```sh theme={"system"} curl -s https://model-{model_id}.api.baseten.co/environments/production/sync/v1/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $BASETEN_API_KEY" \ -d '{ "model": "OpenMOSS-Team/MOSS-Transcribe-Diarize", "messages": [ {"role": "user", "content": [ {"type": "audio_url", "audio_url": {"url": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_en.wav"}} ]} ] }' ``` ## Next steps Endpoint anatomy, authentication, and sync versus async inference Scale replicas with traffic, including scale to zero