audio_url content type, and process both modalities together: they can transcribe speech, answer questions about audio content, or reason over a recording and a text prompt at once.
Not all models support audio. Check the table below before sending audio inputs.
Supported models
Send an audio request
Use theaudio_url content type to include audio in your messages. The request shape matches vision requests with audio_url in place of image_url.
Each model has its own audio requirements:
The shared multimodal input limits cover file size, combined media size, JSON payload size, and download time. A model can impose lower limits. Large combinations can also exceed its input-token or preprocessing budget.
Baseten retrieves audio URLs from the inference service. Prefer presigned object-storage URLs reachable over HTTPS from Baseten’s environment; network policies may block other hosts. Muse Spark Open follows redirects and rejects localhost, internal, and cloud-metadata IP addresses. Base64 data URLs also work, but they count toward the JSON request-body limit.
Send a recording alongside a text prompt like this:
- Python
- JavaScript
- cURL
audio.py
Pricing
There is no additional per-request fee for audio. Audio is converted to input tokens and priced at the model’s standard input rate. Longer recordings produce more tokens and cost more to process. See the Model APIs pricing page for per-model rates.Next steps
Vision
Send images and videos alongside text
Chat Completions reference
Full request and response schema for the
audio_url content type