Audio

Model APIs support audio input on audio-capable models. These models accept audio alongside text in the same request, using the audio_url content type, and process both modalities together: they can transcribe speech, answer questions about audio content, or reason over a recording and a text prompt at once.

To deploy dedicated models for batch or real-time transcription, or for speech generation, see Speech and audio.

Not all models support audio. Check the table below before sending audio inputs.

Supported models

Model	Slug
Inkling	`thinkingmachines/inkling`
Inkling Small	`thinkingmachines/inkling-small`

Send an audio request

Use the audio_url content type to include audio in your messages. The request shape matches vision requests with audio_url in place of image_url. Each model has its own audio requirements:

Model	Requirements
Inkling	WAV at 16 kHz; recordings under 20 minutes give the best results; 45 MB request-body limit
Inkling Small	WAV at 16 kHz; recordings under 2 minutes give the best results; 45 MB request-body limit

Baseten retrieves audio URLs from the inference service, so the URL must be reachable over HTTPS from Baseten’s environment (for example your own object storage or other hosts that allow server-side fetches). Prefer stable, direct HTTPS links. Base64 data: URLs also work, but they count toward the request-body limit, so pass audio by URL when you can. Send a recording alongside a text prompt like this:

Python
JavaScript
cURL

audio.py

from openai import OpenAI
import os

client = OpenAI(
    base_url="https://inference.baseten.co/v1",
    api_key=os.environ["BASETEN_API_KEY"],
)

response = client.chat.completions.create(
    model="thinkingmachines/inkling",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": "Summarize what is said in this recording.",
                },
                {
                    "type": "audio_url",
                    "audio_url": {
                        "url": "https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/jfk.wav",
                    },
                },
            ],
        }
    ],
)

print(response.choices[0].message.content)

audio.js

import OpenAI from "openai";

const client = new OpenAI({
    baseURL: "https://inference.baseten.co/v1",
    apiKey: process.env.BASETEN_API_KEY,
});

const response = await client.chat.completions.create({
    model: "thinkingmachines/inkling",
    messages: [
        {
            role: "user",
            content: [
                {
                    type: "text",
                    text: "Summarize what is said in this recording.",
                },
                {
                    type: "audio_url",
                    audio_url: {
                        url: "https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/jfk.wav",
                    },
                },
            ],
        },
    ],
});

console.log(response.choices[0].message.content);

Request

curl https://inference.baseten.co/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $BASETEN_API_KEY" \
  -d '{
    "model": "thinkingmachines/inkling",
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "text",
            "text": "Summarize what is said in this recording."
          },
          {
            "type": "audio_url",
            "audio_url": {
              "url": "https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/jfk.wav"
            }
          }
        ]
      }
    ]
  }'

Pricing

There is no additional per-request fee for audio. Audio is converted to input tokens and priced at the model’s standard input rate. Longer recordings produce more tokens and cost more to process. See the Model APIs pricing page for per-model rates. The response reports how many input tokens came from audio in usage.prompt_tokens_details.audio_tokens.

Next steps

Vision

Send images and videos alongside text

Chat Completions reference

Full request and response schema for the audio_url content type

Get started

Model APIs

Inference

Training

Sandboxes

Development

Deployment

Engines

Model Labs

Organization

Hosting options

Observability

Troubleshooting

Supported models

Send an audio request

Pricing

Next steps

Vision

Chat Completions reference

​Supported models

​Send an audio request

​Pricing

​Next steps

Vision

Chat Completions reference

Supported models

Send an audio request

Pricing

Next steps