View example on GitHub

 This guide walks through building an audio transcription pipeline using Chains. You’ll break down large media files, distribute transcription tasks across autoscaling deployments, and leverage high-performance GPUs for rapid inference.

​
1. Overview

This Chain enables fast, high-quality transcription by:
  • Partitioning long files (10+ hours) into smaller segments.
  • Detecting silence to optimize split points.
  • Parallelizing inference across multiple GPU-backed deployments.
  • Batching requests to maximize throughput.
  • Using range downloads for efficient data streaming.
  • Leveraging asyncio for concurrent execution.

​
2. Chain Structure

Transcription is divided into two processing layers:
  1. Macro chunks: Large segments (~300s) split from the source media file. These are processed in parallel to handle massive files efficiently.
  2. Micro chunks: Smaller segments (~5–30s) extracted from macro chunks and sent to the Whisper model for transcription.

​
3. Implementing the Chainlets

​
Transcribe (Entrypoint Chainlet)

Handles transcription requests and dispatches tasks to worker Chainlets. Function signature: 
async def run_remote(
  self,
  media_url: str,
  params: data_types.TranscribeParams
) -> data_types.TranscribeOutput:
Steps:
  • Validates that the media source supports range downloads.
  • Uses FFmpeg to extract metadata and duration.
  • Splits the file into macro chunks, optimizing split points at silent sections.
  • Dispatches macro chunk tasks to the MacroChunkWorker for processing.
  • Collects micro chunk transcriptions, merges results, and returns the final text.
Example request: 
curl -X POST $INVOCATION_URL \
    -H "Authorization: Api-Key $BASETEN_API_KEY" \
    -d '<JSON_INPUT>'

{
  "media_url": "http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/TearsOfSteel.mp4",
  "params": {
    "micro_chunk_size_sec": 30,
    "macro_chunk_size_sec": 300
  }
}

​
MacroChunkWorker (Processing Chainlet)

Processes macro chunks by:
  • Extracting relevant time segments using FFmpeg.
  • Streaming audio instead of downloading full files for low latency.
  • Splitting segments at silent points.
  • Encoding audio in base64 for efficient transfer.
  • Distributing micro chunks to the Whisper model for transcription.
This Chainlet runs in parallel with multiple instances autoscaled dynamically.

​
WhisperModel (Inference Model)

A separately deployed Whisper model Chainlet handles speech-to-text transcription.
  • Deployed independently to allow fast iteration on business logic without redeploying the model.
  • Used across different Chains or accessed directly as a standalone model.
  • Supports multiple environments (e.g., dev, prod) using the same instance.
Whisper can also be deployed as a standard Truss model, separate from the Chain.

​
4. Optimizing Performance

Even for very large files, processing time remains bounded by parallel execution.

​
Key performance tuning parameters:

  • micro_chunk_size_sec → Balance GPU utilization and inference latency.
  • macro_chunk_size_sec → Adjust chunk size for optimal parallelism.
  • Autoscaling settings → Tune concurrency and replica counts for load balancing.
Example speedup: 
{
  "input_duration_sec": 734.26,
  "processing_duration_sec": 82.42,
  "speedup": 8.9
}

​
5. Deploy & Run the Chain

​
Deploy WhisperModel first:

truss chains push whisper_chainlet.py
Copy the invocation URL and update WHISPER_URL in transcribe.py.

​
Deploy the transcription Chain:

truss chains push transcribe.py

​
Run transcription on a sample file:

curl -X POST $INVOCATION_URL \
    -H "Authorization: Api-Key $BASETEN_API_KEY" \
    -d '<JSON_INPUT>'

​
Next Steps

  • Learn more about Chains.
  • Optimize GPU autoscaling for peak efficiency.
  • Extend the pipeline with custom business logic.