# Baseten - [Documentation (173 pages)](https://docs.baseten.co/_llms/documentation.txt): Documentation for Documentation. ## Examples ### Examples - [Build with Baseten](https://docs.baseten.co/examples/overview.md): Ready-made examples for deploying, serving, and training models on Baseten. - [Deploy your first model](https://docs.baseten.co/examples/deploy-your-first-model.md): Deploy an open-source LLM to Baseten with just a config file and get an OpenAI-compatible API endpoint. ### Models - [Model recipes](https://docs.baseten.co/examples/models/overview.md): Run popular models through optimized inference engines on Baseten. #### LLMs - [DiffusionGemma](https://docs.baseten.co/examples/models/llm/diffusiongemma.md): Google's DiffusionGemma diffusion language model (26B total, 4B active), served from an FP8 quantized checkpoint. - [Gemma 4](https://docs.baseten.co/examples/models/llm/gemma-4.md): Gemma 4 recipes: 4 variants (E2B, E4B, 26B A4B, 31B), Dense and MoE architectures. - [Holo 3.1](https://docs.baseten.co/examples/models/llm/holo-3.1.md): H Company's Holo 3.1 is a 35B-parameter MoE vision-language model with 3B active parameters per token, built on the Qwen3.6-35B-A3B base. - [Mellum2](https://docs.baseten.co/examples/models/llm/mellum2.md): JetBrains' Mellum2 open MoE code model (12B total, 2.5B active) with a 131k-token context window and tool calling. - [Llama 3.1](https://docs.baseten.co/examples/models/llm/llama-3.1.md): Meta's Llama 3.1 8B instruction-tuned model. - [Llama 3.2](https://docs.baseten.co/examples/models/llm/llama-3.2.md): Meta's compact Llama 3.2 instruction-tuned model. - [Llama 3.3](https://docs.baseten.co/examples/models/llm/llama-3.3.md): Meta's Llama 3.3 70B instruction-tuned model. - [Llama 4](https://docs.baseten.co/examples/models/llm/llama-4.md): Meta's Llama 4 Scout is a 17B-active MoE with native multimodal support and a 10M token context window. - [MiniMax M2.5](https://docs.baseten.co/examples/models/llm/minimax-m2.5.md): Large MoE model with native reasoning and tool calling. - [Nemotron 3](https://docs.baseten.co/examples/models/llm/nemotron-3.md): Nemotron 3 recipes: 2 variants (Nano 30B A3B, Super 120B A12B), Hybrid MoE and MoE architectures. - [Nemotron 3.5](https://docs.baseten.co/examples/models/llm/nemotron-3.5.md): NVIDIA's Nemotron 3.5 Lightning is a hybrid Mamba-2, MoE, and attention reasoning model with 30B total and 3B active parameters. - [GPT-OSS](https://docs.baseten.co/examples/models/llm/gpt-oss.md): GPT-OSS recipes: 2 variants (20B, 120B), Dense and MoE architectures. - [Laguna](https://docs.baseten.co/examples/models/llm/laguna.md): Laguna recipes: 2 variants (M.1, S 2.1), MoE architecture. - [Qwen3](https://docs.baseten.co/examples/models/llm/qwen3.md): Sparse MoE model with 235B total parameters (22B active per token). - [Qwen3.5](https://docs.baseten.co/examples/models/llm/qwen3.5.md): Qwen3.5 recipes: 4 variants (4B, 9B, 35B, 122B), Dense, Hybrid MoE, and MoE architectures. - [Qwen3.6](https://docs.baseten.co/examples/models/llm/qwen3.6.md): Qwen3.6 recipes: 2 variants (27B, 35B-A3B), Dense and Hybrid MoE architectures. - [Qwen3.8](https://docs.baseten.co/examples/models/llm/qwen3.8.md): Dense 27B-parameter multimodal reasoning model with FP8 weights, served on a single H100 with vLLM, MTP speculative decoding, and a data-parallel vision encoder. - [Qwen3-VL](https://docs.baseten.co/examples/models/llm/qwen3-vl.md): Qwen3-VL-32B-Instruct is a 32B-parameter dense vision-language model. - [GLM-4.7](https://docs.baseten.co/examples/models/llm/glm-4.7.md): GLM-4.7 recipes: 2 variants (Standard, Flash), MoE architecture. - [GLM-5](https://docs.baseten.co/examples/models/llm/glm-5.md): Z.ai's GLM-5 reasoning model, served from an FP8 checkpoint on B200:8. - [GLM-5.3 Flash](https://docs.baseten.co/examples/models/llm/glm-5.3-flash.md): Z.ai GLM-5.3 Flash is a multimodal mixture-of-experts model with 321B total and 18B active parameters. - [Ling 3.0 Flash](https://docs.baseten.co/examples/models/llm/ling-3.0-flash.md): InclusionAI Ling 3.0 Flash is a hybrid-linear sparse mixture-of-experts reasoning model with 124B total and 5.1B active parameters. - [Ornith 1.5](https://docs.baseten.co/examples/models/llm/ornith-1.5.md): Ornith 1.5 35B-A3B is an FP8 mixture-of-experts reasoning model for coding and agentic tasks. #### Image generation - [FLUX.1](https://docs.baseten.co/examples/models/image-gen/flux1.md): FLUX.1 recipes: 2 variants (dev, schnell), diffusion-transformer architecture. - [Krea 2 Turbo](https://docs.baseten.co/examples/models/image-gen/krea-2-turbo.md): Krea 2 Turbo is Krea's 12B flow-matching image generation model: a single-stream MMDiT backbone paired with a Qwen3-VL text encoder and the Qwen-Image VAE. #### Transcription - [moss-transcribe-diarize](https://docs.baseten.co/examples/models/transcription/moss-transcribe-diarize.md): MOSS-Transcribe-Diarize transcribes speech, identifies speakers, and adds timestamps in a single pass. - [Qwen3-ASR](https://docs.baseten.co/examples/models/transcription/qwen3-asr.md): Alibaba's Qwen3-ASR is a compact 1.7B speech-to-text model with multilingual transcription support. - [VibeVoice](https://docs.baseten.co/examples/models/transcription/vibevoice.md): Microsoft's VibeVoice-ASR speech-to-text model, returning JSON segments with speaker labels and timestamps through an OpenAI-compatible API. - [Voxtral](https://docs.baseten.co/examples/models/transcription/voxtral.md): Voxtral recipes: 2 variants (Mini 4B, Mini 3B Batch), encoder-decoder architecture. #### Embedding - [Nemotron 3 Embed](https://docs.baseten.co/examples/models/embedding/nemotron-3-embed.md): Nemotron 3 Embed recipes: 2 variants (1B, 8B), Dense architecture. - [Qwen3 Embedding](https://docs.baseten.co/examples/models/embedding/qwen3-embedding.md): Qwen3 Embedding recipes: 3 variants (0.6B, 4B, 8B), Dense architecture. - [Qwen3 Reranker](https://docs.baseten.co/examples/models/embedding/qwen3-reranker.md): Qwen3 Reranker recipes: 3 variants (0.6B, 4B, 8B), Dense architecture. - [Zembed-1](https://docs.baseten.co/examples/models/embedding/zembed-1.md): ZeroEntropy's Zembed-1 is a 4B-parameter text embedding model built for high-quality semantic search and retrieval. ### Engines - [Deploy LLMs with TensorRT-LLM](https://docs.baseten.co/examples/tensorrt-llm.md): Optimize LLMs for low latency and high throughput. - [Serve embeddings with BEI](https://docs.baseten.co/examples/bei.md): Deploy embedding, reranking, and classification models on Baseten Embeddings Inference. - [Deploy a model with Eagle speculative decoding](https://docs.baseten.co/examples/speculative-decoding.md): Deploy Qwen3-4B on BIS-LLM (v2) with an Eagle draft head for faster generation, then call it from an OpenAI-compatible endpoint. ### Custom Docker servers - [Deploy a Hugging Face model](https://docs.baseten.co/examples/deploy-a-hugging-face-model.md): Deploy Gemma 4 26B on Baseten with vLLM, BDN-cached weights, EAGLE3 speculative decoding, and prefix caching. - [Deploy LLMs with vLLM](https://docs.baseten.co/examples/vllm.md): Run any open-source LLM on vLLM's serving framework. - [Deploy LLMs with SGLang](https://docs.baseten.co/examples/sglang.md): Run LLMs on SGLang's high-performance serving framework. - [Deploy LLMs with Ollama](https://docs.baseten.co/examples/ollama.md): Run LLMs on Ollama as a custom Docker server. - [Deploy a Dockerized model](https://docs.baseten.co/examples/docker.md): Deploy any model in a pre-built Docker container. ### Custom Python models - [Build and deploy an LLM](https://docs.baseten.co/examples/deploy-a-llm.md): Package and deploy an LLM with Truss, from model setup to inference. - [Customize a model](https://docs.baseten.co/examples/customize-a-model.md): Deploy a model with custom Python code using the Truss Model class. - [Generate images with Flux](https://docs.baseten.co/examples/image-generation.md): Deploy Flux Schnell as a text-to-image endpoint. - [Generate speech with Kokoro](https://docs.baseten.co/examples/text-to-speech.md): Deploy Kokoro as a text-to-speech endpoint. - [Stream LLM responses](https://docs.baseten.co/examples/streaming.md): Stream LLM output token by token. - [Add system packages](https://docs.baseten.co/examples/system-packages.md): Deploy a model with both Python and system dependencies. ### Chains - [Build a RAG pipeline with Chains](https://docs.baseten.co/examples/chains-build-rag.md): Combine retrieval and generation into a single compound workflow. - [Transcribe audio with Chains](https://docs.baseten.co/examples/chains-audio-transcription.md): Process hours of audio in seconds using efficient chunking, distributed inference, and optimized GPU resources. ### API workflows - [Create a model with the REST API](https://docs.baseten.co/examples/create-a-model-with-rest.md): Deploy a model archive using the Management API. ### Baseten for Model Labs - [Deploy a Model Labs listing](https://docs.baseten.co/examples/model-labs/deploy-a-published-model.md): Deploy a Baseten for Model Labs listing into your workspace or team using the REST API. - [Reference (376 pages)](https://docs.baseten.co/_llms/reference.txt): Documentation for Reference. ## OpenAPI Specs - [inference-api-spec](/reference/inference-api/inference-api-spec.json) - [llm-openapi-spec](/reference/inference-api/llm-openapi-spec.json) - [messages-openapi-spec](/reference/inference-api/messages-openapi-spec.json) - [spec](https://api.baseten.co/v1/spec) - [spec](/spec) > The links below point to documentation indexes. Follow each `/_llms/` index recursively until you reach documentation pages. ## Indexes - [Documentation (173 pages)](https://docs.baseten.co/_llms/documentation.txt): Documentation for Documentation. - [Reference (376 pages)](https://docs.baseten.co/_llms/reference.txt): Documentation for Reference.