> ## Documentation Index
> Fetch the complete documentation index at: https://docs.baseten.co/llms.txt
> Use this file to discover all available pages before exploring further.

# mimo-v2.5

> Deploy MiMo-V2.5 with vLLM on four B200 GPUs and call its chat completions endpoint.

[Reasoning](https://docs.baseten.co/examples/models/capabilities/reasoning)
[Tool calling](https://docs.baseten.co/examples/models/capabilities/tool-calling)
[Long context](https://docs.baseten.co/examples/models/capabilities/long-context)
[Multimodal (image)](https://docs.baseten.co/examples/models/capabilities/multimodal-image)

Deploy MiMo-V2.5 with vLLM on four B200 GPUs and call its chat completions endpoint.

## Setup

Install the Baseten CLI and sign in, then install the OpenAI SDK.

**Install and sign in to Baseten**

> [tab] macOS or Linux
>
> ```bash Terminal theme={"system"}
> brew tap basetenlabs/baseten
> brew trust --formula basetenlabs/baseten/baseten
> brew install baseten
> ```

> [tab] Windows
>
> Download and extract the Windows archive for the latest Baseten CLI release:
>
> ```powershell Terminal theme={"system"}
> $release = Invoke-RestMethod https://api.github.com/repos/basetenlabs/baseten-cli/releases/latest
> $asset = $release.assets | Where-Object { $_.name -like "*_windows_amd64.zip" }
> Invoke-WebRequest $asset.browser_download_url -OutFile baseten.zip
> Expand-Archive -Force baseten.zip .
> ```
>
> Move `baseten.exe` to a directory on your `PATH`.

For other platforms or a specific version, see the [Baseten CLI install reference](https://docs.baseten.co/reference/cli/baseten/overview.md#install).

```sh theme={"system"}
baseten auth login
```

**Install the OpenAI SDK**

```sh theme={"system"}
uv pip install openai
```

To install without Homebrew, see the [Baseten CLI install reference](https://docs.baseten.co/reference/cli/baseten/overview.md#install).

This preset serves MiMo-V2.5 FP8 on four B200 GPUs with tensor parallelism.

> [card] Hardware
>
> B200 × 4

> [card] Engine
>
> vLLM (mimov25-cu129 build)

> [card] Context
>
> 1M

> [card] Concurrency
>
> 64

## Write the config

Create and move into the project directory:

```sh theme={"system"}
mkdir mimo-v2.5-latency && cd mimo-v2.5-latency
```

Then create a file named `config.yaml` and paste the following:

```yaml config.yaml theme={"system"}
model_name: "model:mimo-v2.5 preset:latency"
model_metadata:
  description: >-
    Xiaomi MiMo-V2.5, a 310B-total / 15B-active omnimodal mixture-of-experts model
    with a 1,048,576-token context window, served with vLLM on four B200 GPUs with
    tensor parallelism. OpenAI-compatible chat with MiMo tool-calling and reasoning
    parsers and MTP speculative decoding. Thinking mode is on by default; pass
    chat_template_kwargs.enable_thinking=false to disable.
  repo_id: XiaomiMiMo/MiMo-V2.5
  example_model_input:
    model: XiaomiMiMo/MiMo-V2.5
    messages:
      - role: user
        content: "What is 84 * 3 / 2?"
    max_tokens: 512
    temperature: 1.0
  tags:
    - openai-compatible
base_image:
  # Stable vLLM does not support the mimo_v2 architecture yet; the vLLM team's
  # pre-built MiMo-V2.5 image is pinned instead. Replace with the first stable
  # vllm/vllm-openai release containing mimo_v2 support once one is available.
  image: vllm/vllm-openai:mimov25-cu129
weights:
  - source: "hf://XiaomiMiMo/MiMo-V2.5@63651580ca774f8504f676040460aed3e1244ac1"
    mount_location: "/app/checkpoint/model"
resources:
  instance_type: B200:4
environment_variables:
  VLLM_LOGGING_LEVEL: WARNING
  VLLM_ENGINE_READY_TIMEOUT_S: "3600"
docker_server:
  # TP is pinned to 4: TP=8 hits an attention-projection shape mismatch with this
  # FP8 checkpoint (per the upstream vLLM recipe), so do not scale it to GPU count.
  start_command: >-
    vllm serve /app/checkpoint/model
    --served-model-name XiaomiMiMo/MiMo-V2.5
    --tensor-parallel-size 4
    --trust-remote-code
    --generation-config vllm
    --enable-prefix-caching
    --max-model-len auto
    --gpu-memory-utilization 0.95
    --limit-mm-per-prompt.image 1
    --tool-call-parser mimo
    --enable-auto-tool-choice
    --reasoning-parser mimo
    --speculative-config.method mtp
    --speculative-config.num_speculative_tokens 1
    --load-format runai_streamer
  readiness_endpoint: /health
  liveness_endpoint: /health
  predict_endpoint: /v1/chat/completions
  server_port: 8000

runtime:
  predict_concurrency: 64
  health_checks:
    startup_threshold_seconds: 1800
    restart_threshold_seconds: 1200
    stop_traffic_threshold_seconds: 120
```

## Flags

The `start_command` passes these flags to the engine. Each one controls a runtime or serving behavior:

| Flag | Value | What it does |
| - | - | - |
| `--tensor-parallel-size` | `4` | Number of GPUs to shard the model across. |
| `--trust-remote-code` | (no value) | Execute model-specific Python from the checkpoint (required for many Qwen, Phi, and custom architectures). |
| `--generation-config` | `vllm` | Source of the default generation (sampling) settings. **vllm:** Use vLLM's own defaults instead of the checkpoint's `generation_config.json`. |
| `--enable-prefix-caching` | (no value) | Reuse KV cache across requests that share a prefix. |
| `--max-model-len` | `auto` | Maximum context length (tokens) the server accepts per request. |
| `--gpu-memory-utilization` | `0.95` | Fraction of GPU memory vLLM may use for weights and KV cache. |
| `--limit-mm-per-prompt.image` | `1` | Maximum number of image inputs per prompt. |
| `--tool-call-parser` | `mimo` | Server-side parser that emits structured `tool_calls` on the response. |
| `--enable-auto-tool-choice` | (no value) | Let the model choose when to call tools without requiring `tool_choice: "required"`. |
| `--reasoning-parser` | `mimo` | Server-side parser that separates reasoning output into `reasoning_content`. |
| `--speculative-config.method` | `mtp` | Speculative decoding method. **mtp:** Multi-token prediction head speculation. |
| `--speculative-config.num_speculative_tokens` | `1` | Number of tokens the draft speculator proposes per step. |
| `--load-format` | `runai_streamer` | Weight loading backend. **runai\_streamer:** Stream weights from object storage without materializing to disk. |

## Deploy

Push the config to Baseten with the Baseten CLI, or with the Truss CLI if you prefer it:

```sh Baseten CLI theme={"system"}
baseten model push
```

```sh Truss CLI theme={"system"}
uvx truss push
```

You should see output similar to:

```output theme={"system"}
Pushing model "mimo-v2.5-latency"...
Uploading model...
Uploaded model in 0s
✨ Model mimo-v2.5-latency was successfully pushed ✨

  Model:       mimo-v2.5-latency (abc1d2ef)
  Deployment:  xyz123
  Environment: production

🪵 View logs:
   deployment:   baseten model deployment logs --model-id abc1d2ef --deployment-id xyz123
   environment:  baseten model environment logs --model-id abc1d2ef --environment production  (once deployed)
   app:          https://app.baseten.co/models/abc1d2ef/logs/xyz123

🚀 Invoke your model:
   URL:  https://model-abc1d2ef.api.baseten.co/deployment/xyz123/predict
   CLI:  baseten model predict --model-id abc1d2ef
```

`baseten model push` prints your **model ID** (`abc1d2ef` in the example). The examples below use it wherever you see `{model_id}`, and read your API key from the `BASETEN_API_KEY` environment variable.

## Call the model

Your deployment serves an OpenAI-compatible API.

Now call your deployment to run inference:

> [tab] Python
>
> ```python main.py theme={"system"}
> import os
> from openai import OpenAI
>
> client = OpenAI(
>     api_key=os.environ["BASETEN_API_KEY"],
>     base_url="https://model-{model_id}.api.baseten.co/environments/production/sync/v1",
> )
>
> response = client.chat.completions.create(
>     model="XiaomiMiMo/MiMo-V2.5",
>     messages=[
>         {"role": "user", "content": "What is machine learning?"}
>     ],
> )
>
> print(response.choices[0].message.content)
> ```

> [tab] cURL
>
> ```sh theme={"system"}
> curl -s https://model-{model_id}.api.baseten.co/environments/production/sync/v1/chat/completions \
>   -H "Content-Type: application/json" \
>   -H "Authorization: Bearer $BASETEN_API_KEY" \
>   -d '{
>     "model": "XiaomiMiMo/MiMo-V2.5",
>     "messages": [
>       {"role": "user", "content": "What is machine learning?"}
>     ]
>   }'
> ```

The server parses the model's chain of thought into a separate `reasoning_content` field on the response. Read it alongside the final answer:

```python theme={"system"}
response = client.chat.completions.create(
    model="XiaomiMiMo/MiMo-V2.5",
    messages=[
        {"role": "user", "content": "How many r's in strawberry?"}
    ],
)
print(response.choices[0].message.reasoning_content)  # chain of thought
print(response.choices[0].message.content)            # final answer
```

To let the model call tools, pass a `tools` array. The server returns structured `tool_calls` on the response:

```python theme={"system"}
tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "parameters": {
            "type": "object",
            "properties": {"location": {"type": "string"}},
            "required": ["location"],
        },
    },
}]

response = client.chat.completions.create(
    model="XiaomiMiMo/MiMo-V2.5",
    messages=[
        {"role": "user", "content": "What's the weather in Paris?"}
    ],
    tools=tools,
)
print(response.choices[0].message.tool_calls)
```

## Next steps

> [card] [Call your model](https://docs.baseten.co/inference/calling-your-model.md)
>
> Endpoint anatomy, authentication, and sync versus async inference

> [card] [Autoscaling](https://docs.baseten.co/deployment/autoscaling/overview.md)
>
> Scale replicas with traffic, including scale to zero
