> ## Documentation Index
> Fetch the complete documentation index at: https://docs.baseten.co/llms.txt
> Use this file to discover all available pages before exploring further.

# Dedicated Loops

> Run Loops trainers and samplers on GPU deployments allocated to your run, and scale them with replicas, spot capacity, and warm reuse.

Dedicated Loops runs your trainer and sampler on GPU deployments allocated to your run. Every [supported base model](https://docs.baseten.co/loops/supported-models.md) trains in this mode, and it's the mode to scale out a run, such as asynchronous RL, with more trainer and sampler replicas. Loops uses it by default for models and configurations serverless doesn't serve; to choose it for any model, set `scope` to `deployment`. See [choose a mode](https://docs.baseten.co/loops/availability-modes/overview.md#choose-a-mode).

The following controls apply to trainer and sampler deployments. Deployments bill for GPU time while they run, including time between training operations. See [How a dedicated run ends](#how-a-dedicated-run-ends).

## Scale the trainer

For more training throughput in the dedicated mode, run the trainer as multiple data-parallel replicas. You set the replica count when you create the trainer: the Python SDK takes a `replicas` argument (default `1`) on [`create_lora_training_client()`](https://docs.baseten.co/reference/sdk/loops/service-client), the Baseten CLI takes a `--replicas` flag, and the HTTP API takes a `replicas` field on `POST /v1/loops/trainers`.

> [tab] Python SDK
>
> **To create a multi-replica trainer**:
>
> Pass `replicas` when you create the training client:
>
> ```python theme={"system"}
> from baseten.loops import ServiceClient
>
> service_client = ServiceClient()
> training_client = service_client.create_lora_training_client(
>     base_model="Qwen/Qwen3-0.6B",
>     replicas=2,
> )
> ```
>
> A trainer with more than one replica always starts new deployments; it doesn't reuse a warm run.

> [tab] Baseten CLI
>
> **To create a multi-replica trainer**:
>
> Create a run with the replica count set. The CLI creates a session and run; sampler creation depends on the run's availability mode:
>
> ```sh Push a multi-replica run theme={"system"}
> baseten loops run create --base-model Qwen/Qwen3-0.6B --replicas 2
> ```
>
> ```text Output theme={"system"}
> ✨ Loops deployment for Qwen/Qwen3-0.6B provisioned.
>    Trainer and sampler will finish coming up in the background
> ```
>
> The command doesn't print the run ID. If you don't already have it, look it up with `baseten loops run list`; you need it to attach your training script:
>
> ```sh View your runs theme={"system"}
> baseten loops run list
> ```
>
> ```text Output theme={"system"}
>                              Loops Runs
> ╭─────────┬─────────────────┬────────┬─────────────────────╮
> │ Run ID  │ Base Model      │ Status │ Created At          │
> ├─────────┼─────────────────┼────────┼─────────────────────┤
> │ def4567 │ Qwen/Qwen3-0.6B │ ACTIVE │ 2026-07-20 15:51:58 │
> ╰─────────┴─────────────────┴────────┴─────────────────────╯
> ```

> [tab] REST API
>
> **To create a multi-replica trainer**:
>
> 1. Create a session with [`POST /v1/loops/sessions`](https://docs.baseten.co/reference/loops-api/sessions/create-a-session) and capture its ID:
>
>    ```sh Create a session theme={"system"}
>    SESSION_ID=$(curl -s https://api.baseten.co/v1/loops/sessions \
>      -H "Authorization: Bearer $BASETEN_API_KEY" \
>      -H "Content-Type: application/json" -d '{}' | jq -r .session.id)
>    ```
>
> 2. Create the trainer with `replicas` set:
>
>    ```sh Create the trainer theme={"system"}
>    curl -s https://api.baseten.co/v1/loops/trainers \
>      -H "Authorization: Bearer $BASETEN_API_KEY" \
>      -H "Content-Type: application/json" \
>      -d "{
>        \"session_id\": \"$SESSION_ID\",
>        \"base_model\": \"Qwen/Qwen3-0.6B\",
>        \"replicas\": 2
>      }"
>    ```
>
>    ```json Output theme={"system"}
>    {
>      "run": {
>        "id": "def4567",
>        "session_id": "abc1234",
>        "deployment_id": "def456",
>        "name": "Qwen/Qwen3-0.6B",
>        "base_model": "Qwen/Qwen3-0.6B",
>        "base_url": "https://trainer-def456.api.baseten.co/trainer",
>        "created_at": "2026-07-20T22:51:58.115Z",
>        "sampler": null
>      }
>    }
>    ```

**To attach your training script to the trainer**:

Start your training script with the run ID in its environment, so [`ServiceClient`](https://docs.baseten.co/reference/sdk/loops/service-client) reuses that trainer instead of starting a new single-replica one:

```sh theme={"system"}
LOOPS_REUSE_FROM_RUN_ID=def4567 python train.py
```

Each replica is a full copy of the trainer with its own GPUs, so GPU time and cost scale with `replicas`: `replicas=4` uses four times the GPUs of a single replica.

Your training loop doesn't change. Each [`forward_backward()`](https://docs.baseten.co/reference/sdk/loops/training-client) batch is sharded across the replicas automatically, so batches don't need to divide evenly by the replica count: a 3-datum batch on 2 replicas puts 2 datums on one worker and 1 on the other, and the result still contains one entry per datum, in order. Gradients synchronize across replicas at [`optim_step()`](https://docs.baseten.co/reference/sdk/loops/training-client), which advances `policy_version` once for the whole trainer. Size batches to at least the replica count; a smaller batch works but leaves the extra replicas idle for that step.

Samplers scale the same way. Pass `sampler_num_replicas` to [`create_lora_training_client()`](https://docs.baseten.co/reference/sdk/loops/service-client) to set the replica count for the run's paired sampler, or `num_replicas` to [`ServiceClient.create_sampling_client()`](https://docs.baseten.co/reference/sdk/loops/service-client) for a standalone sampler. The count applies as both the minimum and maximum, and takes effect when the sampler is created. Omit it and the backend default applies.

## Spot capacity

For Loops deployments, spot capacity uses GPUs that Baseten can reclaim. Use it for interruptible training that can continue from a checkpoint. Baseten can reclaim the GPUs at any time, which stops the run.

The `availability_model` setting controls whether deployments use non-preemptible or spot capacity. It defaults to `dedicated`. To allow spot capacity for deployments, configure `ServiceClient`:

> [tab] Python SDK
>
> **To allow spot capacity**:
>
> Pass `availability_model` when you create the client:
>
> ```python theme={"system"}
> from baseten.loops import AvailabilityModel, ServiceClient
>
> service_client = ServiceClient(
>     availability_model=AvailabilityModel.spot,
> )
> ```

> [tab] Environment variable
>
> **To allow spot capacity**:
>
> Set `LOOPS_AVAILABILITY_MODEL` for the process:
>
> ```bash theme={"system"}
> LOOPS_AVAILABILITY_MODEL=spot python train.py
> ```

If you set both options, the value passed to `ServiceClient` takes precedence over `LOOPS_AVAILABILITY_MODEL`. Loops reuses a warm trainer only if it has the same capacity type. Otherwise, it provisions a new spot trainer.

For deployments, the setting applies to trainers and standalone samplers created by the client. A paired sampler inherits its run's capacity type. This setting doesn't choose the availability mode; set `scope` to `deployment` to make sure it applies.

Baseten can preempt spot trainers, so save checkpoints during training. Loops doesn't restart a preempted run automatically, but you can create a new trainer from the latest checkpoint. See [Checkpoints](https://docs.baseten.co/loops/concepts.md#checkpoints).

Spot capacity must be enabled for your workspace. [Contact support](mailto:support@baseten.co) to enable it. For Training Jobs, see [Spot capacity](https://docs.baseten.co/training/management.md#spot-capacity).

If spot trainer creation fails, follow [capacity and quota errors](https://docs.baseten.co/troubleshooting/loops.md#resolve-capacity-and-quota-errors) to distinguish unavailable GPUs from your organization's trainer limit.

## Reuse a warm run

Each new `ServiceClient` creates a session. A new dedicated run starts its own trainer and sampler deployments and waits for them to come up.

A new run can attach to a prior run's warm trainer and paired sampler instead of starting new ones.

> [tab] Python SDK
>
> **To reuse a warm run**:
>
> Pass the earlier run's ID when you create the client:
>
> ```python theme={"system"}
> from baseten.loops import ServiceClient
>
> service_client = ServiceClient(reuse_from_run_id="<run_id>")
> ```

> [tab] Environment variable
>
> **To reuse a warm run**:
>
> Set `LOOPS_REUSE_FROM_RUN_ID` for the process. `ServiceClient` reads it when you don't pass `reuse_from_run_id`:
>
> ```bash theme={"system"}
> LOOPS_REUSE_FROM_RUN_ID="<run_id>" python train.py
> ```

Session-based trainer reuse still works but is deprecated in favor of the run ID: `ServiceClient(reuse_from_session_id=...)`, the `LOOPS_REUSE_FROM_SESSION_ID` environment variable, and the `reuse_from_session_id` field on `POST /v1/loops/trainers`. When both are set, the run ID wins and the session ID is ignored. Standalone sampler creation ignores session reuse and creates a new logical sampler.

Reuse is best-effort. The named run or session must belong to the same team. If the earlier run's trainer isn't available, for example because it's still starting, stopped, or failed, Loops starts a new one and the call still succeeds. Reuse controls deployment startup. It doesn't replace saving and loading checkpoints to preserve training state.

## Overlap trainer and sampler startup

In the dedicated mode, an RL loop can reach its first rollout while the sampler still needs to start. To bring the trainer and sampler up at roughly the same time, kick off the first `save_weights_and_get_sampling_client()` as an async task as soon as the trainer is ready, and train while the sampler provisions:

```python overlap_startup.py theme={"system"}
import asyncio
from baseten.loops import ServiceClient

async def main():
    service_client = ServiceClient()
    training_client = await service_client.create_lora_training_client_async(
        base_model="Qwen/Qwen3-0.6B",
    )
    # The sampler serves the base model until the trainer publishes trained weights.
    sampler_task = asyncio.create_task(
        training_client.save_weights_and_get_sampling_client_async()
    )
    # ... run forward_backward / optim_step steps here ...
    sampling_client = await sampler_task

asyncio.run(main())
```

If you'd rather not manage the async task, pass `with_sampler=True` to `create_lora_training_client()`: the paired sampler starts with the trainer and serves the base model until the trainer publishes weights. By default, the sampler starts at the first sampling-client request, so a run that only trains doesn't pay for sampler GPUs.

Every long-running SDK method has an `*_async` counterpart with the same arguments, so the rest of the loop works the same way; see the note on [`TrainingClient`](https://docs.baseten.co/reference/sdk/loops/training-client).

## Usage and logs

Use [`baseten loops usage`](https://docs.baseten.co/reference/cli/baseten/loops#usage) to inspect dedicated GPU allocations. Deployment logs cover the run's trainer and sampler deployments. See [Loops troubleshooting](https://docs.baseten.co/troubleshooting/loops.md) for request errors and recovery.

## How a dedicated run ends

A dedicated run bills for GPU time while its trainer and sampler deployments run, including idle time. [Deactivate the run](https://docs.baseten.co/loops/deactivate-a-run.md) when you finish to shut down those deployments. Closing the training client stops its keepalive but doesn't deactivate the run.

Without explicit deactivation, the run ends after a period of inactivity, 15 minutes by default. Saved checkpoints remain available after the run ends. See [Deactivate a run](https://docs.baseten.co/loops/deactivate-a-run.md) for shutdown commands and status checks.

## Configure the inactivity timeout

Set the timeout when you create a dedicated trainer with [`POST /v1/loops/trainers`](https://docs.baseten.co/reference/loops-api/runs/create-a-trainer). The timeout starts when the run stops receiving requests or client keepalives.

**scale\_down\_delay\_seconds** `integer` (default: 900)

Inactivity period before the dedicated run ends, in seconds. Accepts values from `1` to `3600`. Explicit deactivation ends the run without waiting for this timeout.

**To create a run with a one-hour inactivity timeout**:

1. Install [jq](https://jqlang.org/download/), set `BASETEN_API_KEY` to your [workspace API key](https://docs.baseten.co/organization/api-keys.md), then create a session:

   ```bash theme={"system"}
   SESSION_ID=$(curl --fail-with-body --silent --show-error \
     https://api.baseten.co/v1/loops/sessions \
     --header "Authorization: Bearer $BASETEN_API_KEY" \
     --header "Content-Type: application/json" \
     --data '{}' | jq -er '.session.id')
   ```

2. Create the dedicated trainer using the returned session ID:

   ```bash theme={"system"}
   curl --fail-with-body --silent --show-error \
     https://api.baseten.co/v1/loops/trainers \
     --header "Authorization: Bearer $BASETEN_API_KEY" \
     --header "Content-Type: application/json" \
     --data "{
       \"session_id\": \"$SESSION_ID\",
       \"base_model\": \"zai-org/GLM-5.3-Flash\",
       \"scope\": \"deployment\",
       \"scale_down_delay_seconds\": 3600
     }"
   ```

   The response includes `run.id`. Keep it to attach your training script with `LOOPS_REUSE_FROM_RUN_ID` or to [deactivate the run](https://docs.baseten.co/loops/deactivate-a-run.md) when you finish. This run bills for GPU time, including the inactivity period.