Dedicated Loops

Dedicated Loops runs your trainer and sampler on GPU deployments allocated to your run. Every supported base model trains in this mode, and it’s the mode to scale out a run, such as asynchronous RL, with more trainer and sampler replicas. Loops uses it by default for models and configurations serverless doesn’t serve; to choose it for any model, set scope to deployment. See choose a mode. The following controls apply to trainer and sampler deployments. Deployments bill for GPU time while they run, including time between training operations. See How a dedicated run ends.

Scale the trainer

For more training throughput in the dedicated mode, run the trainer as multiple data-parallel replicas. You set the replica count when you create the trainer: the Python SDK takes a replicas argument (default 1) on create_lora_training_client(), the Baseten CLI takes a --replicas flag, and the HTTP API takes a replicas field on POST /v1/loops/trainers.

Python SDK
Baseten CLI
REST API

To create a multi-replica trainer:Pass replicas when you create the training client:

from baseten.loops import ServiceClient

service_client = ServiceClient()
training_client = service_client.create_lora_training_client(
    base_model="Qwen/Qwen3-0.6B",
    replicas=2,
)

A trainer with more than one replica always starts new deployments; it doesn’t reuse a warm run.

To create a multi-replica trainer:Create a run with the replica count set. The CLI creates a session and run; sampler creation depends on the run’s availability mode:

Push a multi-replica run

baseten loops run create --base-model Qwen/Qwen3-0.6B --replicas 2

Output

✨ Loops deployment for Qwen/Qwen3-0.6B provisioned.
   Trainer and sampler will finish coming up in the background

The command doesn’t print the run ID. If you don’t already have it, look it up with baseten loops run list; you need it to attach your training script:

View your runs

baseten loops run list

Output

                             Loops Runs
╭─────────┬─────────────────┬────────┬─────────────────────╮
│ Run ID  │ Base Model      │ Status │ Created At          │
├─────────┼─────────────────┼────────┼─────────────────────┤
│ def4567 │ Qwen/Qwen3-0.6B │ ACTIVE │ 2026-07-20 15:51:58 │
╰─────────┴─────────────────┴────────┴─────────────────────╯

To create a multi-replica trainer:

Create a session with POST /v1/loops/sessions and capture its ID:

Create a session

SESSION_ID=$(curl -s https://api.baseten.co/v1/loops/sessions \
  -H "Authorization: Bearer $BASETEN_API_KEY" \
  -H "Content-Type: application/json" -d '{}' | jq -r .session.id)

Create the trainer with replicas set:

Create the trainer

curl -s https://api.baseten.co/v1/loops/trainers \
  -H "Authorization: Bearer $BASETEN_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{
    \"session_id\": \"$SESSION_ID\",
    \"base_model\": \"Qwen/Qwen3-0.6B\",
    \"replicas\": 2
  }"

Output

{
  "run": {
    "id": "def4567",
    "session_id": "abc1234",
    "deployment_id": "def456",
    "name": "Qwen/Qwen3-0.6B",
    "base_model": "Qwen/Qwen3-0.6B",
    "base_url": "https://trainer-def456.api.baseten.co/trainer",
    "created_at": "2026-07-20T22:51:58.115Z",
    "sampler": null
  }
}

To attach your training script to the trainer: Start your training script with the run ID in its environment, so ServiceClient reuses that trainer instead of starting a new single-replica one:

LOOPS_REUSE_FROM_RUN_ID=def4567 python train.py

Each replica is a full copy of the trainer with its own GPUs, so GPU time and cost scale with replicas: replicas=4 uses four times the GPUs of a single replica. Your training loop doesn’t change. Each forward_backward() batch is sharded across the replicas automatically, so batches don’t need to divide evenly by the replica count: a 3-datum batch on 2 replicas puts 2 datums on one worker and 1 on the other, and the result still contains one entry per datum, in order. Gradients synchronize across replicas at optim_step(), which advances policy_version once for the whole trainer. Size batches to at least the replica count; a smaller batch works but leaves the extra replicas idle for that step. Samplers scale the same way. Pass sampler_num_replicas to create_lora_training_client() to set the replica count for the run’s paired sampler, or num_replicas to ServiceClient.create_sampling_client() for a standalone sampler. The count applies as both the minimum and maximum, and takes effect when the sampler is created. Omit it and the backend default applies.

Spot capacity

For Loops deployments, spot capacity uses GPUs that Baseten can reclaim. Use it for interruptible training that can continue from a checkpoint. Baseten can reclaim the GPUs at any time, which stops the run. The availability_model setting controls whether deployments use non-preemptible or spot capacity. It defaults to dedicated. To allow spot capacity for deployments, configure ServiceClient:

Python SDK
Environment variable

To allow spot capacity:Pass availability_model when you create the client:

from baseten.loops import AvailabilityModel, ServiceClient

service_client = ServiceClient(
    availability_model=AvailabilityModel.spot,
)

To allow spot capacity:Set LOOPS_AVAILABILITY_MODEL for the process:

LOOPS_AVAILABILITY_MODEL=spot python train.py

If you set both options, the value passed to ServiceClient takes precedence over LOOPS_AVAILABILITY_MODEL. Loops reuses a warm trainer only if it has the same capacity type. Otherwise, it provisions a new spot trainer. For deployments, the setting applies to trainers and standalone samplers created by the client. A paired sampler inherits its run’s capacity type. This setting doesn’t choose the availability mode; set scope to deployment to make sure it applies. Baseten can preempt spot trainers, so save checkpoints during training. Loops doesn’t restart a preempted run automatically, but you can create a new trainer from the latest checkpoint. See Checkpoints. Spot capacity must be enabled for your workspace. Contact support to enable it. For Training Jobs, see Spot capacity. If spot trainer creation fails, follow capacity and quota errors to distinguish unavailable GPUs from your organization’s trainer limit.

Reuse a warm run

Each new ServiceClient creates a session. A new dedicated run starts its own trainer and sampler deployments and waits for them to come up. A new run can attach to a prior run’s warm trainer and paired sampler instead of starting new ones.

Python SDK
Environment variable

To reuse a warm run:Pass the earlier run’s ID when you create the client:

from baseten.loops import ServiceClient

service_client = ServiceClient(reuse_from_run_id="<run_id>")

To reuse a warm run:Set LOOPS_REUSE_FROM_RUN_ID for the process. ServiceClient reads it when you don’t pass reuse_from_run_id:

LOOPS_REUSE_FROM_RUN_ID="<run_id>" python train.py

Session-based trainer reuse still works but is deprecated in favor of the run ID: ServiceClient(reuse_from_session_id=...), the LOOPS_REUSE_FROM_SESSION_ID environment variable, and the reuse_from_session_id field on POST /v1/loops/trainers. When both are set, the run ID wins and the session ID is ignored. Standalone sampler creation ignores session reuse and creates a new logical sampler. Reuse is best-effort. The named run or session must belong to the same team. If the earlier run’s trainer isn’t available, for example because it’s still starting, stopped, or failed, Loops starts a new one and the call still succeeds. Reuse controls deployment startup. It doesn’t replace saving and loading checkpoints to preserve training state.

Overlap trainer and sampler startup

In the dedicated mode, an RL loop can reach its first rollout while the sampler still needs to start. To bring the trainer and sampler up at roughly the same time, kick off the first save_weights_and_get_sampling_client() as an async task as soon as the trainer is ready, and train while the sampler provisions:

overlap_startup.py

import asyncio
from baseten.loops import ServiceClient

async def main():
    service_client = ServiceClient()
    training_client = await service_client.create_lora_training_client_async(
        base_model="Qwen/Qwen3-0.6B",
    )
    # The sampler serves the base model until the trainer publishes trained weights.
    sampler_task = asyncio.create_task(
        training_client.save_weights_and_get_sampling_client_async()
    )
    # ... run forward_backward / optim_step steps here ...
    sampling_client = await sampler_task

asyncio.run(main())

If you’d rather not manage the async task, pass with_sampler=True to create_lora_training_client(): the paired sampler starts with the trainer and serves the base model until the trainer publishes weights. By default, the sampler starts at the first sampling-client request, so a run that only trains doesn’t pay for sampler GPUs. Every long-running SDK method has an *_async counterpart with the same arguments, so the rest of the loop works the same way; see the note on TrainingClient.

Usage and logs

Use baseten loops usage to inspect dedicated GPU allocations. Deployment logs cover the run’s trainer and sampler deployments. See Loops troubleshooting for request errors and recovery.

How a dedicated run ends

A dedicated run bills for GPU time while its trainer and sampler deployments run, including idle time. Deactivate the run when you finish to shut down those deployments. Closing the training client stops its keepalive but doesn’t deactivate the run. Without explicit deactivation, the run ends after a period of inactivity, 15 minutes by default. Saved checkpoints remain available after the run ends. See Deactivate a run for shutdown commands and status checks.

Configure the inactivity timeout

Set the timeout when you create a dedicated trainer with POST /v1/loops/trainers. The timeout starts when the run stops receiving requests or client keepalives.

integer

default:900

Inactivity period before the dedicated run ends, in seconds. Accepts values from 1 to 3600. Explicit deactivation ends the run without waiting for this timeout.

To create a run with a one-hour inactivity timeout:

Install jq, set BASETEN_API_KEY to your workspace API key, then create a session:

SESSION_ID=$(curl --fail-with-body --silent --show-error \
  https://api.baseten.co/v1/loops/sessions \
  --header "Authorization: Bearer $BASETEN_API_KEY" \
  --header "Content-Type: application/json" \
  --data '{}' | jq -er '.session.id')

Create the dedicated trainer using the returned session ID:

curl --fail-with-body --silent --show-error \
  https://api.baseten.co/v1/loops/trainers \
  --header "Authorization: Bearer $BASETEN_API_KEY" \
  --header "Content-Type: application/json" \
  --data "{
    \"session_id\": \"$SESSION_ID\",
    \"base_model\": \"zai-org/GLM-5.3-Flash\",
    \"scope\": \"deployment\",
    \"scale_down_delay_seconds\": 3600
  }"

The response includes run.id. Keep it to attach your training script with LOOPS_REUSE_FROM_RUN_ID or to deactivate the run when you finish. This run bills for GPU time, including the inactivity period.

Get started

Model APIs

Baseten Grid

Inference

Frontier Gateway

Training

Sandboxes

Development

Deployment

Engines

Model Labs

Organization

Hosting options

Observability

Troubleshooting

Dedicated Loops

Scale the trainer

Spot capacity

Reuse a warm run

Overlap trainer and sampler startup

Usage and logs

How a dedicated run ends

Configure the inactivity timeout

​Scale the trainer

​Spot capacity

​Reuse a warm run

​Overlap trainer and sampler startup

​Usage and logs

​How a dedicated run ends

​Configure the inactivity timeout

Scale the trainer

Spot capacity

Reuse a warm run

Overlap trainer and sampler startup

Usage and logs

How a dedicated run ends

Configure the inactivity timeout