Skip to main content
A Loops session pairs a trainer with a sampler so that trained weights move to the sampler as soon as they exist. The trainer runs forward, backward, and optimizer steps; the sampler generates from current weights. Both live inside the same session and share a weight-sync path. Each spins up on its own. Creating a training client provisions only the trainer; Loops creates the paired sampler when you first create a sampling client. Unlike offline training, where you finish a run, save a checkpoint, and then reload weights into a separate inference process, Loops keeps the sampler in sync throughout. When the trainer saves weights, the sampler picks them up without restarting. The sampler you query at step 100 is running the same weights the trainer committed.

​
Sessions

A Loops session is the container resource that scopes a training project’s work. It holds the trainer and sampler for a given base model and links them to a Baseten training project. Everything you create within a session (trainers, samplers, checkpoints) is queryable through that session’s ID. For the full route reference, see the Loops API overview.

​
Trainers

A trainer is the process that runs the training computation: forward pass, backward pass, and optimizer step. It owns the model weights for the duration of the session and writes checkpoints to a dedicated storage path under a bt://loops:… URI; the Checkpoints section covers the format. There is one trainer per session per base model. You don’t pick the hardware yourself. The trainer defaults to the longest sequence length the base model supports, and Baseten picks the GPU type, GPU count, and node topology (single-node or multi-node) to match. Creating a training client provisions the trainer and blocks until it’s ready. Samplers come up separately; see Samplers. Over HTTP, POST /v1/loops/trainers provisions the trainer. The one sizing knob you control is the replica count; see Scale the trainer. The API route calls a trainer a “run”. Both the HTTP API and the SDK identify it by its run ID: the API takes a run_id query parameter, and the SDK exposes the same value as TrainingClient.run_id.

​
Scale the trainer

For more training throughput, run the trainer as multiple data-parallel replicas. You set the replica count when you create the trainer: the Python SDK takes a replicas argument (default 1) on create_lora_training_client(), the truss CLI takes a --replicas flag, and the HTTP API takes a replicas field on POST /v1/loops/trainers.
To provision a multi-replica trainer:Pass replicas when you create the training client:
Replica counts above 1 always provision fresh capacity, so reuse_from_session_id is ignored for these calls.
To attach your training script to the trainer: Start your training script with the session ID in its environment, so ServiceClient reuses that trainer instead of provisioning a fresh single-replica one:
Each replica is one full copy of the base model’s node group, so the trainer runs the preset node count times replicas nodes. replicas=2 on a single-node preset runs 2 nodes; replicas=4 on a 4-node preset runs 16 nodes with 4 data-parallel workers. Your training loop doesn’t change. Each forward_backward() batch is sharded across the replicas automatically, so batches don’t need to divide evenly by the replica count: a 3-datum batch on 2 replicas puts 2 datums on one worker and 1 on the other, and the result still contains one entry per datum, in order. Gradients synchronize across replicas at optim_step(), which advances policy_version once for the whole trainer. Size batches to at least the replica count; a smaller batch works but leaves the extra replicas idle for that step.

​
Samplers

A sampler generates text. How you create one determines whether it follows your training run’s weights. A paired sampler serves a run’s latest trained weights. You don’t create it directly. The first time you request a sampling client from a training client, with save_weights_and_get_sampling_client() or create_sampling_client(), Loops provisions a sampler and links it to the run. That first call kicks off the sampler’s cold start, and your first sample() calls wait it out; later calls return a client on the same sampler. After that, the sampler picks up new weights whenever the trainer saves them (see How weight sync works). It doesn’t restart when weights change, so generation latency stays low and you can interleave training steps and rollout calls without coordinating reloads. A run that never samples never provisions a sampler, so you don’t pay for GPUs it would sit on. A standalone sampler serves fixed weights with no trainer behind it and no weight sync. Create one with ServiceClient.create_sampling_client(), from a base model (base_model="...") or from a saved checkpoint (model_path="bt://..."). Use it to evaluate a checkpoint after a run ends, or to generate from a base model without starting a run. In the Training UI, a standalone sampler appears on its own rather than under a run.

​
Overlap trainer and sampler startup

An RL loop that trains for a while before its first rollout hits the sampler cold start mid-run, right when you first want generations. To bring the trainer and sampler up at roughly the same time, kick off the first save_weights_and_get_sampling_client() as an async task as soon as the trainer is ready, and train while the sampler provisions:
overlap_startup.py
Every long-running SDK method has an *_async counterpart with the same arguments, so the rest of the loop works the same way; see the note on TrainingClient.

​
Checkpoints

Every time the trainer saves weights, Loops creates a checkpoint identified by a bt://loops:<run_id>/(weights|sampler_weights)/<checkpoint_name> URI. The URI encodes the run ID, the checkpoint target (trainer weights or sampler weights), and the checkpoint name, for example, bt://loops:k4q95w5/weights/step-100. You pass this URI to create a trainer or sampler from a prior checkpoint, or to deploy weights to inference. Loops stores checkpoints as folders on disk, not as single archives. Listing checkpoint files returns a paginated response of presigned URLs, one URL per file in the folder; control pagination with the page_size and page_token query parameters. Fetch each file individually rather than downloading and unpacking one archive. The route is GET /v1/loops/checkpoints/{checkpoint_id}/files.

​
Start and shut down from the CLI

Trainers and samplers stay live as you train, and weights you commit stream into the run’s paired sampler in place, with no separate deploy step for inference. You’re billed for their GPUs until you shut them down. Start a trainer with truss loops push <base_model>. Shut a trainer or sampler down with truss loops deactivate <deployment_id>, using the deployment ID from truss loops view.

​
Reuse infrastructure across sessions

By default, every new ServiceClient creates a fresh session, and the trainers and samplers it creates are provisioned fresh. Each re-run of a script pays the full cold-start cost. A session can opt in to reusing a prior session’s trainer and sampler instead of provisioning new ones. Three equivalent surfaces:
  • SDK kwarg: ServiceClient(reuse_from_session_id="2qjl22w").
  • Environment variable: LOOPS_REUSE_FROM_SESSION_ID=2qjl22w. ServiceClient reads this when no kwarg is passed.
  • HTTP request: reuse_from_session_id field on POST /v1/loops/trainers and POST /v1/loops/samplers.
Reuse is best-effort. The named session must belong to the same team. If the prior trainer is still deploying, stopped, failed, or unhealthy, the backend falls back to provisioning fresh and the call still succeeds. See Skip the cold start on re-runs for the script workflow.

​
How weight sync works

When a trainer saves weights, the paired sampler picks them up through a vLLM plugin. The plugin handles the sync without restarting the sampler, so generation can resume immediately at the new weights.

​
Supported base models

Loops supports a curated set of Hugging Face base models with verified LoRA configurations. See Supported base models for the current list and sequence-length limits.