Installation
Installbaseten-loops with Python 3.12 or later:
baseten.loops.
Authentication
Create a workspace API key for your workspace, then setBASETEN_API_KEY:
ServiceClient reads the key from the environment:
ServiceClient starts a session. The client creates runs and samplers when you request them. Loops picks an availability mode for each run and sampler. See the ServiceClient reference to pass an API key, select a team, or request infrastructure reuse.
See Supported base models for model IDs and sequence lengths.
Closing a Python client doesn’t deactivate the run. Deactivate the run to end it and its paired sampler. This stops a dedicated run’s GPU billing; an idle serverless run incurs no token charges.
Clients
ServiceClient: Create sessions, runs, and samplers, and retrieve checkpoints.TrainingClient: Run forward and backward passes, apply optimizer steps, and save weights.SamplingClient: Generate completions from current or version-pinned weights.
Commonly used methods
create_lora_training_client(): Create a training run and get aTrainingClient.forward_backward()andoptim_step(): Run one training step.save_state(): Save a checkpoint.save_weights_and_get_sampling_client(): Publish weights and get aSamplingClientthat selects them.sample(): Generate from the trained model.list_checkpoints(): List a run’s checkpoints.download_checkpoint(): Download a checkpoint to a local directory.
Reference
- Types: Training inputs, configuration, and result handles.
- Errors: SDK exception types and their causes.
- Helpers: Functions such as
attach_reference_logprobsandloops_log_kl_sample_train.
Related documentation
- Loops quickstart: Train five steps, generate a sample, and save a checkpoint.
- Programmatic training: Run training and evaluation from your application.
- Loops API: Manage runs, trainers, samplers, and checkpoints over HTTP.
- Loops run commands: Manage runs from your terminal.