Skip to main content
Run settings last for a Loops run. Optimizer, loss, and sampling settings apply to each call, so your training script can change them between steps. Start from the defaults, change one setting at a time, and compare checkpoints on a held-out evaluation.

​
Run settings

Set these when you create the training client with create_lora_training_client():
​
integer
default:32
LoRA rank: the width of the adapter’s low-rank matrices. Creating a run above the model’s maximum rank fails with an error that names the limit. With automatic mode selection, a rank serverless doesn’t support uses dedicated capacity. With scope="serverless", the request fails instead. See Availability modes.
​
integer
Longest sequence the run handles, in tokens. Omit it to use the model’s default. Check the model catalog for maximum sequence lengths and your workspace’s enabled limits.
​
integer
Random seed for reproducibility. Defaults to None.

​
Optimizer settings

Pass AdamParams to optim_step() on every step. The trainer uses Adam:
​
number
default:0.0001
Optimizer step size.
​
number
default:0.9
Decay rate of the first-moment estimate.
​
number
default:0.95
Decay rate of the second-moment estimate.
​
number
default:1e-12
Value added to the denominator for numerical stability.
​
number
default:0
Weight decay applied to the adapter weights.
​
number
default:1
Clip the gradient’s global norm to this value. Set 0.0 to turn off clipping.

​
Apply a learning-rate schedule

The trainer applies the values you pass on each call and has no learning-rate schedule of its own. Compute the rate in your script to warm up and then decay it. In SFT with the SDK, replace the training-loop block with the following code. It uses the training_client and tokenized data prepared earlier in that example, derives the step count from the dataset, and prints the rate and loss for each batch:
For eight batches, the first two rates are 2e-5 and 4e-5. The remaining six decay to zero. Use your evaluation to choose the peak rate and warmup length. Batch size is also a choice your loop makes. forward_backward() accumulates gradients until the next optim_step(), so the examples you pass across those calls make up one batch.

​
Loss settings

Choose a loss with loss_fn on forward_backward(), and tune it with loss_fn_config. Loss functions defines each loss’s keys and defaults.
The trainer ignores loss_fn_config keys it doesn’t recognize, so a misspelled key leaves the default in place. Check key names against Loss functions.

​
Sampling settings

Pass SamplingParams to sample(), with num_samples for the number of completions per prompt:
​
number
default:1
Sampling temperature.
​
number
default:1
Nucleus sampling threshold.
​
integer
default:-1
Sample from the top_k most likely tokens. Zero or a negative value turns off this filter.
​
integer
Maximum tokens to generate. Defaults to None.
​
integer
Random seed for reproducible samples. Defaults to None.
​
string | string[] | integer[]
Stop sequences, as strings or token IDs. Defaults to None.
​
object
Constrain generation to a structure, such as a JSON schema or regular expression. Defaults to None, which leaves generation unconstrained. See SamplingParams for the accepted format.

​
Settings Loops fixes

  • LoRA alpha is fixed at 32, so the adapter’s output is scaled by 32 / rank. A higher rank lowers that scale, so retune the learning rate when you change the rank.
  • LoRA target modules are fixed for each model family.
  • Parallelism and hardware are chosen for each model and sequence length.

​
Next steps