> ## Documentation Index
> Fetch the complete documentation index at: https://docs.baseten.co/llms.txt
> Use this file to discover all available pages before exploring further.

# Hyperparameters

> Choose run, optimizer, loss, and sampling settings, and apply a learning-rate schedule in your training loop.

Run settings last for a Loops run. Optimizer, loss, and sampling settings apply to each call, so your training script can change them between steps. Start from the defaults, change one setting at a time, and compare checkpoints on a [held-out evaluation](https://docs.baseten.co/post-training/evaluate.md).

## Run settings

Set these when you create the training client with [`create_lora_training_client()`](https://docs.baseten.co/reference/sdk/loops/service-client):

**rank** `integer` (default: 32)

LoRA rank: the width of the adapter's low-rank matrices. Creating a run above the model's maximum rank fails with an error that names the limit. With automatic mode selection, a rank serverless doesn't support uses dedicated capacity. With `scope="serverless"`, the request fails instead. See [Availability modes](https://docs.baseten.co/loops/availability-modes/overview.md#choose-a-mode).

**max\_seq\_len** `integer`

Longest sequence the run handles, in tokens. Omit it to use the model's default. Check the [model catalog](https://docs.baseten.co/loops/supported-models.md) for maximum sequence lengths and your workspace's enabled limits.

**seed** `integer`

Random seed for reproducibility. Defaults to `None`.

```python theme={"system"}
from baseten.loops import ServiceClient

training_client = ServiceClient().create_lora_training_client(
    base_model="zai-org/GLM-5.3-Flash",
    rank=32,
    max_seq_len=16384,
    seed=42,
    scope="serverless",
)
```

## Optimizer settings

Pass [`AdamParams`](https://docs.baseten.co/reference/sdk/loops/types) to [`optim_step()`](https://docs.baseten.co/reference/sdk/loops/training-client) on every step. The trainer uses Adam:

**learning\_rate** `number` (default: 0.0001)

Optimizer step size.

**beta1** `number` (default: 0.9)

Decay rate of the first-moment estimate.

**beta2** `number` (default: 0.95)

Decay rate of the second-moment estimate.

**eps** `number` (default: 1e-12)

Value added to the denominator for numerical stability.

**weight\_decay** `number` (default: 0)

Weight decay applied to the adapter weights.

**grad\_clip\_norm** `number` (default: 1)

Clip the gradient's global norm to this value. Set `0.0` to turn off clipping.

### Apply a learning-rate schedule

The trainer applies the values you pass on each call and has no learning-rate schedule of its own. Compute the rate in your script to warm up and then decay it.

In [SFT with the SDK](https://docs.baseten.co/examples/sft-sdk-loop.md#run-the-training-loop), replace the training-loop block with the following code. It uses the `training_client` and tokenized `data` prepared earlier in that example, derives the step count from the dataset, and prints the rate and loss for each batch:

```python theme={"system"}
import math

from baseten.loops import AdamParams

BATCH_SIZE = 8
total_steps = math.ceil(len(data) / BATCH_SIZE)
warmup_steps = min(2, total_steps)
peak_learning_rate = 4e-5

for step, start in enumerate(range(0, len(data), BATCH_SIZE)):
    if step < warmup_steps:
        rate = peak_learning_rate * (step + 1) / warmup_steps
    else:
        progress = (step - warmup_steps + 1) / (total_steps - warmup_steps)
        rate = peak_learning_rate * 0.5 * (1 + math.cos(math.pi * progress))

    batch = data[start : start + BATCH_SIZE]
    fb = training_client.forward_backward(data=batch).result(timeout=600.0)
    training_client.optim_step(AdamParams(learning_rate=rate)).result(timeout=600.0)
    print(f"step {step + 1} learning_rate={rate:.2e} loss={fb.loss:.4f}")
```

For eight batches, the first two rates are `2e-5` and `4e-5`. The remaining six decay to zero. Use your evaluation to choose the peak rate and warmup length.

Batch size is also a choice your loop makes. [`forward_backward()`](https://docs.baseten.co/reference/sdk/loops/training-client) accumulates gradients until the next `optim_step()`, so the examples you pass across those calls make up one batch.

## Loss settings

Choose a loss with `loss_fn` on [`forward_backward()`](https://docs.baseten.co/reference/sdk/loops/training-client), and tune it with `loss_fn_config`. [Loss functions](https://docs.baseten.co/loops/loss-functions.md) defines each loss's keys and defaults.

> [!WARNING]
>
> The trainer ignores `loss_fn_config` keys it doesn't recognize, so a misspelled key leaves the default in place. Check key names against [Loss functions](https://docs.baseten.co/loops/loss-functions.md).

## Sampling settings

Pass [`SamplingParams`](https://docs.baseten.co/reference/sdk/loops/types) to [`sample()`](https://docs.baseten.co/reference/sdk/loops/sampling-client), with `num_samples` for the number of completions per prompt:

**temperature** `number` (default: 1)

Sampling temperature.

**top\_p** `number` (default: 1)

Nucleus sampling threshold.

**top\_k** `integer` (default: -1)

Sample from the `top_k` most likely tokens. Zero or a negative value turns off this filter.

**max\_tokens** `integer`

Maximum tokens to generate. Defaults to `None`.

**seed** `integer`

Random seed for reproducible samples. Defaults to `None`.

**stop** `string | string[] | integer[]`

Stop sequences, as strings or token IDs. Defaults to `None`.

**structured\_outputs** `object`

Constrain generation to a structure, such as a JSON schema or regular expression. Defaults to `None`, which leaves generation unconstrained. See [`SamplingParams`](https://docs.baseten.co/reference/sdk/loops/types) for the accepted format.

## Settings Loops fixes

* **LoRA alpha** is fixed at `32`, so the adapter's output is scaled by `32 / rank`. A higher rank lowers that scale, so retune the learning rate when you change the rank.
* **LoRA target modules** are fixed for each model family.
* **Parallelism and hardware** are chosen for each model and sequence length.

## Next steps

* [Loss functions](https://docs.baseten.co/loops/loss-functions.md): Look up the keys and defaults for each loss.
* [Evaluate a model](https://docs.baseten.co/post-training/evaluate.md): Compare runs with different settings on a held-out set.
* [Check whether an RL run is learning](https://docs.baseten.co/loops/rl-run-health.md): Read the metrics that show whether a setting change helped.