> ## Documentation Index
> Fetch the complete documentation index at: https://docs.baseten.co/llms.txt
> Use this file to discover all available pages before exploring further.

# Train a coding agent on Terminal-Bench

> Train GLM-5.3-Flash with asynchronous reinforcement learning as a multi-turn bash agent on Terminal-Bench 2.1 tasks in Blaxel sandboxes.

This example trains [GLM-5.3-Flash](https://docs.baseten.co/loops/supported-models.md) with reinforcement learning (RL) as a coding agent. The agent works through [Terminal-Bench 2.1](https://github.com/harbor-framework/terminal-bench-2-1) tasks with one `bash` tool, each task in its own [Blaxel](https://blaxel.ai) sandbox. Each task's own tests grade the result.

The tasks are [Harbor](https://github.com/laude-institute/harbor) tasks, so Harbor builds each environment and runs each task's verifier. You write only the agent. Start with the [tool-using agent example](https://docs.baseten.co/examples/rl-tool-agent.md) if you haven't run RL with the Loops cookbook before. This run is much larger, and some tasks take hours.

## Prerequisites

* A Baseten workspace with Loops access to `zai-org/GLM-5.3-Flash`, and its [API key](https://docs.baseten.co/organization/api-keys.md). GLM-5.3-Flash trains in either [availability mode](https://docs.baseten.co/loops/availability-modes/overview.md).
* A Blaxel workspace and API key, with capacity for about 50 concurrent sandboxes at 8 GB of memory each.
* [uv](https://docs.astral.sh/uv/) and Python 3.12 or later.

## Install the cookbook

Install the [Loops cookbook](https://pypi.org/project/loops-cookbook/) from PyPI into a [uv](https://docs.astral.sh/uv/) project. The cookbook needs Python 3.12 or later, installs the [`baseten-loops`](https://pypi.org/project/baseten-loops/) SDK, and provides the `loops` command. `loops` runs both recipes; each training config names its recipe with `recipe = "rl"` or `recipe = "sft"`. Create a project first if you don't have one:

```sh theme={"system"}
uv init --python 3.12 loops-project
cd loops-project
uv add loops-cookbook
```

The package includes the cookbook's examples. Run one using its bundled config path, such as `examples/counting/train.toml`, from your project's directory. To change an example, copy its directory into your project at the same path. A config in your project wins over the installed one, and `loops check` prints the directory of the config it used.

Create a `.env` file in your project's directory and set `BASETEN_API_KEY` to a [workspace API key](https://docs.baseten.co/organization/api-keys.md). The commands load `.env` from the training config's directory or the directory you run them from. Never commit `.env`.

A run uses the serverless [availability mode](https://docs.baseten.co/loops/availability-modes/overview.md) when your workspace has serverless access to the model, and the dedicated mode otherwise. To choose a mode, add `BASETEN_LOOPS_SCOPE=serverless` or `BASETEN_LOOPS_SCOPE=deployment` to `.env`.

Install Harbor with its Blaxel environment, and add `BL_WORKSPACE` and `BL_API_KEY` to `.env`:

```sh theme={"system"}
uv add "loops-cookbook[harbor-blaxel]"
```

The config mirrors metrics to Weights & Biases. To use it, install the extra with the following command, then add `WANDB_API_KEY` to `.env`:

```sh theme={"system"}
uv add "loops-cookbook[harbor-blaxel,wandb]"
```

To turn it off, start the run with the following command:

```sh theme={"system"}
uv run loops run examples/terminal_bench/train.toml --output.wandb_project ""
```

Every setting in the config has a command-line flag named by its section and key, so you can change the example without copying it. `uv run loops run --help` lists the flags.

## Read the agent

`examples/terminal_bench/terminal_agent.py` defines `TerminalAgent`, a Harbor agent that calls the policy directly. Its system prompt describes a software engineer in a sandbox. Its one tool runs a bash command in the task's environment and returns the exit code, stdout, and stderr, each truncated to `max_output_bytes`. The agent loops until the model replies without a tool call or reaches `max_turns`.

The agent decides what happens when a generation runs out of tokens. `TerminalAgent` discards a reply the token limit cut and samples the turn again, up to three times, so a cut generation never trains. After three cut attempts, the episode ends as `truncated` and is still graded. The [tool-using agent](https://docs.baseten.co/examples/rl-tool-agent.md) makes the opposite choice: it ends and grades the episode, and the cut generation trains with the rest. Neither is a default; choose what fits your task.

`examples/terminal_bench/harbor.yaml` pins the Terminal-Bench 2.1 dataset to one commit, selects the cookbook's Blaxel environment, and passes the agent's limits:

```yaml examples/terminal_bench/harbor.yaml wrap theme={"system"}
datasets:
  - repo: https://github.com/harbor-framework/terminal-bench-2-1@7131e4375048a0e408a8fb404b5f499d726b695b
    path: tasks
agents:
  - import_path: terminal_agent:TerminalAgent
    kwargs:
      max_turns: 64
      command_timeout: 120
      max_output_bytes: 16384
```

The job must use `loops_cookbook.harbor.blaxel:BlaxelEnvironment` rather than Harbor's own Blaxel environment. The cookbook's class records every sandbox in the run's resource journal, so `loops cleanup` can release them after a crash.

## Read the training config

`examples/terminal_bench/train.toml` trains asynchronously: the sampler collects rollouts for the next update while the trainer applies the current one. The settings that shape the run:

```toml examples/terminal_bench/train.toml theme={"system"}
[model]
base_model = "zai-org/GLM-5.3-Flash"
# Prime would auto-detect GLM-5.1.
renderer = "loops_cookbook.llm.renderer_extensions.glm53:renderer"
max_seq_len = 262144
lora_rank = 32
renderer_chat_template_kwargs = { reasoning_effort = "low", clear_thinking = false }

[training]
mode = "async"
max_steps = 50
learning_rate = 0.00001
adam_epsilon = 0.00000001
router_replay = false
group_size = 8
max_group_losses = 2
groups_per_step = 4
max_consecutive_unusable_groups = 100
max_steps_off_policy = 3
max_policy_delta_in_trajectory = 3

[compute]
# Both of these are account-specific opt-ins. Spot capacity can be reclaimed mid-run
# (the run stops; nothing restarts it). Sticky sessions need sampler affinity routing
# enabled for your Loops workspace; without it, requests fail. Set both to false if
# you are not sure your account has them.
spot_job = true
sampler_replica_affinity = true
```

* Each update trains 4 groups of 8 attempts, for 50 updates.
* `max_steps_off_policy = 3` lets a trained rollout come from a policy up to three updates behind the trainer. Async runs trade that staleness for throughput. Read `freshness/lag_mean` to see the lag the run actually trained on.
* `max_group_losses = 2` lets a group train when up to 2 of its 8 sandboxes fail.

Turn both `[compute]` options off unless your workspace has spot capacity and sampler affinity routing enabled. With affinity on and routing off, sampling requests fail. To turn them off, start the run with the following command:

```sh theme={"system"}
uv run loops run examples/terminal_bench/train.toml \
  --compute.spot_job false \
  --compute.sampler_replica_affinity false
```

`spot_job` requests spot capacity for deployments; it doesn't choose between the serverless and dedicated modes.

## Scale out the run

GLM-5.3-Flash uses the serverless mode by default when your workspace has access, which needs no capacity request and suits a first run. A long asynchronous run that needs more training and rollout throughput belongs in the dedicated mode, where you set the trainer and sampler replica counts.

To use the dedicated mode, add the following line to `.env`:

```sh .env theme={"system"}
BASETEN_LOOPS_SCOPE=deployment
```

Then raise the replica counts when you start the run, for example as follows. `trainer_replicas` adds data-parallel trainer replicas for update throughput, and `sampler_replicas` adds sampler replicas for rollout throughput:

```sh theme={"system"}
uv run loops run examples/terminal_bench/train.toml \
  --compute.trainer_replicas 2 \
  --compute.sampler_replicas 4
```

Each trainer replica is a full copy of the model's GPU allocation, and deployments bill for GPU time, including idle time. To plan capacity for a large run in the dedicated mode, [request more access](https://docs.baseten.co/loops/overview.md#request-more-access). [Scale the trainer](https://docs.baseten.co/loops/availability-modes/dedicated.md#scale-the-trainer) explains how batches split across replicas.

## Run the example

To check the configuration, run the following command. It downloads the dataset, validates the job and agent options, and launches nothing. It warns about any task that declares more disk than the sandbox provides:

```sh theme={"system"}
uv run loops check examples/terminal_bench/train.toml
```

To train, run the following command. The run uses the serverless mode, billed for tokens, when your workspace has serverless access to the model, and the dedicated mode, billed for GPU time, otherwise. The run also starts one Blaxel sandbox per running attempt: 48 at a time, plus up to 5 started ahead so a freed slot is filled at once. Sandboxes bill while they run:

```sh theme={"system"}
uv run loops run examples/terminal_bench/train.toml
```

Tasks vary from minutes to hours. The config allows each rollout up to 6 hours (`rollout_timeout_seconds = 21600`) and reports pipeline progress every 60 seconds, so a slow phase is distinguishable from a stall.

## Read the results

Monitor `reward/mean`, `discard/groups/no_variance`, and `rollout/*` in W\&B or in the run directory's `steps.jsonl`. Terminal-Bench tasks vary widely in difficulty, so expect many `no_variance` groups from tasks the starting model always fails. See [Check whether an RL run is learning](https://docs.baseten.co/loops/rl-run-health.md).

To tell whether the trained agent is a better coding agent, score a saved checkpoint on tasks it didn't train on. Training reward on Terminal-Bench measures the tasks in the training set. See [Evaluate a model](https://docs.baseten.co/post-training/evaluate.md).

## Add a loss on command output

`examples/terminal_bench/train_echo.toml` runs the same job with [ECHO](https://arxiv.org/abs/2605.24517), a custom loss defined in `echo_loss.py`. It adds a next-token loss on the output the agent's commands print, weighted by `observation_weight`, to the importance-sampling loss on the agent's own tokens. The module's docstring describes where it differs from the paper.

To install the loss's extra, run the following command:

```sh theme={"system"}
uv add "loops-cookbook[harbor-blaxel,custom-loss]"
```

To check the configuration, run the following command:

```sh theme={"system"}
uv run loops check examples/terminal_bench/train_echo.toml
```

To start the run, run the following command:

```sh theme={"system"}
uv run loops run examples/terminal_bench/train_echo.toml
```

## Shut down the run

`loops run` leaves the Loops run active. On serverless, deactivation is optional because idle runs incur no token charges. On dedicated capacity, [deactivate the run](https://docs.baseten.co/loops/deactivate-a-run.md) when you finish to stop GPU billing without waiting for automatic shutdown. To end the run and its paired sampler in either mode, run the following command with your run directory:

```sh theme={"system"}
uv run loops deactivate "runs/terminal-bench/<run directory>"
```

The command prints the deactivated run ID. Saved checkpoints remain available.

## Release sandboxes after a crash

Sandboxes bill until they're deleted. If the run crashed, release its sandboxes with the following command:

```sh theme={"system"}
uv run loops cleanup examples/terminal_bench/train.toml \
  --state-dir "runs/terminal-bench/<run directory>/state"
```

## Next steps

* [Design RL environments and rewards](https://docs.baseten.co/loops/rl-environments.md): Graders, task sizing, and exploits.
* [Check whether an RL run is learning](https://docs.baseten.co/loops/rl-run-health.md): Metrics for long, asynchronous runs.
* [Deploy a checkpoint](https://docs.baseten.co/loops/deploy-checkpoints.md): Serve the trained adapter.