Skip to main content
This example fine-tunes Qwen3.6-35B-A3B with supervised fine-tuning (SFT) on 2,000 tool-use conversations from Agent-Ark/Toucan-1.5M. The conversations use real MCP servers and include parallel calls, multi-step chains, multi-turn dialogues, and requests no offered tool can serve. It then samples the base model and the trained adapter on 200 held-out conversations and scores whether each called the right tools with the right arguments. It follows Prepare data for supervised fine-tuning and Evaluate a model on a public dataset, using the Loops cookbook.

​
Prerequisites

  • A Baseten workspace with Loops and its API key.
  • uv and Python 3.12 or later.

​
Install the cookbook

Install the Loops cookbook from PyPI into a uv project. The cookbook needs Python 3.12 or later, installs the baseten-loops SDK, and provides the loops command. loops runs both recipes; each training config names its recipe with recipe = "rl" or recipe = "sft". Create a project first if you don’t have one:
The package includes the cookbook’s examples. Run one using its bundled config path, such as examples/counting/train.toml, from your project’s directory. To change an example, copy its directory into your project at the same path. A config in your project wins over the installed one, and loops check prints the directory of the config it used. Create a .env file in your project’s directory and set BASETEN_API_KEY to a workspace API key. The commands load .env from the training config’s directory or the directory you run them from. Never commit .env. A run uses the serverless availability mode when your workspace has serverless access to the model, and the dedicated mode otherwise. To choose a mode, add BASETEN_LOOPS_SCOPE=serverless or BASETEN_LOOPS_SCOPE=deployment to .env.

​
Build the dataset

examples/sft_toucan/make_data.py downloads part of Toucan and rewrites each conversation into the chat JSONL format described in Format conversations as chat JSONL. It splits by tool set, so the validation conversations use tool combinations the training data never saw. That keeps near-duplicate conversations from landing on both sides of the split. make_data.py writes the dataset beside itself, and evaluate.py writes its results there too, so copy the example into your project first. To copy it, run the following command:
To build the training and validation files, run the following command from your project’s directory:
The script reports the split and every row it left out, with the reason:
The irrelevant conversations are requests no offered tool can serve, where the right answer calls no tool. They’re 15% of each split, so the model learns when not to call a tool as well as when to call one.

​
Check what trains

examples/sft_toucan/sft.toml sets the model, data, and schedule:
examples/sft_toucan/sft.toml
Toucan has no reasoning, so thinking is off. With it on, Qwen3.6 would learn to write an empty reasoning block before every answer. To render every conversation with Qwen3.6’s chat template and report what trains, run the following command. It launches nothing:
The report starts with the files the check used. Confirm that Config: names your copy in examples/sft_toucan/, not the installed one. The rest reports what trains:
About a third of the tokens train: the assistant turns. The longest sequence, 3,582 tokens, fits within max_seq_len = 4096. The trained spans of the first row end with <|im_end|>, so each turn learns to stop.

​
Train

To train for one epoch, run the following command. The run uses the serverless availability mode, billed for the tokens you train, when your workspace has serverless access to Qwen3.6-35B-A3B, and the dedicated mode, billed for GPU time, otherwise.
The run scores validation loss before the first update and every 10 updates. The run’s log prints the Loops run ID and the final checkpoint, and the run directory’s run.json holds the run ID. The last update’s sampler weights are always kept, as step-<N>. The cookbook’s recorded run of this config took 125 updates, about 12 minutes of training once the trainer was up. val/nll fell from 0.914 before the first update to 0.795, mostly in the first 20 updates, and was flat over the last 20.

​
Compare the adapter with the base model

Validation loss shows the adapter fits the data. To check that it does the task better, sample both models on the validation conversations. examples/sft_toucan/evaluate.py takes each conversation’s first assistant turn, samples it greedily from the base model and from the adapter, and scores four checks against the data:
  • decision: calls a tool exactly when the data does.
  • tools: calls the same tools.
  • arguments: calls the same tools with the same arguments.
  • well_formed: stops, and every call parses.
To run it, pass the adapter’s checkpoint path, with the run ID and final step from the training run:
The script starts a sampler for each model and deactivates both when it finishes. On the 200 validation conversations, the recorded run scored: The data’s own replies score 98.5% well_formed under Qwen3.6’s parser, so the adapter reached that ceiling. The largest change is on requests no tool can serve: the base model reaches for an unrelated tool anyway, and the adapter answers directly. evaluate.py measures agreement with the data on first turns. It doesn’t run tools or score later turns. Before you ship a model trained this way, build an evaluation that covers your own tools and the rules your task needs. See Evaluate a model.

​
Shut down the run

loops run leaves the Loops run active. On serverless, deactivation is optional because idle runs incur no token charges. On dedicated capacity, deactivate the run when you finish to stop GPU billing without waiting for automatic shutdown. To end the run and its paired sampler in either mode, run the following command with your run directory:
The command prints the deactivated run ID. Saved checkpoints remain available.

​
Next steps