> ## Documentation Index
> Fetch the complete documentation index at: https://docs.baseten.co/llms.txt
> Use this file to discover all available pages before exploring further.

# Prepare data for supervised fine-tuning

> Choose trustworthy targets, format conversations, decide which turns train, match serving, and render every row before you pay for a trainer.

Supervised fine-tuning (SFT) teaches a model to reproduce the assistant turns in your data. It can only be as good as those turns: a run that fits the data perfectly also learns every wrong answer, formatting slip, and missing tool call the data contains. Decide what a good answer is with an [evaluation](https://docs.baseten.co/post-training/evaluate.md) first, then use the same rules to build the data.

This guide continues the tool-calling assistant from the evaluation guide. The [SFT example](https://docs.baseten.co/examples/sft-tool-calling.md) runs it end to end.

## Choose targets you trust

Every assistant turn you train on is a target. Know where each one came from, because the source sets the ceiling:

* **Production outputs:** the model learns to reproduce what production does today, mistakes included. Training on them distills the current system; it doesn't improve on it. Filter them with your evaluation rules first.
* **Human-written or human-corrected outputs:** the strongest targets, and the most expensive. A verified slice is also what you calibrate an LLM judge against.
* **Outputs from a stronger model:** a common way to train a smaller, cheaper model on a task. Score them with your evaluation and keep only the ones that pass.
* **Public datasets:** useful for general skills such as tool use. Check that the format and behavior match what you serve.

Use your evaluation as a filter. Drop rows whose target breaks a rule, and drop rows that would teach one rule by breaking another. Remove every held-out row and its near-duplicates before training: a held-out example in the training data makes the evaluation measure memory instead of the task.

## Check the label balance

For classification and extraction tasks, count the distinct target values before training. When one target dominates, typically "no issue" or an empty extraction, the cheapest way to lower the loss is to output it every time. The training loss looks excellent and the model flags nothing.

Rebalance at the source by oversampling the rarer targets, and evaluate with per-class false-positive and false-negative rates rather than overall accuracy. On a dataset where 95% of rows are "no issue," a model that never flags anything is 95% accurate.

## Format conversations as chat JSONL

Write one conversation per line in OpenAI's chat format. OpenAI fine-tuning files, Hugging Face conversational datasets, and most trace exports from tools such as LangSmith and Langfuse already use it or convert in a few lines:

```jsonl train.jsonl wrap theme={"system"}
{"messages": [{"role": "system", "content": "You convert units."}, {"role": "user", "content": "12 miles in km?"}, {"role": "assistant", "content": "", "tool_calls": [{"id": "c1", "type": "function", "function": {"name": "convert", "arguments": "{\"value\": 12, \"from\": \"mi\", \"to\": \"km\"}"}}]}, {"role": "tool", "tool_call_id": "c1", "content": "19.312"}, {"role": "assistant", "content": "About 19.31 km.", "weight": 1}], "tools": [{"type": "function", "function": {"name": "convert", "description": "Convert a value between units.", "parameters": {"type": "object", "properties": {"value": {"type": "number"}, "from": {"type": "string"}, "to": {"type": "string"}}, "required": ["value", "from", "to"]}}}]}
```

* `tools` lists the tools offered in that conversation, in OpenAI's function format. Rows can offer different tools.
* `tool_calls` and `tool` messages follow OpenAI's format. Answer every call exactly once.
* `weight` on an assistant message sets whether that turn trains: `1` trains it, `0` keeps it as context only.
* `reasoning_content` on an assistant message carries that turn's reasoning, when your data has it.

The conversations can come from any model. Training re-renders the text with the trained model's own chat template.

To convert a Hugging Face dataset that already has `messages` and `tools` columns, run the following script. Reshape the columns inside the loop when your dataset names them differently:

```python make_data.py theme={"system"}
import json

from datasets import load_dataset

rows = load_dataset("<org>/<dataset>", split="train")
with open("train.jsonl", "w") as out:
    for row in rows:
        out.write(
            json.dumps({"messages": row["messages"], "tools": row.get("tools") or []})
            + "\n"
        )
```

For data in Amazon S3, copy the file down first, then convert it the same way. To copy it, run the following command:

```bash theme={"system"}
aws s3 cp "s3://<bucket>/<path>/train.jsonl" .
```

## Decide which turns train

User, system, and tool-result messages are never targets. Among assistant turns, choose deliberately, because the choice changes what the model learns:

| Train | When |
| - | - |
| Only the final assistant turn | The task is one decision or one answer, such as a classification or an extraction. Earlier turns are context. |
| Every assistant turn | Every step is behavior you want, such as each tool call in an agent trajectory. |
| Selected turns | Some steps are good and others aren't. Set `weight` to `0` on the turns you don't want to reinforce. |

When a conversation repeats a mistake that a later turn corrects, train the correction and set `weight` to `0` on the mistake.

## Match how you serve

Train on the exact inputs the model sees in production. The system prompt, the tool definitions, and the chat template settings in training must match the deployment. A model trained with one system prompt and served with another is being evaluated on a task it didn't train on.

Treat reasoning as a setting to test, not a default. Reasoning tends to help with multi-step planning, tool ordering, and calculation. It tends to cost latency and tokens without benefit on direct extraction, classification, and formatting. Score the base model both ways on your evaluation before you choose.

Turning reasoning on in the template doesn't create reasoning supervision. If your data has no reasoning, train with thinking off, for example `enable_thinking = false` for Qwen3.6. With thinking on, the model learns to write an empty reasoning block. Don't generate reasoning text after the fact to fill the gap: the model learns to imitate traces that never produced the answers.

## Render every row before you train

Render the whole dataset with the model's chat template before you provision a trainer. Rendering finds rows that are too long, tool calls whose arguments don't parse, and turns the template can't end. It also shows exactly which tokens train.

The Loops cookbook does this with `loops check`, which reads a training config, renders every row, and launches nothing:

Install the [Loops cookbook](https://pypi.org/project/loops-cookbook/) from PyPI into a [uv](https://docs.astral.sh/uv/) project. The cookbook needs Python 3.12 or later, installs the [`baseten-loops`](https://pypi.org/project/baseten-loops/) SDK, and provides the `loops` command. `loops` runs both recipes; each training config names its recipe with `recipe = "rl"` or `recipe = "sft"`. Create a project first if you don't have one:

```sh theme={"system"}
uv init --python 3.12 loops-project
cd loops-project
uv add loops-cookbook
```

The package includes the cookbook's examples. Run one using its bundled config path, such as `examples/counting/train.toml`, from your project's directory. To change an example, copy its directory into your project at the same path. A config in your project wins over the installed one, and `loops check` prints the directory of the config it used.

Create a `.env` file in your project's directory and set `BASETEN_API_KEY` to a [workspace API key](https://docs.baseten.co/organization/api-keys.md). The commands load `.env` from the training config's directory or the directory you run them from. Never commit `.env`.

A run uses the serverless [availability mode](https://docs.baseten.co/loops/availability-modes/overview.md) when your workspace has serverless access to the model, and the dedicated mode otherwise. To choose a mode, add `BASETEN_LOOPS_SCOPE=serverless` or `BASETEN_LOOPS_SCOPE=deployment` to `.env`.

A minimal config names the recipe, the model, the data, and the training settings:

```toml sft.toml theme={"system"}
recipe = "sft"

[model]
base_model = "Qwen/Qwen3.6-35B-A3B"
lora_rank = 32
max_seq_len = 4096
renderer_chat_template_kwargs = { enable_thinking = false }  # the data has no reasoning

[data]
train = "data/train.jsonl"
validation = "data/validation.jsonl"

[training]
learning_rate = 2e-4
lr_schedule = "cosine"
warmup_steps = 10
batch_size = 16
epochs = 1
```

To render the dataset and report what trains, run the following command:

```sh theme={"system"}
uv run loops check sft.toml
```

On the 2,000 conversations from the SFT example, the report reads:

```text wrap theme={"system"}
Renderer: Qwen36Renderer
train: 2000 rows -> 2000 sequences, 1171026 of 3447520 tokens trained (34.0%); tokens per sequence median 1618.5, p95 3281, max 3582
validation: 200 rows -> 200 sequences, 118908 of 330915 tokens trained (35.9%); tokens per sequence median 1623, p95 3099, max 3550
Schedule: 125 update(s), 125 per epoch of up to 16 sequences; learning rate 0.0002, cosine after 10 warmup update(s)
Trained text of train.jsonl:1, span by span:
  ...
Configuration and data checked; no Loops trainer launched.
```

Read three things before you run:

* **The share of tokens that train.** Here about a third of each conversation is assistant output. A share near 0% means the weights or roles are wrong; near 100% means user or tool text is being trained.
* **Sequence lengths against `max_seq_len`.** Set `max_seq_len` above the longest sequence. Rows over it are refused and listed, not truncated.
* **The trained text of the first row.** Confirm the spans are the assistant turns you meant to train, that each ends with the model's end-of-turn token, and that tool calls render in the model's own format.

To start the run, run the following command:

```bash theme={"system"}
uv run loops run sft.toml
```

The run uses the serverless [availability mode](https://docs.baseten.co/loops/availability-modes/overview.md), billed for the tokens you train, when your workspace has serverless access to the model, and the dedicated mode, billed for GPU time, otherwise. `Qwen/Qwen3.6-35B-A3B` trains in either mode; the [model catalog](https://docs.baseten.co/loops/supported-models.md) shows which models do.

## Next steps

* [Fine-tune tool calling with SFT](https://docs.baseten.co/examples/sft-tool-calling.md): Train on this data and evaluate the adapter against the base model.
* [Train on a chat dataset](https://docs.baseten.co/examples/sft-sdk-loop.md): Write the SFT loop yourself with the Loops SDK.
* [Evaluate a model](https://docs.baseten.co/post-training/evaluate.md): Score the result on your held-out set.