Prepare data for supervised fine-tuning

Supervised fine-tuning (SFT) teaches a model to reproduce the assistant turns in your data. It can only be as good as those turns: a run that fits the data perfectly also learns every wrong answer, formatting slip, and missing tool call the data contains. Decide what a good answer is with an evaluation first, then use the same rules to build the data. This guide continues the tool-calling assistant from the evaluation guide. The SFT example runs it end to end.

Choose targets you trust

Every assistant turn you train on is a target. Know where each one came from, because the source sets the ceiling:

Production outputs: the model learns to reproduce what production does today, mistakes included. Training on them distills the current system; it doesn’t improve on it. Filter them with your evaluation rules first.
Human-written or human-corrected outputs: the strongest targets, and the most expensive. A verified slice is also what you calibrate an LLM judge against.
Outputs from a stronger model: a common way to train a smaller, cheaper model on a task. Score them with your evaluation and keep only the ones that pass.
Public datasets: useful for general skills such as tool use. Check that the format and behavior match what you serve.

Use your evaluation as a filter. Drop rows whose target breaks a rule, and drop rows that would teach one rule by breaking another. Remove every held-out row and its near-duplicates before training: a held-out example in the training data makes the evaluation measure memory instead of the task.

Check the label balance

For classification and extraction tasks, count the distinct target values before training. When one target dominates, typically “no issue” or an empty extraction, the cheapest way to lower the loss is to output it every time. The training loss looks excellent and the model flags nothing. Rebalance at the source by oversampling the rarer targets, and evaluate with per-class false-positive and false-negative rates rather than overall accuracy. On a dataset where 95% of rows are “no issue,” a model that never flags anything is 95% accurate.

Format conversations as chat JSONL

Write one conversation per line in OpenAI’s chat format. OpenAI fine-tuning files, Hugging Face conversational datasets, and most trace exports from tools such as LangSmith and Langfuse already use it or convert in a few lines:

train.jsonl

{"messages": [{"role": "system", "content": "You convert units."}, {"role": "user", "content": "12 miles in km?"}, {"role": "assistant", "content": "", "tool_calls": [{"id": "c1", "type": "function", "function": {"name": "convert", "arguments": "{\"value\": 12, \"from\": \"mi\", \"to\": \"km\"}"}}]}, {"role": "tool", "tool_call_id": "c1", "content": "19.312"}, {"role": "assistant", "content": "About 19.31 km.", "weight": 1}], "tools": [{"type": "function", "function": {"name": "convert", "description": "Convert a value between units.", "parameters": {"type": "object", "properties": {"value": {"type": "number"}, "from": {"type": "string"}, "to": {"type": "string"}}, "required": ["value", "from", "to"]}}}]}

tools lists the tools offered in that conversation, in OpenAI’s function format. Rows can offer different tools.
tool_calls and tool messages follow OpenAI’s format. Answer every call exactly once.
weight on an assistant message sets whether that turn trains: 1 trains it, 0 keeps it as context only.
reasoning_content on an assistant message carries that turn’s reasoning, when your data has it.

The conversations can come from any model. Training re-renders the text with the trained model’s own chat template. To convert a Hugging Face dataset that already has messages and tools columns, run the following script. Reshape the columns inside the loop when your dataset names them differently:

make_data.py

import json

from datasets import load_dataset

rows = load_dataset("<org>/<dataset>", split="train")
with open("train.jsonl", "w") as out:
    for row in rows:
        out.write(
            json.dumps({"messages": row["messages"], "tools": row.get("tools") or []})
            + "\n"
        )

For data in Amazon S3, copy the file down first, then convert it the same way. To copy it, run the following command:

aws s3 cp "s3://<bucket>/<path>/train.jsonl" .

Decide which turns train

User, system, and tool-result messages are never targets. Among assistant turns, choose deliberately, because the choice changes what the model learns:

Train	When
Only the final assistant turn	The task is one decision or one answer, such as a classification or an extraction. Earlier turns are context.
Every assistant turn	Every step is behavior you want, such as each tool call in an agent trajectory.
Selected turns	Some steps are good and others aren’t. Set `weight` to `0` on the turns you don’t want to reinforce.

When a conversation repeats a mistake that a later turn corrects, train the correction and set weight to 0 on the mistake.

Match how you serve

Train on the exact inputs the model sees in production. The system prompt, the tool definitions, and the chat template settings in training must match the deployment. A model trained with one system prompt and served with another is being evaluated on a task it didn’t train on. Treat reasoning as a setting to test, not a default. Reasoning tends to help with multi-step planning, tool ordering, and calculation. It tends to cost latency and tokens without benefit on direct extraction, classification, and formatting. Score the base model both ways on your evaluation before you choose. Turning reasoning on in the template doesn’t create reasoning supervision. If your data has no reasoning, train with thinking off, for example enable_thinking = false for Qwen3.6. With thinking on, the model learns to write an empty reasoning block. Don’t generate reasoning text after the fact to fill the gap: the model learns to imitate traces that never produced the answers.

Render every row before you train

Render the whole dataset with the model’s chat template before you provision a trainer. Rendering finds rows that are too long, tool calls whose arguments don’t parse, and turns the template can’t end. It also shows exactly which tokens train. The Loops cookbook does this with loops check, which reads a training config, renders every row, and launches nothing: Install the Loops cookbook from PyPI into a uv project. The cookbook needs Python 3.12 or later, installs the baseten-loops SDK, and provides the loops command. loops runs both recipes; each training config names its recipe with recipe = "rl" or recipe = "sft". Create a project first if you don’t have one:

uv init --python 3.12 loops-project
cd loops-project
uv add loops-cookbook

The package includes the cookbook’s examples. Run one using its bundled config path, such as examples/counting/train.toml, from your project’s directory. To change an example, copy its directory into your project at the same path. A config in your project wins over the installed one, and loops check prints the directory of the config it used. Create a .env file in your project’s directory and set BASETEN_API_KEY to a workspace API key. The commands load .env from the training config’s directory or the directory you run them from. Never commit .env. A run uses the serverless availability mode when your workspace has serverless access to the model, and the dedicated mode otherwise. To choose a mode, add BASETEN_LOOPS_SCOPE=serverless or BASETEN_LOOPS_SCOPE=deployment to .env. A minimal config names the recipe, the model, the data, and the training settings:

sft.toml

recipe = "sft"

[model]
base_model = "Qwen/Qwen3.6-35B-A3B"
lora_rank = 32
max_seq_len = 4096
renderer_chat_template_kwargs = { enable_thinking = false }  # the data has no reasoning

[data]
train = "data/train.jsonl"
validation = "data/validation.jsonl"

[training]
learning_rate = 2e-4
lr_schedule = "cosine"
warmup_steps = 10
batch_size = 16
epochs = 1

To render the dataset and report what trains, run the following command:

uv run loops check sft.toml

On the 2,000 conversations from the SFT example, the report reads:

Renderer: Qwen36Renderer
train: 2000 rows -> 2000 sequences, 1171026 of 3447520 tokens trained (34.0%); tokens per sequence median 1618.5, p95 3281, max 3582
validation: 200 rows -> 200 sequences, 118908 of 330915 tokens trained (35.9%); tokens per sequence median 1623, p95 3099, max 3550
Schedule: 125 update(s), 125 per epoch of up to 16 sequences; learning rate 0.0002, cosine after 10 warmup update(s)
Trained text of train.jsonl:1, span by span:
  ...
Configuration and data checked; no Loops trainer launched.

Read three things before you run:

The share of tokens that train. Here about a third of each conversation is assistant output. A share near 0% means the weights or roles are wrong; near 100% means user or tool text is being trained.
Sequence lengths against max_seq_len. Set max_seq_len above the longest sequence. Rows over it are refused and listed, not truncated.
The trained text of the first row. Confirm the spans are the assistant turns you meant to train, that each ends with the model’s end-of-turn token, and that tool calls render in the model’s own format.

To start the run, run the following command:

uv run loops run sft.toml

The run uses the serverless availability mode, billed for the tokens you train, when your workspace has serverless access to the model, and the dedicated mode, billed for GPU time, otherwise. Qwen/Qwen3.6-35B-A3B trains in either mode; the model catalog shows which models do.

Next steps

Fine-tune tool calling with SFT: Train on this data and evaluate the adapter against the base model.
Train on a chat dataset: Write the SFT loop yourself with the Loops SDK.
Evaluate a model: Score the result on your held-out set.

Get started

Model APIs

Baseten Grid

Inference

Frontier Gateway

Training

Sandboxes

Development

Deployment

Engines

Model Labs

Organization

Hosting options

Observability

Troubleshooting

Prepare data for supervised fine-tuning

Choose targets you trust

Check the label balance

Format conversations as chat JSONL

Decide which turns train

Match how you serve

Render every row before you train

Next steps

​Choose targets you trust

​Check the label balance

​Format conversations as chat JSONL

​Decide which turns train

​Match how you serve

​Render every row before you train

​Next steps

Choose targets you trust

Check the label balance

Format conversations as chat JSONL

Decide which turns train

Match how you serve

Render every row before you train

Next steps