Choose SFT when you have answers
SFT is the simpler and cheaper method, and usually the first to try. It fits when correct outputs exist or can be produced: production outputs you’ve verified, corrections from reviewers, or outputs from a stronger model that pass your evaluation. Distilling a large model’s behavior on one task into a smaller model is SFT on the large model’s filtered outputs. SFT can’t exceed its targets. If your best outputs come from production, the trained model learns production’s behavior, mistakes included. See choose targets you trust.Choose RL when you can score outcomes
RL fits when checking a result is easier than writing it. Unit tests can check code no one has written yet. A shell command can check that an agent left the files in the right state. A rubric can check a summary against its source. RL also trains on the model’s own attempts, so it learns from the situations the model actually gets into, such as a failed tool call in the middle of a long task. RL needs a starting model that sometimes succeeds. The model learns from the difference between its better and worse attempts at the same task, so a task it always fails teaches nothing. See size tasks so attempts disagree.Combine them
A common sequence is SFT first, then RL. SFT teaches the format and the basic behavior from examples, so the model succeeds often enough for RL to have a signal. RL then improves on the cases where the examples ran out. In the Loops cookbook, start the RL run from the SFT run’s trainer weights with[training] init_from = "bt://loops:<run_id>/weights/step-<N>". The base model and LoRA rank must match the SFT run.
If SFT plateaus below your bar, look at what kind of errors remain before you add RL. Errors that come from the data, such as inconsistent labels, need better data. Errors the model makes in situations the data never showed, such as recovering from a failed step, are what RL trains on.
Next steps
- Fine-tune tool calling with SFT: An SFT run with a before-and-after evaluation.
- Train a model to count circles with RL: A first RL run on serverless Loops with an image generator and a correctness reward.
- Train a tool-using agent with RL: Build a custom environment with a sandbox and a hidden grader.
- Ship a post-trained model: Check, compare, deploy, and roll out the result.