Skip to main content
Post-training is worth it when a smaller, faster model trained on your task can replace a larger general model, or do the task better. It costs setup time, an evaluation you trust, and GPU hours, so check that it’s the smallest fix that closes your gap before you start.

​
Measure the gap first

You can’t decide whether training helps without a number to improve. Before anything else, write down what a good answer is and score the models you use today. That evaluation tells you whether there’s a gap, how big it is, and later whether a trained model closed it.

​
Try the fixes that don’t need training

Score each of these on your evaluation before you train. Each is cheaper to try and easier to undo:
  • The prompt. Clearer instructions, the rules from your evaluation stated explicitly, and a few worked examples.
  • Retrieval. If the model lacks facts rather than skill, give it the facts at request time.
  • The agent harness. Better tool descriptions, fewer tools, smaller steps, or validation and retries around a step that fails.
  • A different existing model. Try a larger model, or a smaller one if latency or cost is the problem, through Model APIs or your own deployment.
If one of these closes the gap, ship it. Train when the remaining gap is a skill or a behavior the model needs to learn, or when you need a smaller model to match a larger one on this task.

​
Check that the task is ready

Post-training pays off on a task that’s specific and stable:
  • One identifiable input and output. “Turn this call transcript into these JSON fields” is trainable. A whole product isn’t. For an agent with several steps, train for the step that costs the most or fails the most, or train one model for several steps of the same kind.
  • A stable definition of quality. If the prompt, output format, or product changes every week, a trained model goes stale as fast. Wait until the task settles.
  • Real inputs. You need inputs from production, or ones that match production closely. Synthetic inputs miss the cases that fail in production.
  • Something to learn from. For SFT, that’s outputs you trust. For RL, that’s a way to score an outcome automatically, such as tests, a checker, or a rubric a judge applies reliably.
If a requirement is missing, the fix is usually more work on the evaluation or the data, not a different training method.

​
Estimate the payoff

Training aims at one of three outcomes. Decide which one you need, because it decides the model size and the bar the trained model must clear:
  • Lower serving cost: a smaller model matches the current one on your evaluation. The bar is no regression.
  • Lower latency: the task has a latency budget the current model can’t meet. The budget caps the model size. For example, a 1.3-second ceiling in a voice pipeline set a 4B model as the upper bound in EliseAI’s case.
  • Higher quality: the trained model beats every model you can buy on your evaluation. The bar is a measured improvement on held-out data.
Compare the cost of the model you use today with an alternative that meets your evaluation. Enter the prices you currently pay and your monthly usage, then select a Model API, enter another provider’s rates, or estimate dedicated capacity for a trained checkpoint. Include any training and setup costs to estimate break-even time. Model API presets price hosted base models you can use without post-training. To serve a trained checkpoint, estimate its dedicated deployment using a capacity configuration you’ve load-tested. The calculator doesn’t infer which hardware or replica count can handle your traffic. Use billed output tokens, including reasoning tokens. Output lengths, tokenizer differences, and cache reuse can change actual token usage. Set the alternative’s token volume separately when your measurements differ. The estimate covers token or instance charges; account for other recurring costs separately. Compare the savings with the cost of getting there: the engineering time to build the evaluation and data, and the training runs. Training in the serverless mode bills for the tokens you train and sample; training in the dedicated mode bills for GPU time. See where each model trains. An inference deployment bills for GPU time rather than tokens, so for a model you serve yourself, use the deployment’s hourly cost and the number of replicas your traffic needs.

​
Next steps