Skip to main content
The base model sets the ceiling for what training can reach and the floor for what serving costs. Narrow the supported base models by your constraints first, then let your evaluation choose among the rest.

​
Apply hard constraints

Rule out models that can’t ship, whatever their scores:
  • Publisher and license. The organization in the model ID, such as Qwen or google, names the publisher. Read the license on the model’s Hugging Face page, and check both against any policy your organization has on model origin or license terms.
  • Sequence length. Your longest training example, and for RL your longest agent conversation including tool results, must fit the model’s maximum training sequence length. The maximum context a model serves at inference doesn’t set what it can train on. The model catalog lists each limit, and some differ between SFT and RL.
  • Inputs. If your task includes images, choose a model marked for vision.
  • Availability mode. Some models also train in the self-serve serverless mode, billed for tokens; every model trains in the dedicated mode. Filter the model catalog by Serverless to see which. See Availability modes.
Check workspace access with the Baseten CLI or REST API to list the models and sequence lengths enabled for SFT or RL.

​
Account for serving cost and latency

You pay to serve the trained model on every request for as long as it’s in production, so weigh serving cost and latency alongside quality. Two properties in the model ID predict them:
  • Total parameters set the GPU memory the deployment needs, and so the hardware it runs on.
  • Active parameters set the compute per token. A mixture-of-experts model activates part of its weights for each token: Qwen/Qwen3.6-35B-A3B has 35B parameters, and about 3B are active per token. It generates faster than a 35B dense model and needs more memory than a 3B one.
Time to first token grows with prompt length and model size, so measure it with your own prompts. Deploy a candidate, or call it through Model APIs when it’s available there, and time requests at your production prompt lengths. If your task has a latency budget, it can rule out every model above a size, as a 1.3-second budget did for EliseAI’s 4B model.

​
Baseline the candidates

Score the remaining candidates on your evaluation before you train any of them, with identical prompts, tools, sampling settings, and output limits:
  1. Score the strongest model you can call. It shows the best result the task allows today and whether your evaluation can tell good models from weak ones.
  2. Score the smallest model that fits your constraints and one larger challenger. A base model that starts closer to the bar needs less training to reach it.
  3. Score each reasoning setting as its own candidate. Turning reasoning on changes quality, latency, and output tokens. Reasoning tends to help with multi-step planning, tool ordering, and calculation, and to cost tokens without benefit on extraction, classification, and formatting.
Choose the smallest candidate that can credibly reach your bar after training. When two candidates score within your evaluation’s run-to-run variation, choose the cheaper one. Stop adding candidates once the choice is clear. Record the model ID, reasoning setting, and baseline scores. The untrained base model’s score is the comparison that shows what training added.

​
Next steps