batch_system_one() in Python or batchSystemOne() in Node.js.
Request body
string
required
The Model API slug, such as
decision-models/baseten-elephant-9b-latest. See supported models.string
Set to
"flex" for flex processing or "priority" for priority processing with decision models. The default is "default". This field is optional. For pricing differences, see Decision models.string | object | array
required
The input to evaluate: plain text, a JSON object or array, or a conversation as
{"messages": [...]}. The API renders structured data as text for the model; you don’t need to serialize it yourself. It interprets objects containing messages as conversations.Each conversation message must have a role of user or assistant; any other role returns a validation error. Both roles accept text content. User messages also accept arrays containing text and image parts. Assistant messages containing images return a validation error. See Image inputs.Numbers, booleans, and null aren’t accepted as the top-level state. The state counts toward each question’s context limit, so a long state leaves less room for the question text.Structured state
Conversation state
object
required
A nonempty map of question names to question definitions. The response uses the same names in
answers. Include questions about the same state together to reuse its shared input prefix. The model evaluates each question independently; it can’t use another question’s answer.A request supports at most 512 options across all questions. Each noul question counts as two options.integer
An optional positive token limit for the complete prompt for each question, including the shared state, question, criteria, formatting, and image tokens. When omitted, the model’s context limit applies. Larger values don’t increase it. Prompts that exceed the effective limit return a
4xx error (400, 413, or 422); they aren’t truncated.Question definitions
Each question containstype. Use instructions to define the question precisely. The criteria format depends on its type.
string
required
One of
choice, noul, or score.string | object | array | null
The question to evaluate against
state. The API renders objects and arrays as JSON text. When omitted, null, or an empty string, the model uses a default question for the selected type:choice: “Which option applies to the state?”noul: “Is the statement true, or is the answer to the question yes?”score: “Which level describes the state?”
object | array | null
Required for
choice and score:choice: An object mapping 1 to 255 option names to descriptions. The response returns the selected option name inchoice.score: An array of 1 to 10 level descriptions, ordered from lowest to highest. Level numbering starts at0.noul: Omit this field to use yes/no defaults, or provide an object containing exactly"true"and"false"to describe each outcome. Empty objects andnullalso use the defaults.
null. The API renders objects and arrays as JSON text. A null choice description uses the option name without a description. A null yes/no description uses the default for that outcome.Questions
Image inputs
Send images in thecontent array of a user message under state.messages. See Evaluate images for a complete request. Text parts have type: "text" and a text field. Image parts have type: "image_url" and an image_url object with these fields:
string
required
The image source: a base64 PNG, JPEG, or WebP data URL, or an HTTPS URL hosted on Amazon S3, Google Cloud Storage, or Azure Blob Storage. Other remote image hosts aren’t accepted.Given the low-latency nature of decision models, we strongly recommend to use base64 URIs for quick resolution and usage of popular image encoding schemes.
string
default:"auto"
Set to
"auto" or omit it. Other values return a validation error.<image:N> or <video:N> placeholders into state text or questions.
Limits
The context limit depends on the selected model. See Supported models for each model’s limit. Each question’s complete prompt must fit within that limit, or withinmax_len if you set a smaller value. Image tokens and prompt formatting count toward the limit, so leave room beyond the state text.
If any question’s prompt exceeds the limit, the API rejects the entire request with a 4xx error (400, 413, or 422) and returns no answers. It doesn’t truncate the state, questions, or images. Shorten the state or criteria, or remove images, then retry. Requests that exceed the total option budget also return a validation error; split them into smaller requests.
Response body
string
The backend model identifier. This can differ from the Model API slug in the request.
object
A map of question names to typed answers. Each answer includes its
type and the fields described below. Responses can include additional model-specific fields.integer
The shared input prefix counted once, plus the question-specific tokens for every question. Includes prompt formatting and image tokens. This can be lower than the sum of input tokens from sending each question in a separate request.
integer
One token per question in the request.
Answer fields
These fields appear inside each named entry inanswers.
string
The question type:
choice, noul, or score.object
For
choice, each option’s name mapped to its probability. For score, each zero-based level number, represented as a string key, mapped to its probability.number
For
choice and score, the concentration of probabilities on a scale from 0 to 1. 0 means no more concentrated than a uniform distribution. 1 means all probability is on one option or level. To apply a probability threshold to a choice, use probabilities[choice].See Apply a decision threshold.Choice answers
string
The selected option’s name.
Yes/no answers
number
The probability of yes, from 0 to 1. Apply a threshold in your application to make a yes/no decision.
Score answers
number
The rating on your rubric, starting at
0 for the first level. A score of 1.5 falls between the second and third levels.object
Level numbers, represented as string keys starting at
0, mapped to their descriptions.Errors
Invalid requests or requests that exceed size limits return a4xx error (400, 413, or 422). Examples include an unknown request field, a missing questions field, a number in state, a message role other than user or assistant, or an unsupported question type. Validation responses include an error message describing the problem.
HTTP 429 indicates rate limiting or insufficient serving capacity. Retry with backoff and reduce concurrent requests if errors persist. Correct invalid requests or reduce their size before retrying HTTP 400, 413, or 422 responses.
Invalid question type