Decision models

Decision models evaluate questions about your input and return probabilities and scores. Use them to route support tickets, detect spam, check requests for safety, or score content against a rubric. Baseten hosts decision models through the System One API. You don’t need to deploy a model. Send text, structured data, or images in state and define the questions to answer in questions.

Choose how to start

Your goal	Start here
Evaluate text or structured input.	Send a decision request with choice, yes/no, and score questions.
Turn a probability into an application action.	Apply a decision threshold to route a ticket or send it for review.
Check visual content.	Evaluate an image with a question about a screenshot.
Look up fields and limits.	Use the System One API reference.

Supported models

Model	Slug	Context limit
elephant-1	`decision-models/baseten-elephant-9b-latest`	16,384 tokens
mammoth-1	`decision-models/baseten-mammoth-27b-latest`	16,384 tokens
joey-1	`decision-models/baseten-joey-2b-latest`	16,384 tokens

Question types

Question	API type	Read the answer from
Choose one option, such as a support team.	`choice`	`choice` and `probabilities`
Evaluate a yes/no question.	`noul`	`noul`, the probability of yes from 0 to 1
Rate text against ordered levels you define.	`score`	`score`, `legend`, and `probabilities`

Include questions about the same state in one request to reuse the shared input prefix. Each answer appears under its question’s name in the response. The model evaluates each question independently; use application logic to combine the answers.

Send a decision request

Create a Baseten API key and set the BASETEN_API_KEY environment variable. This request routes a support ticket, checks whether the customer wants a refund, and scores urgency. Send plain text, a JSON object or array, or a conversation as {"messages": [...]} in state. Define the decisions in questions; the response returns each answer under its question’s name. For more information, see the accepted input formats.

Python
JavaScript
cURL

Install the Python Performance Client with python3 -m pip install baseten-performance-client before running this example.To evaluate the support request:Save the following code as decide.py, then run python3 decide.py:

import os

from baseten_performance_client import PerformanceClient

client = PerformanceClient(
    base_url="https://inference.baseten.co",
    api_key=os.environ["BASETEN_API_KEY"],
)

body = {
  "model": "decision-models/baseten-elephant-9b-latest",
  "service_tier": "default",
  "state": "I was charged twice for my subscription. Please refund the duplicate charge.",
  "questions": {
    "route": {
      "type": "choice",
      "instructions": "Which team should handle this request?",
      "criteria": {
        "billing": "Billing, payments, and refunds",
        "technical_support": "Technical troubleshooting"
      }
    },
    "refund": {
      "type": "noul",
      "instructions": "Is the customer asking for a refund?"
    },
    "urgency": {
      "type": "score",
      "instructions": "How urgent is the request?",
      "criteria": [
        "Routine request",
        "Needs attention today",
        "Immediate emergency"
      ]
    }
  }
}

response = client.batch_system_one(requests=[body])
result = response.data[0]

print(f"Team: {result['answers']['route']['choice']}")
print(f"Refund probability: {result['answers']['refund']['noul']:.4f}")
print(f"Urgency score: {result['answers']['urgency']['score']:.4f}")

Team: billing
Refund probability: 0.9901
Urgency score: 0.5125

Use Node.js and install the JavaScript Performance Client with npm install @basetenlabs/performance-client before running this example.To evaluate the support request:Save the following code as decide.mjs, then run node decide.mjs:

import { PerformanceClient } from "@basetenlabs/performance-client";

const client = new PerformanceClient(
  "https://inference.baseten.co",
  process.env.BASETEN_API_KEY,
);

const body = {
  "model": "decision-models/baseten-elephant-9b-latest",
  "service_tier": "default",
  "state": "I was charged twice for my subscription. Please refund the duplicate charge.",
  "questions": {
    "route": {
      "type": "choice",
      "instructions": "Which team should handle this request?",
      "criteria": {
        "billing": "Billing, payments, and refunds",
        "technical_support": "Technical troubleshooting"
      }
    },
    "refund": {
      "type": "noul",
      "instructions": "Is the customer asking for a refund?"
    },
    "urgency": {
      "type": "score",
      "instructions": "How urgent is the request?",
      "criteria": [
        "Routine request",
        "Needs attention today",
        "Immediate emergency"
      ]
    }
  }
};

const response = await client.batchSystemOne([body]);
const result = response.data[0];

console.log(`Team: ${result.answers.route.choice}`);
console.log(`Refund probability: ${result.answers.refund.noul.toFixed(4)}`);
console.log(`Urgency score: ${result.answers.urgency.score.toFixed(4)}`);

Team: billing
Refund probability: 0.9901
Urgency score: 0.5125

To evaluate the support request:Run the following command:

curl --fail-with-body https://inference.baseten.co/v1/systemone \
  -H "Authorization: Bearer $BASETEN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "decision-models/baseten-elephant-9b-latest",
    "service_tier": "default",
    "state": "I was charged twice for my subscription. Please refund the duplicate charge.",
    "questions": {
      "route": {
        "type": "choice",
        "instructions": "Which team should handle this request?",
        "criteria": {
          "billing": "Billing, payments, and refunds",
          "technical_support": "Technical troubleshooting"
        }
      },
      "refund": {
        "type": "noul",
        "instructions": "Is the customer asking for a refund?"
      },
      "urgency": {
        "type": "score",
        "instructions": "How urgent is the request?",
        "criteria": [
          "Routine request",
          "Needs attention today",
          "Immediate emergency"
        ]
      }
    }
  }'

{
  "model": "decision-models/baseten-elephant-9b-latest",
  "answers": {
    "route": {
      "type": "choice",
      "choice": "billing",
      "confidence": 0.9984472837920018,
      "probabilities": {
        "billing": 0.9992236418960009,
        "technical_support": 0.0007763581039990197
      }
    },
    "refund": {
      "type": "noul",
      "noul": 0.9900829878334823
    },
    "urgency": {
      "type": "score",
      "score": 0.5125449725145267,
      "confidence": 0.23118254122820991,
      "legend": {
        "0": "Routine request",
        "1": "Needs attention today",
        "2": "Immediate emergency"
      },
      "probabilities": {
        "0": 0.5087426775022438,
        "1": 0.4699696724809857,
        "2": 0.021287650016770514
      }
    }
  },
  "usage": {
    "input_tokens": 231,
    "output_tokens": 3
  }
}

Score levels start at 0, so the example’s score of 0.5125 falls between “Routine request” and “Needs attention today”. A noul value is a probability, so your application chooses the threshold for a yes/no decision. Probabilities and scores can vary between requests. The examples use "service_tier": "default". Set service_tier to "flex" for flex processing or "priority" for priority processing. Flex requests cost approximately 90% of the default tier price. By default, priority requests cost 2x the default tier, unless stated otherwise in the pricing overview. For more information, see service_tier and Model API pricing. For more information, see the System One API reference.

Apply a decision threshold

Route a ticket automatically when the selected option’s probability is at least 0.9. Otherwise, send it for review. Set the BASETEN_API_KEY environment variable before running an example.

Python
JavaScript
cURL

Install the Python Performance Client with python3 -m pip install baseten-performance-client before running this example.To route a ticket using a probability threshold:Save the following code as route_ticket.py, then run python3 route_ticket.py:

import os

from baseten_performance_client import PerformanceClient

client = PerformanceClient(
    base_url="https://inference.baseten.co",
    api_key=os.environ["BASETEN_API_KEY"],
)

body = {
  "model": "decision-models/baseten-elephant-9b-latest",
  "service_tier": "default",
  "state": "I was charged twice for my subscription. Please refund the duplicate charge.",
  "questions": {
    "route": {
      "type": "choice",
      "instructions": "Which team should handle this request?",
      "criteria": {
        "billing": "Billing, payments, and refunds",
        "technical_support": "Technical troubleshooting"
      }
    }
  }
}

response = client.batch_system_one(requests=[body])
result = response.data[0]

route = result["answers"]["route"]
team = route["choice"]
probability = route["probabilities"][team]

if probability >= 0.9:
    print(team)
else:
    print("manual_review")

billing

Use Node.js and install the JavaScript Performance Client with npm install @basetenlabs/performance-client before running this example.To route a ticket using a probability threshold:Save the following code as route_ticket.mjs, then run node route_ticket.mjs:

import { PerformanceClient } from "@basetenlabs/performance-client";

const client = new PerformanceClient(
  "https://inference.baseten.co",
  process.env.BASETEN_API_KEY,
);

const body = {
  "model": "decision-models/baseten-elephant-9b-latest",
  "service_tier": "default",
  "state": "I was charged twice for my subscription. Please refund the duplicate charge.",
  "questions": {
    "route": {
      "type": "choice",
      "instructions": "Which team should handle this request?",
      "criteria": {
        "billing": "Billing, payments, and refunds",
        "technical_support": "Technical troubleshooting"
      }
    }
  }
};

const response = await client.batchSystemOne([body]);
const result = response.data[0];

const route = result.answers.route;
const team = route.choice;
const probability = route.probabilities[team];

console.log(probability >= 0.9 ? team : "manual_review");

billing

Install jq before running this example.To route a ticket using a probability threshold:Run the following command:

set -o pipefail

curl --fail-with-body https://inference.baseten.co/v1/systemone \
  -H "Authorization: Bearer $BASETEN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "decision-models/baseten-elephant-9b-latest",
    "service_tier": "default",
    "state": "I was charged twice for my subscription. Please refund the duplicate charge.",
    "questions": {
      "route": {
        "type": "choice",
        "instructions": "Which team should handle this request?",
        "criteria": {
          "billing": "Billing, payments, and refunds",
          "technical_support": "Technical troubleshooting"
        }
      }
    }
  }' | jq -r '
    .answers.route |
    if .probabilities[.choice] >= 0.9 then .choice else "manual_review" end
  '

billing

Use probabilities[choice] for the selected option’s probability. Test the threshold against labeled examples and adjust it for the mistakes your application can tolerate. The confidence field measures how concentrated the distribution is; it isn’t the selected option’s probability. For more information, see choice answer fields.

Evaluate images

Send images in a user message under state.messages, with an array of text and image_url parts in content. You can ask several questions about the same images in one request. For more information, see supported image sources and limits. Download the example screenshot as screenshot.png in your working directory, or use your own PNG. The sample shows an Error: Payment failed message. Set the BASETEN_API_KEY environment variable before running an example.

Python
JavaScript
cURL

Install the Python Performance Client with python3 -m pip install baseten-performance-client before running this example.To check whether a screenshot shows an error:Save the following code as check_image.py, then run python3 check_image.py:

import base64
import os
from pathlib import Path

from baseten_performance_client import PerformanceClient

client = PerformanceClient(
    base_url="https://inference.baseten.co",
    api_key=os.environ["BASETEN_API_KEY"],
)

image = base64.b64encode(Path("screenshot.png").read_bytes()).decode("ascii")
body = {
  "model": "decision-models/baseten-elephant-9b-latest",
  "service_tier": "default",
  "state": {
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "text",
            "text": "Evaluate this screenshot."
          },
          {
            "type": "image_url",
            "image_url": {
              "url": f"data:image/png;base64,{image}"
            }
          }
        ]
      }
    ]
  },
  "questions": {
    "error_visible": {
      "type": "noul",
      "instructions": "Is an error message visible?"
    }
  }
}

response = client.batch_system_one(requests=[body])
result = response.data[0]

print(f"Error probability: {result['answers']['error_visible']['noul']:.4f}")

Error probability: 0.9994

Use Node.js and install the JavaScript Performance Client with npm install @basetenlabs/performance-client before running this example.To check whether a screenshot shows an error:Save the following code as check_image.mjs, then run node check_image.mjs:

import { PerformanceClient } from "@basetenlabs/performance-client";
import { readFileSync } from "node:fs";

const client = new PerformanceClient(
  "https://inference.baseten.co",
  process.env.BASETEN_API_KEY,
);

const image = readFileSync("screenshot.png").toString("base64");
const body = {
  "model": "decision-models/baseten-elephant-9b-latest",
  "service_tier": "default",
  "state": {
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "text",
            "text": "Evaluate this screenshot."
          },
          {
            "type": "image_url",
            "image_url": {
              "url": `data:image/png;base64,${image}`
            }
          }
        ]
      }
    ]
  },
  "questions": {
    "error_visible": {
      "type": "noul",
      "instructions": "Is an error message visible?"
    }
  }
};

const response = await client.batchSystemOne([body]);
const result = response.data[0];

console.log(`Error probability: ${result.answers.error_visible.noul.toFixed(4)}`);

Error probability: 0.9994

To check whether a screenshot shows an error:Run the following command:

IMAGE_DATA=$(base64 < screenshot.png | tr -d '\n')

curl --fail-with-body https://inference.baseten.co/v1/systemone \
  -H "Authorization: Bearer $BASETEN_API_KEY" \
  -H "Content-Type: application/json" \
  --data-binary @- <<EOF
{
  "model": "decision-models/baseten-elephant-9b-latest",
  "service_tier": "default",
  "state": {
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "text",
            "text": "Evaluate this screenshot."
          },
          {
            "type": "image_url",
            "image_url": {
              "url": "data:image/png;base64,$IMAGE_DATA"
            }
          }
        ]
      }
    ]
  },
  "questions": {
    "error_visible": {
      "type": "noul",
      "instructions": "Is an error message visible?"
    }
  }
}
EOF

{
  "model": "decision-models/baseten-elephant-9b-latest",
  "answers": {
    "error_visible": {
      "type": "noul",
      "noul": 0.9994401989013798
    }
  },
  "usage": {
    "input_tokens": 394,
    "output_tokens": 1
  }
}

The answers.error_visible.noul value is the probability that the image shows an error message. The output above comes from the example screenshot; your result depends on the image. Apply a threshold in your application as you would for text decisions.

Usage

The response reports input tokens in usage.input_tokens and one output token per question in usage.output_tokens. Input usage counts the shared prefix once, plus each question’s tokens. Sending related questions together avoids repeating the state in separate requests. For more information, see Model API usage to track requests and tokens.

Get started

Model APIs

Baseten Grid

Inference

Frontier Gateway

Training

Sandboxes

Development

Deployment

Engines

Model Labs

Organization

Hosting options

Observability

Troubleshooting

Decision models

Choose how to start

Supported models

Question types

Send a decision request

Apply a decision threshold

Evaluate images

Usage

​Choose how to start

​Supported models

​Question types

​Send a decision request

​Apply a decision threshold

​Evaluate images

​Usage

Choose how to start

Supported models

Question types

Send a decision request

Apply a decision threshold

Evaluate images

Usage