Decision models

Decision models evaluate questions about your input and return typed answers your application can use. Use them to classify requests, choose a handler, or score content against a rubric. Baseten hosts Laya, a decision model developed by ConvAI Innovations, through the System One API. You don’t need to deploy a model. Send text in state and define the questions to answer in questions.

Supported models

Model	Slug
Laya	`decision-models/laya`

Question types

Type	Use it to	Read the answer from
`choice`	Choose one option, such as a support team.	`choice` and `probabilities`
`noul`	Evaluate a yes/no question.	`noul`, the probability of yes from 0 to 1
`score`	Rate text against ordered levels you define.	`score`, `legend`, and `probabilities`

You can include several named questions in one request. Each answer appears under its question’s name in the response. Define narrow questions and use application logic to combine the results.

Send a decision request

Create a Baseten API key and set the BASETEN_API_KEY environment variable. This example routes a support request, checks whether the customer wants a refund, and scores urgency. Send plain text in state.

Python
cURL

To evaluate the support request:Install the Baseten Performance Client:

python3 -m pip install baseten-performance-client

Save this example as decide.py and run python3 decide.py.

decide.py

import os

from baseten_performance_client import PerformanceClient

client = PerformanceClient(
    base_url="https://inference.baseten.co",
    api_key=os.environ["BASETEN_API_KEY"],
)

body = {
    "model": "decision-models/laya",
    "state": "I was charged twice for my subscription. Please refund the duplicate charge.",
    "questions": {
        "route": {
            "type": "choice",
            "instructions": "Which team should handle this request?",
            "criteria": {
                "billing": "Billing, payments, and refunds",
                "technical_support": "Technical troubleshooting"
            }
        },
        "refund": {
            "type": "noul",
            "instructions": "Is the customer asking for a refund?"
        },
        "urgency": {
            "type": "score",
            "instructions": "How urgent is the request?",
            "criteria": [
                "Routine request",
                "Needs attention today",
                "Immediate emergency"
            ]
        }
    }
}

response = client.batch_system_one(requests=[body])
result = response.data[0]

print(result["answers"]["route"]["choice"])
print(result["answers"]["refund"]["noul"])
print(result["answers"]["urgency"]["score"])

Pass one request in a list and read its result from response.data[0].

Output

billing
0.7195
1.5515

To evaluate the support request:

Request

curl --fail-with-body https://inference.baseten.co/v1/systemone \
  -H "Authorization: Bearer $BASETEN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "decision-models/laya",
  "state": "I was charged twice for my subscription. Please refund the duplicate charge.",
  "questions": {
    "route": {
      "type": "choice",
      "instructions": "Which team should handle this request?",
      "criteria": {
        "billing": "Billing, payments, and refunds",
        "technical_support": "Technical troubleshooting"
      }
    },
    "refund": {
      "type": "noul",
      "instructions": "Is the customer asking for a refund?"
    },
    "urgency": {
      "type": "score",
      "instructions": "How urgent is the request?",
      "criteria": [
        "Routine request",
        "Needs attention today",
        "Immediate emergency"
      ]
    }
  }
}'

The answers object includes these typed values (other response fields are omitted):

Answers

{
  "route": {
    "type": "choice",
    "choice": "billing",
    "probabilities": {
      "billing": 0.8044,
      "technical_support": 0.1956
    }
  },
  "refund": {
    "type": "noul",
    "noul": 0.7195
  },
  "urgency": {
    "type": "score",
    "score": 1.5515,
    "legend": {
      "0": "Routine request",
      "1": "Needs attention today",
      "2": "Immediate emergency"
    },
    "probabilities": {
      "0": 0.0488,
      "1": 0.3509,
      "2": 0.6003
    }
  }
}

A score can fall between rubric levels. A noul value is a probability, so your application chooses the threshold for a yes/no decision. See the System One reference for request fields and response shapes.

Usage

The response reports input tokens in usage.input_tokens and reports usage.output_tokens as 0. See Model API usage to track requests and tokens.

Get started

Model APIs

Inference

Training

Sandboxes

Development

Deployment

Engines

Model Labs

Organization

Hosting options

Observability

Troubleshooting

Decision models

Supported models

Question types

Send a decision request

Usage

​Supported models

​Question types

​Send a decision request

​Usage

Supported models

Question types

Send a decision request

Usage