Interacting with models
Invoke models via multiple methods to integrate them into your applications.
We provide three ways to engage with a deployed model:
  1. 1.
    Call the model directly via a REST API endpoint
  2. 2.
    Use the model in our worklet engine
  3. 3.
    Interface with the model via our Baseten Python client
The following examples will illustrate how to call the model in each of these ways.

Type of inputs

Model inputs are expected to be JSON serializable into a Python list type.

Motivating example: Custom model

Assume there is a custom model defined by the following code. It will be a very simple model that given a list of dictionary inputs, will return the sum of the keys 'a', 'b', and 'c' for each item.
class InteractionModel:
def load(self):
pass
def predict(self, inputs: list) -> dict:
results = []
for model_input in inputs:
model_result = model_input['a'] + model_input['b'] + model_input['c']
results.append(model_result)
return {
'predictions': results
}

REST API

Once this model is deployed on Baseten infrastructure, to call it directly via the REST API one would execute the following to call the model.
$ curl -X POST https://app.baseten.co/model_versions/<model_version_id>/predict \
-H 'Authorization: Api-Key <api-key>' \
-d '{"inputs": [{"a": 1, "b": 2, "c": 3}]}'
# Response
{
"model_id": <model_id>,
"model_version_id": <model_version_id>,
"predictions": [
{
"predictions": [
6
]
}
]
}

Worklet graph

In an application generated on Baseten create a Model block referencing the deployed model and a Code block preceding it.
Worklet Graph
Then in the Code block implement the following Python:
def model_input(block_input, env, context):
return [{'a': 1, 'b': 2, 'c': 3}]
Once saved, the entire application can be run to see the following output.
Worklet Output

Baseten client

With a deployed model in Baseten infrastructure one can query with the Baseten client package in Python as so.
model = baseten.deployed_model_version_id(<model_version_id>)
model.predict([{"a": 1, "b": 2, "c": 3}])
# ->
[{'predictions': [6]}]

Motivating example: Iris RFC

In this example we will demonstrate interacting with the Iris Random Forest Classifier from the Baseten Pre-trained Models. The Iris model expects a list of lists with dimensions Nx4 and features that correspond to sepal length, sepal width, petal length, and pedal width. But for this example, we are focusing on structure more than information. So the input will be simple, like [[1, 2, 3, 4]].

REST API

Here we will call the model via the REST endpoint and cURL.
$ curl -X POST https://app.baseten.co/model_versions/<model_version_id>/predict \
-H 'Authorization: Api-Key <api-key>' \
-d '{"inputs": [1, 2, 3, 4]]}'
# Response
{
"model_id": <model_id>,
"model_version_id": <model_version_id>,
"predictions": [
2
],
"probabilities": [
[
0,
0.37,
0.63
]
]
}

Worklet graph

The sample demo application for Iris RFC demonstrates the call for a 10 x 4 input.
import numpy as np
NUMBER_OF_IRISES = 10
SEPAL_LENGTH_MEAN = 5.84
SEPAL_WIDTH_MEAN = 3.06
PETAL_LENGTH_MEAN = 3.76
PETAL_WIDTH_MEAN = 1.20
SEPAL_LENGTH_STD = 0.83
SEPAL_WIDTH_STD = 0.44
PETAL_LENGTH_STD = 1.77
PETAL_WIDTH_STD = 0.76
def generate_random_irises(number_of_irises):
random_irises = np.array([
np.random.randn(number_of_irises)*SEPAL_LENGTH_STD+SEPAL_LENGTH_MEAN,
np.random.randn(number_of_irises)*SEPAL_WIDTH_STD+SEPAL_WIDTH_MEAN,
np.random.randn(number_of_irises)*PETAL_LENGTH_STD+PETAL_LENGTH_MEAN,
np.random.randn(number_of_irises)*PETAL_WIDTH_STD+PETAL_WIDTH_MEAN,
]).T.tolist()
return random_irises
def process_input(block_input, env, context):
inputs = block_input.get('inputs')
if not inputs:
number_of_irises = block_input.get('number_of_irises', NUMBER_OF_IRISES)
inputs = generate_random_irises(number_of_irises)
env['model_input'] = inputs
return inputs
The output of the worklet produces the classes for each of the 10 examples
Worklet Output

Baseten client

Here we will query the Iris model with two inputs.
import baseten
model = baseten.deployed_model_version_id(<model_version_id>)
model.predict([[1, 2, 3, 4], [5, 6, 7, 8]])
# ->
[2, 2]