Upgrade notes

Notes for upgrading away from legacy usage of Baseten.
If you first installed Truss and Baseten in October 2022 or later, meaning you installed truss==0.1.1 baseten==0.2.7 or later, you can skip this page entirely!
At Baseten, we're constantly working on improving our product and user experience. These improvements sometimes require meaningful changes to developer tools like our Python client. As part of our commitment to backward compatibility, we make sure these changes do not break existing models and workflows. However, we highly recommend that all developers using Baseten use the newest version of our Python client, Truss, and other tools.
This page guides you through upgrading legacy usage of Baseten.
Upgrading model interface
This is required when moving from truss==0.1.0 baseten==0.2.6 or earlier to truss==0.1.1 baseten==0.2.7.
If you first installed Truss and Baseten in October 2022 or later, meaning you installed truss==0.1.1 baseten==0.2.7 or later, you can skip this page entirely!
At Baseten, we're constantly working on improving our product and user experience. These improvements sometimes require meaningful changes to developer tools like our Python client. As part of our commitment to backward compatibility, we make sure these changes do not break existing models and workflows. However, we highly recommend that all developers using Baseten use the newest version of our Python client, Truss, and other tools.
This page guides you through upgrading legacy usage of Baseten.
Upgrading model interface
By default, when you deploy a model to Baseten, like in this XGBoost classifier example, the deployed model expects to receive a dictionary with the key inputs and a list of input values, and will return a dictionary with the key predictions and a list of model results.
model_input = {"inputs": [[0, 0, 0, 0, 0, 0]]}
model_output = {'predictions': [0.21339938044548035]}
Until now, that default behavior has not been changeable. All models deployed to Baseten had to follow that spec. However, this interface was too inflexible, so as of the most recent version of the Baseten Python package (0.2.7), you can set your own interface for your models.
Setting your model interface
Baseten uses Truss under the hood for model deployment. You can customize your model's interface by editing the predict function in models/model.py in Truss. The auto-generated predict function for the aforementioned XGBoost example looks like this:
def predict(self, request: Dict) -> Dict[str, List]:
        response = {}
        inputs = request["inputs"]
        dmatrix_inputs = xgb.DMatrix(inputs)
        result = self._model.predict(dmatrix_inputs)
        response["predictions"] = result
        return response
Modify this function to parse whatever request and response you want, and remember that Truss also supports pre- and post-processing functions that can further modify input and output when more complicated parsing is needed.
Backwards compatibility
This change does not modify the behavior of existing deployed models, nor the default behavior of future models. However, it does change how deployed models are invoked through the Baseten Python client.
Previously, the predict() function wrapped its argument in a dictionary with the inputs key. Now that said key is not required, the predict() function passes its inputs as-is, which means you have to enter the entire model input yourself:
Before (1.0 spec):
baseten.predict([[0,0,0,0,0,0]])
Now (2.0 spec):
baseten.predict({"inputs": [[0,0,0,0,0,0]]})
The syntax for invoking a model via an API call has not changed.
However, to prevent breaking existing scripts using baseten.predict, a new spec_version flag is now included in Truss. This parameter is set to 2.0 by default for all new models, so they will use the new input spec, but all existing models will continue to function exactly as they have been on the 1.0 spec. You can upgrade your model to the latest interface spec by changing the flag in the config.yaml file in Truss.
Enjoy this unrestricted interface by installing the newest versions of Truss and the Baseten client.
pip install --upgrade baseten truss
Deploying pre-Truss custom models
This requires the Python package baseten==0.1.28 or earlier and is relevant to users who deployed models before July 2022.
Standard supported framework models work out of the box with no need for customization. If needed, you have complete control over the python environment and execution of your model, this is where custom models are useful. In order to deploy a custom model you'll need to provide:
A list of all files to be packaged with your model, including:
Serialized objects such as model binaries, embeddings, and datasets,
Python files defining your model and all supporting files,
Anything else that your model needs to run,
A requirements.txt file specifying the dependencies of your model.
Your model must be a Python class that implements two methods:
load, a method that will be called upon initialization of the model in the deployment environment.
predict, a method which consumes deserialized JSON input from a web request or from a Baseten worklet. This is the integration point for the underlying model object. It must return data in a JSON-serializable format.
Additionally, ensure that your model file is not called model.py or another name that conflicts with the Python namespace.
For example:
my_model.py
import pickle
​
​
class MyCustomModel:
    def __init__(self):
        self.model = None
        self.encoder = None
​
    def load(self):
        self.model = pickle.load(open('model/my_model.pkl', 'rb'))
        self.encoder = pickle.load(open('model/my_encoder.pkl', 'rb'))
​
    def predict(self, inputs: list) -> dict:
        # Encode the inputs
        model_inputs = self.encoder(inputs)
        # Run predict on the encoded inputs 
        predictions = self.model.predict(model_inputs)
        return {
            "predictions": predictions,
            "encoded_values": model_inputs,
        }
In order to deploy a custom model, use the deploy_custom method of the Baseten API. You need to provide a name, the model class, the complete set of files supporting the model, and the requirements file.
import baseten
​
baseten.login("*** INSERT API KEY ***")  # https://docs.baseten.co/settings/api-keys
baseten.deploy_custom(
    model_name='encoder model',
    model_class='MyCustomModel',
    model_files=['my_model.py', 'my_model.pkl', 'my_encoder.pkl'],
    requirements_file='requirements.txt'
)
Baseten will package your files and deploy your custom model.
For a more in-depth look at the technology powering our model serving, please look at Truss, an open-source model serving tool built by Baseten.
APPENDIX - Previous
Python packages
Last modified 5mo ago