BaseTen
Search…
Custom Models
When your model isn't a pure object from a standard ML library, you can still deploy your model via our custom models API.

Deploying a custom model

Standard supported framework models work out of the box with no need for customization. If needed, you have complete control over the python environment and execution of your model, this is where custom models are useful. In order to deploy a custom model you'll need to provide:
  • A list of all files to be packaged with your model, including:
    • Serialized objects such as model binaries, embeddings, and datasets,
    • Python files defining your model and all supporting files,
    • Anything else that your model needs to run,
  • A requirements.txt file specifying the dependencies of your model.
Your model must be a python class that implements two methods:
  • load , a method that will be called upon initialization of the model in the deployment environment.
  • predict which takes the raw JSON input of the the predict call to the model. It must return data in a JSON-serializable format.
For example:
my_model.py
1
import pickle
2
3
4
class MyCustomModel:
5
def __init__(self):
6
self.model = None
7
self.encoder = None
8
9
def load(self):
10
self.model = pickle.load(open('model/my_model.pkl', 'rb'))
11
self.encoder = pickle.load(open('model/my_encoder.pkl', 'rb'))
12
13
14
def predict(self, inputs) -> dict:
15
# Encode the inputs
16
model_input = self.encoder(inputs)
17
# Run predict on the encoded inputs
18
predictions = self.model.predict(model_input)
19
return {
20
"predictions": predictions,
21
"encoded_values": model_input,
22
}
23
Copied!
In order to deploy a custom model, use the deploy_custom method of the BaseTen API. You need to provide a name, the model class, the complete set of files supporting the model, and the requirements file.
1
baseten.deploy_custom(
2
model_name='encoder model',
3
model_class='MyCustomModel',
4
model_files=['my_model.py', 'my_model.pkl', 'my_encoder.pkl'],
5
requirements_file='requirements.txt'
6
)
Copied!
BaseTen will package your files and deploy your custom model.
For a more in-depth look at the technology powering our model serving, please look at BaseTen Scaffolds
Last modified 1mo ago
Copy link