MODELS
Tracking model metadata
Use artifacts, configs, and metrics to store and interact with model metadata.
Once you've started using Baseten for model deployment, you might wish to associate pieces of information with models that are deployed in the Baseten infrastructure. We provide three pieces of functionality in our Python client to support model metadata information: artifacts, configs, and metrics.
Artifacts
Artifacts are effectively general purpose BLOB storage. They can be created and associated to models and model versions. Constructing an artifact entails giving it a name and a list of files. The files will be compressed and stored and can be subsequently retrieved. The artifact can be associated with a model as a whole (using
model_id) or a particular version of a model (using model_version_id).1
import baseten
2
​
3
model = baseten.deployed_model_version_id(model_version_id='a12b34c')
4
# artifacts can be associated directly on the model object
5
model.add_artifact(
6
name='Train/Test Data'
7
list_of_files=['data/train.h5', 'data/test.h5', 'README.txt'],
8
description='All the files needed for model reproduction'
9
)
10
​
11
# or an artifact can be created with just a model (or model version) ID
12
artifact = baseten.create_artifact(
13
name='Model binary',
14
list_of_files=['model.joblib'],
15
description='Model object stored with joblib',
16
model_version_id=model_version_id
17
)
18
​
19
# artifacts can be linked to many models
20
artifact.create_link(model_version_id='a_different_model_version_id')
Copied!
Once created and uploaded, interacting with artifacts and their associated models is easy.
1
# a presigned URL to download the artifact's .tgz
2
artifact_url = artifact.url()
3
​
4
# extracts the artifact binary into the given directory
5
artifact.download(target_directory='model_binary_download/')
6
​
7
# update the artifact with new data
8
artifact.update(
9
description='Model object stored as h5',
10
list_of_files=['model.h5'],
11
name='Model h5 binary'
12
)
13
​
14
# see what models are linked to this artifact
15
artifact.links()
16
​
17
# or see what artifacts are linked to a model
18
model.artifacts()
19
​
20
# delete the artifact and it's data
21
artifact.delete()
Copied!
The
url method on the artifact will give the end user a presigned URL to download and manage the archive created by the upload. The update method will replace all the existing artifact data with the data provided. The delete method will destroy the artifact and delete all the data associated with it. Use the methods carefully.Model configuration
Baseten provides an API for storing model configuration. This feature will store JSON serializable data and provide access to that information. As such it is a good place to store information such as hyperparameters and other configuration information to track with particular models.
1
model = baseten.deployed_model_version_id(model_version_id='a12b34c')
2
​
3
# a sample configuration
4
model.log_config({
5
'batch_size': 16,
6
'layers': 64,
7
'epochs': 20,
8
'learning_rate': 0.0001
9
})
10
​
11
# the latest configuration associated with the model
12
model.get_config()
Copied!
Model metrics
Model metrics and other offline evaluation metrics can also be stored via JSON serialization. To track evaluation metrics with a deployed model pass a JSON serializable object to the
log_metrics method in the client library.1
model = baseten.deployed_model_version_id(model_version_id='a12b34c')
2
​
3
# a sample set of metrics to track
4
model.log_metrics({
5
'accuracy': 94.5,
6
'log_loss': 0.222222
7
})
8
​
9
# the latest metrics associated with the model
10
model.get_metrics()
Copied!
Programmatic model metadata access
To facilitate use of these features we provide a high level solution for listing your models and their versions. Simply call
models_summary() from the client library and observe a list of dictionaries that describe the models and model versions you currently have deployed on Baseten.1
import baseten
2
​
3
​
4
baseten.models_summary()
5
​
6
# sample response
7
[{'id': 'YNBEDqa',
8
'name': 'Iris Random Forest Classifier',
9
'description': 'Predict Iris class with Random Forest Classifier',
10
'versions': [{'id': 'prwny3y',
11
'created': '2022-02-16T01:05:35.563227+00:00',
12
'configuration': {'sha': 'foobarbaz'}}]}]
Copied!