Set the behavior of your model server.
Truss is configurable to its core. Every Truss must include a file
config.yamlin its root directory, which is automatically generated when the Truss is created. However, configuration is optional. Every configurable value has a sensible default, and a completely empty config file is valid.
description: An instruction-following LLM Using Evol-Instruct.
This reference covers the settings in
config.yamlthat may need to be modified in the course of packaging a model as a Truss.
YAML syntax can be a bit non-obvious when dealing with empty lists and dictionaries. You may notice the following in the default Truss config file:
When you fill them in with values, lists and dictionaries should look like this:
Describe your model for documentation purposes.
Any environment variables can be provided here as key value pairs and are exposed to the environment that the model executes in. Many Python libraries can be customized using environment variables, so this field can be quite handy in those scenarios.
Set any additional metadata in this catch-all field. The entire contents of the config file are available to the model at runtime, so this is a good place to store any custom information that model needs. For example, scikit-learn models include a flag here that indicates whether the model supports returning probabilities alongside predictions.
The model's name, for documentation purposes.
Set the version of Python to use for serving the model. Currently supported Python version are:
py38-- Python 3.8
py39-- Python 3.9
py310-- Python 3.10
py311-- Python 3.11
YAML syntax can be a bit non-obvious when dealing with empty lists. You may notice the following in the default Truss config file:
When you fill them in with values, lists should look like this:
This field can be used to specify the keys for such secrets and dummy default values -- Never store actual secret values in the config. Dummy default values are instructive of what the actual values look like and thus act as documentation of the format.
A model may depend on certain secret values that can't be bundled with the model and need to be bound securely at runtime. For example, a model may need to download information from s3 and may need access to AWS credentials for that.
Specify any system packages that you would typically install using
apton a Debian operating system.