ML models often have dependencies on external libraries, data, and other resources. These models also typically have particular hardware configurations.

In this guide, we’ll cover the basics of how to configure your model to specify this information.

Configuration for models is specified in the config.yaml file. Here are some of the common configuration options:

​

Environment variables

You can specify environment variables to be set in the model serving environment using the environment_variables key.

environment_variables:
  MY_ENV_VAR: my_value

​

Python Packages

Python packages can be specified in two ways in the config.yaml file:

1. requirements: A list of Python packages to install.
2. requirements_file: A requirements.txt file to install pip packages from.

For example, if you have a simple list of packages, you can specify them as follows:

requirements:
  - package_name
  - package_name2

Note that you can pin versions using the == operator.

requirements:
  - package_name==1.0.0
  - package_name2==2.0.0

If you need more control over the installation process and want to use different pip options or repositories, you can specify a requirements_file instead.

requirements_file: ./requirements.txt

​

System Packages

Truss also has support for installing apt-installable Debian packages. to add system packages to your model serving environment, add the following to your config.yaml file:

system_packages:
  - package_name
  - package_name2

For a more concrete examples,

system_packages:
  - tesseract-ocr

​

Resources

Another key part of configuring your model is specifying hardware resources needed. You can use the resources key to specify these. For a CPU model, your resources configuration might look something like:

resources:
  accelerator: null
  cpu: "1"
  memory: 2Gi
  use_gpu: false

For a GPU model, your resources configuration might look like:

resources:
  accelerator: "A10G"

When you push your model, it will be assigned an instance type matching the specifications required.

See the Resources page for more information on options available.

​

Advanced configuration

There are numerous other options for configuring your model. See some of the other guides:

• Secrets
• Data
• Custom Build Commands
• Base Docker Images
• Custom Servers
• Custom Health Checks