Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.baseten.co/llms.txt

Use this file to discover all available pages before exploring further.

ML models depend on external libraries, data files, and specific hardware configurations. This guide shows you how to configure your model’s dependencies and resources. The config.yaml file defines your model’s configuration. Common options include:

​
Environment variables

To set environment variables in the model serving environment, use the environment_variables key:
config.yaml
environment_variables:
  MY_ENV_VAR: my_value

​
Python packages

Specify Python packages in config.yaml using either requirements (an inline list) or requirements_file (a path to a file). These two options are mutually exclusive.

​
Inline list

List packages directly in config.yaml:
config.yaml
requirements:
  - package_name
  - package_name2
Pin package versions with ==:
config.yaml
requirements:
  - package_name==1.0.0
  - package_name2==2.0.0

​
Requirements file

Point requirements_file at a dependency file. Truss supports three formats:
Use a standard pip requirements file for full control over pip options and repositories.
config.yaml
requirements_file: ./requirements.txt

​
Dependency constraints

Truss uses a constraints.txt file to enforce version bounds on base server dependencies. If you specify a package that overlaps with base dependencies (for example, numpy or fastapi), your version is respected but must fall within the bounds defined in constraints.txt. If you specify a version outside these bounds, the build will fail with an unsatisfiable error. This applies to both requirements (inline list) and requirements_file.

​
Chains

Chains supports the same three formats via DockerImage.requirements_file. Use make_abs_path_here to resolve the path relative to the source file: 
import truss_chains as chains

class MyChainlet(chains.ChainletBase):
    remote_config = chains.RemoteConfig(
        docker_image=chains.DockerImage(
            requirements_file=chains.make_abs_path_here("requirements.txt"),
        ),
    )
pyproject.toml and uv.lock work the same way: 
docker_image=chains.DockerImage(
    requirements_file=chains.make_abs_path_here("pyproject.toml"),
)

docker_image=chains.DockerImage(
    requirements_file=chains.make_abs_path_here("uv.lock"),
)
pip_requirements_file is deprecated. Use requirements_file instead. You can’t combine pip_requirements with pyproject.toml or uv.lock files — manage all dependencies in your pyproject.toml.

​
System packages

Truss also has support for installing apt-installable Debian packages. To add system packages to your model serving environment, add the following to your config.yaml file:
config.yaml
system_packages:
  - package_name
  - package_name2
For example, to install Tesseract OCR:
config.yaml
system_packages:
  - tesseract-ocr

​
Resources

Specify hardware resources in the resources section. Option 1: Specify individual resource fields For a CPU model:
config.yaml
resources:
  cpu: "1"
  memory: 2Gi
For a GPU model:
config.yaml
resources:
  accelerator: "L4"
When you push your model, it will be assigned an instance type matching the specifications required. Option 2: Specify an exact instance type
config.yaml
resources:
  instance_type: "L4:4x16"
Using instance_type lets you select an exact SKU. When specified, other resource fields are ignored. See the Resources page for more information on options available.

​
Advanced configuration

There are numerous other options for configuring your model. See some of the other guides: