Base Docker images

Truss uses containerized environments to ensure consistent model execution across deployments. While the default Truss image works for most cases, you may need a custom base image to meet specific package or system requirements.

​

Setting a base image inconfig.yaml

Specify a custom base image in config.yaml:

base_image:
  image: <image_name:tag>
  python_executable_path: <path-to-python>

• image: The Docker image to use.
• python_executable_path: The path to the Python binary inside the container.

​

Example: NVIDIA NeMo Model

Using a custom image to deploy NVIDIA NeMo TitaNet model:

base_image:
  image: nvcr.io/nvidia/nemo:23.03
  python_executable_path: /usr/bin/python
apply_library_patches: true
requirements:
  - PySoundFile
resources:
  accelerator: T4
  cpu: 2500m
  memory: 4512Mi
  use_gpu: true
secrets: {}
system_packages:
  - python3.8-venv

​

Using Private Base Images

To pull private images from a container registry, store authentication credentials as secrets.

​

Example: Docker Hub Authentication

Add this to config.yaml:

secrets:
  DOCKER_REGISTRY_https://index.docker.io/v1/: null

Then, set a Base64-encoded secret in Baseten:

echo -n 'username:password' | base64

For Google Cloud Artifact Registry, use an access token:

secrets:
  DOCKER_REGISTRY_us-west2-docker.pkg.dev: null

​

Creating a custom base image

You can build a new base image using Truss’s base images as a foundation. Available images are listed on Docker Hub.

​

Example: Customizing a Truss Base Image

FROM baseten/truss-server-base:3.11-gpu-v0.7.16
RUN pip uninstall cython -y
RUN pip install cython==0.29.30

​

Building & Pushing Your Custom Image

Ensure Docker is installed and running. Then, build, tag, and push your image:

docker build -t my-custom-base-image:0.1 .
docker tag my-custom-base-image:0.1 your-docker-username/my-custom-base-image:0.1
docker push your-docker-username/my-custom-base-image:0.1