Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.baseten.co/llms.txt

Use this file to discover all available pages before exploring further.

Truss uses containerized environments to ensure consistent model execution across deployments. While the default Truss image works for most cases, you may need a custom base image to meet specific package or system requirements.

​
Setting a base image inconfig.yaml

Specify a custom base image in config.yaml:
config.yaml
base_image:
  image: <image_name:tag>
  python_executable_path: <path-to-python>
  • image: The Docker image to use.
  • python_executable_path: The path to the Python binary inside the container.

​
Example: NVIDIA NeMo model

Using a custom image to deploy NVIDIA NeMo TitaNet model:
config.yaml
base_image:
  image: nvcr.io/nvidia/nemo:23.03
  python_executable_path: /usr/bin/python
apply_library_patches: true
requirements:
  - PySoundFile
resources:
  accelerator: T4
  cpu: 2500m
  memory: 4512Mi
  use_gpu: true
secrets: {}
system_packages:
  - python3.8-venv

​
Using private base images

If your base image is private, ensure that you have configured your model to use a private registry

​
Creating a custom base image

You can build a new base image using Truss’s base images as a foundation. Available images are listed on Docker Hub.

​
Example: Customizing a Truss base image

Dockerfile
FROM baseten/truss-server-base:3.11-gpu-v0.7.16
RUN pip uninstall cython -y
RUN pip install cython==0.29.30

​
Building & pushing your custom image

Ensure Docker is installed and running. Then, build, tag, and push your image: 
docker build -t my-custom-base-image:0.1 .
docker tag my-custom-base-image:0.1 your-docker-username/my-custom-base-image:0.1
docker push your-docker-username/my-custom-base-image:0.1