Bundling external code
Bundle utilities and libraries with your ML model.
If your model depends on internal packages that are not available on PyPi, you can bundle this code with the Truss in the
packages folder. These bundled packages can then be imported and used in your model/model.py file.When you import your Truss, the import mechanism add the packages in the Truss'
packages directory to the path.Make sure to avoid namespace conflicts. The model serving environment depends on standard Python modules and popular packages, so don't give your bundled packages names like
requests, torch, transformers, or anything else that conflicts with standard or popular Python packages.If you want to include helper functions or other Python code directly with the Truss, create a file in
packages/. For example, we'll create my_utils.py at packages/my_utils.py.There, we can write any code we want, for example:
class MyObject:
a = 1
b = 2
def my_func():
pass
By default,
config.yaml includes the line bundled_packages_dir: packages, so all you have to do to access my_func() in model/model.py is import it:from my_utils import MyObject, my_func
And the bundled code will be available just like packages installed from PyPi.
There are situations where you may need to maintain code separately from your Truss. For example, you may want to share code across multiple Trusses without publishing it to PyPi. External packages are built for this use case.
Last modified 3mo ago