Links

Using secrets securely

Keep sensitive information out of your Git repository.
Some models need access to APIs, databases, AWS resources, or other secured information. With Truss, you can securely reference access keys, API tokens, passwords, secrets, and more from your model.
Never commit secret values to source control.

Configuring secrets with Truss

One common use case is giving a Truss access to a private model on HuggingFace. To give it that access, we'd need to store the hf_access_token provided by HuggingFace and reference it securely.
Start by defining secrets in the config.yaml file. YAML is a key-value store, but you don't want to ever actually store the secret values in this file, even when running the Truss locally. Instead, set default values for the secrets, like null.
secrets:
hf_access_token: null
YAML syntax can be a bit non-obvious when dealing with empty dictionaries. You may notice the following in the default Truss config file:
secrets: {}
When you fill them in with values, dictionaries should look like this:
secrets:
key1: default_value1
key2: default_value2

Accessing secrets in your model

You can access the secrets in the model/model.py file by referencing them via kwargs in the init function.
def __init__(self, **kwargs) -> None:
self._secrets = kwargs.get("secrets")
self._tokenizer = None
From there, you can use the secrets from your config.yaml as a dictionary within load(), predict(), or any other function in model/model.py.
# Still in __init__
def load(self):
self._tokenizer = T5Tokenizer.from_pretrained(
"google/flan-t5-xl", use_auth_token=self._secrets["hf_access_token"]
)

Setting secrets in production

First, securely store secrets in your Baseten account by following this documentation.
When deploying your model to Baseten, set is_trusted=True in the deploy() command to enable your model to access secrets:
import baseten
import truss
my_model = truss.load("my-model")
baseten.deploy(
my_model,
model_name="My model",
is_trusted=True
)