Using secrets securely
Keep sensitive information out of your Git repository.
Some models need access to APIs, databases, AWS resources, or other secured information. With Truss, you can securely reference access keys, API tokens, passwords, secrets, and more from your model.
Never commit secret values to source control.
One common use case is giving a Truss access to a private model on HuggingFace. To give it that access, we'd need to store the
hf_access_token provided by HuggingFace and reference it securely.Start by defining secrets in the
config.yaml file. YAML is a key-value store, but you don't want to ever actually store the secret values in this file, even when running the Truss locally. Instead, set default values for the secrets, like null.secrets:
hf_access_token: null
YAML syntax can be a bit non-obvious when dealing with empty dictionaries. You may notice the following in the default Truss config file:
secrets: {}
When you fill them in with values, dictionaries should look like this:
secrets:
key1: default_value1
key2: default_value2
You can access the secrets in the
model/model.py file by referencing them via kwargs in the init function.def __init__(self, **kwargs) -> None:
self._secrets = kwargs.get("secrets")
self._tokenizer = None
From there, you can use the secrets from your
config.yaml as a dictionary within load(), predict(), or any other function in model/model.py.# Still in __init__
def load(self):
self._tokenizer = T5Tokenizer.from_pretrained(
"google/flan-t5-xl", use_auth_token=self._secrets["hf_access_token"]
)
When deploying your model to Baseten, set
is_trusted=True in the deploy() command to enable your model to access secrets:import baseten
import truss
my_model = truss.load("my-model")
baseten.deploy(
my_model,
model_name="My model",
is_trusted=True
)
Last modified 21d ago