Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.baseten.co/llms.txt

Use this file to discover all available pages before exploring further.

truss predict [OPTIONS]
Calls the packaged model with the provided input data. Use this to test your model locally or remotely.

​
Options

​
--target-directory
TEXT
A Truss directory. Defaults to current directory.
​
-d, --data
TEXT
JSON string representing the request payload.
​
-f, --file
PATH
Path to a JSON file containing the request payload.
​
--remote
TEXT
Name of the remote in .trussrc to invoke.
​
--model
TEXT
ID of the model to invoke.
​
--model-deployment
TEXT
ID of the model deployment to invoke.
​
--model-version
TEXT
Deprecated: Use --model-deployment instead. ID of the model deployment to invoke.
​
--published
Invoke the published (production) deployment.
Examples: Call a deployed model with inline JSON data: 
truss predict --remote baseten --model YOUR_MODEL_ID -d '{"prompt": "What is the meaning of life?"}'
The response is printed as formatted JSON. For streaming models, output is printed as chunks arrive. Call a model using a JSON file: 
truss predict -f request.json
Call the production deployment: 
truss predict --published -d '{"prompt": "Hello, world!"}'