truss predict

truss predict [OPTIONS]

Calls the packaged model with the provided input data. Use this to test your model locally or remotely.

--target-directory

TEXT

A Truss directory. Defaults to current directory.

-d, --data

TEXT

JSON string representing the request payload.

-f, --file

PATH

Path to a JSON file containing the request payload.

--remote

TEXT

Name of the remote in .trussrc to invoke.

--model

TEXT

ID of the model to invoke.

--model-deployment

TEXT

ID of the model deployment to invoke.

--model-version

TEXT

Deprecated: Use --model-deployment instead. ID of the model deployment to invoke.

--published

Invoke the published (production) deployment.

Examples: Call a deployed model with inline JSON data:

truss predict --remote baseten --model YOUR_MODEL_ID -d '{"prompt": "What is the meaning of life?"}'

The response is printed as formatted JSON. For streaming models, output is printed as chunks arrive. Call a model using a JSON file:

truss predict -f request.json

Call the production deployment:

truss predict --published -d '{"prompt": "Hello, world!"}'