Invoking models
Sample invocations for Baseten Python client and API
You can invoke any deployed model that is deployed to your Baseten workspace. This doc provides sample invocations via the Baseten Python client and via an API.
If your model is scaled to zero, there will be a brief cold start on the first invocation while the model server spins up.
Use the latest version of the Baseten Python client for invocation:
pip install --upgrade baseten
To make invocations from the Python client, authenticate with an API key if you have not done so already. Run:
baseten login
Primary version invocation
Specific version invocation
Example invocation for WizardLM:
import baseten
model = baseten.deployed_model_id("abcd1234") # Replace with your actual model ID
model.predict({"prompt": "What is the difference between a wizard and a sorcerer?"})
Example invocation for WizardLM:
import baseten
model = baseten.deployed_model_version_id("qwerty12") # Replace with your actual version ID
model.predict({"prompt": "What is the difference between a wizard and a sorcerer?"})
Each API invocation must be individually authenticated via an API key in the authorization header.
Primary version invocation
Specific version invocation
Example invocation for WizardLM:
curl -X POST " https://app.baseten.co/models/abcd1234/predict" \
-H "Content-Type: application/json" \
-H 'Authorization: Api-Key WLieNJms.5CjCtfqXGyUsuvFcyXcIXnnRNGK2WgkB' \
-d '{
"prompt": "What is the difference between a wizard and a sorcerer?",
"temperature": 0.3
}'
Example invocation for WizardLM:
curl -X POST " https://app.baseten.co/model_versions/qwerty12/predict" \
-H "Content-Type: application/json" \
-H 'Authorization: Api-Key WLieNJms.5CjCtfqXGyUsuvFcyXcIXnnRNGK2WgkB' \
-d '{
"prompt": "What is the difference between a wizard and a sorcerer?",
"temperature": 0.3
}'