A guide to setting concurrency for your model
config.yaml
model_name: "My model with concurrency limits" runtime: predict_concurrency: 2 # Default is 1
predict_concurrency
Concurrency Target
Was this page helpful?