Skip to main content
PATCH
cURL

Authorizations

Authorization
string
header
required

Send Authorization: Bearer <api_key>. The legacy Authorization: Api-Key <api_key> scheme is also accepted.

Path Parameters

model_id
string
required

Body

application/json

A request to update autoscaling settings for a deployment. All fields are optional, and we only update ones passed in.

min_replica
integer | null

Minimum number of replicas

Example:

0

max_replica
integer | null

Maximum number of replicas

Example:

7

autoscaling_window
integer | null

Timeframe of traffic considered for autoscaling decisions

Example:

600

scale_down_delay
integer | null

Waiting period before scaling down any active replica

Example:

120

concurrency_target
integer | null

Number of requests per replica before scaling up

Example:

2

target_utilization_percentage
integer | null

Target utilization percentage for scaling up/down.

Example:

70

target_in_flight_tokens
integer | null

Target number of in-flight tokens for autoscaling decisions. Early access only.

Example:

40000

max_scale_down_rate
integer | null

Maximum percentage of replicas that can be removed per autoscaling window (1–50). E.g. 20 means at most 20% of replicas are removed per window.

Required range: 1 <= x <= 50
Example:

20

Response

200 - application/json

The response to a request to update autoscaling settings.

status
enum<string>
required

Status of the request to update autoscaling settings

Available options:
ACCEPTED,
QUEUED,
UNCHANGED
message
string
required

A message describing the status of the request to update autoscaling settings