Schedule autoscaling for traffic

Autoscaling responds to traffic that has already arrived, so requests at the start of a traffic increase wait behind a cold start while new replicas start up. When your traffic follows known clock times, you can change the autoscaling settings before the traffic arrives rather than after. An autoscaling schedule is a recurring window on a model environment, carrying its own autoscaling settings. While the window is active, the environment runs the schedule’s settings. Outside every window, it runs the environment’s default configuration. Schedules do this natively, so you don’t need an external cron job or GitHub Action calling the Management API on a timer.

​

How schedules work

Autoscaling defaults to the configuration values you set on the environment. To vary those values by time of day or day of week, define up to 10 schedules, each pairing a recurring window with its own set of autoscaling settings. The schedule whose window covers the current time is the one in effect. A schedule’s window recurs on one of two frequencies:

• Daily: one window a day, between a start and end time, on the days you choose, such as 09:00 to 18:00 on Monday through Friday.
• Hourly: a window in every hour, optionally limited to a range of hours, such as :55 to :20 during hours 09 to 17.

Windows include their start and exclude their end: in a 09:00 to 17:00 window, 09:00 is inside and 17:00 is not. An end time at or before the start wraps past midnight, so a 22:00 to 06:00 window covers the overnight hours. Every schedule on an environment shares one timezone, set once for the whole set, and the times you enter are local to it.

​

Configure a schedule

You configure schedules per environment, from the same dialog as the rest of your autoscaling settings. To add an autoscaling schedule:

1. Sign in to your workspace at app.baseten.co and choose Dedicated Inference in the sidebar.
2. Select your model, then select the deployment for the environment you want to schedule.
3. Choose Configure autoscaling.
4. For Config to edit, choose Schedules.
5. Choose Add schedule, then set its window and replicas:
   • For Frequency, choose Daily or Hourly.
   • For On days, select the days the schedule runs.
   • For Starts at and Ends at, set the window. For an hourly schedule, use Limit to hours to restrict which hours it runs in.
   • Set Min and Max replicas for the window. To override the remaining autoscaling settings, choose More options.
6. Choose Save.

If the new schedule’s window covers the current time, the environment switches to it as soon as you save. An Active badge marks whichever configuration is running.

​

Validation

Saving fails if any schedule breaks one of these rules:

• Count: up to 10 schedules per environment.
• Window length: at least 5 minutes, long enough to scale up and back down.
• Overlap: two windows can’t cover the same time.
• Gaps: windows either touch exactly or leave at least 5 minutes between them.
• Coverage: schedules can’t fill the entire week, so the default configuration always has time to run.
• Timezone: one shared timezone per environment.

Windows that touch exactly hand off directly. A 05:40 to 09:00 window can hand straight to a 09:00 to 18:00 window with no gap and no return to the default in between.

​

Edit or remove a schedule

Changes to the schedule that’s currently running apply immediately, not at its next start. Disabling or removing it falls back to the default, or to another schedule whose window covers the current time. To edit, rename, duplicate, disable, or remove a schedule:

1. Sign in to your workspace at app.baseten.co and choose Dedicated Inference in the sidebar.
2. Select your model, then select the deployment for the environment.
3. Choose Configure autoscaling.
4. For Config to edit, choose Schedules.
5. Expand the schedule and make your changes:
   • To rename, duplicate, or remove it, use the actions menu on its header.
   • To pause it without removing it, use the Enabled switch.
6. Choose Save.

A disabled schedule still has to pass validation, so you can re-enable it later.

​

Next steps