Scaling Down Runners

Powered Off Servers

The program scales down runners by first cleaning up powered-off servers. The scaled-down service relies on the fact that the startup script starts an ephemeral runner that will pick up only 1 job and then power itself off after the job is complete.

The powered-off servers are deleted after the max-powered-off-time interval, which can be specified using the --max-powered-off-time option, which by default is set to 20 sec.

Unused Runners

The scale-down service also monitors all the runners that have unused status and tries to delete any servers associated with such runners if the runner is unused for more than the max-unused-runner-time period. This is needed in case a runner never gets a job assigned to it, and the server will stay in the power-on state. This cycle relies on the fact that the runner's name is the same as the server's name. The max-unused-runner-time can be specified using the --max-unused-runner-time option, which by default is set to 180 sec.

Zombie Servers

The scale-down service will delete any zombie servers. A zombie server is defined as any server that fails to register its runner within the max-runner-registration-time. The max-runner-registration-time can be specified using the --max-runner-registration-time option which by default is set to 180 sec.