-
Notifications
You must be signed in to change notification settings - Fork 32
Description
It would be a great addition if the autoscaler could be configured to listen on multiple queue names.
For instance, I have a queue named private
that has been in place for a long time and a lot of repos target this queue. Now, I have introduced private-arm64
as we expand into multi-arch builds.
I have used the BuildkiteAgentTags
input on the elastic-ci stack to add additional queue tags to private
: queue: private-amd64
and queue: private-x86_64
. This mostly works. If a job is created and asks for private-amd64
it will be assigned. But if there are no private
queue workers running none will be created because the autoscaler is only watching for BUILDKITE_AGENT: private
and not private-amd64
.
Also, being able to target queue: private-{{matrix.arch}}
is VERY useful to avoid duplication in pipelines.
This means I either need to:
- Switch over to queue names such as
private-amd64
andprivate-arm64
, which would require updating a lot of repos fromprivate
->private-amd64
- Introduce an entirely new
private-amd64
queue. Slightly inefficient on resource usage with extra agents hanging around.
I think the ideal would be for the autoscaler to support multiple queue names to trigger a scale event, eg: BUILDKITE_QUEUE: private,private-amd64,private-x86_64
. This would also match the behavior of the buildkite-agent
which is able to listen for and accept jobs on multiple queue
tags.