Update autoscaling from zero enhancement proposal with support for platform-aware autoscale from zero

This commit updates the contract between the cluster-autoscaler Cluster API provider and the infrastructure provider's controllers that reconcile the Infrastructure Machine Template to support platform-aware autoscale from 0 in clusters consisting of nodes heterogeneous in CPU architecture and OS.

With this commit, the infrastructure providers implementing controllers to reconcile the status of their Infrastructure Machine Templates for supporting autoscale from 0 will be able to fill the status.nodeInfo stanza with additional information about the nodes.

The status.nodeInfo stanza has type corev1.NodeSystemInfo to reflect the same content, the rendered nodes' objects would store in their status field.

The cluster-autoscaler can use that information to build the node template labels `kubernetes.io/arch` and `kubernetes.io/os` if that information is present.

Suppose the pending pods that trigger the cluster autoscaler have a node selector or a requiredDuringSchedulingIgnoredDuringExecution node affinity concerning the architecture or operating system of the node where they can execute. In that case, the autoscaler will be able to filter the nodes groups options according to the architecture or operating system requested by the pod.

The users could already provide this information to the cluster autoscaler through the labels capacity annotation. However, there is no similar capability to support future labels/taints through information set by the reconcilers of the status of Infrastructure Machine Templates.

Author: aleskandro

docs/proposals/20210310-opt-in-autoscaling-from-zero.md

@@ -107,8 +107,8 @@ node group. But, during a scale from zero situation (ie when a node group has ze
 autoscaler needs to acquire this information from the infrastructure provider.
 
 An optional status field is proposed on the Infrastructure Machine Template which will be populated
-by infrastructure providers to contain the CPU, memory, and GPU capacities for machines described by that
-template. The cluster autoscaler will then utilize this information by reading the appropriate
+by infrastructure providers to contain the CPU, CPU architecture, memory, and GPU capacities for machines
+described by that template. The cluster autoscaler will then utilize this information by reading the appropriate
 infrastructure reference from the resource it is scaling (MachineSet or MachineDeployment).
 
 A user may override the field in the associated infrastructure template by applying annotations to the
@@ -160,6 +160,10 @@ the template.  Internally, this field will be represented by a Go `map` type uti
 for the keys and `k8s.io/apimachinery/pkg/api/resource.Quantity` as the values (similar to how resource
 limits and requests are handled for pods).
 
+Additionally, the status field could contain information about the node, such as the architecture and 
+operating system. This information is not required for the autoscaler to function, but it can be useful in 
+scenarios where the autoscaler needs to make decisions for clusters with heterogeneous node groups in architecture, OS, or both.
+
 It is worth mentioning that the Infrastructure Machine Templates are not usually reconciled by themselves.
 Each infrastructure provider will be responsible for determining the best implementation for adding the
 status field based on the information available on their platform.
@@ -175,6 +179,9 @@ const (
 // DockerMachineTemplateStatus defines the observed state of a DockerMachineTemplate
 type DockerMachineTemplateStatus struct {
     Capacity corev1.ResourceList `json:"capacity,omitempty"`
+    
+    // +optional
+    NodeInfo *platform.NodeInfo `json:"nodeInfo,omitempty"`
 }
 
 // DockerMachineTemplate is the Schema for the dockermachinetemplates API.
@@ -188,6 +195,40 @@ type DockerMachineTemplate struct {
 ```
 _Note: the `ResourceList` and `ResourceName` referenced are from k8s.io/api/core/v1`_
 
+`platform.NodeInfo` is a struct that contains the architecture and operating system information of the node, to implement 
+in the providers integration code.
+Its definition would look like this:
+
+```go
+package platform
+
+// Architecture represents the CPU architecture of the node. Its underlying type is a string.
+// Its underlying type is a string and its value can be any of amd64, arm64, s390x, ppc64le.
+// +kubebuilder:validation:Enum=amd64;arm64;s390x;ppc64le
+// +enum
+type Architecture string
+
+// Architecture constants defined for clarity.
+const (
+    ArchitectureAmd64 Architecture = "amd64"
+    ArchitectureArm64 Architecture = "arm64"
+    ArchitectureS390x Architecture = "s390x"
+    ArchitecturePpc64le Architecture = "ppc64le"
+)
+
+// NodeInfo contains information about the node's architecture and operating system.
+type NodeInfo struct {
+    // architecture is the CPU architecture of the node. 
+    // Its underlying type is a string and its value can be any of amd64, arm64, s390x, ppc64le.
+    // +optional
+    Architecture Architecture `json:"architecture,omitempty"`
+    // operatingSystem is a string representing the operating system of the node.
+    // This may be a string like 'linux' or 'windows'.
+    // +optional
+    OperatingSystem string `json:"operatingSystem,omitempty"`
+}
+```
+
 When used as a manifest, it would look like this:
 
 ```
@@ -204,8 +245,13 @@ status:
     memory: 500mb
     cpu: "1"
     nvidia.com/gpu: "1"
+  nodeInfo:
+    architecture: arm64
+    operatingSystem: linux
 ```
 
+The information stored in the `status.nodeInfo` field will be used by the cluster autoscaler's scheduler simulator to determine the simulated node's labels `kubernetes.io/arch` and `kubernetes.io/os`. This logic will be implemented in the cluster autoscaler's ClusterAPI cloud provider code.
+
 #### MachineSet and MachineDeployment Annotations
 
 In cases where a user needs to provide specific resource information for a
@@ -229,6 +275,8 @@ metadata:
       capacity.cluster-autoscaler.kubernetes.io/memory: "500mb"
       capacity.cluster-autoscaler.kubernetes.io/cpu: "1"
       capacity.cluster-autoscaler.kubernetes.io/ephemeral-disk: "100Gi"
+      node-info.cluster-autoscaler.kubernetes.io/architecture: "arm64"
+      node-info.cluster-autoscaler.kubernetes.io/operating-system: "linux"
 ```
 _Note: the annotations will be defined in the cluster autoscaler, not in cluster-api._
 
@@ -246,6 +294,9 @@ metadata:
     capacity.cluster-autoscaler.kubernetes.io/taints: "key1=value1:NoSchedule,key2=value2:NoExecute"
 ```
 
+The `capacity.cluster-autoscaler.kubernetes.io/labels` annotation takes precedence over other sources when conflicts occur.
+For instance, if the `kubernetes.io/arch` label is specified in `capacity.cluster-autoscaler.kubernetes.io/labels`, its value supersedes that specified by `node-info.cluster-autoscaler.kubernetes.io/architecture`.
+
 ### Security Model
 
 This feature will require the service account associated with the cluster autoscaler to have
@@ -318,6 +369,7 @@ office hours meeting:
 
 ## Implementation History
 
+- [X] 05/08/2025: Updated proposal to enable architecture- and OS- aware auto-scale from 0
 - [X] 09/12/2024: Added section on Implementation Status
 - [X] 01/31/2023: Updated proposal to include annotation changes
 - [X] 06/10/2021: Proposed idea in an issue or [community meeting]