Issue with Thumbprint updates when Thumbprint is set in VSphereMachineTemplate

[alexzimmer96]

/kind bug

What steps did you take and what happened:
When trying to update the vCenter Certificate Thumbprint in our environments, we discovered that changing the Thumbprint only in the VSphereCluster resource is not sufficient to get the reconcilation to start working again. The Thumbprint is included in the VSphereCluster and the VSphereMachineTemplate. Therefore, its not possible to update the the VSphereVM resource as discussed here (even if it should not be required).

What did you expect to happen:
From the discussion in other issues we expected to get the reconcilation loop back working when changing the Thumbprint only in the VSphereCluster resource. The "official" way of updating the Thumbprint would be therefore include creating patched VSphereMachine templates, update the reference and perform a full rolling upgrade of all nodes in every cluster.

Anything else you would like to add:
We played around with disabling the ValidationWebhook for UPDATE actions on the VSphereMachine and VSphereMachineTemplate. We can enable the reconcilation for a cluster again, when disabling the webhook and just patching the Thumbprint in at least on VSphereMachine of that cluster. After one VSphereMachine has the correct Thumbprint set, the reconcilation of the whole cluster (of the machine) is working again. Patching the Thumbprint in the VSphereMachineTemplate seems to have no effect at all.

Environment:

• Cluster-api-provider-vsphere version: 1.10.0
• Kubernetes version: (use kubectl version): 1.29.4
• OS (e.g. from /etc/os-release): Talos Linux