Refactor schedule-precedence to allow user overrides

[black-dragon74]

This patch refactors the schedule-precedence logic
to allow the value to be overridden by a ConfigMap.

The default mode of operation still prefers PVCs first
as source of key rotation and reclaimspace annotations.

The values for schedule-precedence keys are now concretely
defined as either pvc or storageclass.

[Rakshith-R]

This patch modifies the manager deployment to include the default ConfigMap for csi-addons.

Having it this way will enable us to configure default values on Install without requiring any code changes. It will also enable us to have different values if needed between upstream and downstream.

If it comes in as part of csv or install yamls, won't the default values override whatever values the user has set in the configmap during upgrade ?

[Madhu-1]

This patch modifies the manager deployment to include the default ConfigMap for csi-addons.
Having it this way will enable us to configure default values on Install without requiring any code changes. It will also enable us to have different values if needed between upstream and downstream.

If it comes in as part of csv or install yamls, won't the default values override whatever values the user has set in the configmap during upgrade ?

If we are not creating the configmap by default, we should not enforce for the creation, whoever is creating the CM should add the value. For upstream, we can update the document if required to update it.

What are the side effect and possibilities of updating the default in the commandline itself?

[Rakshith-R]

@black-dragon74 ,
Can you please add a PendingReleaseNotes.md file similar to the one we have at CephCSI and document this change in it ?

[Rakshith-R]

Mention possible values and default in this point itself.
Add a sentence mentioned "sc-only" option is deprecated and will be removed in next release.

[Rakshith-R]

Thanks