Broken and outdated requirements.txt file

[sean-freeman]

SUMMARY

Broken and outdated requirements.txt file

Requirements file should track MS Azure CLI releases, aiding compatibility and end-user experience. Notably as there is a hidden dependency for the main MS Azure CLI azure-cli Python Package.

At the moment the requirements file version locks various Python Packages to much older versions that cause conflict/incompatibility with later releases of the MS Azure CLI.

Most of the commits for the requirements file are to version bump the MS Azure CLI (see file history https://github.com/ansible-collections/azure/commits/dev/requirements.txt), and others fear to amend other version locked packages.

In addition, this requirements file is effectively a pip freeze and can be much shorter if importing only the "top" packages which would automatically resolve their own dependencies.

ISSUE TYPE

• Bug Report

COLLECTION VERSION

ALL

STEPS TO REPRODUCE

Setup isolated test

brew install python@3.12
python -m venv /venv
source /venv/bin/activate
pip install ansible-core

Option 1 - lazy

# Due to Python Package conflicts, MS Azure CLI must always be installed first
# and then the version-locked Python Packages required by the Ansible Collection for MS Azure
# which will downgrade various Python Packages.

# Append 'msrestazure' Python Package that is hidden dependency for azure-common Python Package,
# which would cause Ansible Module util 'azure_rm_common' to break login

pip install --resume-retries 10 \
azure-cli==2.75.0 \
msrestazure \
msgraph-sdk \
azure-mgmt-resourcehealth

pip install -r https://raw.githubusercontent.com/ansible-collections/azure/refs/heads/dev/requirements.txt --resume-retries 10

Option 2 - detailed

# Due to Python Package conflicts, MS Azure CLI must always be installed first
# and then the version-locked Python Packages required by the Ansible Collection for MS Azure
# which will downgrade various Python Packages.

# Append 'msrestazure' Python Package that is hidden dependency for azure-common Python Package,
# which would cause Ansible Module util 'azure_rm_common' to break login

pip install --resume-retries 10 \
azure-cli==2.75.0 \
msrestazure \
msgraph-sdk \
azure-mgmt-resourcehealth

pip after initial Azure CLI 2.75.0 installation

adal==1.2.7
aiohappyeyeballs==2.6.1
aiohttp==3.12.15
aiosignal==1.4.0
antlr4-python3-runtime==4.13.2
anyio==4.10.0
applicationinsights==0.11.10
argcomplete==3.5.3
attrs==25.3.0
azure-appconfiguration==1.7.1
azure-batch==15.0.0b2
azure-cli==2.75.0
azure-cli-core==2.75.0
azure-cli-telemetry==1.1.0
azure-common==1.1.28
azure-core==1.35.1
azure-cosmos==3.2.0
azure-data-tables==12.4.0
azure-datalake-store==1.0.1
azure-identity==1.25.0
azure-keyvault-administration==4.4.0b2
azure-keyvault-certificates==4.7.0
azure-keyvault-keys==4.11.0b1
azure-keyvault-secrets==4.7.0
azure-keyvault-securitydomain==1.0.0b1
azure-mgmt-advisor==9.0.0
azure-mgmt-apimanagement==4.0.0
azure-mgmt-appconfiguration==3.1.0
azure-mgmt-appcontainers==2.0.0
azure-mgmt-applicationinsights==1.0.0
azure-mgmt-authorization==4.0.0
azure-mgmt-batch==17.3.0
azure-mgmt-batchai==7.0.0b1
azure-mgmt-billing==6.0.0
azure-mgmt-botservice==2.0.0
azure-mgmt-cdn==12.0.0
azure-mgmt-cognitiveservices==13.5.0
azure-mgmt-compute==34.1.0
azure-mgmt-containerinstance==10.2.0b1
azure-mgmt-containerregistry==14.1.0b1
azure-mgmt-containerservice==37.0.0
azure-mgmt-core==1.6.0
azure-mgmt-cosmosdb==9.8.0
azure-mgmt-databoxedge==1.0.0
azure-mgmt-datalake-store==1.1.0b1
azure-mgmt-datamigration==10.0.0
azure-mgmt-dns==8.0.0
azure-mgmt-eventgrid==10.2.0b2
azure-mgmt-eventhub==10.1.0
azure-mgmt-extendedlocation==1.0.0b2
azure-mgmt-hdinsight==9.0.0b3
azure-mgmt-imagebuilder==1.3.0
azure-mgmt-iotcentral==10.0.0b2
azure-mgmt-iothub==3.0.0
azure-mgmt-iothubprovisioningservices==1.1.0
azure-mgmt-keyvault==11.0.0
azure-mgmt-loganalytics==13.0.0b4
azure-mgmt-managementgroups==1.0.0
azure-mgmt-maps==2.0.0
azure-mgmt-marketplaceordering==1.1.0
azure-mgmt-media==9.0.0
azure-mgmt-monitor==7.0.0
azure-mgmt-msi==7.0.0
azure-mgmt-mysqlflexibleservers==1.0.0b3
azure-mgmt-netapp==10.1.0
azure-mgmt-policyinsights==1.1.0b4
azure-mgmt-postgresqlflexibleservers==1.1.0b2
azure-mgmt-privatedns==1.0.0
azure-mgmt-rdbms==10.2.0b17
azure-mgmt-recoveryservices==3.1.0
azure-mgmt-recoveryservicesbackup==9.2.0
azure-mgmt-redhatopenshift==1.5.0
azure-mgmt-redis==14.5.0
azure-mgmt-resource==23.3.0
azure-mgmt-resourcehealth==1.0.0b6
azure-mgmt-search==9.2.0
azure-mgmt-security==6.0.0
azure-mgmt-servicebus==8.2.1
azure-mgmt-servicefabric==2.1.0
azure-mgmt-servicefabricmanagedclusters==2.1.0b1
azure-mgmt-servicelinker==1.2.0b3
azure-mgmt-signalr==2.0.0b2
azure-mgmt-sql==4.0.0b21
azure-mgmt-sqlvirtualmachine==1.0.0b5
azure-mgmt-storage==23.0.0
azure-mgmt-synapse==2.1.0b5
azure-mgmt-trafficmanager==1.0.0
azure-mgmt-web==7.3.1
azure-monitor-query==1.2.0
azure-multiapi-storage==1.4.1
azure-storage-common==1.4.2
azure-synapse-accesscontrol==0.5.0
azure-synapse-artifacts==0.20.0
azure-synapse-managedprivateendpoints==0.4.0
azure-synapse-spark==0.7.0
bcrypt==4.3.0
certifi==2025.8.3
cffi==2.0.0
chardet==5.2.0
charset-normalizer==3.4.3
colorama==0.4.6
cryptography==46.0.1
decorator==5.2.1
Deprecated==1.2.18
fabric==3.2.2
frozenlist==1.7.0
h11==0.16.0
h2==4.3.0
hpack==4.1.0
httpcore==1.0.9
httpx==0.28.1
humanfriendly==10.0
hyperframe==6.1.0
idna==3.10
importlib_metadata==8.7.0
invoke==2.2.0
isodate==0.7.2
javaproperties==0.5.2
jmespath==1.0.1
jsondiff==2.0.0
knack==0.11.0
microsoft-kiota-abstractions==1.9.7
microsoft-kiota-authentication-azure==1.9.7
microsoft-kiota-http==1.9.7
microsoft-kiota-serialization-form==1.9.7
microsoft-kiota-serialization-json==1.9.7
microsoft-kiota-serialization-multipart==1.9.7
microsoft-kiota-serialization-text==1.9.7
microsoft-security-utilities-secret-masker==1.0.0b4
msal==1.33.0b1
msal-extensions==1.2.0
msgraph-core==1.3.8
msgraph-sdk==1.45.0
msrest==0.7.1
msrestazure==0.6.4.post1
multidict==6.6.4
oauthlib==3.3.1
opentelemetry-api==1.37.0
opentelemetry-sdk==1.37.0
opentelemetry-semantic-conventions==0.58b0
packaging==25.0
paramiko==3.5.1
pip==25.1.1
pkginfo==1.12.1.2
portalocker==2.10.1
propcache==0.3.2
psutil==7.1.0
py-deviceid==0.1.1
pycomposefile==0.0.34
pycparser==2.23
PyGithub==1.59.1
Pygments==2.19.2
PyJWT==2.10.1
PyNaCl==1.5.0
pyOpenSSL==25.3.0
PySocks==1.7.1
python-dateutil==2.9.0.post0
PyYAML==6.0.2
requests==2.32.5
requests-oauthlib==2.0.0
scp==0.13.6
semver==3.0.4
setuptools==80.9.0
six==1.17.0
sniffio==1.3.1
sshtunnel==0.1.5
std-uritemplate==2.0.5
tabulate==0.9.0
typing_extensions==4.15.0
urllib3==2.5.0
websocket-client==1.3.3
wrapt==1.17.3
xmltodict==0.15.1
yarl==1.20.1
zipp==3.23.0

# Remove 'azure-iot-hub' Python Package that is abandoned, which will require 'azure-macro-utils', and which uses 'uamqp' that has a compile cmake error - https://github.com/ansible-collections/azure/issues/1511

# Install downgraded Python Packages for Ansible Collection to avoid errors
pip install --resume-retries 10 \
azure-mgmt-core==1.4.0 \
azure-mgmt-automation==1.1.0b4 \
azure-mgmt-compute==33.0.0 \
azure-mgmt-containerregistry==10.3.0 \
azure-mgmt-containerservice==32.1.0 \
azure-mgmt-datafactory==9.0.0 \
azure-mgmt-managedservices==7.0.0b2 \
azure-mgmt-monitor==6.0.2 \
azure-mgmt-network==28.0.0 \
azure-mgmt-notificationhubs==8.1.0b1 \
azure-mgmt-search==9.2.0b2 \
azure-mgmt-storage==21.2.1 \
azure-storage-blob==12.23.0b1

# Ignore - seemingly unnecessary packages
# pip install --resume-retries 10 \
# azure-containerregistry \
# azure-keyvault \
# azure-nspkg \
# azure-servicebus
# azure-mgmt-nspkg \
# azure-mgmt-resourcegraph \
# azure-mgmt-devtestlabs \
# netaddr \
# oras \
# pandas

pip after downgraded Python Packages are installed

adal==1.2.7
aiohappyeyeballs==2.6.1
aiohttp==3.12.15
aiosignal==1.4.0
antlr4-python3-runtime==4.13.2
anyio==4.10.0
applicationinsights==0.11.10
argcomplete==3.5.3
attrs==25.3.0
azure-appconfiguration==1.7.1
azure-batch==15.0.0b2
azure-cli==2.75.0
azure-cli-core==2.75.0
azure-cli-telemetry==1.1.0
azure-common==1.1.28
azure-core==1.35.1
azure-cosmos==3.2.0
azure-data-tables==12.4.0
azure-datalake-store==1.0.1
azure-identity==1.25.0
azure-keyvault-administration==4.4.0b2
azure-keyvault-certificates==4.7.0
azure-keyvault-keys==4.11.0b1
azure-keyvault-secrets==4.7.0
azure-keyvault-securitydomain==1.0.0b1
azure-mgmt-advisor==9.0.0
azure-mgmt-apimanagement==4.0.0
azure-mgmt-appconfiguration==3.1.0
azure-mgmt-appcontainers==2.0.0
azure-mgmt-applicationinsights==1.0.0
azure-mgmt-authorization==4.0.0
azure-mgmt-automation==1.1.0b4
azure-mgmt-batch==17.3.0
azure-mgmt-batchai==7.0.0b1
azure-mgmt-billing==6.0.0
azure-mgmt-botservice==2.0.0
azure-mgmt-cdn==12.0.0
azure-mgmt-cognitiveservices==13.5.0
azure-mgmt-compute==33.0.0
azure-mgmt-containerinstance==10.2.0b1
azure-mgmt-containerregistry==10.3.0
azure-mgmt-containerservice==32.1.0
azure-mgmt-core==1.4.0
azure-mgmt-cosmosdb==9.8.0
azure-mgmt-databoxedge==1.0.0
azure-mgmt-datafactory==9.0.0
azure-mgmt-datalake-store==1.1.0b1
azure-mgmt-datamigration==10.0.0
azure-mgmt-dns==8.0.0
azure-mgmt-eventgrid==10.2.0b2
azure-mgmt-eventhub==10.1.0
azure-mgmt-extendedlocation==1.0.0b2
azure-mgmt-hdinsight==9.0.0b3
azure-mgmt-imagebuilder==1.3.0
azure-mgmt-iotcentral==10.0.0b2
azure-mgmt-iothub==3.0.0
azure-mgmt-iothubprovisioningservices==1.1.0
azure-mgmt-keyvault==11.0.0
azure-mgmt-loganalytics==13.0.0b4
azure-mgmt-managedservices==7.0.0b2
azure-mgmt-managementgroups==1.0.0
azure-mgmt-maps==2.0.0
azure-mgmt-marketplaceordering==1.1.0
azure-mgmt-media==9.0.0
azure-mgmt-monitor==6.0.2
azure-mgmt-msi==7.0.0
azure-mgmt-mysqlflexibleservers==1.0.0b3
azure-mgmt-netapp==10.1.0
azure-mgmt-network==28.0.0
azure-mgmt-notificationhubs==8.1.0b1
azure-mgmt-policyinsights==1.1.0b4
azure-mgmt-postgresqlflexibleservers==1.1.0b2
azure-mgmt-privatedns==1.0.0
azure-mgmt-rdbms==10.2.0b17
azure-mgmt-recoveryservices==3.1.0
azure-mgmt-recoveryservicesbackup==9.2.0
azure-mgmt-redhatopenshift==1.5.0
azure-mgmt-redis==14.5.0
azure-mgmt-resource==23.3.0
azure-mgmt-resourcehealth==1.0.0b6
azure-mgmt-search==9.2.0b2
azure-mgmt-security==6.0.0
azure-mgmt-servicebus==8.2.1
azure-mgmt-servicefabric==2.1.0
azure-mgmt-servicefabricmanagedclusters==2.1.0b1
azure-mgmt-servicelinker==1.2.0b3
azure-mgmt-signalr==2.0.0b2
azure-mgmt-sql==4.0.0b21
azure-mgmt-sqlvirtualmachine==1.0.0b5
azure-mgmt-storage==21.2.1
azure-mgmt-synapse==2.1.0b5
azure-mgmt-trafficmanager==1.0.0
azure-mgmt-web==7.3.1
azure-monitor-query==1.2.0
azure-multiapi-storage==1.4.1
azure-storage-blob==12.23.0b1
azure-storage-common==1.4.2
azure-synapse-accesscontrol==0.5.0
azure-synapse-artifacts==0.20.0
azure-synapse-managedprivateendpoints==0.4.0
azure-synapse-spark==0.7.0
bcrypt==4.3.0
certifi==2025.8.3
cffi==2.0.0
chardet==5.2.0
charset-normalizer==3.4.3
colorama==0.4.6
cryptography==46.0.1
decorator==5.2.1
Deprecated==1.2.18
fabric==3.2.2
frozenlist==1.7.0
h11==0.16.0
h2==4.3.0
hpack==4.1.0
httpcore==1.0.9
httpx==0.28.1
humanfriendly==10.0
hyperframe==6.1.0
idna==3.10
importlib_metadata==8.7.0
invoke==2.2.0
isodate==0.7.2
javaproperties==0.5.2
jmespath==1.0.1
jsondiff==2.0.0
knack==0.11.0
microsoft-kiota-abstractions==1.9.7
microsoft-kiota-authentication-azure==1.9.7
microsoft-kiota-http==1.9.7
microsoft-kiota-serialization-form==1.9.7
microsoft-kiota-serialization-json==1.9.7
microsoft-kiota-serialization-multipart==1.9.7
microsoft-kiota-serialization-text==1.9.7
microsoft-security-utilities-secret-masker==1.0.0b4
msal==1.33.0b1
msal-extensions==1.2.0
msgraph-core==1.3.8
msgraph-sdk==1.45.0
msrest==0.7.1
msrestazure==0.6.4.post1
multidict==6.6.4
oauthlib==3.3.1
opentelemetry-api==1.37.0
opentelemetry-sdk==1.37.0
opentelemetry-semantic-conventions==0.58b0
packaging==25.0
paramiko==3.5.1
pkginfo==1.12.1.2
portalocker==2.10.1
propcache==0.3.2
psutil==7.1.0
py-deviceid==0.1.1
pycomposefile==0.0.34
pycparser==2.23
PyGithub==1.59.1
Pygments==2.19.2
PyJWT==2.10.1
PyNaCl==1.5.0
pyOpenSSL==25.3.0
PySocks==1.7.1
python-dateutil==2.9.0.post0
PyYAML==6.0.2
requests==2.32.5
requests-oauthlib==2.0.0
scp==0.13.6
semver==3.0.4
setuptools==80.9.0
six==1.17.0
sniffio==1.3.1
sshtunnel==0.1.5
std-uritemplate==2.0.5
tabulate==0.9.0
typing_extensions==4.15.0
urllib3==2.5.0
websocket-client==1.3.3
wrapt==1.17.3
xmltodict==0.15.1
yarl==1.20.1
zipp==3.23.0

Create test file ansible_playbook_test.yml

---

- name: Ansible Task to test MS Azure connectivity
  hosts: localhost
  any_errors_fatal: true
  environment:
    ANSIBLE_AZURE_AUTH_SOURCE: "auto" # Set to auto to use module parameters

  tasks:
    - name: Get all SSH Public Keys in MS Azure
      azure.azcollection.azure_rm_sshpublickey_info:
        subscription_id: ""
        tenant: ""

Execute Ansible Playbook test

export ANSIBLE_COLLECTIONS_PATH="$PWD/ansible_collections"
ansible-galaxy collection install -p ./ansible_collections azure.azcollection
ansible-playbook ansible_playbook_test.yml -e "@./ansible_extravars.yml"

[sean-freeman]

Related to #988 #1363 #1463 #1511 #1739 #1762 #1779 #1925 #1938

Edit: If you're wondering, yes I tested those downgraded Python Packages one-by-one from the entire requirements list until I got to the problematic Python Packages that must be those specific versions otherwise the Ansible Collection would fail consistently

[Fred-sun]

Thank you for your feedback. I will explore this issue. Thank you!

[Fred-sun]

azure.azcollection relies on azure-cli-core. So 'https://github.com/ansible-collections/azure/blob/dev/plugins/module_utils/azure_rm_common.py#L1862', It is the azure-cli-core package import exception. And not azure-cli. Also. Because the installation of azure cli also affects the version of python sdk, it may lead to version compatibility issues. So we are thinking that azure cli and azure.azCollection can be placed in different virtual environments. azure.azcollection can still obtain the corresponding token under the '.azure' file generated by 'az login' without affecting the operation of azure.azcollection.

[Fred-sun]

Also, I'm considering installing azure-cli through other means, such as in ubuntu(via apt-get: https://learn.microsoft.com/en-us/cli/azure/install-azure-cli-linux?view=azure-cli-latest&pivots=apt), This way, it should not conflict with the python-sdk installed in the local environment. Thank you!

[sean-freeman]

@Fred-sun

Correct, azure-mgmt-core is version locked in the requirements to 1.4.0 (https://github.com/ansible-collections/azure/blob/dev/requirements.txt#L14).

And the azure-cli-core is version locked in the requirements to 2.75.0 (https://github.com/ansible-collections/azure/blob/dev/requirements.txt#L5)

But the collection requires usage of the az CLI command (even if line I highlighted [again] provides a misleading error message), which can be easily tested by not providing passing any login credential vars and the Ansible Collection will auto default to usage of the az CLI command and it's cached login..... if the az is not there, failure will occur.

And we can see that the azure-cli 2.75.0 imports azure-mgmt-core at version lock 1.6.0 (https://github.com/Azure/azure-cli/blob/dev/src/azure-cli/requirements.py3.Darwin.txt#L36)

Therefore, a conflict that I've demonstrated in the above repeatable steps.

The requirements for the Ansible Collection of MS Azure could be a significantly smaller list and easier to control, if the Ansible Collection version locks periodically against a specific MS Azure CLI release version.

---

Put aside the investigation above provides evidence of conflicts and downgrades, or that the requirements file leads to various errors as I traced across 9+ GH Issues (probably more if I read every GH Issue for the repo).

In some environments, it is unacceptable to have multiple Python environments for security purposes.

But also it leads to issues with the Shell Environment Variables (e.g. for proxies etc) passing through correctly and resulting in errors; I have witnessed those and cannot provide further detail.

So, these would not be allowed:

• Install isolated OS Packages of MS Azure CLI and Ansible

  • OS Package of the MS Azure CLI (e.g. via macOS Homebrew which will use an isolated Python 3.13 > azure-cli.rb)
  • OS Package of the Ansible Core / Community Edition (e.g. via macOS Homebrew which will us another isolated Python 3.13) > ansible.rb

• Install isolated OS Package of MS Azure CLI and Python, and create Python VENV with Ansible + other Python Packages

  • OS Package of the MS Azure CLI (e.g. via macOS Homebrew which will use an isolated Python 3.13 > azure-cli.rb)
  • OS Package of Python with version-lock to 3.12 (e.g. via macOS Homebrew> ansible.rb), create python -m venv
    • Python Package for ansible-core and more

But this would be permitted in a controlled environment:

• Install isolated OS Package of Python and create Python VENV with all Python Packages
  • OS Package of Python with version-lock to 3.12 (e.g. via macOS Homebrew> ansible.rb), create python -m venv
    • Python Package for ansible-core
    • Python Package for azure-cli
    • .... and more

---

That's many reasons to take action once and for all:

• Closes 9+ GH Issues immediately
• Easier maintenance long-term by version locking to specific MS Azure CLI releases
  • Avoids requirements conflicts going forward
• Ensures Ansible Collection performs correctly when MS Azure CLI is installed and used for login
• Allows Ansible Collection usage in secure/controlled environments or pipelines (e.g. AWX as can be seen in one of the GH Issues)
• Aids Ansible Collection usage when behind proxies

Side note for potential motivation - don't have any issues like this when using AWS.....

[Fred-sun]

Just as you mentioned above, if we use an isolation environment, we will indirectly rely on 'msrestazure'. However, this is an old certification method, but it does not align with Microsoft's mission of eliminating reliance on 'msrstazure'. This is also the purpose for which we use the 'az 'command to obtain the 'az login' authentication.

So my idea is, for example, in the Ubuntu system, to install it through the apt-get method, it is installed globally in the system and won't interfere with our pip3 install -r ~/requirments.txt. In this way, the az command can be used in the same environment. You can also run the azure.azcollection module. Thank you!

Reference link for apt-get installation: https://learn.microsoft.com/en-us/cli/azure/install-azure-cli-linux?view=azure-cli-latest&pivots=apt

[SQLCanuck]

• version locking to specific MS Azure CLI releases

If the version locking to Azure CLI releases was done, it may make sense to follow the LTS releases. The current LTS release of Azure CLI being v2.77.

https://learn.microsoft.com/en-us/cli/azure/azure-cli-support-lifecycle?view=azure-cli-latest#long-term-support-releases-lts