lookup for keyvault_secret not compatible with CLI and AWX/AAP simultaneously #2110
Description
SUMMARY
Continuation of #2103
Starting in 3.10.0 changes were made to the lookup for azure_keyvault_secret that has broken the ability to use the same playbook at the CLI and in AWX (and assuming AAP as well)
If adding use_cli=true to fix CLI az login authentication, it does not work in AWX using a Credential type of "Microsoft Azure Resource Manager" or a custom credential type that adds environment variables for AZURE_CLIENT_ID/AZURE_TENANT_ID/AZURE_CLIENT_SECRET
This results in playbooks either working in AWX or at CLI but not both, when this worked in 3.9.0 and earlier
Currently we have pinned our collection to 3.9.0 until we find a way to have both execution styles work with the same code and the new modifications.
I like the ability for AWX credential type "Microsoft Azure Resource Manager" to be functional with these lookups, but making our playbooks not be able to run in 1 of the 2 styles we regularly use is not a sustainable option.
Desired outcome is for all of the authentication methods to work together with whatever is provided.
ISSUE TYPE
- Bug Report
COMPONENT NAME
azure_keyvault_secret as a lookup
ANSIBLE VERSION
ansible [core 2.18.11]
config file = /runner/ansible.cfg
configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/local/lib/python3.12/site-packages/ansible
ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections:/runner/collections
executable location = /usr/local/bin/ansible
python version = 3.12.9 (main, Aug 14 2025, 00:00:00) [GCC 14.2.1 20250110 (Red Hat 14.2.1-7)] (/usr/bin/python3.12)
jinja version = 3.1.6
libyaml = True
COLLECTION VERSION
azure.azcollection 3.10.1
or
azure.azcollection 3.11.0
CONFIGURATION
COLLECTIONS_PATHS(/runner/ansible.cfg) = ['/root/.ansible/collections', '/usr/share/ansible/collections', '/runner/collections']
CONFIG_FILE() = /runner/ansible.cfg
DEFAULT_FILTER_PLUGIN_PATH(/runner/ansible.cfg) = ['/runner/custom_filters', '/runner/collections/ansible_collections/mts/common/plugins/filter']
DEFAULT_ROLES_PATH(/runner/ansible.cfg) = ['/runner/roles']
DEFAULT_STDOUT_CALLBACK(/runner/ansible.cfg) = community.general.timestamp
DEFAULT_TIMEOUT(/runner/ansible.cfg) = 40
HOST_KEY_CHECKING(/runner/ansible.cfg) = False
INTERPRETER_PYTHON(/runner/ansible.cfg) = auto_silent
OS / ENVIRONMENT
EE based on Red Hat UBI 10 in Docker
STEPS TO REPRODUCE
- name: Debug playbook on localhost
hosts: localhost
gather_facts: false
tasks:
- name: Show debug message
ansible.builtin.debug:
msg: "{{ lookup('azure.azcollection.azure_keyvault_secret', 'my-secret', vault_url='https://kv-myvault-001.vault.azure.net', use_msi=false) }}"
- name: Show debug message with use_cli
ansible.builtin.debug:
msg: "{{ lookup('azure.azcollection.azure_keyvault_secret', 'my-secret', vault_url='https://kv-myvault-001.vault.azure.net', use_msi=false, use_cli=true) }}"EXPECTED RESULTS
Tasks succeed in AWX with either environment variables for a service principal or with Resource Manager credential type.
Tasks also succeed using the same playbooks when running at CLI with az login authentication
ACTUAL RESULTS
After changes in 3.10.0, AWX custom credentials using environment variables AZURE_CLIENT_ID/AZURE_TENANT_ID/AZURE_CLIENT_SECRET stopped working, but could be resolved by switching to the built in credential type Microsoft Azure Resource Manager which was a change but a positive one as it worked like other Azure related modules
At the same time, running from the CLI and authenticating with az login stopped working. It was mentioned that we had to add use_cli=true to all lookup instances to get CLI authentication working with collection 3.10.0+ which does allow CLI authentication to work, but committing this code and attempting to run in AWX with either the Microsoft Azure Resource Manager credential or the environment variables no longer work.
AWX with Resource Manager credentials provided, but use_cli=true defined
{
"msg": "An unhandled exception occurred while running the lookup plugin 'azure.azcollection.azure_keyvault_secret'. Error was a <class 'ansible.errors.AnsibleError'>, original message: Failed to fetch secret my-secret from https://kv-myvault-001.vault.azure.net.. Failed to fetch secret my-secret from https://kv-myvault-001.vault.azure.net.",
"_ansible_no_log": false
}