[Bug] [Migrating to workload identity from Pod identity] [AKS]'The error response was either empty or could not be parsed.. Error response received from the server: no azure identity found for request'

[shahriaak]

Library version used

4.70.1.0

.NET version

8.0

Scenario

ManagedIdentityClient - managed identity

Is this a new or an existing app?

The app is in production, and I have upgraded to a new version of MSAL

Issue description and reproduction steps

We are working on migrating our service to workload identity from pod identity Workload identity migration but we ran into the following error with Redis failing to auth with managed identity.
The code used to connect to redis already has the changes to use UserAsssignedManagedIdentity: https://learn.microsoft.com/en-us/azure/azure-cache-for-redis/cache-azure-active-directory-for-authentication#client-library-support

Exception encountered starting the service System.Exception: Failed to acquire token
 ---> MSAL.NetCore.4.70.1.0.MsalServiceException:
        ErrorCode: managed_identity_request_failed
Microsoft.Identity.Client.MsalServiceException: [Managed Identity] The error response was either empty or could not be parsed.. Error response received from the server: no azure identity found for request clientID 7277##### REDACTED #####752d
.
   at Microsoft.Azure.StackExchangeRedis.AzureCacheOptionsProviderWithToken.AcquireTokenAsync(Boolean throwOnFailure) in C:\__w\1\s\src\AzureCacheOptionsProviderWithToken.cs:line 193
   at StackExchange.Redis.AzureCacheForRedis.ConfigureForAzureAsync(ConfigurationOptions configurationOptions, AzureCacheOptions azureCacheOptions) in C:\__w\1\s\src\AzureCacheForRedis.cs:line 108
        StatusCode: 404 
        ResponseBody:  
        Headers: 
   --- End of inner exception stack trace ---
   at StackExchange.Redis.AzureCacheForRedis.ConfigureForAzureAsync(ConfigurationOptions configurationOptions, AzureCacheOptions azureCacheOptions) in C:\__w\1\s\src\AzureCacheForRedis.cs:line 112
   at StackExchange.Redis.AzureCacheForRedis.ConfigureForAzureWithUserAssignedManagedIdentityAsync(ConfigurationOptions configurationOptions, String clientId) in C:\__w\1\s\src\AzureCacheForRedis.cs:line 46
   at Microsoft.MarketplaceServicesCore.Core.Throttling.AzureRedisClient.Connect()
   at Microsoft.MarketplaceServicesCore.Core.Throttling.AzureThrottlerBuilder.CreateThrottlerAsync(AzureThrottlerSettings settings, ILogger logger)
   at Microsoft.MarketplaceServices.Mcapi.EntitlementsSynthesizer.RegistrationExtensions.<>c__DisplayClass0_0.<RegisterLicenseWriters>b__1(IServiceProvider sp)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)

Relevant code snippets

Code used to connect to Redis:

    public class AzureRedisClient : IRedisClient
    {
        private static readonly TimeSpan DriftRefresh = TimeSpan.FromSeconds(60);

        private readonly AzureThrottlerSettings configuration;
        private ConnectionMultiplexer redisConnection;
        private readonly ILogger logger;

        private readonly CancellationTokenSource timeSyncCts;
        private Task timeSyncTask;

        private bool connected = false;

        public long OffsetInTicks
        {
            get; private set;
        }

        public AzureRedisClient(AzureThrottlerSettings configuration, ILogger logger)
        {
            this.configuration = configuration ?? throw new ArgumentNullException(nameof(configuration));
            this.logger = logger ?? throw new ArgumentNullException(nameof(logger));
            this.timeSyncCts = new CancellationTokenSource();
        }

        /// <summary>
        /// Connects to a Redis instance. Starts task to read latency to Redis and adjust calculations appropriately.
        /// </summary>
        /// <returns>A task representing the execution of this operation.</returns>
        public async Task Connect()
        {
            if (this.connected) { return; }

            TimeSpan timeout = this.configuration.DefaultTimeoutToRedis ?? TimeSpan.FromSeconds(1);

            var options = ConfigurationOptions.Parse(this.configuration.RedisConnectionString);
            options.SyncTimeout = (int)timeout.TotalMilliseconds;
            options.AsyncTimeout = (int)timeout.TotalMilliseconds;

            await this.ConfigureIdentityAsync(options);

            this.redisConnection = ConnectionMultiplexer.Connect(options);

            // Calculate latency to Redis for time series calculation.
            this.timeSyncTask = this.SetOffsetAsync();
            this.connected = true;
        }

        private Task ConfigureIdentityAsync(ConfigurationOptions options)
        {
            if (String.IsNullOrEmpty(this.configuration.PrincipalId))
            {
                return Task.CompletedTask;
            }

            if (String.IsNullOrEmpty(this.configuration.ClientId))
            {
                return options.ConfigureForAzureWithSystemAssignedManagedIdentityAsync();
            }

            return options.ConfigureForAzureWithUserAssignedManagedIdentityAsync(this.configuration.ClientId);
        }

        public void Dispose()
        {
            if (this.timeSyncTask != null && !this.timeSyncTask.IsCompleted)
            {
                this.timeSyncCts.Cancel();
                this.timeSyncTask.Wait();
            }

            this.timeSyncCts?.Dispose();
            this.redisConnection?.Dispose();
        }

        /// <summary>
        /// Executes a Redis Command.
        /// </summary>
        /// <typeparam name="T">Type of response.</typeparam>
        /// <param name="key">Key of the entry. Required by Xbox Throttling interface.</param>
        /// <param name="command">Redis Command.</param>
        /// <returns>Return value of command.</returns>
        /// <exception cref="InvalidOperationException">Connect must be run prior to executing commands.</exception>
        public Task<T> ExecuteDatabaseCommandAsync<T>(string key, Func<IDatabase, Task<T>> command)
        {
            if (!this.connected) { throw new InvalidOperationException("Connection not yet attempted."); }

            return command(this.redisConnection.GetDatabase());
        }

        private async Task SetOffsetAsync()
        {
            if (this.redisConnection == null) { return; }

            EndPoint endpoint = this.redisConnection.GetEndPoints().FirstOrDefault();

            if (endpoint == null) { return; }

            IServer server = this.redisConnection.GetServer(endpoint);

            CancellationToken ct = this.timeSyncCts.Token;
            while (!ct.IsCancellationRequested)
            {
                try
                {
                    DateTime serverTime = await server.TimeAsync();
                    this.OffsetInTicks = DateTime.UtcNow.Ticks - serverTime.Ticks;
                    await Task.Delay(DriftRefresh, ct);
                }
                catch (TaskCanceledException tce) when (tce.CancellationToken == ct)
                {
                    return;
                }
                catch (Exception exception)
                {
                    this.logger.LogException(exception);
                }
            }
        }
    }

Expected behavior

No response

Identity provider

Microsoft Entra ID (Work and School accounts and Personal Microsoft accounts)

Regression

No response

Solution and workarounds

No response

[github-actions]

Here are some similar issues that might help you. Please check if they can solve your problem.

• [Bug] Unable to Authenticate with a User Assigned Managed Identity In Azure Deployment Scripts (Azure Containers) #5272

---

Possible solution (Extracted from existing issue, might be incorrect; please verify carefully)

Hi, this has been fixed in MSAL 4.67.2 - please update to the latest version. If you are using Azure SDK, please wait for them to release a patch, it should happen soon.

Reference:

• [Bug] Unable to Authenticate with a User Assigned Managed Identity In Azure Deployment Scripts (Azure Containers) #5272 (comment)

Powered by issue-sentinel