[BUG] Incorrect Retry Handling for Document Intelligence - 429 Status regardless of Retry options

[kikaragyozov]

Library name and version

Azure.AI.DocumentIntelligence.1.0.0

Describe the bug

When running under the free plan for Document Intelligence, regardless of what settings you use for your Retry Policy - be it Exponential, be it Fixed, you'll always get hit with a 429 too many requests when for example you invoke the following method of the SDK:

Operation<Azure.AI.DocumentIntelligence.AnalyzeResult> classification = await _client.ClassifyDocumentAsync(WaitUntil.Completed, new ClassifyDocumentOptions(classifier.AzureId, data)
                {
                    Pages = "1",
                    Split = SplitMode.None
                }, cancellationToken);

Because the free plan has a Get request limit of 1 second, either the SDK has a faulty logic in where it does the first request "fast" and then immediately does a retry, before applying any of the settings the user has configured for a subsequent retry, or the SDK in no way, shape, or form, uses the RetryPolicy a user has defined for Polling and instead, does something else.

The behavior is only observed when the option WaitUntil.Completed is specified. You always get hit with a 429 too many requests, regardless of what you set up via ConfigureOptions like so:

.ConfigureOptions(options =>
{
    options.Retry.Mode = RetryMode.Fixed;
    options.Retry.MaxRetries = 157;
    options.Retry.Delay = TimeSpan.FromSeconds(15);
    options.Retry.MaxDelay = TimeSpan.FromSeconds(30);
});

With the given options above, we can still observe polling from Azure's side. We also tried using Exponential with similar settings, to no avail.

Expected behavior

I expect that when I correctly know my limits imposed by Azure, that when I configure the Retry Policy to conform with them, and additionally, limit any outbound requests with an application-wide semaphore, to never be hit with a 429 too many requests, ever.

This is not the case, and it fails immediately with that single method.

What's going on?

Actual behavior

Upon invoking the method, you get hit with a 429 too many requests when running on a Free Plan.

Reproduction Steps

Call the method on a free plan environment and you'll see a 429 in the logs.

Environment

No response

[jsquire]

Hi @kikaragyozov. Thanks for reaching out and we regret that you're experiencing difficulties. To start, I'd like to make sure that we have the same understanding of how things are intended to work.

How things work

In the code snippet that you shared, the ClassifyDocumentAsync call invokes a service operation and uses that response as a basis of polling. In the scenario that this initial call, then the retry policy that you've configured will be invoked and the initial retry will detail for the interval. If that call succeeds, the result is used to create a poller.

Pollers for long running operations in Azure SDK packages, including those used by DocumentAnalysisClient are not governed by the retry settings passed to a client. Document Analysis uses a fixed 1-second polling pattern. The initial poll is immediate, and then subsequent polling takes place at 1 second intervals when no error is encountered. This why you'll see an immediate second request for LRO endpoints and is an intentional pattern across the Azure SDK to prioritize returning results as quickly as possible.

In the scenario where the service responds with a 429 to the poller, it will honor the Retry-After header of the service, delaying for that period and then an extra second before making another attempt. So long as Document Intelligence is following its documented contract, this would ensure that the poller does not overwhelm the service and cause repeated failures.

Mitigations

It sounds like your plan is seeing an initial 429 from the service because you've started an LRO and then used WaitUntil.Completed to request automatic polling. Because there's no delay between the "start running" and initial status polling, it is exceeding your allowable calls.

You can mitigate this by using WaitUntil.Started and then having your application take control of polling, such that you can apply the delay strategy that makes the most sense for our plan and application scenarios.

Follow-up questions

• I'm not entirely clear on the pattern of calls that you're seeing. My assumption is the 429 is the result of the start/poll combination. If that isn't the case, could you please help me understand the pattern that you're seeing.

• You mention polling specifically, with respect to "429s" which leads me to wonder if you're seeing multiple failures. Occasional throttles may just be the normal clock skew between service and caller, since the automatic polling happens at the same interval that throttling does. However, it may also indicate that the service is not sending a Retry-After header with its 429s, which is likely a service error. I'd appreciate understanding the pattern that you're seeing over time.

[github-actions]

Hi @kikaragyozov. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.

[github-actions]

Hi @kikaragyozov, we're sending this friendly reminder because we haven't heard back from you in 7 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!