[QUERY] [Duplicate] Azure.RequestFailedException: Multiple condition headers are not supported

[ErezCohenn]

Library name and version

Azure.Messaging.Eventhubs.Processor.5.11.5

Describe the bug

We are encountering a recurring issue within our high-throughput data pipeline service that utilizes Azure Event Hubs. The problem manifests as an unhandled exception during the processing of Event Hub messages, specifically during the operation "Executing a load balancing cycle." The inner exception is an Azure.RequestFailedException with the message "Multiple condition headers are not supported," resulting in a 400 status code.

Impact:

When this error occurs frequently, it causes the affected pod to lock the partition for an extended period. Consequently, data accumulates in the Event Hub until the transient error resolves automatically, at which point the checkpoint is released. This behavior leads to significant data latency, sometimes extending to several days, which severely impacts our data processing timelines.

Troubleshooting Steps Taken:

• SDK Logs Analysis: We have enabled and reviewed the Event Hubs SDK logs but did not find any relevant traces that indicate the root cause or potential fixes for this issue.
• Configuration Review: We have verified that we are not adding any manual headers to the storage checkpoint operations. Additionally, there are no proxy services in our architecture that could be injecting additional headers into the requests.
• Reviewed this ticket: [BUG] Azure.RequestFailedException: Multiple condition headers are not supported on stopping EventProcessorClient #33635, we dont want to override the srouce code of the SDK in order to add the property CacheControl to the header of the blobClient (And also if there is another way to add this header - what is the impact and the meaning of that? ).

Expected behavior

The EventProcessorClient should handle load balancing without encountering exceptions of "Multiple condition headers are not supported", ensuring continuous event processing.

Actual behavior

The process fails with a RequestFailedException indicating that multiple condition headers are not supported.

Additional Context:

Inner Exception: Azure.RequestFailedException: Multiple condition headers are not supported.
Stack Trace:
scss
Copy code
at Azure.Storage.Blobs.BlockBlobRestClient.UploadAsync(...)
at Azure.Storage.Blobs.Specialized.BlockBlobClient.UploadInternal(...)
at Azure.Storage.PartitionedUploader2.UploadInternal(...) at Azure.Storage.Blobs.BlobClient.StagedUploadInternal(...) at Azure.Messaging.EventHubs.Primitives.BlobCheckpointStoreInternal.ClaimOwnershipAsync(...) at Azure.Messaging.EventHubs.Primitives.EventProcessor1.DelegatingCheckpointStore.ClaimOwnershipAsync(...)
at Azure.Messaging.EventHubs.Primitives.PartitionLoadBalancer.RenewOwnershi

Reproduction Steps

Troubleshooting Steps Taken:

SDK Logs Analysis: We have enabled and reviewed the Event Hubs SDK logs but did not find any relevant traces that indicate the root cause or potential fixes for this issue.

Configuration Review: We have verified that we are not adding any manual headers to the storage checkpoint operations. Additionally, there are no proxy services in our architecture that could be injecting additional headers into the requests.

Environment

Linux OS
.NET 8
Azure EventHub premium tier
It happens in all environments we have

[github-actions]

Thank you for your feedback. Tagging and routing to the team member best able to assist.

[github-actions]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @xgithubtriage.

[amnguye]

This is definitely a duplicate of #33635 which was mentioned in the github issue.

I don't see why you would need to edit the source code of the SDK to accomplish the resolution that was concluded in the other github issue. I would highly suggest, doing a Set Properties call on the blob and setting the Cache Control property with a max age of 0.

#33635 (comment)

Also in the github issue we discussed looking into your environment to check for any firewalls, proxy or anything that could be causing a cache to happen in the first place.

This is not a bug in the SDK of eventhubs or storage.