How to prevent an Azure website from processing the same message multiple times at once

Question

I have an Azure WebJob project that I run locally on my dev machine. It listens on the Azure Service Bus message queue. Nothing happens like threads, just the most basic message queue.

It receives / processes the same message several times, launching it twice immediately after receiving the message, and then periodically while the message is being processed.

Questions:

• Why do I get the same message several times instantly? It seems to be reused before applying PeekLock?
• How is it that the message is re-received, although it is still being processed? Can I set the PeekLock duration or somehow block the message only for processing only
• How can I guarantee that each message in the queue is processed only once?
• I want to be able to process several messages at the same time, and not the same message several times, so setting MaxConcurrentCalls to 1 does not seem to be my answer or I do not understand this property?

I use the async function, a simple injector and a custom JobActivator, so instead of the void static method, my function signature is:

public async Task ProcessQueueMessage([ServiceBusTrigger("AnyQueue")] MediaEncoderQueueItem message, TextWriter log) {...} 

Inside the job, it moves some files around the blob service and calls (and waits) a media encoder from media services. Thus, although web work itself does not do much processing, it takes quite a lot of time (15 minutes, for some files).

The application starts up, and when I send a message to the queue, it responds. However, it receives the message several times as soon as the message is received:

 Executing: 'Functions.ProcessQueueMessage' - Reason: 'New ServiceBus message detected on 'MyQueue'.' Executing: 'Functions.ProcessQueueMessage' - Reason: 'New ServiceBus message detected on 'MyQueue'.' 

In addition, during the execution of the task (and I see the output from the Media Service function), it will receive another “copy” from the queue.

finally, after completing the task, it still periodically processes the same message.

Answer 1

I suspect what is happening: The maximum DurationLock can be 5 minutes. If the message is processed within 5 minutes, the message is marked as completed and removed from the broker. Otherwise, the message will appear again if the processing takes more than 5 minutes (we have lost the message lock) and will be consumed again. You can verify this by looking at the DeliveryCount your message.

To solve this problem, you can update message blocking before it expires with BrokeredMessage.RenewLockAsync() .