Windows Azure Job Queue Design

Question

In which design is someone successfully used to complete job processing in Windows Azure?

Requirements:

• Ability to transfer jobs to the queue.
• N workers can consume jobs from the queue and process them.
• The quest caster must be warned (press, not interrogate) of the completed quest.

Research so far:

• Create a Work queue using Azure service bus queues ( http://blogs.msdn.com/b/appfabric/archive/2011/05/17/an-introduction-to-service-bus-queues.aspx )
• The web interface pushes jobs to the queue, workers block on Receive () indefinitely (see http://msdn.microsoft.com/en-us/library/microsoft.servicebus.messaging.brokeredmessage.aspx ) until the Job is ready (to avoid a “zero” long poll that costs money due to transaction costs of API calls)

Regarding the completion notification:

• There is no obvious ability to be alerted when a task is completed. I thought I could use the themes / subscriptions of the bus service ( https://www.windowsazure.com/en-us/develop/net/how-to-guides/service-bus-topics/ ) and have the caller "subscribe to "topic" Completed tasks ", However:
  • You apparently cannot subscribe more than once to the same topic, unless you create several Subscription entries (which are not scalable)
  • If we did not create a “Subscription” for each Job ID and did not have a caller block when calling the recipient's API () (using I / O completion ports) in this subscription, we cannot receive real-time notifications about when Work has been processed.

Does anyone have experience implementing this kind of Job system (in real time, with low latency, with completion notifications for the caller) before?

thanks

Answer 1

In fact, the turn is not worth clicking. The whole idea of ​​the queue is that the receiver should not receive the message in real time and wants to periodically check the message. If you need real-time communication, you can create an HTTP / TCP listener on the recipient side and let the sender make an HTTP / TCP request.

Thus, one approach is to create a web service on web roles using internal endpoints. You send a business address along with a job role message using the queue. When the task is completed, the work role calls the service to notify the web role that the task has been completed.

This approach is great, but it does not make much difference. It cannot display anything in the user interface (unless you are implementing a web socket), because the server cannot notify the browser. Therefore, if you want to display a notification in the browser client, I would suggest you use the pull solution (if you do not implement a web socket). If you use a rich client, you can host the web service on the client machine and allow the employee role to notify the client by calling the service.

Best wishes,

Ming Xu.