Mixing pubs / subas with jobs in RabbitMQ

Question

I appreciate the use of RabbitMQ as a message / message queue and look at an example tutorials on the RabbitMQ page.

I am looking for a specific scenario that is not covered by the tutorials, and I am not sure how and how to do this with RabbitMQ.

Setup :

• Suppose I have a service, let me call it “purchase orders,” and I have other services called “logistics” and “accounting.”

• When an order is sent, I want to send it as a message via RabbitMQ.

• There are 2 "accounts" and 3 "logistics" services

What would be the right way to ensure that the “account” and “logistics” process the message only once? Using pub / sub will cause messages to be processed twice (account) or trice (logistics), if I understand correctly.

With work queues and prefetch = 1, it will guarantee that only one will receive it, but I have 2 services and you want each type of service to receive one.

Is there a way to combine both and have work queues for each of the services without sending 2 separate events / messages to two different exchanges?

Answer 1

Using pub / sub will cause messages to be processed twice (account) or trice (logistics), if I understand correctly.

you probably have 1 queue per employee, based on your description, and you send a message to all work queues. therefore, each worker receives a copy of the message because you sent the message to all the queues.

what you need is one queue of "accounts" and one "logistic" queue. You will have several accounts in one queue; the same for the logistic service / queue.

setting prefetch = 1 is also important. this prevents you from immediately reading too many messages to one employee.

Is there a way to combine both and have work queues for each of the services without sending 2 separate events / messages to two different exchanges?

yes - do not use split exchanges. use a topic or direct exchange and use several routing keys to route a single message both in the account queue and in logistics.

What would be the right way to ensure that the “account” and “logistics” process the message only once?

there is no way to guarantee this, 100%. at some point, even with the correct configuration, as I described, you will have a network failure or a worker crash or some other problem, and the message will be processed twice. you should consider this in your design using idempotence in the processing form .

hope this helps!