Geek Answers Handbook

Use it to transfer JSON data

I am trying to use RabbitMQ for a distributed system that will work something like this:

  • the manufacturer queues a JSON-formatted list of order identifiers
  • several consumers exit this queue, do business logic with the identifiers of this order, and the result (formatted JSON) is also placed in another queue
  • another consumer takes data from the second line and transfers it back to the caller

I am still very new to RabbitMQ, and I wonder if this model is suitable, given the fact that the data should be as fast as possible (sometimes in seconds, maximum 5), so there are real time. Also, how large can a message be queued? The JSON that the producer will receive will be large enough based on what the consumer is doing.

Thanks for any ideas!

+4
source share
2 answers

There is nothing wrong with the design you proposed.

A slight wrinkle is that enforcing “real-time requirements” is not easy. For example, it is currently not possible to expire messages in a queue, so when using messages, they must be processed by clients.

The total message size in RabbitMQ <= 1,8,1 was limited by the amount of RAM available. Starting with version 2.0.0, it is limited by the amount of available disk space (i.e., Rabbit will print messages to disk if it works with low memory capacity). Individual message sizes are recorded as 32-bit integers (IIRC), so individual messages cannot exceed ~ 4 GB; if this is a problem, consider storing JSON in NAS and passing them some identifiers in messages. In addition, there are no restrictions.

+1
source

See page 47 in this presentation ( InfoQ ) for an excellent comparison between different message formats.

+2
source

More articles:


All Articles