Why use AMQP / ZeroMQ / RabbitMQ

Question

rather than writing your own library.

We are working on a project here, which will be a self-expanding server pool, if one partition becomes too heavy, the manager will split it and put it on another computer as a separate process. He also warns that all connected clients affect the connection to the new server.

I am curious to use ZeroMQ for inter-server and interprocess communication. My partner would prefer to give up. I am looking for a community to answer this question.

I myself am a beginner programmer and just found out about the messaging queues. As I googled and read, it seems like everyone is using message queues for all kinds of things, but why? What makes them better than writing your own library? Why are they so common and why are there so many?

Answer 1

what makes them better than writing your own library?

When you deploy the first version of your application, probably nothing: your needs are clearly defined, and you will develop a messaging system that will meet your needs: a small list of features, a small source code, etc.

These tools are very useful after the first release, when you really need to expand the application and add additional features to it. Let me give you a few cases:

• your application will have to talk to a large destination machine (sparc / powerpc) from a small destination machine (x86, intel / amd). Your messaging system had some kind of assumption of admission in order: go and fix it.
• you developed your application, so it is not a binary protocol / message system, and now it is very slow because you spend most of your time analyzing it (the number of messages has increased, and parsing has become a bottleneck): adapt it so that it could transport binary / fixed coding

• at the beginning you had 3 cars inside the lan, no noticeable delays that everyone gets to each car. your client / boss / pointy-haired-devil-boss will appear and tell you that you install the application in a WAN that you don’t control, and then you start with connection failures, bad delay, etc. you need to save the message and send an answer later: return to the code and include this material in (and enjoy)

• sent messages should have answers, but not all of them: you send some parameters and expect that a spreadsheet will be displayed instead of them, and not just send and confirm, return to the code and connect this material (and enjoy.)

• Some messages are critical, and proper backup / storage / is required to receive / send. Why do you ask? audit objectives

And many other use cases that I forgot ...

You can implement it yourself, but do not spend a lot of time on it: you will probably replace it later.

Answer 2

This is very similar to the question: why use a database when you can write your own?

The answer is that with a tool that has been around for a while and that is well understood in many different use cases, it pays more and more over time and as your requirements evolve. This is especially true if more than one developer is involved in the project. Do you want to become support staff for the queuing system if you switch to a new project? Using a tool prevents this. This becomes the problem of another.

Example: perseverance. Writing a tool to store one message on disk is very simple. Writing persistors, which scales and works well and stably, in many different use cases is both manageable and cheap to support, heavy. If you want someone to complain about how hard it is, look at this: http://www.lshift.net/blog/2009/12/07/rabbitmq-at-the-skills-matter-functional- programming-exchange

Anyway, I hope this helps. Be sure to write your own tool. Many people have done this. Whatever your problem is good.

Answer 3

I am considering using ZeroMQ myself - so I came across this question.

Assume at the moment that you have the opportunity to implement a message queuing system that meets all your requirements. Why do you accept ZeroMQ (or another third-party library) on a roll-your-own basis? Simple cost.

Assume for a moment that ZeroMQ already meets all your requirements. All you need to do is integrate it into your assembly, read some documentation, and then start using it. It was much less effort than folding your own. In addition, the burden of maintenance was transferred to another company. Since ZeroMQ is free, it looks like you just created your development team to include (part of) the ZeroMQ team.

If you started a software development business, then I think that you would balance the cost / risk of using third-party libraries against your own, in which case using ZeroMQ will win hands.

Perhaps you (or, rather, your partner) suffer, as many developers do, from Not Invented Here ? If so, adjust your attitude and reevaluate the use of ZeroMQ. Personally, I prefer a profitable relationship with other people. I hope I can be proud of finding ZeroMQ ... time will tell.

EDIT: I came across this video from the ZeroMQ developers, which talks about why you should use ZeroMQ.

Answer 4

what makes them better than writing your own library?

Mass messaging systems are transactional, which are conceptually easy to use as a client, but it is difficult to obtain the right as a developer, especially given the constant queues. You might think that you can get rid of writing a library of quick messages, but without transactions and perseverance you would not get all the benefits of a messaging system.

Persistence in this context means that the middleware for messaging keeps raw messages in permanent storage (on disk) in the event of a server failure; messages can be processed after a reboot, and retransmission is not required (the sender does not even know that there was a problem). Transactional means that you can read messages from different queues and write messages to different queues in transactional mode, which means that either all reads and writes succeed, or (if one or more failures) none succeed. This is not very different from a transaction known from interacting with databases and has the same advantages (it simplifies error handling, without transactions, you would have to ensure that each individual record / record is successful, and if one or more failures, you have to undo those changes that have succeeded).

Answer 5

Before writing your own library, read the 0MQ manual here: http://zguide.zeromq.org/page:all

Most likely, you either decide to install RabbitMQ, or you will build your library on top of ZeroMQ, as they have already completed all the hard parts.

Answer 6

If you have some time, try and deploy your own implementation! Learning this exercise will convince you of the wisdom of using an already tested library.