Python IPC - Twisted, RabbitMQ,

Question

I want to create 2 applications in Python that need to interact with each other. One of these applications should behave like a server, and the second should be a graphical client interface. They can run on the same system (on the same computer) or remotely and on different devices.

I want to ask you what technology I should use - AMQP messaging (for example, RabbitMQ ), Twisted as a server (or Tornado ) or ZeroMQ and connect applications to it. In the future I would like to have some authentication, etc.

I read a lot of questions and articles (for example, this one: Why do we need to use rabbitmq ), and many people say "rabbitmq and twisted different." I know what they are. I really like to know the differences and why one of these solutions will be superior to the other in this case.

EDIT: I want to use it with the following requirements:

• More than one user will be connected at that time - I think that 1 to 10 users will be connected to one program, and they will work together
• Data transfer is a “message” telling what the user has done - something like remote calls (but don’t focus on this because GUIS can be written in different languages, so messages will be something like json information).
• The system should provide collaboration, so it should be as interactive as possible. (data will be sent all the time when the user types or performs something).

In addition, I would like to hear why one solution will be better than another, not only in this particular case.

Answer 1

Twisted is used to solve the C10k network problem by providing you with an asynchronous network through the Reactor Pattern . Its also convenient because it provides a nice abstraction of concurrency, since threading / concurrency in Python is not as simple as Erlang say. Consequently, some people use Twisted to send work tasks, but that’s not what it is for.

RabbitMQ is based on the message queue pattern. Its all about reliable messaging and not about networking. I emphasize the reliable part because there are many different asynchronous network frameworks (e.g. Vert.x) that provide messaging (also known as pub / sub).

Most often, most people combine the two templates together to create a “message bus” that will work with different network needs without unnecessarily blocking the network and for great integration and scalability.

The reason the “message queue” goes so well with the network “reactor loop” is because the block should not be blocked in the reactor loop, so you have to send blocking jobs to some other process (stream, lwp, separate process cars, line, etc.). In practice, the cleanest way to do this is through distributed messaging.

According to your requirements, it sounds as if you should use an asynchronous network if you want the results to be displayed instantly and scale, but you could probably get away with a simple system that just did the polls, given that you only have several customers. So, the question is, how many total users (Twisted)? And how reliable do you want updates to be delivered (RabbitMQ)? Finally, you want your architecture to be agnostic of the language and platform ... you might want to use Node.js later (focus on the message queue instead of the asynchronous network ... i.e. RabbitMQ). Personally, I would look at Vert.x , which allows you to write in Python.

Answer 2

When someone tells you that Twisted and RabbitMQ are different from each other, this is because comparing both is comparing two things for a different purpose.

Twisted is an asynchronous structure like Tornadao. RabbitMQ is a message queuing system. You cannot compare each of them directly.

You should include your question in two new questions, the first of which should use the protocol to transfer my process? The answer can be found with words like amqp, Protocol Buffers ...

And another, what structure should I use to write my client and server programs? Here the answer may fall on Twisted, Tornado, ....