RabbitMQ transmission speeds accelerate?

Question

I am looking for ideas on how to speed up messaging through RabbitMQ.

I installed the latest version on a 64-bit version of Windows, running the server on my local machine, on which I also publish and consume to / from the C # implementation. I initially maximized 40,000 messages per second, which is impressive, but does not fit my needs (I am competing with a custom binary reader that can handle 24 million unpacked 16-byte arrays of large bytes per second, obviously I do not expect to come close to this but I'm trying to improve at least). I need to send about 115 million messages as quickly as possible. I do not want to save data, and the connection will be directed to a single consumer. Then I built chunks of my 16-byte arrays and posted them on the bus without any improvements. The maximum transfer rate is 45 mb / s. I find this very slow, given the fact that in the end it should just boil down to raw transmission speed, because I could create arrays of bytes of a few megabytes in size, where the routing efficiency coefficient on the exchange becomes insignificant compared to the transfer speed. Why is my maximum message throughput of 45 Mbps?

Answer 1

Bump ... and Update: have not seen the answer to this question for a longer time. I'm a little surprised that not a single RabbitMQ developer entered the game. I played a lot with RabbitMQ and ZeroMQ. I decided that RabbitMQ was not up to the task when considering high-performance messaging solutions in the process. The broker implementation and especially the parsing logic are the main bottleneck for increasing throughput. I dropped RabbitMQ from the list of possible options.

A white paper was presented that described how they provide a solution for managing financial flows with low latency and high bandwidth, but it seems to me that all they did was throw hardware at it, and not provide a solution that aimed at low latency, high bandwidth.

ZeroMQ did a great job after I studied the documentation more intensively. I can start the interaction in the process, it provides fairly stable push / pull, pub / sub, req / rep patterns and the pairs / pairs that I need. I was looking for locking logic inside a pub / sub template that ZeroMQ does not provide (instead of exceeding a high watermark), but a push / pull template provides locking. So I need everything I need. The only problem I am facing is understanding event handling; the implementation of the event structure through polling / multiplexing is not very satisfactory.