Geek Answers Handbook

RabbitMQ: how to limit consumption rate

I need to limit the speed of consuming messages from the rabbitmq queue.

I found many suggestions, but most of them suggest using the prefetch option. But this option does not do what I need. Even if I set prefetch to 1, the speed is around 6000 messages per second. This is too much for the consumer.

I need to limit, for example, from 70 to 200 messages per second. This means consuming one message every 5-14 ms. There are no simultaneous messages.

I am using Node.JS with the amqp.node library.

+4
source share
4 answers

I have already found a solution.

nanotimer npm .

delay = 1/[message_per_second] .

prefetch = 1

- [processing_message_time]

timeout = ack

.

+1

: https://en.wikipedia.org/wiki/Token_bucket

, " " TTL (, ?) , . , " ", " ", , .

NodeJS + amqplib :

var queueName = 'my_token_bucket';
rabbitChannel.assertQueue(queueName, {durable: true, messageTtl: 1000, maxLength: bucket.ratePerSecond});
writeToken();

function writeToken() {
    rabbitChannel.sendToQueue(queueName, new Buffer(new Date().toISOString()), {persistent: true});
    setTimeout(writeToken, 1000 / bucket.ratePerSecond);
}
+4

. " " RabbitMQ.

, , , , , . , RabbitMQ .

, RabbitMQ , . . n- n- .

, 1. RabbitMQ, . , , , . , .

+1

, RabbitMQ .

, , .

, " ", . , 10 , . 70-200, Redis, , . , Redis, , .

, 1 2 . , .

0

More articles:


All Articles