Geek Answers Handbook

Unknown queue names are displayed on Rabbitmq mgmt. when using celery

I created only the last two queue names that are displayed in the Rabbitmq Webui management in the following table:

enter image description here

The rest of the table has hash-like queues that I don't know about:

1- Who created them? (I know it is celery, but which process, task,etc.) 2- Why they are created, and what they are created for?.

I can notice that as the number of clicked messages increases, the number of these hash-like messages increases.

+6
source share
1 answer

When using celery, Rabbitmq is used as the default backend, as well as for storing errors upon failure (which caused exceptions).

Each new task creates a new queue on the server, with thousands of tasks the broker can be overloaded with queues, and this will affect performance in negative ways.

Each queue in Rabbit will be a separate Erlang process, so if you plan to keep a lot of results at the same time, you may have to increase the Erlang process limit and the maximum number of file descriptors of your OS allows.

Old results will not be cleaned automatically, so we must tell the rabbit to do this.

Below is conf. the line dictates the time for the life of the Queue pace. The default is 1 day.

 CELERY_AMQP_TASK_RESULT_EXPIRES = Number of seconds

OR. We can completely change the backend repository, and not do it with a rabbit.

 CELERY_BACKEND = "amqp"

We can also ignore it:

 CELERY_IGNORE_RESULT = True.

In addition, when ignoring the result, we can also save errors that were saved for later use, which means another queue for unsuccessful tasks.

 CELERY_STORE_ERRORS_EVEN_IF_IGNORED = True.

I will not mark this question as an answer, expecting a better answer.

Rererences:

+5
source

More articles:


All Articles