Using celery as a daemon using a supervisor does not work

Question

I have a django application in which it has celery functionality, so I can successfully manage celery, as shown below.

celery -A tasks worker --loglevel=info 

but as a well-known fact, we need to run it as a daemon, and therefore I wrote the celery.conf file below inside /etc/supervisor/conf.d/

 ; ================================== ; celery worker supervisor example ; ================================== [program:celery] ; Set full path to celery program if using virtualenv command=/root/Envs/proj/bin/celery -A app.tasks worker --loglevel=info user=root environment=C_FORCE_ROOT="yes" environment=HOME="/root",USER="root" directory=/root/apps/proj/structure numprocs=1 stdout_logfile=/var/log/celery/worker.log stderr_logfile=/var/log/celery/worker.log autostart=true autorestart=true startsecs=10 ; Need to wait for currently executing tasks to finish at shutdown. ; Increase this if you have very long running tasks. stopwaitsecs = 600 ; When resorting to send SIGKILL to the program to terminate it ; send SIGKILL to its whole process group instead, ; taking care of its children as well. killasgroup=true ; if rabbitmq is supervised, set its priority higher ; so it starts first priority=998 

but when I tried to update the supervisor like supervisorctl reread and supervisorctl update , I got a message from supervisorctl status

 celery FATAL Exited too quickly (process log may have details) 

So, I went to the worker.log file and saw the error message below

 Running a worker with superuser privileges when the worker accepts messages serialized with pickle is a very bad idea! If you really want to continue then you have to set the C_FORCE_ROOT environment variable (but please think about this before you do). User information: uid=0 euid=0 gid=0 egid=0 

So why did he complain about C_FORCE_ROOT , although we set it as an environment variable inside the confisor confisor file? what am i doing wrong in the above conf file?

Answer 1

You will need to start the celery with a non-superuser account, please remove the following lines from your configuration:

 user=root environment=C_FORCE_ROOT="yes" environment=HOME="/root",USER="root" 

And add these lines to your configuration, I assume that you are using django as non-superuser and developers as a user group:

 user=django group=developers 

Note that subprocesses inherit shell variables used to start the supervisor, with the exception of those overridden here and within the program environment. See supervisor docs .

So note that when changing environment variables through supervisor configuration files for Changes will not be applied by running supervisorctl reread and supervisorctl reload . You must start the supervisor from the very beginning with the following command:

 supervisord -c /path/to/config/file.conf 

Answer 2

I had the same problem, so I added

 environment=C_FORCE_ROOT="yes" 

in my program configuration, but this did not work so I used

 environment=C_FORCE_ROOT="true" 

working

Answer 3

From this other thread from stackoverflow. I was able to add the following settings and it worked for me.

 app.conf.update( CELERY_ACCEPT_CONTENT = ['json'], CELERY_TASK_SERIALIZER = 'json', CELERY_RESULT_SERIALIZER = 'json', )