Celery Tasks Not Handled

Question

I'm trying to handle some tasks using celery and I'm not too lucky. I run with celery and celery like demons. I have a tasks.py file that looks like this: a simple application and a task:

 from celery import Celery app = Celery('tasks', broker='amqp://user: pass@hostname :5672/vhostname') @app.task def process_file(f): # do some stuff # and log results 

And this file is referenced from another process.py file, which I use to monitor file changes that look like this:

 from tasks import process_file file_name = '/file/to/process' result = process_file.delay(file_name) result.get() 

And with this little code, celery cannot see tasks and process them. I can execute similar code in the python interpreter, and celery processes them:

 py >>> from tasks import process_file py >>> process_file.delay('/file/to/process') <AsyncResult: 8af23a4e-3f26-469c-8eee-e646b9d28c7b> 

However, when I run the tasks at the interpreter, beat.log and worker1.log do not show any indication that the tasks were received, but with logging I can confirm that the task code has been completed. There are no obvious errors in the .log files either. Any ideas what might cause this problem?

My /etc/default/celerybeat looks like this:

 CELERY_BIN="/usr/local/bin/celery" CELERYBEAT_CHDIR="/opt/dirwithpyfiles" CELERYBEAT_OPTS="--schedule=/var/run/celery/celerybeat-schedule" 

And /etc/default/celeryd :

 CELERYD_NODES="worker1" CELERY_BIN="/usr/local/bin/celery" CELERYD_CHDIR="/opt/dirwithpyfiles" CELERYD_OPTS="--time-limit=300 --concurrency=8" CELERYD_USER="celery" CELERYD_GROUP="celery" CELERYD_LOG_FILE="/var/log/celery/%N.log" CELERYD_PID_FILE="/var/run/celery/%N.pid" CELERY_CREATE_DIRS=1 

Answer 1

So, I understood my problem here by launching celery from the Kli, and not as a demon, which allowed me to see a more detailed output of the errors that occurred. I did this by doing:

 user@hostname /opt/dirwithpyfiles $ su celery celery@hostname /opt/dirwithpyfiles $ celery -A tasks worker --loglevel=info 

There, I saw that the rights issue was resolved as a celery user, which was not there when I ran the python interpreter commands as a regular user. I fixed this by changing the permissions of /file/to/process so that both users can read from it.