Getting the result from a celery worker constantly

Question

I have a web application in which I am trying to use celery to load background tasks from a database. I am currently loading the database on demand, but I want to load tasks at hourly intervals and work in the background. I use jar and code in python. Redis also works for me.

So far, using celery, I got a worker to process tasks and rhythm to send tasks to an employee at intervals. But I want to get the results [dataframe or query] from the worker, and if the result is not ready, he should load the previous worker result.

Any ideas on how to do this?

Edit

I am retrieving the results from the database using sqlalchemy and I am showing the results on a web page. I have my home page, which has all the different links that all lead to different graphs that I want to download in the background, so the user does not need to wait long for the download.

Answer 1

Celery Task is performed by the Worker, and its result is stored in the Celery Backend .

If I get you right, I think you have several options:

• Ignore the result of the graph loading task, store everything you need in your database as a side effect of the task. If necessary, request the most recent result in this database. If DB is Redis, you can find ZADD and ZRANGE . This way you will get a new one if available, or a previous one if not.

• You can find the result of the task if you specify its id . You can do this when you want to know the status, something like (where celery is the Celery application): result = celery.AsyncResult(<the task id>)

• Use the callback to update further when a new result is ready.

• Let the background thread wait for AsyncResult or native_join , which is supported using Redis, and update accordingly (not recommended).

I personally used option No. 1 in such cases (using MongoDB) and found it very convenient and flexible. But perhaps due to the nature of your user interface, option number 3 will be more suitable for you.