Geek Answers Handbook

Does Celerybeat's execution schedule run multiple times?

I have a calculate_common_locations task that runs once through CELERYBEAT_SCHEDULE. The task simply calls the function in the database:

@app.task
def calculate_common_locations():
    db.execute("SELECT * FROM calculate_centroids('b')")

This is an entry in CELERYBEAT_SCHEDULE:

CELERYBEAT_SCHEDULE = {
   'common_locations': {
        'task': 'clients.tasks.calculate_common_locations',
        'schedule': crontab(hour=23, day_of_week='sun'), #every week
    },
    [..]
}

The schedule includes more tasks that are performed once a day or every 10 seconds. These tasks are not repeated many times. Celery flower shows that the task is completed more than 20 times. The first one started on schedule, it starts ~ 100 s, runs successfully, and then starts again.

enter image description here

There is only one celerybeat move:

ps -Af | grep celerybeat 
foo     24359   779  0 01:53 ?        00:00:04 [celeryd: celery@celery:MainProcess] -active- (worker --beat --app=cloud.celeryapp:app --concurrency=10 -l INFO -s /home/foo/run/celerybeat-schedule --pidfile=/home/foo/run/celerybeat.pid)

Here's how celery begins (through a supervisor):

celery worker --beat --app=cloud.celery app:app --concurrency=10 -l INFO -s /home/foo/run/celerybeat-schedule --pidfile=/home/foo/run/celerybeat.pid

I tested it without a switch - concurrency = 10. The database function is still executed several times.

( > 1 Mil), ( ). Postgres , .

, , - ?

, :

  • django ( .delay()),
  • sql ( * ),
  • (100).

:

  • == 3.1.12
  • psql (PostgreSQL) 9.3.5
+4
1

, , crontab(hour=23, day_of_week='sun'):

>>> crontab(hour=23, day_of_week='sun')
<crontab: * 23 sun * * (m/h/d/dM/MY)>

, , 11 .

, , :

crontab(minute=0, hour=23, day_of_week='sun')
+8

More articles:


All Articles