I am testing my implementation of celery for 3 ec2 machines right now. I am pretty sure about my implementation now, but I am having problems with the actual execution of the employee. My test structure is as follows:
- 1 ec2 machine is designated as a broker, a celery worker also works.
- 1 ec2 machine is assigned by the client (starts the client celery script, which queues all tasks using .delay (), also starts the celery worker
- 1 ec2 machine is purely working.
All machines have 1 celery worker. Previously, I immediately received a message:
"A significant drift from celery @ [other ec2 ip] may mean that the clock is out of sync." Then the drift value in seconds will be printed, which will increase over time. I also get messages: "a missed heartbeat from celery @ [other ec2 ip].
At this moment, the machine will work very little, so my AutoScaling configuration in ec2 will automatically turn off the instance as soon as it reaches the processor load levels very low (<5%)
To try to solve this problem, I tried to synchronize all my machine clocks (although I thought that celery could handle this) using this command, which was executed at startup for all machines:
apt-get -qy install ntp
service ntp start
10 , , ec2 . , , .
, ?
(3.1) rabbitmq
EDIT: , us-west-1a us-west-1c ec2.
EDIT2: , . t2.micro 3 ( 1 ), , .