Geek Answers Handbook

Zookeeper lock remains locked

I use celery and a zoo (kazoo castle) to lock my workers. I have a problem when I kill (-9) one of the workers before releasing the lock, after which the lock will remain locked forever.

So my question is: does the process lock process kill in this process or is this error in zookeeper?

+8
python locking celery apache-zookeeper kazoo
source share
2 answers

Killing a process with the kill signal will not remove the “software locks,” such as ZooKeeper locks.

The only kind of locks killed by the KILL signal is OS-level locks, since all file descriptors are killed, and file descriptor locks are blocked. But as for ZooKeeper, this is not OS level locks (this will only be because the ZooKeeper process, even on the same computer, is not your python process).

Therefore, this is not a bug in ZooKeeper and the expected behavior of your kill -9.

+1
source share

Zookeeper locks use ephemeral nodes . An ephemeral node is a node that lives as long as its created session is alive. Sessions are saved during the process, creating a session, periodically sending a core message to zookeeper.

So, if you kill the process that created the lock, the lock will eventually be released, as the session will die, since the zookeeper no longer receives heart contractions.

Thus, killing a worker before releasing the lock should ultimately release the lock.

If the lock is never released, a couple of things can happen,

  • Someone noticed that the lock was released and received. Presumably you are blocking because there is competition, and some other process will try to get the lock when it is released.
  • Woo is not waiting long enough. When you connect to the zookeeper, the session timeout setting must be set, that is, how long the server will keep the session waiting without any heart rate, you have to wait so long to see the blocked locks.
  • There is a mistake in kazoo. This is possible, but it seems that the kazoo blocking recipe uses ephemeral nodes, and the example you are using is very simple.

It is very unlikely that this is a bug in zookeeper.

How do you know that a lock is not released?

+8
source share

More articles:


All Articles