“Lost Notification” from Nodelute Repair

Question

I often see the following message when starting nodetool repair :

 [2015-02-10 16:19:40,042] Lost notification. You should check server log for repair status of keyspace xxx 

What does this mean (and how to prevent it if it is dangerous)?

I am using Cassandra 2.1.2 in a four-node cluster.

Answer 1

This message is not harmful in itself. This means that nodetool has lost traces of repair status. This does not affect the repair itself. This can be dangerous if you run the following recovery command after completing the previous command, which will lead to several parallel repairs, which will lead to a significantly higher load on the system. I used to have a script (now it no longer exists) that tracked the logs for messages about the start / end of the recovery cycle caused by the message “lost notification”, so as not to make competing repairs.

Answer 2

This seems to be a known bug that has already been fixed in recent versions.

Answer 3

You can always go as suggested by the error message, check the cassandra syslog and collect information about the repair activity.

 $ cd /var/log/cassandra/ $ cat system.log | grep repair 

Please note that I am testing for some purposes cassandra 2.1.15 and still run into a problem. Since the consideration, since this is not a serious mistake, does not affect the repair process, I think that it will be adhered to for some time.