Zookeeper automatic cleaning does not work

Question

After about 6 months of using zookeeper, it’s under development, although it works great, but the size of its data directory has grown to 6 GIG ! and he is still growing. The following are some of the system specifications:

version for zookeeper: 3.4.6
number of customers: <10
number of znodes: <400
also...
There are 90 log files in the dataDir / version-2 file. * no snapshot. * file in dataDir / version-2!

Google search for this problem. I found the automatic cleaning option in the Advanced Settings section of the ZooKeeper Administrator Guide page. Then I flipped the zookeeper using the following conifguration (zoo.cfg):

TickTime = 2000
DATADIR = / home / faghani / Software / Zookeeper / zkdata
ClientPort = 2181
authProvider.1 = org.apache.zookeeper.server.auth.SASLAuthenticationProvider requireClientAuthScheme = SASL
autopurge.snapRetainCount = 3
autopurge.purgeInterval = 1

But no changes happened even when purgeInterval got expired many times, i.e. The size of the zookeeper data directory is 6G and the file has not been deleted. Here's ls -laht on ${dataDir}/version-2 . It's weird here, Nautilus says the data directory size is 6G, but ls -laht says it's just 3.4G!

 faghani@node255 :~/software/zookeeper/zkdata/version-2$ ls -laht total 3.4G -rw-rw-r-- 1 faghani faghani 65M Dec 20 10:09 log.1061d drwx------ 2 faghani faghani 4.0K Dec 20 10:09 . -rw-rw-r-- 1 faghani faghani 65M Dec 19 17:28 log.105f2 -rw-rw-r-- 1 faghani faghani 65M Dec 15 18:37 log.105c1 -rw-rw-r-- 1 faghani faghani 65M Dec 14 16:17 log.105bc -rw-rw-r-- 1 faghani faghani 65M Dec 9 18:08 log.10576 drwx------ 3 faghani faghani 4.0K Dec 9 16:57 .. -rw-rw-r-- 1 faghani faghani 65M Dec 9 16:56 log.10565 -rw-rw-r-- 1 faghani faghani 65M Dec 8 18:31 log.1048c and many more until ... -rw------- 1 faghani faghani 65M Sep 2 16:41 log.1d03 

Also, the following command (as suggested in the Maintenance section) did not affect the files in the data directory.

 java -cp zookeeper.jar:lib/slf4j-api-1.7.5.jar:lib/slf4j-log4j12-1.7.5.jar:lib/log4j-1.2.16.jar:conf org.apache.zookeeper.server.PurgeTxnLog <dataDir> <snapDir> -n <count> 

By the way, I found this question , but, unfortunately, there is no solution on this page for this.

Questions:

1- Where are snapshot files stored? * 2- If SASL settings may prevent automatic cleaning? (I think no)
3- Is there something wrong with the setup?

EDIT: It looks like the solution is something like a snapCount property. The default value of this property is 100000, just reduce it to a very small number, for example. 10 and check the system.

Answer 1

If your problem is directly related to starting ZK using Solr, it would be useful to read the following email chain in the Solr mail list: Automatically clean up ZK transaction logs in Solr .

In a nutshell, Solr has an embedded ZK going through its own SolrZKServer , which does not take into account autopurge.snapRetainCount=3 and autopurge.purgeInterval=1 .

Answer 2

You can use the zkCleanup.sh script, which is located inside the source folder (subcategory ./bin ). If you cannot find it, you can find it here . Using this script:

 zkCleanup.sh <snapshotDir> -n <count> for example: # ./zkCleanup.sh /tmp/zookeeper -n 6 

<snapshotDir> - location of zookeeper snapshot files, in my case my snapshot files are located in the /tmp/zookeeper/version-2/ folder

<count> are the saved log file and snapshot file numbers. The <count> value should usually be greater than 3.

For more information, you can refer to this document: Ongoing cleaning of the data catalog .

This can be run as a cron job on ZooKeeper server machines for daily log cleaning. In my case, I clean once every week:

 0 7 * * 0 ( cd /root/otter/zookeeper/zookeeper-3.4.10/bin && ./zkCleanup.sh /tmp/zookeeper -n 5 ) >> /tmp/zookeeper/cron.log 2>&1 

You can add this with crontab -e , but remember to change the frequency to suit your requirements.