Geek Answers Handbook

How to clear logs using PurgeTxnLog?

Zookeeper quickly resolves its internal binaries across our entire production environment. According to: http://zookeeper.apache.org/doc/r3.3.3/zookeeperAdmin.html as well as http://dougchang333.blogspot.com/2013/02/zookeeper-cleaning-logs-snapshots.html this is the expected behavior, and you should call org.apache.zookeeper.server.PurgeTxnLog to regularly rotate your poop.

So:

% ls -l1rt /tmp/zookeeper/version-2/ total 314432 -rw-r--r-- 1 root root 67108880 Jun 26 18:00 log.1 -rw-r--r-- 1 root root 947092 Jun 26 18:00 snapshot.e99b -rw-r--r-- 1 root root 67108880 Jun 27 05:00 log.e99d -rw-r--r-- 1 root root 1620918 Jun 27 05:00 snapshot.1e266 ... many more % sudo java -cp zookeeper-3.4.6.jar::lib/jline-0.9.94.jar:lib/log4j-1.2.16.jar:lib/netty-3.7.0.Final.jar:lib/slf4j-api-1.6.1.jar:lib/slf4j-log4j12-1.6.1.jar:conf \ org.apache.zookeeper.server.PurgeTxnLog \ /tmp/zookeeper/version-2 /tmp/zookeeper/version-2 -n 3

but I get:

 % ls -l1rt /tmp/zookeeper/version-2/ ... all the existing logs plus a new directory /tmp/zookeeper/version-2/version-2

Am I doing something wrong?

3.4.6-Zookeeper /

+7
apache-zookeeper
source share
2 answers

ZooKeeper now has an Autopurge function with 3.4.0. Take a look at https://zookeeper.apache.org/doc/trunk/zookeeperAdmin.html

It says that you can use autopurge.snapRetainCount and autopurge.purgeInterval


autopurge.snapRetainCount

New in version 3.4.0: if enabled, the ZooKeeper automatic cleaning function saves the latest snapshots of autopurge.snapRetainCount and the corresponding transaction logs to dataDir and dataLogDir respectively and deletes the rest. Default: 3. The minimum value is 3.


autopurge.purgeInterval

New in 3.4.0: time interval in hours for which the cleanup task should be run. Set a positive integer (1 or higher) to enable automatic cleaning. The default is 0.

+10
source share

Since I don’t hear the fix through Zookeeper, this was a difficult decision:

 COUNT=6 DATADIR=/tmp/zookeeper/version-2/ ls -1drt ${DATADIR}/* | head --lines=-${COUNT} | xargs sudo rm -f

Must be run once a day from a cron or jenkins job to prevent zookeeper from exploding.

+4
source share

More articles:


All Articles