Locally change log level for zookeeper C client

Question

I use Apache Spark spark-shell with mesos and zookeeper, which seems to work fine, except that I get more logs from zookeeper than I would like, which is a little distracting:

 Welcome to ____ __ / __/__ ___ _____/ /__ _\ \/ _ \/ _ `/ __/ '_/ /___/ .__/\_,_/_/ /_/\_\ version 1.5.0 /_/ Using Scala version 2.10.4 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_60) Type in expressions to have them evaluated. Type :help for more information. 2015-03-26 20:08:22,115:15978(0x7f83a1fbc740): ZOO_INFO@log _env@712 : Client environment:zookeeper.version=zookeeper C client 3.4.6 2015-03-26 20:08:22,115:15978(0x7f83a1fbc740): ZOO_INFO@log _env@716 : Client environment:host.name=myhost 2015-03-26 20:08:22,115:15978(0x7f83a1fbc740): ZOO_INFO@log _env@723 : Client environment:os.name=Linux 2015-03-26 20:08:22,115:15978(0x7f83a1fbc740): ZOO_INFO@log _env@724 : Client environment:os.arch=3.2.0-34-generic 2015-03-26 20:08:22,115:15978(0x7f83a1fbc740): ZOO_INFO@log _env@725 : Client environment:os.version=#53-Ubuntu SMP Thu Nov 15 10:48:16 UTC 2012 2015-03-26 20:08:22,115:15978(0x7f83a1fbc740): ZOO_INFO@log _env@733 : Client environment:user.name=myname 2015-03-26 20:08:22,115:15978(0x7f83a1fbc740): ZOO_INFO@zookeeper _init@786 : Initiating client connection, host=localhost:9999 sessionTimeout=10000 watcher=0xffffffffff sessionId=0 sessionPasswd=<null> context=0x777777777777 flags=0 2015-03-26 20:08:22,115:15978(0x7f83a1fbc740): ZOO_INFO@check _events@1703 : initiated connection to server [127.0.0.1:9999] 2015-03-26 20:08:22,115:15978(0x7f83a1fbc740): ZOO_INFO@check _events@1750 : session establishment complete on server [127.0.0.1:9999], sessionId=0x11111111111111, negotiated timeout=10000 Spark context available as sc. SQL context available as sqlContext. scala> 

I tried changing $SPARK_CONF_DIR/log4j.properties

 log4j.logger.org.apache.zookeeper=WARN 

Which didn't work, but I'm not surprised, as the mesos documentation mentions that writing to zookeeper goes directly to stderr :

NOTE. Messages from third-party logs (for example, ZooKeeper) are recorded only in stderr!

Adding a jar for intrinsic safety classpath, which has a static block with an external function interface call for the zookeeper C api function to change the log level , it seems that it can but I'm not sure, and I have no experience with calling C code from the JVM.

Answer 1

In addition to

 log4j.logger.org.apache.zookeeper=WARN 

also configure the stderr log level to print only logs with priority ERROR (and higher) .

 # Send WARN or higher to stderr log4j.appender.stderr = org.apache.log4j.ConsoleAppender log4j.appender.stderr.Threshold = ERROR log4j.appender.stderr.Target = System.err 

---

Also, LevelRangeFilter may also be useful, as shown.

Answer 2

Could not find anything on this, but for JNI (Java-to-C) this can be suggested. It seems that neither zookeeper nor mesos have a web service or shells (zookeeper has zkshell, although it’s not applicable in our case), which can be used.