Geek Answers Handbook

Trying to build and run Apache Kafka 0.8 vs Scala 2.9.2 without success

As stated in the topic description, I am trying to run Kafka 0.8 with Scala 2.9.2 .

I managed to get a working version using a quick start for 0.8 ( https://cwiki.apache.org/KAFKA/kafka-08-quick-start.html ), but it was compiled against Scala 2.8.0 by default.

I tried to change the step

 ./sbt package

to

 ./sbt "++2.9.2 package"

it compiles without errors, but during startup it complains that it cannot find the main class.

 /tmp/kafka-8-test/kafka[0.8]$ bin/kafka-server-start.sh onfig/server1.properties Error: Could not find or load main class kafka.Kafka

Any help would be greatly appreciated.

+4
source share
3 answers

The problem is that the bin / kafka-server-start.sh script uses bin / kafka-run-class.sh to execute the generated jar file.

This script has hardcoded versions, so you need to configure it like this:

 ... library=$(echo "$ivyPath/org.scala-lang/scala-library/jars/scala-library-2.9.2.jar") CLASSPATH=$CLASSPATH:$library compiler=~$(echo "$ivyPath/org.scala-lang/scala-compiler/jars/scala-compiler-2.9.2.jar") CLASSPATH=$CLASSPATH:$compiler log4j=$(echo "$ivyPath/log4j/log4j/jars/log4j-1.2.15.jar") CLASSPATH=$CLASSPATH:$log4j slf=$(echo "$ivyPath/org.slf4j/slf4j-api/jars/slf4j-api-1.6.4.jar") CLASSPATH=$CLASSPATH:$slf zookeeper=$(echo "$ivyPath/org.apache.zookeeper/zookeeper/jars/zookeeper-3.3.4.jar") CLASSPATH=$CLASSPATH:$zookeeper jopt=$(echo "$ivyPath/net.sf.jopt-simple/jopt-simple/jars/jopt-simple-3.2.jar") CLASSPATH=$CLASSPATH:$jopt for file in $base_dir/core/target/scala-2.9.2/*.jar; do CLASSPATH=$CLASSPATH:$file done for file in $base_dir/core/lib/*.jar; do CLASSPATH=$CLASSPATH:$file done for file in $base_dir/perf/target/scala-2.9.2/kafka*.jar; do CLASSPATH=$CLASSPATH:$file done ...
+4
source

kafka-run-class.sh hardcoded to Scala 2.8.0. You can change 2.8.0 to 2.9.2, as suggested by prenomenon.

This works for me:

  • Linux | Unix

    sed -i "s/2.8.0/2.9.2/g" bin/kafka-run-class.sh

  • MacOS

    sed -i.bak "s/2.8.0/2.9.2/g" bin/kafka-run-class.sh

+9
source

With the latest version of Kafka 0.8.1.1 and gradlew, "SCALA_VERSION" is a variable in the script.

SCALA_VERSION = 2.10.4

However, somewhere, something goes wrong:

,,,. / gradlew -PscalaVersion = 2.10.4 jar

`` ``

where one of the files does not contain part 2.10.4, but only 2.10:

peter_v@trusty64 :~/data/kafka/kafka-0.8.1.1-src$ find . -name '*.jar' ./perf/build/libs/kafka-perf_2.10-0.8.1.1.jar ./clients/build/libs/kafka-clients-0.8.1.1.jar ./system_test/migration_tool_testsuite/0.7/lib/kafka-perf-0.7.0.jar ./system_test/migration_tool_testsuite/0.7/lib/kafka-0.7.0.jar ./system_test/migration_tool_testsuite/0.7/lib/zkclient-0.1.jar ./examples/build/libs/kafka-examples-0.8.1.1.jar ./core/build/libs/kafka_2.10-0.8.1.1.jar ############ 2.10 instead of 2.10.4 ? ./core/build/dependant-libs-2.10.4/snappy-java-1.0.5.jar ./core/build/dependant-libs-2.10.4/metrics-core-2.2.0.jar ./core/build/dependant-libs-2.10.4/zkclient-0.3.jar ./core/build/dependant-libs-2.10.4/log4j-1.2.15.jar ./core/build/dependant-libs-2.10.4/slf4j-api-1.7.2.jar ./core/build/dependant-libs-2.10.4/zookeeper-3.3.4.jar ./core/build/dependant-libs-2.10.4/jopt-simple-3.2.jar ./core/build/dependant-libs-2.10.4/scala-library-2.10.4.jar ./target/scala-2.10/kafka-0-8-1-1-src_2.10-0.1-SNAPSHOT.jar ./lib/apache-rat-0.8.jar ./contrib/hadoop-consumer/lib/piggybank.jar ./contrib/hadoop-consumer/build/libs/kafka-hadoop-consumer-0.8.1.1.jar ./contrib/hadoop-producer/lib/piggybank.jar ./contrib/hadoop-producer/build/libs/kafka-hadoop-producer-0.8.1.1.jar ./contrib/build/libs/contrib-0.8.1.1.jar ./gradle/wrapper/gradle-wrapper.jar

With a copy of the name 2.10.4 as a workaround, Kafka started correctly.

`` ``

cp core / build / libs / kafka_2.10-0.8.1.1.jar core / build / libs / kafka_2.10.4-0.8.1.1.jar `` `

0
source

More articles:


All Articles