I have Storm 1.1.1 installed on my machine, and on my other machine I use Kafka version 0.10.0.1. Both services are associated with Zookeeper version 3.4.6.
I successfully deployed my topology, which looked like this:
public class SOTopology
{
public static void main (String[] args ) throws Exception
{
final String brokers = args[0];
final String kafkaTopic = args[1];
final String mongo_uri = args[2];
final String mongo_collection = args[3];
TopologyBuilder topology=new TopologyBuilder();
topology.setSpout("KafkaSpout",new KafkaSpout<>(KafkaSpoutConfig.builder(brokers, kafkaTopic).build()), 1);
topology.setBolt("FilterBolt", new Filterbolt(),1).shuffleGrouping("KafkaSpout");
topology.setBolt("TagCountBolt", new TagCountBolt(),1).shuffleGrouping("FilterBolt");
topology.setBolt("TopicBolt", new TopicBolt(),1).shuffleGrouping("FilterBolt");
topology.setBolt("MongoDBBolt",new MongoDBBolt(),1).shuffleGrouping("TagCountBolt").shuffleGrouping("TopicBolt");
Config conf = new Config();
conf.setDebug(true);
conf.put("mongo.uri", mongo_uri);
conf.put("mongo.collection", mongo_collection);
conf.setMaxSpoutPending(40);
conf.setNumWorkers(10);
conf.setDebug(true);
StormSubmitter.submitTopology("StackOverflowTopology", conf, topology.createTopology());
}
}
When I switch to my StormUI, I get the following message Offset lags for kafka not supported for older versions. Please update kafka spout to latest version.I am not using HDP, but I am already using the latest version of the storm, and my storm clients .jar and storm-kafka-client.jar are both for version 1.1.1. Anyone with an idea how I can fix this?
EDIT: In a possible duplicate, they use HDP, and they fixed it by simply updating their HDP cluster. Since I do not run it on HDP, I cannot update my HDP cluster, and I am sure that my storm is the correct version.