Geek Answers Handbook

How to set group name when using messages in kafka using command line

Any idea to set the group name when using messages in kafka using the command line.

I tried the following command:

bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic nil_RF2_P2 --from-beginning --config group.id=test1 'config' is not a recognized option

The goal is to find the offset of the consumed messages with the following command:

 bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --zookeeper localhost:2181 --group test1

Can anyone help in this regard!

Thanks in advance!

+5
source share
4 answers

Got a response to changing the group name from the command line !!

actions:

 1) create a new consumer.properties file, say consumer1.properties. 2) change group.id=<Give a new group name> in the consumer.properties. 3) bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic topicname --from-beginning --consumer.config config/consumer1.properties --delete-consumer-offsets
+5
source

if you want to change the group ID without a lost record offset, you manually got the offset of the current Group.id and set for a new user who has a new identifier. if you do not have offset control in the user instance, you can run this command.

 /bin/kafka-consumer-groups.sh --new-consumer --bootstrap-server <ip_address>:<Broker_port> --describe --group Group_name

and then you can search for data from a specific offset. note that you must call the request after polling the calls, the Assign command does not work. also you can see my sample code on github
enter the link here

+3
source

The simplest solution:

bin / kafka-console-consumer.sh --zookeeper localhost: 2181 --topic nil_RF2_P2 - from the beginning - consumer group group.id = test1

If you specify the flag - from the beginning , just remember that a consumer group should not have consumed any records in the past, otherwise your consumer will start consuming from the earliest unused record of the indicated group (and not from the actual beginning, as you may incorrectly assume) .

+3
source

If you use bash, you can use the ability to replace it with a process.

 bin/kafka-console-consumer.sh --zookeeper localhost:2181 \ --topic nil_RF2_P2 --from-beginning \ --consumer.config <(echo group.id=test1)
0
source

More articles:


All Articles