Kafka 0.11 how to reset offsets

Question

I am trying to use reset consumer bias with the latest CLI tools for Kafka.

kafka-consumer-groups.bat --bootstrap-server kafka-host:9092 --group my-group --reset-offsets --to-earliest --all-topics 

As a result, I see this output:

 TOPIC PARTITION NEW-OFFSET FirstTopic 0 0 SecondTopic 0 0 

But run the command again:

 kafka-consumer-groups.bat --bootstrap-server kafka-host:9092 --group my-group --describe 

outputs the result:

 Consumer group 'my-group' has no active members. TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG FirstTopic 0 1230 1230 0 SecondTopic 0 1022 1022 0 

I tried other options, such as resetting to an explicit offset or directly indicating the topic, but the result is the same. The conclusion suggests that the operation is performed when checking the offsets using a command or debugging, shows that the offset has not changed.

Does anyone succeed in dumping consumer bias in brokers without a zookeeper.

Answer 1

By default, --reset-offsets simply prints the result of the operation. To complete the operation, you need to add --execute to your command:

 kafka-consumer-groups.bat --bootstrap-server kafka-host:9092 --group my-group --reset-offsets --to-earliest --all-topics --execute 

Answer 2

Although the accepted answer answers the OP question perfectly, there are more options available to reset offsets. Therefore, adding this answer extends the accepted answer.

Reset bias of all topics to the earliest in the consumer group

 kafka-consumer-groups.sh --bootstrap-server <kafka_broker_host:9091> --group <group_name> --reset-offsets --to-earliest --all-topics --execute 

Reset an offset of a specific topic to the earliest in a consumer group

 kafka-consumer-groups.sh --bootstrap-server <kafka_broker_host:9091> --group <group_name> --reset-offsets --to-earliest --topic <my-topic> --execute 

Reset bias of a specific topic to a specific bias in a consumer group

 kafka-consumer-groups.sh --bootstrap-server <kafka_broker_host:9091> --group <group_name> --reset-offsets --to-offset 1000 --topic <my-topic> --execute 

Other supported arguments:

--shift-by [positive or negative integer] - offsets the offset forward or backward from the given integer.

- in the current and - in the last the same as - in the offset and - in the earliest .

--to-datetime [Date and time format: yyyy-mm-ddhh: mm: ss.xxx ]

 kafka-consumer-groups.sh --bootstrap-server <kafka_broker_host:9091> --group <group_name> --reset-offsets --to-datetime 2017-08-04T00:00:00.000 [ --all-topics or --topic <topic-name> ] --execute 

- by-duration [Format: PnDTnHnMnS ]

 kafka-consumer-groups.sh --bootstrap-server <kafka_broker_host:9091> --group <group_name> --reset-offsets --by-duration PT0H10M0S [ --all-topics or --topic <topic-name> ] --execute 

Reset to the offset in duration from the current time stamp.

How to check?

Use the command below to check the current / end offsets and confirm the reset.

 kafka-consumer-groups.sh --bootstrap-server <kafka_host:port> --group <group_id> --describe 

Output Example:

 Consumer group 'group1' has no active members. TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG CONSUMER-ID HOST CLIENT-ID intro 0 0 99 99 - - - 

Possible mistakes:

Error: assignments can only be reset if the group '[group_name]' is inactive, but the current state is Stable.

"Stable" means that an active consumer is working in this group. Therefore, you must first stop active consumers and try to reset the bias again.

it is impossible to reset biases if there is an active consumer for a group of consumers.