Geek Answers Handbook

Kafka interrupt reassign section action

I am using kafka 0.8.2 and one of my kafka servers has died (there is no way to recover data on disks). There is a topic with replication 1 with one of the partitions on a dead server. I thought reassignment would move the metadata for this section to a new server without the need for data, but reassignment is bound to in progress .

I ran:

 $ /opt/kafka/kafka/bin/kafka-reassign-partitions.sh --zookeeper myzookeeper.my.com --reassignment-json-file new_assignment.json --verify Status of partition reassignment: Reassignment of partition [topicX,1] is still in progress

This will never be done since a dead server never returns.

In the new server log I saw:

 [2015-05-28 06:25:15,401] INFO Completed load of log topicX-1 with log end offset 0 (kafka.log.Log) [2015-05-28 06:25:15,402] INFO Created log for partition [topicX,1] in /mnt2/data/kafka with properties {segment.index.bytes -> 10485760, file.delete.delay.ms -> 60000, segment.bytes -> 536870912, flush.ms -> 9223372036854775807, delete.retention.ms -> 86400000, index.interval.bytes -> 4096, retention.bytes -> -1, min.insync.replicas -> 1, cleanup.policy -> delete, unclean.leader.election.enable -> true, segment.ms -> 604800000, max.message.bytes -> 1000012, flush.messages -> 9223372036854775807, min.cleanable.dirty.ratio -> 0.5, retention.ms -> 259200000, segment.jitter.ms -> 0}. (kafka.log.LogManager) [2015-05-28 06:25:15,403] WARN Partition [topicX,1] on broker 4151132: No checkpointed highwatermark is found for partition [topicX,1] (kafka.cluster.Partition) [2015-05-28 06:25:15,405] INFO [ReplicaFetcherManager on broker 4151132] Removed fetcher for partitions (kafka.server.ReplicaFetcherManager) [2015-05-28 06:25:15,408] INFO [ReplicaFetcherManager on broker 4151132] Added fetcher for partitions List() (kafka.server.ReplicaFetcherManager) [2015-05-28 06:25:15,411] INFO [ReplicaFetcherManager on broker 4151132] Removed fetcher for partitions (kafka.server.ReplicaFetcherManager) [2015-05-28 06:25:15,413] INFO [ReplicaFetcherManager on broker 4151132] Added fetcher for partitions List() (kafka.server.ReplicaFetcherManager)

Is there a way to get it to complete or abort the reassignment action?

+5
source share
2 answers

You can abort the assignment by deleting the "/ admin / reassign_partitions" zk node in the zookeeper cluster and move the partitions assigned to the dead broker to the new nodes.

+2
source

With kafka 0.8.2.2 , in order to confirm the response of Foo L, rebooting another machine with the same broker identifier solved the problem.

Until we have this new broker with the same broker identifier, the transfer is suspended, and the verify command always gives the same answer

./bin/kafka-reassign-partitions.sh --zookeeper "$ZK_SERVERS" --broker-list "$BROKERS_ID" --reassignment-json-file reassignment.json --verify

Result:

Reassignment of partition [topicName,partitionId] is still in progress

+1
source

More articles:


All Articles