Increasing Partitions For a Kafka Topic
- Stop existing consumers.
- Add paritions:
kafka-topics \
--zookeeper ${zookeeper_hosts} \
--alter \
--topic ${topic} \
--paritions ${new_partition_count}
- Check the reset of the offsets for the new paritions:
kafka-consumer-groups \
--bootstrap-server ${zookeeper_hosts} \
--group ${consumer_group} \
--topic ${topic}[:${new_parition_numbers_separated_by_commas}] \
--reset-offsets \
--to-earliest
- Reset the offsets for the new paritions:
kafka-consumer-groups \
--bootstrap-server ${zookeeper_hosts} \
--group ${consumer_group} \
--topic ${topic}[:${new_parition_numbers_separated_by_commas}] \
--reset-offsets \
--to-earliest \
--execute
- Restart the consumers.