Scenario
Kafka message accumulation scenarios include the following:
l Consumer processing delay: When the processing capacity of the consumer is insufficient or faulty, the consumer cannot consume messages in Kafka in time, causing message accumulation. This may be due to the consumer's complex processing logic, a slow processing speed, or insufficient resources for consumers.
l Network failure: When the network between the Kafka cluster and the consumer is faulty or unstable, message transmission may be delayed or interrupted. This causes messages to accumulate in Kafka and the messages cannot be consumed until the network is recovered.
l Messages are produced faster than they are consumed: When a producer produces messages faster than consumers can process, it causes message accumulation in Kafka. This can be because the producer's speed is too fast, the consumer processing capacity is insufficient, or the consumer is malfunctioning.
l Consumer group changes: When the consumers in the consumer group change, such as the entry or exit of consumers, it will trigger Kafka rebalancing. During rebalancing, the consumers cannot consume messages, resulting in message accumulation. This typically happens with consumer expansions and fault recovery.
l Message influx during peak hours: At certain times or events, a large number of messages may flood into Kafka to the extent that exceeds the processing capacity of consumers. This causes messages accumulation in Kafka until the consumer is able to process this volume of messages.
Procedure
(1) Log in to the management console.
(2) Go to the Kafka console.
(3) In the operation column of the instance list page, click Manage in the target instance row.
(4) Click Consumer Group Management and click Consumption Accumulation and the following window will appear.
