Scenario
Kafka resets the offset in the following scenarios:
l First consumption: When a new consumer joins a Kafka cluster, it needs to start consuming messages from a certain position. In this case, you can reset the consumption position to the oldest message or the most recent message.
l Consumer group is reset: When the consumers in the consumer group change, such as the entry or exit of consumers, you may need to reset the offset. In this case, you can reset the consumption position to the oldest message or the most recent message.
l Consumer fault: When a consumer fails and needs to be replaced or repaired, the offset may need to be reset. In this case, the consumption position can be reset to the oldest message or the most recent message to ensure that the new consumer can start consuming from the correct position.
l Consumer reprocessing of messages: In some cases, consumers may need to reprocess messages that have already been consumed. This may be due to a change in the consumer's processing logic, or the need to recalculate the previous result. In this case, the consumption position can be reset to the specified message for the consumer to reprocess the message.
l Slow consumer consumption: When the consumer has insufficient processing power to consume messages in time, the offset may need to be reset. In this case, the consumption position can be reset to the most recent message so that the consumer can skip the accumulated messages and start consuming from the most recent message.
Procedure
Tips: Currently, you can only reset offset for messages within 72 hours, and you can select a reset time within 72 hours.
(1) Log in to the management console.
(2) Go to the Kafka console.
(3) In the operation column of the instance list page, click Manage in the target instance row.
(4) Click Consumer Group Management to enter the consumer group management page.
(5) In the row of the target consumer group, click Reset Offset.
(6) After the Reset Offset window appears, you can choose to start consumption from the latest point, the oldest point, and a specified time point.
The three reset methods are applicable to the following scenarios:
l Starting consumption from the latest point: Reset the offset to the latest message. This means that consumers will start consuming the latest message on the current Kafka topic and ignore the messages that have already been produced. This method is suitable for scenarios where only the latest messages are needed most, such as real-time monitoring or logging.
l Starting consumption from the oldest point: Resets the offset to the oldest message. This means that consumers will start consuming from the earliest messages on the current Kafka topic, including messages that have already been produced. This method is suitable for scenarios where the entire message history needs to be processed, such as data replay or analysis.
l Resetting offset to time point: Reset the offset to a specified time point. This means that consumers will start consuming messages after a specified time point, and can precisely locate consumption start position. This method is suitable for scenarios that require consumption from a specific time point, such as data backtracking or reprocessing.