Distributed Message Service RocketMQ

Explanation of Terms

2024-07-03 02:51:22

Broker

A broker is also called a server and acts as a message intermediary. It stores and forwards messages. It is called a provider in the JMS specifications. RocketMQ deploys broker clusters across multiple servers to achieve distributed storage, HA and scale-out.

Name Server

The name server is a nearly stateless node that can be deployed in a cluster and data is not synchronized between nodes. It mainly provides broker registration, topic routing management and other functions.

Topic

RocketMQ topics are similar to queues in the JMS specifications, where all messages are stored in different topics, and producers and consumers produce and consume messages in the name of the topic. (Note: RocketMQ topics are not broadcasting consumption topics in JMS specifications).

A topic can exist on multiple brokers, and can be distributed across different brokers.

A topic may have multiple queues or partitions. The messages of a topic are stored in different queues.

Queue

In RocketMQ, a queue is located in a topic and is the smallest unit for storing data. A RocketMQ queue is different from a queue in the JMS specifications and can be deemed as a topic partition.

Producer Group

A producer group is the name of a collection of producers, that send a type of messages with a consistent sending logic, and the messages are generally generated by the business system.

Consumer Group

A consumer group is the name of a collection of consumers, which consume a type of messages with a consistent consumption logic, and the messages are consumed asynchronously by the background system. The consumption progress is stored on the consumer group.

Consumer Instance

A consumer instance represents a member of a consumer group, and different consumers are created using different instance names.

Producer Instance

A producer instance represents a producer, and different producers are created using different instance names.

PUSH Consumption

It is a type of consumers, and the applications typically register a Listener interface with the Consumer object. Once a message is received, the Consumer object immediately calls back the Listener interface. In RocketMQ, the client automatically starts 5-64 threads to consume messages. This means that the listener method will be executed by multiple threads. The client makes internal adjustments based on the amount of accumulation, and you do not need to start or manage consumption threads. A throttling mechanism is established to stop pushing new messages if a certain number of messages are accumulated and cannot be consumed in a timely manner.

PULL Consumption

It is a type of consumer, and applications typically actively call the Consumer's message pulling method to pull messages from the broker. The initiative is controlled by applications, but the timeliness depends on the frequency of active pulling by applications. In PULL consumption, threads are determined by applications.

Broadcasting Consumption

This consumption mode has impacts or limitations in many usage scenarios and is not recommended in RocketMQ. RocketMQ consumers have two ways to consume messages in a topic. One of the two ways is broadcasting consumption. In broadcasting consumption mode, when multiple consumers in a consumer group consume a message, RocketMQ ensures that the message is consumed by each consumer at least once. In version V1.x, the consumption progress of broadcasting consumption is saved on the client and affects many usage scenarios. Therefore, this consumption mode is not recommended in RocketMQ.

Clustering Consumption

In clustering consumption mode, the consumers in one or more consumer groups consume messages from a specific topic. Each consumer group has an independent consumption progress, which is saved on the server. The consumers in a consumer group can evenly consume messages to achieve load balancing. For example, a topic has nine messages and a consumer group has three different consumers (possibly three processes or three machines). Each consumer consumes only three messages. This consumption mode achieves point-to-point consumption and broadcasting consumption in JMS, and is recommended because it meets the requirements of most scenarios.

Ordered Message

Messages that are consumed in the exact order that they are created. RocketMQ has two types of ordered messages.

Normal Ordered Message

Normal ordered messages: Under normal circumstances, ordered messages can be guaranteed. However, once the broker is restarted due to a communication fault, the total number of queues changes, and the queues will change after the modulo hashing and positioning. As a result, a temporary message order inconsistency occurs. If a temporary message disorder is acceptable for the business in case of a cluster fault (such as a broker shutdown or restart), normal ordered messages are applicable.

Strictly Ordered Message

A type of ordered message. The message order is guaranteed in both normal and abnormal conditions, but a failover is not supported. Specifically, if one machine in the broker cluster fails, the entire cluster is unavailable (or the queue corresponding to the hash value is affected), and the service availability is greatly compromised. If the server is deployed in synchronous dual-write mode, the slave node automatically switches to the master node. However, the service will still be unavailable for a few minutes. If the business allows a temporary disorder, normal ordered consumption is recommended.


bC5mP0Nc3S0K