Scenario
The operation of creating a topic in Kafka is applicable to the following scenarios:
l System expansion: When the system needs expansion to process more data, you can increase the capacity of data storage and processing by creating a topic. The administrator can create new topics based on system requirements and configure the appropriate number of partitions and replicas.
l Data distribution: In some cases, data of a particular type or source needs to be distributed to different topics for processing. By creating a topic, you can create specialized topics for these specific data streams and write the corresponding data to those topics.
l Data analysis and real-time processing: You can create a topic that can be used for data analysis and real-time processing. Depending on your analysis needs, you can create new topics for specific analysis tasks and write relevant data to these topics for analysis and processing.
l Business requirements change: When business requirements change, new topics may need to be created to accommodate new data processing processes or data storage requirements. By creating a topic, you can reorganize and manage your data based on the new business requirements.
l Data isolation and permission control: Sometimes it is necessary to isolate and control permissions on different data to ensure data security and privacy. By creating a new topic, you can create independent topics for different data and configure permissions and access control rules as needed.
In short, Kafka topic operation is suitable for various scenarios that require data expansion, distribution, analysis, and processing, as well as the need to handle changes in business requirements and data isolation.
Procedure
(1) Log in to the management console.
(2) Go to the Kafka console.
(3) In the operation column of the instance list page, click Manage in the target instance row.
(4) Click Topic Management and then click Create Topic.
(5) Then, the following window will appear. Enter the topic name and number of partitions, and select whether it is a private topic.
Parameter | Description |
Topic name | The topic name can only be a 4 to 64-character string of a-z, A-Z, 0-9, and -. Topic names cannot be changed after creation. |
Number of Partitions | You can configure the number of partitions in a topic, and the larger the number of partitions, the greater the consumption concurrency. If this parameter is set to 1, messages are consumed in a first-in, first-out order. Value range: 1-20. Default value: 3. |
Number of Replicas | Every topic is configured with a number of replicas. Kafka automatically backs up data on each replica, and the data will still be available when one of the broker nodes fails. The larger the number of replicas, the higher the reliability. Default value: 3. |
Whether it is a private topic | Messages on common topics can be directly produced and consumed, while messages on private topics need to be produced and consumed with certificates. Note: To use any topic created in a Kafka instance, the user must have the topic permissions, no matter whether the topic is private or not.
|
Creating Topics in Batches
(1) Log in to the management console.
(2) Go to the Kafka console.
(3) In the operation column of the instance list page, click Manage in the target instance row.
(4) Click Topic Management and then click Batch Create.
(5) Then, the following file-uploading interface will appear.
(6) Click Upload to complete the batch creation.
Download Bulk Creation Template
(1) Log in to the management console.
(2) Go to the Kafka console.
(3) In the operation column of the instance list page, click Manage in the target instance row.
(4) Click Topic Management and click Download Batch Creation Template.
(5) Download the batch creation template, which is shown below.
(6) Parameter description.
Parameter | Description |
topic | Topic name. |
partition number | Number of partitions. |
factor number | Number of replicas. |
isPrivate(private:1 public:2) | Whether the topic is private, 1: private; 2: public |
description | Description |