Create Cluster
YI-MapReduce supports creating clusters with varied types, component scopes, numbers of each type of nodes, virtual machine specifications, availability zones, VPC networks, and authentication information. Based on the user's selected cluster type, version, and node specifications, YI-MapReduce aids in automatically completing the installation, deployment, and parameter optimization of enterprise-level big data platforms.
YI-MapReduce provides users with a fully controllable big data cluster. Users can set the login method for the virtual machine during its creation. The resources of the created YI-MapReduce cluster are fully allocated to the user.
YI-MapReduce Cluster Types Include Data Analysis and Elastic Search Clusters
1. Data Analysis Cluster: Apache Doris - An open-source MPP architecture OLAP analysis engine that supports sub-second data queries and multi-table joins.
2. Elastic Search Cluster: Delivers low-cost, high-performance, and reliable search and analysis service capabilities for structured/unstructured data.
YI-MapReduce Cluster Consists of Master Nodes, Core Nodes, and Task Nodes
1. Master nodes: The management nodes in a cluster, responsible for normal cluster scheduling, primarily deploying processes such as Doris FE and Elasticsearch Master. The cluster defaults to High Availability (HA) mode, with a fixed number of 3 master nodes. This type of nodes support upgrades to the configuration of these nodes, allowing them to manage larger clusters.
2. Core nodes: The computation and storage nodes in a cluster, primarily deploying processes such as Doris BE、Elasticsearch NodeData. They support configuration upgrades to meet the needs of storage data volume or computation volume expansion. The upgrade process will not affect the normal operations of the current cluster.
3. Task nodes: Purely computation nodes in a cluster. They compute data, but do not store data.
Configuration Upgrade
If the specifications (vCPU and memory) of your master or core node ECS instances fail to meet your business needs, you can enhance these ECS instance specifications through the feature of configuration upgrade.
Cluster O&M Management
MR Manager provides resource overview and supports O&M management, and configuration of cluster services, hosts, tenants and resources, monitoring and alarms.
1. Resource Overview: This feature displays the CPU, memory, network information, etc., for all hosts in this cluster, which includes metrics such as CPU usage, disk usage, memory usage, and network sending rate.
2. Cluster Service: This feature displays all the cluster services in the current cluster and lists them in a list view by component type. It also supports one-click start and one-click stop for all cluster services at the cluster service list.
3. Host: This feature, by default, displays a list of all hosts in the current cluster, allowing you to view the host information of the current O&M platform. It also lets you view role instance allocation and alarm history on each host.
4. Monitoring and Alarm: This feature supports metric queries and alarm history. The metric query feature allows for the querying of monitoring metrics at both the role instance level and host level. This supports the graphing of metric results, enabling users to intuitively understand changes in metrics. The alarm history feature allows for the querying of alarm content at the cluster service level, role instance level, and host level.
5. O&M and Configuration: This feature supports pipeline history, configuration management, configuration history, and configuration synchronization history. Pipeline history displays the operation history of all pipelines, along with the operator involved. Configuration management supports viewing the configuration files of different cluster services and performing operations such as adding, modifying, and deleting configurations. Configuration history supports viewing the configuration content of different versions of configuration files and comparing content between different versions. Configuration synchronization history supports viewing the configuration synchronization history of different environments, the operators involved in configuration synchronization, and the details of the configuration synchronization.