For the component details of various YI-MapReduce cluster versions, please refer to the Version Overview.
See the figure below: YI-MapReduce Architecture
The architecture of YI-MapReduce includes capabilities at all stages of infrastructure and big data processing workflows.
● Infrastructure
The big data cluster, built on the eSurfing Cloud Elastic Cloud Server (ECS), has high reliability and security, fully supported by the underlying virtualization.
• The Virtual Private Cloud (VPC) provides each tenant with a virtual internal network, isolated from other networks by default. This, along with the accompanying security group access control, ensures network-level security.
• The Elastic Volume Service (EVS) offers reliable storage capabilities with different specifications and performance.
• The Elastic Cloud Server (ECS) provides an elastically scalable virtual server. Combined with the VPC, security group, multiple replicas of EVS data, and disaster recovery capabilities, it creates an efficient, reliable, and secure business cluster environment for customers.
● Data Storage
YI-MapReduce supports the storage of structured and unstructured data in the cluster, and supports multiple efficient formats to meet the requirements of different computing engines.
– Doris provides real-time data warehousing service, exhibiting high concurrency and low latency.
– Elasticsearch facilitates the retrieval and analysis of both structured and unstructured data.
● YI-MapReduce Manager
To maintain the high availability of big data component services, it is necessary to deploy the various components of the big data ecosystem based on Hadoop in a distributed manner. This deployment comes with high demands for management and O&M complexity. YI-MapReduce offers a unified platform for O&M management, YI-MapReduce Manager, which includes capabilities for visually guided cluster deployment. In addition, YI-MapReduce Manager provides management capabilities for tenants and resources, O&M for various big data components in YI-MapReduce, and one-stop O&M capabilities such as monitoring, alarms, and configuration.