Technical Architecture
The following figure shows the technical architecture of eSurfing Cloud DTS.
How DTS Works
Data Migration
DTS supports various data migration types, including struct migration, full data migration, and incremental data migration. If you want to migrate data without service downtime, you must select all these migration types when you create a migration task.
Full data migration takes a long time. During this process, business data is continuously written into the source instance. To ensure the consistency of data after migration, the log offset of the source instance is recorded and stored in the local database before full data migration starts.
After full data migration is complete, DTS starts the incremental log playback module. The incremental log playback module obtains the incremental data from the incremental log read module and migrates the incremental data to the destination instance after reverse parsing, filtering, and encapsulation.
Data Synchronization
DTS synchronizes incremental data between two data sources in real time.
Process for Creating a Synchronization Link
Initial synchronization
Initialize the existing data of the source instance on the destination instance.
Real-time synchronization of incremental data
After the initial synchronization is complete, DTS dynamically synchronizes the incremental data of the source instance to the destination instance.
Underlying Implementation of Real-time Incremental Data Synchronization
DTS uses the following modules to implement real-time synchronization of incremental data in the underlying architecture:
Log read module
The log read module reads raw data from the source instance and persists the data locally after parsing, filtering, and standard formatting. The log read module connects to the source instance and reads the incremental logs through the database protocol. If the source database is a RDS MySQL database, the log read module connects to the source database through the binlog dump protocol.
Log playback module
The log playback module requests the incremental data from the log read module, filters the data according to the synchronization objects that you configured, and synchronizes the log records to the destination instance while ensuring the time sequence and consistency of transactions.
