CN111221800B

CN111221800B - Database migration method and device, electronic equipment and storage medium

Info

Publication number: CN111221800B
Application number: CN201911319464.8A
Authority: CN
Inventors: 曾培贵; 朱定宝
Original assignee: Ping An Life Insurance Company of China Ltd
Current assignee: Ping An Life Insurance Company of China Ltd
Priority date: 2019-12-19
Filing date: 2019-12-19
Publication date: 2024-09-13
Anticipated expiration: 2039-12-19
Also published as: CN111221800A

Abstract

The invention provides a database migration method, a database migration device, electronic equipment and a storage medium. According to the method, the whole amount of data in the old library can be synchronized to the new library, the generated new data is written into the old library, when the whole amount of synchronization is determined to be completed, the new data is incrementally synchronized from the old library to the new library, the total amount of migration data when the new library is online is reduced, the waiting time of the new library is shortened, after the new library is online, real-time data are respectively written into the old library and the new library, and further when the new library is not completely stabilized, the old library is still combined for use, so that data read-write is ensured to be normal, data in the new library is further verified, when verification is passed, the new library is controlled to execute a read-write instruction, and the old library is controlled to be offline, so that migration of the database is completed, the speed of data processing is improved, normal execution of read-write operation is ensured, and the stability of the new library is further ensured through verification.

Description

Database migration method and device, electronic equipment and storage medium

Technical Field

The present invention relates to the field of data processing technologies, and in particular, to a database migration method, a database migration device, an electronic device, and a storage medium.

Background

Database migration is a problem frequently encountered in project development, such as migration of some old systems, performance of the database encountering bottlenecks, the database being used being too expensive, and the like, and the database migration is required. Currently, the removal of O (from Oracle) has become a major trend, and more enterprises migrate data from the Oracle database to the MySQL database.

In the prior art, in order to realize database migration, a new database is firstly built, then codes are changed, when the new database is online, all application programs are required to be stopped, data are synchronized again, after the data synchronization is completed, a new version is released, and the data is directly migrated to the new database.

The above method has several drawbacks. Firstly, all data need to be synchronized, so that the time is long, and a few hours are often spent, and the usability of the system is greatly reduced; secondly, the data is directly migrated to a new database, some format errors or subtle changes of the data are difficult to discover in time due to the difference between the databases, if verification is problematic after online, a new version is released to solve, and risks are high for systems with high requirements on the importance of the data and the availability of the system.

Disclosure of Invention

In view of the foregoing, it is necessary to provide a database migration method, apparatus, electronic device, and storage medium, which not only can increase the speed of data migration, but also ensure normal execution of read-write operations, and further ensure the stability of a new database through verification.

A database migration method, the method comprising:

Receiving a database migration instruction, wherein the database migration instruction is used for triggering migration of data from an old library to a new library;

synchronizing the data volume in the old library to the new library;

detecting whether new data are generated in real time;

When new data is generated, writing the generated new data into the old library;

incrementally synchronizing the new data from the old library to the new library upon determining that full synchronization is complete;

Acquiring real-time data after the new library is online;

writing the real-time data into the old library and the new library respectively;

Checking the data in the new library;

And when the verification is passed, controlling the new library to execute a read-write instruction and controlling the old library to be offline.

According to a preferred embodiment of the present invention, the incremental synchronization of the new data from the old library to the new library comprises:

Obtaining an operation log of the old library;

analyzing the operation log to obtain changed data;

determining the changed data as the new data;

synchronizing the new data to the new library.

In accordance with a preferred embodiment of the present invention, upon incremental synchronization of the new data from the old library to the new library, the method further comprises:

when incremental synchronization is performed, real-time monitoring is performed based on an exception handling mechanism;

Capturing abnormal data when abnormal data is detected;

the captured data is skipped.

According to a preferred embodiment of the invention, the method further comprises:

recording the captured data as abnormal data;

Determining a type of the anomalous data after incrementally synchronizing the new data from the old library to the new library;

when the abnormal data is in writing failure, calling data corresponding to the abnormal data from the old library based on the data identification of the abnormal data, and complementarily recording the called data to the new library; or alternatively

When the abnormal data is a write-in error, based on the data identification of the abnormal data, the data corresponding to the abnormal data is called from the old library, and the abnormal data is updated according to the called data.

According to a preferred embodiment of the present invention, when writing the real-time data into the old library and the new library, respectively, the method further comprises:

and stopping incremental synchronization of data from the old library to the new library.

before a configuration time point in preset time, carrying out full comparison on the data in the new library and the data in the old library to obtain a full comparison result;

After the configuration time point in the preset time, performing incremental comparison on the data in the new library and the data in the old library to obtain an incremental comparison result;

Monitoring the log of the new library in a preset time to obtain a monitoring result;

And when the total comparison result, the increment comparison result and the monitoring result are not abnormal, determining that the new library passes the verification.

According to a preferred embodiment of the present invention, the controlling the new library to execute the read-write instruction, and controlling the old library to be offline includes:

Acquiring a configuration list, wherein the configuration list stores the corresponding relation between the read-write instruction type and the database;

determining a target database from the new database and the old database according to the configuration list;

executing a read-write instruction in the target database;

After the configuration time, when the reading and writing of the new library are determined to be normal, controlling the new library to execute a reading and writing instruction;

and cleaning the old library data and controlling the old library to be offline.

A database migration apparatus, the apparatus comprising:

The receiving unit is used for receiving a database migration instruction, wherein the database migration instruction is used for triggering migration of data from an old database to a new database;

A synchronization unit, configured to synchronize the full amount of data in the old library to the new library;

A detection unit for detecting whether new data is generated in real time;

A writing unit for writing new data generated when new data is generated into the old library;

the synchronization unit is further configured to incrementally synchronize the new data from the old library to the new library when it is determined that full synchronization is complete;

The acquisition unit is used for acquiring real-time data after the new library is online;

The writing unit is further used for writing the real-time data into the old library and the new library respectively;

the verification unit is used for verifying the data in the new library;

And the control unit is used for controlling the new library to execute the read-write instruction and controlling the old library to be offline when the verification is passed.

According to a preferred embodiment of the present invention, the synchronizing unit incrementally synchronizing the new data from the old library to the new library comprises:

Obtaining an operation log of the old library;

analyzing the operation log to obtain changed data;

determining the changed data as the new data;

synchronizing the new data to the new library.

According to a preferred embodiment of the invention, the device further comprises:

The monitoring unit is used for carrying out real-time monitoring based on an exception handling mechanism when carrying out increment synchronization on the new data from the old library to the new library;

The capturing unit is used for capturing abnormal data when the abnormal data is detected;

And a skip unit for skipping the captured data.

A recording unit configured to record the captured data as abnormal data;

a determining unit configured to determine a type of the abnormal data after incrementally synchronizing the new data from the old library to the new library;

The supplementary record unit is used for calling the data corresponding to the abnormal data from the old library based on the data identification of the abnormal data when the abnormal data is in writing failure, and supplementary recording the called data to the new library; or alternatively

And the updating unit is used for calling the data corresponding to the abnormal data from the old library based on the data identification of the abnormal data when the abnormal data is the write-in error, and updating the abnormal data according to the called data.

and the stopping unit is used for stopping incremental synchronization data from the old library to the new library when the real-time data are written into the old library and the new library respectively.

the comparison unit is used for performing full comparison on the data in the new library and the data in the old library before the configuration time point in the preset time to obtain a full comparison result;

the comparison unit is further configured to perform incremental comparison on the data in the new library and the data in the old library after the configuration time point in the preset time, so as to obtain an incremental comparison result;

the monitoring unit is used for monitoring the log of the new library in a preset time to obtain a monitoring result;

and the determining unit is further used for determining that the new library passes the verification when the full comparison result, the increment comparison result and the monitoring result are not abnormal.

According to a preferred embodiment of the invention, the control unit is specifically adapted to:

executing a read-write instruction in the target database;

An electronic device, the electronic device comprising:

A memory storing at least one instruction; and

And the processor executes the instructions stored in the memory to realize the database migration method.

A computer-readable storage medium having stored therein at least one instruction that is executed by a processor in an electronic device to implement the database migration method.

According to the technical scheme, when a database migration instruction is received, the old database and the new database can be obtained, whether new data are generated or not is detected in real time by synchronizing the whole data in the old database to the new database, when new data are generated, the generated new data are written into the old database, when the whole data synchronization is confirmed to be completed, the new data are incrementally synchronized from the old database to the new database, the total amount of migration data when the new database is online is reduced, the waiting time of online of the new database is further shortened, after the new database is online, real-time data are obtained, the real-time data are respectively written into the old database and the new database, and when the new database is not completely stable, the old database is still combined for ensuring that data read-write is normal, when the new database is verified, the new database is controlled to execute a read-write instruction, and the old database is controlled to be offline, and further the migration of the database is completed, the data is not only improved, and the normal read-write operation is ensured, and the normal read-write operation is further ensured.

Drawings

FIG. 1 is a flow chart of a preferred embodiment of the database migration method of the present invention.

FIG. 2 is a functional block diagram of a preferred embodiment of the database migration apparatus of the present invention.

FIG. 3 is a schematic diagram of an electronic device implementing a database migration method according to a preferred embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in detail with reference to the accompanying drawings and specific embodiments.

FIG. 1 is a flow chart of a preferred embodiment of the database migration method of the present invention. The order of the steps in the flowchart may be changed and some steps may be omitted according to various needs.

The database migration method is applied to one or more electronic devices, wherein the electronic devices are devices capable of automatically performing numerical calculation and/or information processing according to preset or stored instructions, and the hardware of the electronic devices comprises, but is not limited to, microprocessors, application SPECIFIC INTEGRATED Circuits (ASICs), programmable gate arrays (Field-Programmable GATE ARRAY, FPGA), digital processors (DIGITAL SIGNAL processors, DSPs), embedded devices and the like.

The electronic device may be any electronic product that can interact with a user in a human-computer manner, such as a Personal computer, a tablet computer, a smart phone, a Personal digital assistant (Personal DIGITAL ASSISTANT, PDA), a game console, an interactive internet protocol television (Internet Protocol Television, IPTV), a smart wearable device, etc.

The electronic device may also include a network device and/or a user device. Wherein the network device includes, but is not limited to, a single network server, a server group composed of a plurality of network servers, or a Cloud based Cloud Computing (Cloud Computing) composed of a large number of hosts or network servers.

The network in which the electronic device is located includes, but is not limited to, the internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (Virtual Private Network, VPN), and the like.

S10, receiving a database migration instruction.

The database migration instruction is used for triggering the migration of data from an old database to a new database.

In at least one embodiment of the present invention, the database migration instructions may be triggered by an associated worker, such as: developer, etc.

In at least one embodiment of the present invention, the old library may comprise an Oracle database or the like, and the new library may comprise a MySQL database or the like, as the present invention is not limited.

In at least one embodiment of the present invention, the electronic device may obtain the new library and the old library from the database migration instruction.

For example: the database migration instructions may be used to migrate data from the Oracle database to the MySQL database.

S11, synchronizing the data total in the old library to the new library.

In at least one embodiment of the present invention, in order to save time spent on data synchronization after the new library is online, the electronic device synchronizes the full amount of data in the old library to the new library before the new library is online.

In particular, the electronic device may synchronize the full amount of data in the old library to the new library using a designated tool, and the invention is not limited.

Wherein the designation tool may include, but is not limited to: kettale (KETTLE), HKROnline SyncNavigator, etc.

S12, detecting whether new data are generated in real time.

In at least one embodiment of the present invention, since each system or terminal connected to the old library is always in operation, new data is continuously generated, and the newly generated data needs to be written into the old library and the new library, so as to facilitate calling.

Therefore, in order to avoid that newly generated data is not written, the electronic device detects in real time whether new data is generated.

S13, when new data are generated, the generated new data are written into the old library.

In at least one embodiment of the invention, the electronic device first writes the new data generated to the old library, since the new library is not already formally in use online.

By the implementation mode, the old library can still be used for storing data before the new library is formally online.

And S14, when the full synchronization is determined to be completed, incrementally synchronizing the new data from the old library to the new library.

In at least one embodiment of the present invention, incremental synchronization is performed after full synchronization is completed, thereby avoiding that there is data to be repeatedly synchronized to the new library, resulting in redundancy of the data in the new library.

In the prior art, a full-scale synchronization mode is generally adopted to perform database migration, and the migration mode not only needs to spend a great deal of time for data writing before the new database is formally online, reduces the usability of the system, but also needs to stop the application program in use for data initialization, and influences the normal operation of the application program.

In comparison, the method and the device for the online synchronization of the data of the new library further perform incremental synchronization on the data before the new library is online, so that when the new library is online, the new data does not need to be migrated, the total amount of migration data is reduced, the waiting time of the online of the new library is further shortened, the usability of the system is improved, and meanwhile, an application program corresponding to the new data does not need to be stopped, and the user experience is improved.

Preferably, the electronic device incrementally synchronizing the new data from the old library to the new library comprises:

The electronic equipment acquires the operation log of the old library, analyzes the operation log to obtain changed data, determines the changed data as new data, and synchronizes the new data to the new library.

In particular, the electronic device may employ a configuration tool to synchronize the new data to the new library, and the invention is not limited.

For example: the configuration tool may include, but is not limited to: oracle Golden Gate, etc.

Specifically, the electronic device employs Oracle Golden Gate to establish a synchronization link (gg link) between the old library to the new library.

Further, the electronic device analyzes the operation log, determines changed data from the operation log, determines the changed data as the new data, and then opens Oracle Golden Gate the electronic device, and performs incremental synchronization of the new data based on the gg link.

By the above embodiment, since incremental synchronization is performed based on the operation log, no data is missed, and since Oracle Golden Gate is a log-based structured data copy, and the operation log of the old library is read as a synchronization basis, consumption of a CPU (Central Processing Unit ) of the system is low, and the system performance is less affected.

In at least one embodiment of the invention, the electronic device may add transactions to the write operations of the old and new libraries, respectively.

Wherein the transaction is the smallest unit of work for a database operation, is a series of operations performed as a single logical unit of work that are committed together as a whole to the system, either all performed or none performed, i.e.: the transaction is a set of operation sets (working logic units) which can not be subdivided, the electronic equipment respectively and independently adds the transaction to the branch code for operating the new library and the branch code for operating the old library, so that the new library and the old library can not be mutually influenced, when the data writing of the new library is abnormal, the data writing of the old library can not be influenced, and the operations of the new library and the old library are mutually independent.

In at least one embodiment of the invention, in incrementally synchronizing the new data from the old library to the new library, the method further comprises:

and when the electronic equipment detects that the data is abnormal, the electronic equipment captures the abnormal data and further skips the captured data.

Specifically, during incremental synchronization, the electronic device may execute the exception handling mechanism using a try-catch statement.

Further, the electronic device adopts a try statement to monitor in real time, and adopts a catch statement to capture the abnormal data.

In the prior art, when the abnormal data is detected, incremental synchronization is terminated, so that the efficiency of the incremental synchronization is influenced, and the migration efficiency of the database is further influenced.

In this embodiment, the electronic device skips the captured data and continues to perform incremental synchronization, so as to improve efficiency of data synchronization.

However, since the abnormality is captured, the electronic device also processes the abnormality, and thus, the quality of the data is not affected.

Specifically, the method further comprises:

The electronic device records the captured data as anomalous data, which the electronic device determines the type of anomalous data, in particular after incremental synchronization of the new data from the old library to the new library:

(1) And when the abnormal data is writing failure, the electronic equipment retrieves the data corresponding to the abnormal data from the old library based on the data identification of the abnormal data, and supplements the retrieved data to the new library.

(2) When the abnormal data is a write-in error, the electronic equipment invokes data corresponding to the abnormal data from the old library based on the data identification of the abnormal data, and updates the abnormal data according to the invoked data.

Types of the abnormal data include, but are not limited to: write failure, write error, etc.

Further, by the data identification, a piece of data can be uniquely determined.

It can be understood that if the abnormal data is a write failure, it is indicated that corresponding data may not be written in the new library, so that the electronic device retrieves data corresponding to the abnormal data from the old library based on the data identifier, and supplements the retrieved data to the new library, thereby avoiding data omission and ensuring the comprehensiveness of the data in the new library.

If the abnormal data is a writing error, the electronic equipment indicates that the error data possibly occurs in the new library, so that the electronic equipment invokes the data corresponding to the abnormal data from the old library based on the data identification of the abnormal data, and updates the abnormal data according to the invoked data, thereby avoiding data errors and ensuring the accuracy of the data in the new library.

Of course, in other embodiments, the electronic device may further obtain an execution log of incremental synchronization, obtain information such as writing time of the abnormal data from the execution log, and call data corresponding to the abnormal data from the old library according to the obtained information.

S15, after the new library is online, acquiring real-time data.

In at least one embodiment of the present invention, since new data is generated (e.g., application data, operation data, etc.) after the new library is online, the electronic device needs to write the data into the database in real time in order to ensure the integrity of the data.

Therefore, after the new library is online, the electronic device acquires real-time data so as to store the data acquired in real time in time.

S16, writing the real-time data into the old library and the new library respectively.

In at least one embodiment of the present invention, since the new library is already online, the electronic device does not need to use other tools to write the real-time data to the new library, but only needs to use the built-in code of the new library to write directly.

In at least one embodiment of the present invention, when writing the real-time data to the old library and the new library, respectively, the method further comprises:

The electronic device stops incrementally synchronizing data from the old library to the new library.

It can be understood that after the new database is online, the electronic device performs synchronous writing of data to both the new database and the old database through the code layer, and if incremental synchronization of data from the old database to the new database is not stopped, repeated data is synchronized from the old database to the new database, so that the new database has redundant data.

Further, in this embodiment, the electronic device may also stop the related application program, so as to avoid data loss caused by the running of the program.

By the implementation mode, data redundancy can be effectively avoided, and the quality of data in the new library is improved.

S17, checking the data in the new library.

In at least one embodiment of the present invention, in order to ensure accuracy and comprehensiveness of the data in the new library, the electronic device performs verification on the data in the new library, and a specific verification manner will be described later.

And S18, when the verification is passed, controlling the new library to execute a read-write instruction and controlling the old library to be offline.

In at least one embodiment of the present invention, the electronic device may further wait a period of time when the verification is passed, control the new library to execute the read/write command, and control the old library to be offline, so as to further ensure that the new library is completely stable.

In at least one embodiment of the invention, the method further comprises:

and before a configuration time point in preset time, the electronic equipment performs full comparison on the data in the new library and the data in the old library to obtain a full comparison result, and further, after the configuration time point in the preset time, the electronic equipment performs increment comparison on the data in the new library and the data in the old library to obtain an increment comparison result, and in the preset time, the electronic equipment monitors the log of the new library to obtain a monitoring result, and when the full comparison result, the increment comparison result and the monitoring result are not abnormal, the electronic equipment determines that the new library passes verification.

The preset time and the configuration time point can be determined by analyzing big data according to historical data, and the invention is not limited.

Specifically, the electronic device acquires all the verification time periods from the historical data, determines the latest verification time and the earliest verification time from the all the verification time periods, and determines the preset time by taking the earliest verification time as the starting time and the latest verification time as the ending time.

Further, the electronic device may process the data in the preset time by using a least square method to obtain an optimal solution, and determine the optimal solution as the configuration time point.

Through the implementation mode, firstly, the full quantity comparison technology and the increment comparison technology are combined, the detection of the data in the new library is realized, and then, the log monitoring technology is combined, so that the verification process of the data is further perfected, and the accurate verification of the data in the new library is realized.

In at least one embodiment of the present invention, the controlling the new library to execute the read-write instruction, and controlling the old library to be offline includes:

The electronic equipment acquires a configuration list, wherein the configuration list stores the corresponding relation between the read-write instruction types and the databases, determines a target database from the new database and the old database according to the configuration list, executes the read-write instruction in the target database, and after the configuration time, when the new database is determined to be read and written normally, the electronic equipment controls the new database to execute the read-write instruction, cleans the old database data and controls the old database to be offline.

For example: the electronic equipment modifies parameters of the configuration file to enable one part of read-write instructions to be executed in the new library and the other part of read-write instructions to be executed in the old library, then the execution range of the read-write instructions by the new library is gradually enlarged until the whole system reads and writes from the new library, and after the read-write functions are normal, the configuration parameters are modified, and data is written into the new library only and is not written into the old library.

Wherein, the configuration list can be configured according to big data analysis of historical data, and the invention is not limited.

Specifically, the electronic device obtains historical configuration data, calculates the probability of each read-write instruction type operating in a certain database according to the historical configuration data, determines a database corresponding to the maximum probability of each read-write instruction type operating from the calculated probabilities, records the corresponding relation between the corresponding database and each read-write instruction type, and configures the configuration list according to the corresponding relation.

For example: when the probability of the read-write operation of the task A in the old library is maximum, the electronic equipment records that the read-write operation of the task A is executed in the old library.

The configuration time can be configured in a self-defined manner.

Unlike the mode of directly transferring data to the new library in the prior art, the scheme can gradually open the new library, and avoid abnormal data generation after the new library is directly opened.

FIG. 2 is a functional block diagram of a preferred embodiment of the database migration apparatus of the present invention. The database migration apparatus 11 includes a receiving unit 110, a synchronizing unit 111, a detecting unit 112, a writing unit 113, an acquiring unit 114, a verifying unit 115, a control unit 116, a monitoring unit 117, a capturing unit 118, a skipping unit 119, a recording unit 120, a determining unit 121, a complement unit 122, an updating unit 123, a stopping unit 124, a comparing unit 125, and a monitoring unit 126. The module/unit referred to in the present invention refers to a series of computer program segments capable of being executed by the processor 13 and of performing a fixed function, which are stored in the memory 12. In the present embodiment, the functions of the respective modules/units will be described in detail in the following embodiments.

The receiving unit 110 receives a database migration instruction.

In at least one embodiment of the present invention, the new library and the old library may be obtained from the database migration instruction.

The synchronization unit 111 synchronizes the full amount of data in the old library to the new library.

In at least one embodiment of the present invention, in order to save time spent on data synchronization after the new library is online, the synchronization unit 111 synchronizes the full amount of data in the old library to the new library before the new library is online.

Specifically, the synchronization unit 111 may use a designated tool to synchronize the full amount of data in the old library to the new library, which is not limited by the present invention.

The detection unit 112 detects whether new data is generated in real time.

Therefore, in order to avoid that newly generated data is not written, the detecting unit 112 detects whether new data is generated in real time.

When new data is generated, the writing unit 113 writes the generated new data to the old library.

In at least one embodiment of the present invention, the writing unit 113 first writes the generated new data to the old library because the new library is not formally in use on-line.

When it is determined that full synchronization is complete, the synchronization unit 111 incrementally synchronizes the new data from the old library to the new library.

Preferably, the synchronizing unit 111 incrementally synchronizing the new data from the old library to the new library includes:

The synchronization unit 111 obtains the operation log of the old library, analyzes the operation log, and obtains changed data, and the synchronization unit 111 determines the changed data as the new data and synchronizes the new data to the new library.

In particular, the synchronization unit 111 may employ a configuration tool to synchronize the new data to the new library, and the present invention is not limited.

Specifically, the synchronization unit 111 employs the Oracle Golden Gate to build a synchronization link (gg link) between the old library to the new library.

Further, the synchronization unit 111 parses the operation log and determines changed data from the operation log, the synchronization unit 111 determines the changed data as the new data, and then the synchronization unit 111 starts up Oracle Golden Gate and performs incremental synchronization of the new data based on the gg link.

In at least one embodiment of the present invention, the write unit 113 may add transactions to the write operations of the old bank and the new bank, respectively.

Wherein the transaction is the smallest unit of work for a database operation, is a series of operations performed as a single logical unit of work that are committed together as a whole to the system, either all performed or none performed, i.e.: the transaction is a set of operation sets (working logic units) which can not be subdivided, the writing unit 113 adds the transaction to the branch code operating the new library and the branch code operating the old library separately, so that the new library and the old library will not affect each other, and thus, when the data writing of the new library is abnormal, the data writing of the old library will not be affected, and the operations of the new library and the old library are independent.

In at least one embodiment of the present invention, the monitoring unit 117 performs real-time monitoring based on an exception handling mechanism while the synchronization unit 111 incrementally synchronizes the new data from the old library to the new library, and the capturing unit 118 captures the data of the exception when the data exception is detected, and the skip unit 119 further skips the captured data.

Specifically, when performing delta synchronization, the exception handling mechanism may be performed using a try-catch statement.

Further, the monitoring unit 117 performs real-time monitoring using try statements, and the capturing unit 118 captures the abnormal data using catch statements.

In this embodiment, the skip unit 119 skips the captured data and continues to perform incremental synchronization to improve the efficiency of data synchronization.

However, since the anomaly is captured, the anomaly is handled without affecting the quality of the data.

Specifically, the recording unit 120 records the captured data as abnormal data, and after incrementally synchronizing the new data from the old library to the new library, the determining unit 121 determines the type of the abnormal data, specifically:

(1) When the abnormal data is a write failure, the supplementary recording unit 122 retrieves data corresponding to the abnormal data from the old library based on the data identifier of the abnormal data, and supplementary records the retrieved data to the new library.

(2) When the abnormal data is a write error, the updating unit 123 retrieves data corresponding to the abnormal data from the old library based on the data identification of the abnormal data, and updates the abnormal data according to the retrieved data.

It can be understood that if the abnormal data is a write failure, it is indicated that the new library may not have corresponding data written therein, so that the supplementary recording unit 122 retrieves the data corresponding to the abnormal data from the old library based on the data identifier, and supplementary records the retrieved data to the new library, thereby avoiding data omission and ensuring the comprehensiveness of the data in the new library.

If the abnormal data is a write error, it is indicated that the error data may occur in the new library, so the updating unit 123 retrieves data corresponding to the abnormal data from the old library based on the data identifier of the abnormal data, and updates the abnormal data according to the retrieved data, thereby avoiding data errors and ensuring the accuracy of the data in the new library.

Of course, in other embodiments, it is also possible to obtain an execution log of incremental synchronization, obtain information such as writing time of the abnormal data from the execution log, and call data corresponding to the abnormal data from the old library according to the obtained information.

After the new library is brought online, the acquisition unit 114 acquires real-time data.

In at least one embodiment of the present invention, since new data (such as application data, operation data, etc.) is generated after the new library is online, the writing unit 113 needs to write the data into the database in real time in order to ensure the integrity of the data.

Therefore, after the new library is online, the acquiring unit 114 acquires real-time data so as to save the real-time acquired data in time.

The writing unit 113 writes the real-time data to the old library and the new library, respectively.

In at least one embodiment of the present invention, since the new library is already online, the writing unit 113 does not need to use other tools to write the real-time data to the new library, but only needs to use the built-in code of the new library to write directly.

In at least one embodiment of the present invention, the stopping unit 124 stops incrementally synchronizing data from the old library to the new library while writing the real-time data to the old library and the new library, respectively.

It will be appreciated that, after the new database is online, the writing unit 113 performs synchronous writing of data to both the new database and the old database through the code layer, and if incremental synchronization of data from the old database to the new database is not stopped, repeated data is synchronized from the old database to the new database, so that the new database has redundant data.

Further, in this embodiment, the stopping unit 124 may also stop the related application program to avoid data loss caused by the running of the program.

The verification unit 115 verifies the data in the new library.

In at least one embodiment of the present invention, in order to ensure accuracy and comprehensiveness of the data in the new library, the verification unit 115 performs verification on the data in the new library, and a specific verification manner will be described later.

When passing the verification, the control unit 116 controls the new library to execute the read-write instruction and controls the old library to be offline.

In at least one embodiment of the present invention, the control unit 116 may also wait a period of time when the verification is passed, control the new library to execute the read/write command, and control the old library to be offline to further ensure that the new library is completely stable.

In at least one embodiment of the present invention, before a configuration time point within a preset time, the comparing unit 125 performs a full-scale comparison on the data in the new library and the data in the old library to obtain a full-scale comparison result, further, after the configuration time point within the preset time, the comparing unit 125 performs an incremental comparison on the data in the new library and the data in the old library to obtain an incremental comparison result, and in the preset time, the monitoring unit 126 monitors the log of the new library to obtain a monitoring result, and when none of the full-scale comparison result, the incremental comparison result and the monitoring result is abnormal, the determining unit 121 determines that the new library passes the verification.

Specifically, the determining unit 121 obtains all the verification time periods from the historical data, determines the latest verification time and the earliest verification time from the all the verification time periods, and determines the preset time by using the earliest verification time as the start time and the latest verification time as the end time.

Further, the determining unit 121 may process the data in the preset time by using a least square method to obtain an optimal solution, and determine the optimal solution as the configuration time point.

In at least one embodiment of the present invention, the controlling unit 116 controls the new library to execute the read/write instruction, and controls the old library to be offline includes:

The control unit 116 obtains a configuration list, in which a correspondence between a read-write instruction type and a database is stored, the control unit 116 determines a target database from the new database and the old database according to the configuration list, and executes the read-write instruction in the target database, after a configuration time, when it is determined that the read-write of the new database is normal, the control unit 116 controls the new database to execute the read-write instruction, clears the old database data, and controls the old database to be offline.

For example: the control unit 116 modifies parameters of the configuration file, so that a part of types of read-write instructions are executed in the new library, another part of types of read-write instructions are executed in the old library, and then the execution range of the new library for the read-write instructions is gradually enlarged until the whole system reads and writes from the new library, and after the read-write functions are normal, the configuration parameters are modified, and data is written into the new library only and is not written into the old library.

Specifically, the control unit 116 obtains the historical configuration data, calculates the probability that each read-write instruction type operates in a certain database according to the historical configuration data, determines the database corresponding to the maximum probability of each read-write instruction type operation from the calculated probabilities, records the corresponding relation between the corresponding database and each read-write instruction type, and the control unit 116 configures the configuration list according to the corresponding relation.

For example: when the probability of the read-write operation of task a in the old library is maximum, the control unit 116 records that the read-write operation of task a is performed in the old library.

The configuration time can be configured in a self-defined manner.

Fig. 3 is a schematic structural diagram of an electronic device according to a preferred embodiment of the present invention for implementing a database migration method.

The electronic device 1 may comprise a memory 12, a processor 13 and a bus, and may further comprise a computer program, such as a database migration program, stored in the memory 12 and executable on the processor 13.

It will be appreciated by those skilled in the art that the schematic diagram is merely an example of the electronic device 1 and does not constitute a limitation of the electronic device 1, the electronic device 1 may be a bus type structure, a star type structure, the electronic device 1 may further comprise more or less other hardware or software than illustrated, or a different arrangement of components, for example, the electronic device 1 may further comprise an input-output device, a network access device, etc.

It should be noted that the electronic device 1 is only used as an example, and other electronic products that may be present in the present invention or may be present in the future are also included in the scope of the present invention by way of reference.

The memory 12 includes at least one type of readable storage medium including flash memory, a removable hard disk, a multimedia card, a card memory (e.g., SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, etc. The memory 12 may in some embodiments be an internal storage unit of the electronic device 1, such as a mobile hard disk of the electronic device 1. The memory 12 may also be an external storage device of the electronic device 1 in other embodiments, such as a plug-in mobile hard disk, a smart memory card (SMART MEDIA CARD, SMC), a Secure Digital (SD) card, a flash memory card (FLASH CARD) or the like, which are provided on the electronic device 1. Further, the memory 12 may also include both an internal storage unit and an external storage device of the electronic device 1. The memory 12 may be used not only for storing application software installed in the electronic device 1 and various types of data, such as codes of database migration programs, etc., but also for temporarily storing data that has been output or is to be output.

The processor 13 may be comprised of integrated circuits in some embodiments, for example, a single packaged integrated circuit, or may be comprised of multiple integrated circuits packaged with the same or different functions, including one or more central processing units (Central Processing unit, CPU), microprocessors, digital processing chips, graphics processors, various control chips, and the like. The processor 13 is a Control Unit (Control Unit) of the electronic device 1, connects the respective components of the entire electronic device 1 using various interfaces and lines, and executes various functions of the electronic device 1 and processes data by running or executing programs or modules (e.g., executing database migration programs, etc.) stored in the memory 12, and calling data stored in the memory 12.

The processor 13 executes the operating system of the electronic device 1 and various types of applications installed. The processor 13 executes the application program to implement the steps in the various database migration method embodiments described above, such as steps S10, S11, S12, S13, S14, S15, S16, S17, S18 shown in fig. 1.

Or the processor 13, when executing the computer program, performs the functions of the modules/units in the above-described device embodiments, for example:

synchronizing the data volume in the old library to the new library;

detecting whether new data are generated in real time;

Acquiring real-time data after the new library is online;

Checking the data in the new library;

Illustratively, the computer program may be partitioned into one or more modules/units that are stored in the memory 12 and executed by the processor 13 to complete the present invention. The one or more modules/units may be a series of instruction segments of a computer program capable of performing a specific function for describing the execution of the computer program in the electronic device 1. For example, the computer program may be divided into a receiving unit 110, a synchronizing unit 111, a detecting unit 112, a writing unit 113, an acquiring unit 114, a verifying unit 115, a controlling unit 116, a monitoring unit 117, a capturing unit 118, a skipping unit 119, a recording unit 120, a determining unit 121, a complementing unit 122, an updating unit 123, a stopping unit 124, a comparing unit 125, and a monitoring unit 126.

The integrated units implemented in the form of software functional modules described above may be stored in a computer readable storage medium. The software functional modules described above are stored in a storage medium and include instructions for causing a computer device (which may be a personal computer, a computer device, or a network device, etc.) or processor (processor) to perform portions of the methods described in the various embodiments of the invention.

The integrated modules/units of the electronic device 1 may be stored in a computer readable storage medium if implemented in the form of software functional units and sold or used as separate products. Based on this understanding, the present invention may also be implemented by a computer program for instructing a relevant hardware device to implement all or part of the procedures of the above-mentioned embodiment method, where the computer program may be stored in a computer readable storage medium and the computer program may be executed by a processor to implement the steps of each of the above-mentioned method embodiments.

Wherein the computer program comprises computer program code which may be in source code form, object code form, executable file or some intermediate form etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM).

The bus may be a peripheral component interconnect standard (PERIPHERAL COMPONENT INTERCONNECT, PCI) bus, or an extended industry standard architecture (extended industry standard architecture, EISA) bus, among others. The bus may be classified as an address bus, a data bus, a control bus, etc. For ease of illustration, only one arrow is shown in FIG. 3, but only one bus or one type of bus is not shown. The bus is arranged to enable a connection communication between the memory 12 and at least one processor 13 or the like.

Although not shown, the electronic device 1 may further comprise a power source (such as a battery) for powering the various components, which may preferably be logically connected to the at least one processor 13 via a power management means, so as to perform functions such as charge management, discharge management, and power consumption management via the power management means. The power supply may also include one or more of any of a direct current or alternating current power supply, recharging device, power failure detection circuit, power converter or inverter, power status indicator, etc. The electronic device 1 may further include various sensors, bluetooth modules, wi-Fi modules, etc., which will not be described herein.

Further, the electronic device 1 may also comprise a network interface, optionally the network interface may comprise a wired interface and/or a wireless interface (e.g. WI-FI interface, bluetooth interface, etc.), typically used for establishing a communication connection between the electronic device 1 and other electronic devices.

The electronic device 1 may optionally further comprise a user interface, which may be a Display, an input unit, such as a Keyboard (Keyboard), or a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, or the like. The display may also be referred to as a display screen or display unit, as appropriate, for displaying information processed in the electronic device 1 and for displaying a visual user interface.

It should be understood that the embodiments described are for illustrative purposes only and are not limited to this configuration in the scope of the patent application.

Fig. 3 shows only an electronic device 1 with components 12-13, it being understood by a person skilled in the art that the structure shown in fig. 3 does not constitute a limitation of the electronic device 1, and may comprise fewer or more components than shown, or may combine certain components, or a different arrangement of components.

In connection with fig. 1, the memory 12 in the electronic device 1 stores a plurality of instructions to implement a database migration method, the processor 13 being executable to implement:

synchronizing the data volume in the old library to the new library;

detecting whether new data are generated in real time;

Acquiring real-time data after the new library is online;

Checking the data in the new library;

Specifically, the specific implementation method of the above instructions by the processor 13 may refer to the description of the relevant steps in the corresponding embodiment of fig. 1, which is not repeated herein.

In the several embodiments provided in the present invention, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is merely a logical function division, and there may be other manners of division when actually implemented.

The modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical units, may be located in one place, or may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional module in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units can be realized in a form of hardware or a form of hardware and a form of software functional modules.

It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof.

The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference signs in the claims shall not be construed as limiting the claim concerned.

Furthermore, it is evident that the word "comprising" does not exclude other elements or steps, and that the singular does not exclude a plurality. A plurality of units or means recited in the system claims can also be implemented by means of software or hardware by means of one unit or means. The terms second, etc. are used to denote a name, but not any particular order.

Finally, it should be noted that the above-mentioned embodiments are merely for illustrating the technical solution of the present invention and not for limiting the same, and although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications and equivalents may be made to the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A method of database migration, the method comprising:

synchronizing the data volume in the old library to the new library;

detecting whether new data are generated in real time;

Acquiring real-time data after the new library is online;

Checking the data in the new library, including: before a configuration time point in preset time, carrying out full comparison on the data in the new library and the data in the old library to obtain a full comparison result; after the configuration time point in the preset time, performing incremental comparison on the data in the new library and the data in the old library to obtain an incremental comparison result; monitoring the log of the new library in a preset time to obtain a monitoring result; when the total comparison result, the increment comparison result and the monitoring result are not abnormal, determining that the new library passes the verification;

when the verification is passed, controlling the new library to execute a read-write instruction and controlling the old library to be offline;

in incrementally synchronizing the new data from the old library to the new library, the method further comprises:

Capturing abnormal data when abnormal data is detected;

Skipping the captured data;

recording the captured data as abnormal data;

When the abnormal data is in writing failure, calling data corresponding to the abnormal data from the old library based on the data identification of the abnormal data, and complementarily recording the called data to the new library;

2. The database migration method of claim 1, wherein the incrementally synchronizing the new data from the old library to the new library comprises:

Obtaining an operation log of the old library;

analyzing the operation log to obtain changed data;

determining the changed data as the new data;

synchronizing the new data to the new library.

3. The database migration method of claim 1, wherein when writing the real-time data to the old library and the new library, respectively, the method further comprises:

4. The database migration method of claim 1, wherein controlling the new library to execute read-write instructions and controlling the old library to be offline comprises:

executing a read-write instruction in the target database;

5. A database migration apparatus, the apparatus comprising:

A detection unit for detecting whether new data is generated in real time;

The verification unit is used for verifying the data in the new library, and comprises the following steps: before a configuration time point in preset time, carrying out full comparison on the data in the new library and the data in the old library to obtain a full comparison result; after the configuration time point in the preset time, performing incremental comparison on the data in the new library and the data in the old library to obtain an incremental comparison result; monitoring the log of the new library in a preset time to obtain a monitoring result; when the total comparison result, the increment comparison result and the monitoring result are not abnormal, determining that the new library passes the verification;

The control unit is used for controlling the new library to execute a read-write instruction and controlling the old library to be offline when the verification is passed;

in incrementally synchronizing the new data from the old library to the new library, the apparatus further comprises:

the monitoring unit is used for carrying out real-time monitoring based on an exception handling mechanism when incremental synchronization is carried out;

a skipping unit for skipping the captured data;

A recording unit configured to record the captured data as abnormal data;

The supplementary record unit is used for calling the data corresponding to the abnormal data from the old library based on the data identification of the abnormal data when the abnormal data is in writing failure, and supplementary recording the called data to the new library;

6. An electronic device, the electronic device comprising:

A memory storing at least one instruction; and

A processor executing instructions stored in the memory to implement the database migration method of any one of claims 1 to 4.

7. A computer-readable storage medium, characterized by: the computer-readable storage medium having stored therein at least one instruction for execution by a processor in an electronic device to implement the database migration method of any one of claims 1-4.