Nothing Special   »   [go: up one dir, main page]

CN115543969A - Data migration method, device, equipment and medium - Google Patents

Data migration method, device, equipment and medium Download PDF

Info

Publication number
CN115543969A
CN115543969A CN202211495843.4A CN202211495843A CN115543969A CN 115543969 A CN115543969 A CN 115543969A CN 202211495843 A CN202211495843 A CN 202211495843A CN 115543969 A CN115543969 A CN 115543969A
Authority
CN
China
Prior art keywords
data
identifier
target
migration
source data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202211495843.4A
Other languages
Chinese (zh)
Other versions
CN115543969B (en
Inventor
邢金港
张颖
王帅
钱程
尹涛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
CCB Finetech Co Ltd
Original Assignee
CCB Finetech Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by CCB Finetech Co Ltd filed Critical CCB Finetech Co Ltd
Priority to CN202211495843.4A priority Critical patent/CN115543969B/en
Publication of CN115543969A publication Critical patent/CN115543969A/en
Application granted granted Critical
Publication of CN115543969B publication Critical patent/CN115543969B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21Design, administration or maintenance of databases
    • G06F16/214Database migration support
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2455Query execution
    • G06F16/24564Applying rules; Deductive queries

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The present application relates to the field of data storage technologies, and in particular, to a data migration method, apparatus, device, and medium. In the embodiment of the application, the first configuration file carrying the first identifier of the source data to be migrated and the corresponding relationship between the first identifier to be migrated and the corresponding first migration rule can be written into the pre-stored data migration program to obtain the first target program, and then the first target program is operated to realize data migration, so that face-to-face butting of a butting person and a developer can be avoided, the efficiency of data migration is improved, the error rate of data migration is reduced, and the labor cost of development of the developer is reduced.

Description

Data migration method, device, equipment and medium
Technical Field
The present application relates to the field of data storage technologies, and in particular, to a data migration method, apparatus, device, and medium.
Background
In the construction of a new software system, data migration work often needs to be carried out in a matched manner, and the data migration refers to that when a new software system is used for replacing an old software system, the data of the old software system is cleaned, managed and processed according to the requirements of the new software system, so that the data of the old software system meets the requirements of the new software system, namely, the source data in a source database is modified according to the requirements of the new software system and is migrated to a target database. The data migration work is time-consuming, labor-consuming and error-prone, especially for a financial system, data is used as the most important asset, the migration accuracy is particularly emphasized, various supervision requirements need to be met, and for a business process adjusted in a new software system, the data also needs to be correspondingly adjusted to adapt to a new caliber.
In the related art, in order to implement data migration, a butt-joint person and a developer need to be in face-to-face butt joint, the developer performs subsequent development work according to the data migration requirement after determining the data migration requirement of the butt-joint person, the labor cost consumed by the data migration work is high, the risk of data migration work errors is also high, and the efficiency of data migration is also low.
Disclosure of Invention
The application provides a data migration method, a data migration device, data migration equipment and a data migration medium, which are used for solving the problems of low data migration efficiency and high risk of data migration errors in the related technology.
The embodiment of the application provides a data migration method, which comprises the following steps:
receiving a generated first configuration file, wherein the first configuration file carries a first identifier of source data to be migrated and a corresponding relation between the first identifier of the source data to be migrated and a corresponding first migration rule;
determining a parameter position corresponding to the first configuration file from a pre-stored data migration program, writing the first configuration file into the corresponding parameter position, and generating a first target program; and running the first target program, processing the source data of the first identifier by adopting the first migration rule, and migrating the processed first target data to a target database.
Further, the process of generating the first configuration file includes:
receiving a second identifier of a source database where the selected source data to be migrated is located;
identifying a target database type corresponding to the source database of the second identifier, and displaying a first candidate migration rule corresponding to the target database type according to a corresponding relation between a prestored database type and a migration rule;
receiving an input first identifier of source data to be migrated;
and receiving a migration rule corresponding to the source data selected from the first candidate migration rules, and generating a first configuration file containing a first identifier of the source data and a corresponding relation between the identifier of the source data to be migrated and the migration rule corresponding to the identifier.
Further, after receiving the input first identifier of the source data to be migrated and before receiving the migration rule corresponding to the source data selected from the candidate migration rules, the method further includes:
determining whether the first identified source data is primary key data;
if yes, displaying a second candidate migration rule corresponding to the primary key data according to a corresponding relation between the pre-stored primary key data and the migration rule;
the receiving a migration rule corresponding to the source data selected from the first candidate migration rules comprises:
receiving a migration rule corresponding to the source data selected from the first candidate migration rule and/or the second candidate migration rule.
Further, the method further comprises:
and if the first target program is operated and the source data is not migrated to the target database, determining that the first target program is a non-executable program.
Further, the method further comprises:
and outputting prompt information for prompting the developer to develop for the second time.
Further, the method further comprises:
receiving a second target program input after secondary development; a second configuration file is written in the second target program, and the second configuration file carries a first identifier of source data to be migrated and a corresponding relation between the identifier of the source data to be migrated and a second migration rule corresponding to the identifier;
executing the second target program, processing the source data of the first identifier by adopting the second migration rule, and migrating the second target data obtained after processing to a target database.
Further, the configuration file also carries a target data table identifier in a target database to be migrated;
the migrating the processed target data to the target database comprises:
and migrating the processed target data to a target data table corresponding to the target data table identification in the target database.
An embodiment of the present application further provides a data migration apparatus, the apparatus includes:
the receiving module is used for receiving the generated first configuration file, wherein the first configuration file carries a first identifier of source data to be migrated and a corresponding relation between the first identifier of the source data to be migrated and a first migration rule corresponding to the first identifier;
the migration module is used for determining a parameter position corresponding to the first configuration file from a pre-stored data migration program, writing the first configuration file into the corresponding parameter position and generating a first target program; and running the first target program, processing the source data of the first identifier by adopting the first migration rule, and migrating the processed first target data to a target database.
Further, the apparatus further comprises:
the processing module is used for receiving a second identifier of a source database where the selected source data to be migrated is located; identifying a target database type corresponding to the source database of the second identifier, and displaying a first candidate migration rule corresponding to the target database type according to a corresponding relation between a prestored database type and a migration rule; receiving an input first identifier of source data to be migrated; receiving a migration rule from the first candidate
And generating a first configuration file containing the first identifier of the source data and the corresponding relation between the identifier of the source data to be migrated and the migration rule corresponding to the identifier of the source data to be migrated.
Further, the processing module is further configured to determine whether the source data of the first identifier is primary key data; if so, displaying a second candidate migration rule corresponding to the primary key data according to the corresponding relation between the pre-stored primary key data and the migration rule;
the migration module is specifically configured to receive a migration rule corresponding to the source data selected from the first candidate migration rule and/or the second candidate migration rule.
Further, the processing module is further configured to determine that the first target program is a non-executable program if the source data is not migrated to the target database by running the first target program.
Further, the processing module is further configured to output prompt information for prompting a developer to perform secondary development.
Further, the processing module is further configured to receive a second target program input after the secondary development; a second configuration file is written in the second target program, and the second configuration file carries a first identifier of the source data to be migrated and a corresponding relation between the identifier of the source data to be migrated and a second migration rule corresponding to the identifier; and executing the second target program, processing the source data of the first identifier by adopting the second migration rule, and migrating the second target data obtained after processing to a target database.
Further, the configuration file also carries a target data table identifier in a target database to be migrated;
the migration module is specifically configured to migrate the processed target data to a target data table corresponding to the target data table identifier in the target database.
An embodiment of the present application further provides an electronic device, where the electronic device includes a processor, and the processor is configured to implement the steps of the data migration method according to any one of the above when executing the computer program stored in the memory.
The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program, and the computer program, when executed by a processor, implements the steps of the data migration method as described in any one of the above.
An embodiment of the present application further provides a computer program product, where the computer program product includes: computer program code for causing a computer to perform the steps of the business data migration method as described above when said computer program code is run on a computer.
In the embodiment of the application, a generated first configuration file is received, wherein the first configuration file carries a first identifier of source data to be migrated and a corresponding relation between the identifier of the source data to be migrated and a first migration rule corresponding to the identifier; determining a parameter position corresponding to a first configuration file from a pre-stored data migration program, writing the first configuration file into the corresponding parameter position, and generating a first target program; and operating the first target program, processing the source data of the first identifier by adopting a first migration rule, and migrating the processed first target data to a target database. In the embodiment of the application, the first configuration file carrying the first identifier of the source data to be migrated and the corresponding relationship between the first identifier to be migrated and the corresponding first migration rule can be written into the pre-stored data migration program to obtain the first target program, and then the first target program is operated to realize data migration, so that the face-to-face butt joint of butt joint personnel and development personnel can be avoided, the efficiency of data migration is improved, the error rate of data migration is reduced, and the labor cost of development of the development personnel is reduced.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the description below are only some embodiments of the present application, and it is obvious for those skilled in the art that other drawings may be obtained according to these drawings without inventive labor.
Fig. 1 is a schematic diagram of a data migration process according to an embodiment of the present application;
FIG. 2 is a schematic illustration of a display of a saved migration rule provided by some embodiments of the present application;
FIG. 3 is a schematic illustration of a database display provided by some embodiments of the present application;
fig. 4 is a schematic display diagram of a selected identifier of data to be migrated and a correspondence between the identifier of the data to be migrated and a migration rule provided in some embodiments of the present application;
FIG. 5 is a schematic illustration of a display of source data converted to target data according to some embodiments of the present disclosure;
FIG. 6 is a schematic diagram illustrating a process for migrating data to be migrated to a target data table in a target database according to some embodiments of the present application;
FIG. 7 is a process diagram of data migration according to some embodiments of the present application;
FIG. 8 is a schematic diagram of a data migration apparatus according to some embodiments of the present application;
fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be described in further detail with reference to the accompanying drawings, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
In order to improve the efficiency of data migration and reduce the risk of data migration errors, embodiments of the present application provide a data migration method, apparatus, device, and medium.
In the embodiment of the application, a generated first configuration file is received, wherein the first configuration file carries a first identifier of source data to be migrated and a corresponding relation between the identifier of the source data to be migrated and a first migration rule corresponding to the identifier; determining a parameter position corresponding to a first configuration file from a pre-stored data migration program, writing the first configuration file into the corresponding parameter position, and generating a first target program; and operating the first target program, processing the source data of the first identifier by adopting a first migration rule, and migrating the processed first target data to a target database.
Example 1:
fig. 1 is a schematic diagram of a data migration process provided in an embodiment of the present application, where the process includes the following steps:
s101: receiving a generated first configuration file, wherein the first configuration file carries a first identifier of source data to be migrated and a corresponding relation between the first identifier of the source data to be migrated and a corresponding first migration rule.
The data migration method provided by the embodiment of the application is applied to electronic equipment, and the electronic equipment can be equipment such as an intelligent terminal, a PC (personal computer) or a server.
In this embodiment of the present application, in order to implement data migration, an electronic device receives a generated first configuration file, where the first configuration file carries a first identifier of source data to be migrated and a correspondence between the first identifier of the source data to be migrated and a first migration rule corresponding to the first identifier, where the first identifier may be a name corresponding to the source data, a name of a field corresponding to the source data, location information corresponding to the source data, and the like, and the first migration rule may include: convert date in aaaa-bb-cc data format to date in aaaaabbcc data format, convert date in timestamp data format to aaababcc data format, etc., map field T1 of A table to field T2 of B table, repeat client cleanup, etc. The first configuration file may carry one source data to be migrated and a correspondence between a first identifier of the source data and a first migration rule corresponding to the first identifier, and may also carry a plurality of source data to be migrated and a correspondence between a first identifier of the source data and a first migration rule corresponding to the first identifier for each of the plurality of source data. The first identifier of one source data may correspond to one first migration rule, and the first identifier of one source data may also correspond to a plurality of first migration rules.
S102: determining a parameter position corresponding to the first configuration file from a pre-stored data migration program, and writing the first configuration file into the corresponding parameter position to generate a first target program; and running the first target program, processing the source data of the first identifier by adopting the first migration rule, and migrating the processed first target data to a target database.
In order to facilitate that the electronic device can perform data migration on source data to be migrated according to a first migration rule corresponding to the source data, in the embodiment of the present application, a data migration program is pre-stored, where the data migration program is a non-executable code of a parameter value to be input. Where executable means that the code may run based on specific environmental conditions, as distinguished from the document; non-executable is the opposite of executable, and non-executable code is used to describe the way program logic is easy for an engineer to read, but cannot be run directly.
In this embodiment of the application, in order to migrate source data to be migrated to a target database, after receiving a first configuration file carrying a first identifier of the source data to be migrated and a source data of the first identifier of the source data to be migrated and a first migration rule corresponding to the source data, an electronic device determines a parameter position of the first configuration file corresponding to a pre-stored data migration program, and then writes the first configuration file into the corresponding parameter position to obtain a first target program. The first configuration file corresponds to which parameter position in the pre-saved data migration program is preset and known.
And then the electronic equipment runs the first target program, processes the source data of the first identifier by adopting a first migration rule to obtain first target data, and migrates the first target data obtained after processing into a target database. For example, the source data with the data format of "2022, 9, month, and 27" is changed to the first target data with the data format of "2022-9-17", and then the first target data with the data format of "2022-9-17" is migrated to the target database.
The target database is used for storing first target data and supports functions of data addition, updating, deletion, query and the like. The first target data are data which are generated after the source data are processed according to the corresponding first migration rule and meet the requirements of a specific data format and data quality.
In the embodiment of the application, the first configuration file which can carry the first identifier of the source data to be migrated and the corresponding relation between the first identifier to be migrated and the corresponding first migration rule is written into the pre-stored data migration program to obtain the first target program, and then the first target program is operated to realize data migration, so that the face-to-face butting of butting personnel and developers can be avoided, the efficiency of data migration is improved, the error rate of data migration is reduced, and the labor cost of development of developers is reduced.
Example 2:
to generate the first configuration file, on the basis of the foregoing embodiments, in an embodiment of the present application, a process of generating the first configuration file includes:
receiving a second identifier of a source database where the selected source data to be migrated is located;
identifying a target database type corresponding to the source database of the second identifier, and displaying a first candidate migration rule corresponding to the target database type according to a corresponding relation between a prestored database type and a migration rule;
receiving an input first identifier of source data to be migrated;
and receiving a migration rule corresponding to the source data selected from the first candidate migration rules, and generating a first configuration file containing a first identifier of the source data and a corresponding relation between the identifier of the source data to be migrated and the migration rule corresponding to the identifier.
In this embodiment of the present application, in order to facilitate obtaining a target data type corresponding to a source database where source data to be migrated is located, all source databases that can be selected may be displayed on a display page of the electronic device, a dockee selects a source database where source data to be migrated is located from all source databases, and at this time, the electronic device receives a second identifier of the source database where the selected source data to be migrated is located, where the number of the selected source data to be migrated may be 1 or multiple.
In order to determine the target database type corresponding to the source database with the second identifier, in this embodiment of the application, a corresponding relationship between the identifier of each source database and the database type is pre-stored in the electronic device, and the target database type corresponding to the source database with the second identifier is determined based on the pre-stored corresponding relationship between the identifier of each source database and the database type.
In order to determine which migration rules correspond to a target database type corresponding to a source database where source data to be migrated is located, in this embodiment of the application, a corresponding relationship between a database type and a migration rule is pre-stored in an electronic device, a first candidate migration rule corresponding to the target database type is displayed according to the target database type and the corresponding relationship between the pre-stored database type and the migration rule, and specifically, the first candidate migration rule is displayed on a display page of the electronic device.
Specifically, in the embodiment of the present application, for each database type in the electronic device, all migration rules corresponding to the database type may be stored in one database, and for convenience of management, the migration rules may also be stored in multiple databases, for example, in three databases, and the corresponding migration rules stored in each database may be the same or different. For convenience, the four databases are described as examples, and are referred to as a general rule maintenance library, a solution maintenance library, a general rule library, and a special rule library, respectively. The general rule maintenance library is mainly used for supporting the collection and maintenance of some migration general rules, and for a specific service system, particularly for a relatively stable target software system, there are many general migration rules, such as code mapping, date format conversion and the like, wherein the general rule database is generally an Msql type database; the solution maintenance library is mainly used for storing migration rules of programs accumulated by software systems in some specific fields during data migration, such as repeat client cleaning and the like, for example, client merge migration rules, specifically, for repeat clients existing in a source database, the rules are specified by a business department, a fixed format document is provided, and the fixed format document includes: the solution database comprises a client number, a combined group number (different clients are the same group if combined, and the same group number should be allocated), a reserved identification (the same combined group client only takes data of one valid client as a source, and partial data of other clients, such as account information, is incorporated into the client), wherein the solution database can be a mysql type or oracle type database; the general rule base is mainly used for storing more common processing rules, such as field mapping and the like; the special rule base mainly stores other migration rules which have specific requirements for customers and cannot be solved through the migration rules stored in the three databases.
Fig. 2 is a schematic display diagram of a saved migration rule according to some embodiments of the present application, and is described with reference to fig. 2.
The following description will be given by taking information such as the database in which the specific migration rules are stored, the applicable database type, the applicable operating system type, the applicable environment, the solution order, the rule type, and the execution time point as examples.
And storing a conversion migration rule corresponding to target data for converting source data of other date format classes into YYYYMMDD date format classes in a universal rule database, wherein the type of the applicable database is Oracle, the type of the applicable operating system is ALL, namely ALL operating systems are applicable, the applicable environment is ALL, namely ALL environments are applicable, no solution order exists, the type of the rule is Check, and no execution time point exists.
The primary key data migration rule is stored in a general rule database, the type of the applicable database is Oracle, the type of the applicable operating system is ALL, namely ALL operating systems are applicable, no solution sequence exists, the type of the rule is Check, and no execution time point exists.
And storing a conversion migration rule corresponding to target data for converting the source data of the YYYYMMDD period format class into the YYYYY-MM-DD date format class in a general rule base, wherein the applicable database type is Oracle, the applicable operating system type is ALL, namely ALL operating systems are applicable, the applicable environment is ALL, namely ALL environments are applicable, no solution sequence is provided, the rule type is Function, and no execution time point is provided.
The client merging and migrating rules are stored in a solution maintenance rule base, the applicable database type is ALL, namely ALL the database types are applicable, the applicable operating system type is Linux, the applicable environment is Shell, the solution sequence is 1, the rule type is Program, and the execution time point is AfterTrans.
In order to determine the source data to be migrated, all source data which can be selected from the source database are displayed on the electronic device, a dockee can select the source data to be migrated from all the source data displayed on the display page, and at this time, the electronic device can receive the first identifier of the input source data to be migrated, where the source data is data before processing and before migration, and the number of the source data to be migrated may be one or multiple, and specifically, the number of the source data to be migrated may be set according to a requirement.
In order to accurately migrate source data to a target database, a dockee selects a migration rule corresponding to the source data to be migrated from first candidate migration rules displayed on a display page of an electronic device, where the migration rule may include: convert date in aaaa-bb-cc data format to date in aaaaabbcc data format, convert date in timestamp data format to aaababcc data format, etc., map field T1 of A table to field T2 of B table, repeat client cleanup, etc. For each source data to be migrated, the migration rule corresponding to the source data to be migrated may be 1, or multiple.
After receiving an input first identifier of source data to be migrated and a migration rule corresponding to the source data selected from first candidate migration rules, the electronic device generates a first configuration file containing the first identifier of the source data and a corresponding relationship between the first identifier of the source data to be migrated and the migration rule corresponding to the first identifier of the source data to be migrated.
When the dockee selects the source data to be migrated and selects the migration rule corresponding to the source data to be migrated from the first candidate migration rule, the dockee supports simultaneous access and simultaneous operation by multiple persons, that is, the method can be implemented by a typical network application including a front end, a back end and a database based on a browser service.
In addition, if there is no predefined content on the display page, for example, there is no migration rule that the dockee wants to select on the display page, the predefined content may be temporarily recorded in the display page of the electronic device in the remark form, and finally recorded in the database of the electronic device, so that the subsequent developer can develop the migration rule and add the corresponding migration rule.
Fig. 3 is a schematic display diagram of a database according to some embodiments of the present application, and is now described with reference to fig. 3.
The database used for storing the migration rules comprises a general rule maintenance library, a solution maintenance library, a general rule library and a special rule library, the docking personnel can select the migration rules on the display page of the electronic equipment, when the display page does not have the migration rules which the docking personnel wants to select, the migration rules can be recorded on the display page of the electronic equipment temporarily in a remark mode and finally recorded in the special rule database of the electronic equipment, so that the subsequent developers can develop the migration rules, and the newly added migration rules are stored in the special rule database.
Fig. 4 is a schematic display diagram of a selected identifier of data to be migrated and a correspondence between the identifier of the data to be migrated and a migration rule according to some embodiments of the present application, and now description is given to fig. 4.
The selected source data to be migrated is the source data in the client table (TB 1010) table.
The source data to be migrated, which is identified as "Custno", has a corresponding chinese name of "client number", and a field format of "Varchar (20)", and has no corresponding migration rule, that is, the source data is directly determined as target data and is migrated to the target database, and the source data is primary key data.
The source data to be migrated, which is identified as "Create _ date", is named as "creation time" in the corresponding chinese language, and is in the field format of "Varchar (8)", and the corresponding migration rule is to convert the source data in other date format classes into target data in the yyymmdd date format class, where the source data is non-primary key data.
The source data to be migrated, which is identified as "Create _ user", is named as "Create user" in the corresponding chinese name, has a field format of "Varchar (8)", has no corresponding migration rule, and is non-primary key data.
Fig. 5 is a schematic display diagram illustrating conversion of source data into destination data according to some embodiments of the present application, and is now described with reference to fig. 5.
The source data to be migrated, which is identified as "Custno", is converted into target data, which is identified as "Cst _ id", the corresponding chinese name is "client number", the field format is "Varchar (20)", there is no corresponding migration rule, that is, the source data is directly determined as the target data and migrated to the target database, the source table corresponding to the source data is TB1010, and the source data is primary key data.
The source data to be migrated, which is identified as "Create _ user", is converted into target data, which is identified as "Begin _ time", the corresponding Chinese name is "creation time", the field format is "Varchar (20)", the corresponding migration rule is to convert the source data in other date format classes into the target data in the YYYYMMDD date format class, the source table corresponding to the source data is TB1010, and the source data is non-primary key data.
The source data to be migrated, which is identified as "Create _ user", is converted into target data, the corresponding identifier is "Belong _ user", the corresponding name of the Chinese is "Create user", the field format is "Varchar (20)", no corresponding migration rule exists, the source table corresponding to the source data is TB1010, and the source data is non-primary key data. Wherein the target data is saved to a client table (tbcch us 0) in the target database.
Example 3:
in order to accurately select a migration rule corresponding to source data to be migrated, on the basis of the foregoing embodiments, in this embodiment of the present application, after receiving an input first identifier of the source data to be migrated, and before receiving a migration rule corresponding to the source data selected from the candidate migration rules, the method further includes:
determining whether the first identified source data is primary key data;
if so, displaying a second candidate migration rule corresponding to the primary key data according to the corresponding relation between the pre-stored primary key data and the migration rule;
the receiving a migration rule corresponding to the source data selected from the first candidate migration rule includes:
receiving a migration rule corresponding to the source data selected from the first candidate migration rule and/or the second candidate migration rule.
In the embodiment of the present application, migration rules corresponding to different types of source data to be migrated are different, for example, the source data of primary key data is different from the migration rules corresponding to the source data of non-primary key data, in order to accurately select the migration rule corresponding to the source data to be migrated, a correspondence between the primary key data and the migration rules is pre-stored in the electronic device, the electronic device determines whether the source data of the first identifier is the primary key data, if so, determines a second candidate migration rule corresponding to the primary key data according to the correspondence between the pre-stored primary key data and the migration rules, and displays the second candidate migration rule corresponding to the primary key data, and specifically, displays the second candidate migration rule on a display page of the electronic device.
At this time, after receiving the input first identifier of the source data to be migrated, when receiving a migration rule corresponding to the source data selected from the first candidate rules, receiving a migration rule corresponding to the source data selected from the first candidate migration rule and/or the second candidate migration rule.
Example 4:
in order to migrate source data to a target database, on the basis of the foregoing embodiments, in an embodiment of the present application, the method further includes:
and if the first target program is operated and the source data is not migrated to the target database, determining that the first target program is a non-executable program, and outputting prompt information for secondary development of developers.
In the embodiment of the application, after the first configuration file is written into the corresponding parameter position of the pre-stored data migration program, the obtained first target program may be an executable program or a non-executable program, and if the first target program is operated, the source data to be migrated can be migrated into the target database, and then the first target program is determined to be the executable program; and if the source data to be migrated cannot be migrated to the target database after the first target program is operated, determining that the first target program is a non-executable program.
Example 5:
for convenience of reminding, on the basis of the above embodiments, in the embodiment of the present application, the method further includes:
and outputting prompt information for prompting the developer to develop for the second time.
If the first target program is a non-executable program, in order to facilitate subsequent migration of the source data to be migrated to the target database, the electronic device may output prompt information for prompting the developer to perform secondary development, so that the subsequent developer performs secondary development based on the first target program. The prompt message may be a text message or a voice message.
Specifically, a voice of "the current program is not executable and please develop for the second time" may be output, and a text of "the current program is not executable and please develop for the second time" may be displayed on a display page of the electronic device.
Example 6:
in order to ensure that the source data to be migrated can be subsequently migrated to the target database when the first target program is a non-executable program, on the basis of the foregoing embodiments, the method further includes:
receiving a second target program input after secondary development; a second configuration file is written in the second target program, and the second configuration file carries a first identifier of the source data to be migrated and a corresponding relation between the identifier of the source data to be migrated and a second migration rule corresponding to the identifier;
and executing the second target program, processing the source data of the first identifier by adopting the second migration rule, and migrating the second target data obtained after processing to a target database.
In this embodiment of the application, after the electronic device outputs prompt information for prompting a developer to perform secondary development, the developer may perform secondary development according to a current non-executable first object program to obtain an executable second object program, and input the second object program into the electronic device, where the electronic device receives the second object program input after the secondary development. The second configuration file carries a first identifier of the source data to be migrated and a corresponding relationship between the identifier of the source data to be migrated and a second migration rule corresponding to the identifier of the source data to be migrated, wherein, for each source data to be migrated, the identifier of the source data to be migrated and the second migration rule corresponding to the identifier of the source data to be migrated may be the same as or different from the first migration rule corresponding to the identifier of the source data to be migrated in the first configuration file.
In order to migrate the source data to be migrated to the target database, the electronic device executes the second target program, processes the source data of the first identifier by using the second migration rule, and migrates the second target data obtained after the processing to the target database.
Example 7:
in order to accurately migrate the source data to be migrated to the target database, on the basis of the above embodiments, the configuration file also carries a target data table identifier in the target database to be migrated;
the migrating the processed target data to the target database comprises:
and migrating the processed target data to a target data table corresponding to the target data table identification in the target database.
In the embodiment of the present application, for convenience of management, in migrating source data to a target database, the source data is generally migrated to a specific data table in the target database, and some source data should be migrated to the same data table and some source data should be migrated to a different data table.
In order to facilitate determining to which data table in the target database the source data to be migrated is migrated, in the embodiment of the present application, a dockee in a configuration file may also carry an identifier of the target data table in the target database to be migrated, in addition to a correspondence between the identifier of the source data to be migrated and a migration rule corresponding thereto, where the configuration file may be a first configuration file or a second configuration file, and if the configuration file is the first configuration file, the first configuration file carries a first identifier of the source data to be migrated, a correspondence between the identifier of the source data to be migrated and a first migration rule corresponding thereto, and a target data table identifier in the target database to be migrated; and if the configuration file is a second configuration file, the second configuration file carries a second identifier of the source data to be migrated, a corresponding relation between the identifier of the source data to be migrated and a second migration rule corresponding to the identifier of the source data to be migrated, and a target data table identifier in the target database to be migrated.
After the electronic equipment runs a first target program written in a first configuration file or a second target program written in a second configuration file, the electronic equipment processes the source data of the first identifier by adopting a first migration rule or a second migration rule, and migrates the processed target data to a target data table corresponding to the target data table identifier in a target database of the docking personnel. If the target data is obtained by processing the source data of the first identifier by adopting a first migration rule if the target data is a first target program, the target data is the first target data, and if the target data is a second target program, the target data is the second target data if the target data is obtained by processing the source data of the first identifier by adopting a second migration rule.
It should be noted that, in the above method, the target data table to which the source data of the first identifier is to be migrated is preset before data migration and stored in the target database, a display page of the subsequent electronic device displays all data tables that can be migrated in the target database, and a user only needs to select which data table to which the source data of the first identifier is to be migrated, and the electronic device receives the identifier of the target data table to be migrated and carries the identifier of the target data table to be migrated in the configuration file.
In addition, the configuration file may carry an identifier of source data to be migrated, a correspondence between the identifier of the source data to be migrated and a migration rule corresponding thereto, and may also carry identifier information of a target database to be migrated and attribute information of a target data table to be generated, the configuration file is written into a pre-stored data migration program, a target program is obtained, the target program is run, the source data of the first identifier is processed by using the migration rule, and a target data table serving as the attribute information is generated in the target database, and finally the processed obtained target data is stored in the target data table generated in the target database. The attribute information may be the size of the table, the color of the table, and the like.
It should be noted that, in the foregoing method, the target data table to which the source data of the first identifier is to be migrated is not preset before data migration, but a target data table is generated in the target database according to the attribute information of the target data table to be generated during data migration.
Fig. 6 is a schematic diagram of a process for migrating data to be migrated to a target data table in a target database according to some embodiments of the present application, and is described with reference to fig. 6. For convenience of description, the generated first target program is taken as an example of an executable program.
Scheme 1: the electronic equipment comprises data tables for storing data to be migrated in a target database in advance, and each data table has a corresponding identifier. The method comprises the steps that a butt joint person selects data to be migrated, a migration rule corresponding to an identifier of source data to be migrated and a target data table in a target database to be migrated based on a display page of electronic equipment, at the moment, the electronic equipment receives the identifier of the source data to be migrated, a corresponding relation between the identifier of the source data to be migrated and the migration rule corresponding to the identifier of the source data to be migrated and the identifier of the target data table in the target database to be migrated, the electronic equipment generates a first configuration file according to the identifier of the source data to be migrated, the corresponding relation between the identifier of the source data to be migrated and the migration rule corresponding to the identifier of the source data to be migrated and the identifier of the target data table in the target database to be migrated, the first configuration file is written into a pre-stored data migration program to obtain a first target program, the first target program is operated, the source data of the first identifier is processed based on the first migration rule, and the target data obtained after processing is migrated into the pre-stored target data table in the target database.
Scheme 2: the method comprises the steps that a butt-joint person selects migration rules corresponding to data to be migrated and source data to be migrated based on a display page of electronic equipment, further selects an identifier of a target database to be migrated and attribute information of a target data table to be migrated, at the moment, the electronic equipment receives the identifier of the source data to be migrated, the corresponding relation between the identifier of the source data to be migrated and the migration rules corresponding to the identifier of the source data to be migrated, the identifier of the target database to be migrated and the attribute information of the target data table to be migrated, the electronic equipment generates a first configuration file according to the identifier of the source data to be migrated and the corresponding relation between the identifier of the source data to be migrated and the migration rules corresponding to the identifier of the target database to be migrated and the attribute information of the target data table to be migrated, writes the first configuration file into a pre-stored data migration program, obtains a first target program, runs the first target program, processes the source data of the first identifier based on the first migration rules, generates a target data table in the target database after the target data table is finally generated.
Fig. 7 is a schematic diagram of a data migration process according to some embodiments of the present application, and is now described with reference to fig. 7.
Displaying all the source databases which can be selected on a display page of the electronic device, enabling a docking worker to select the source database where the source data to be migrated is located from the display page of the electronic device, receiving a second identifier of the source database where the selected source data to be migrated is located by the electronic device, and determining and displaying a first candidate migration rule corresponding to a target database type according to the target database type corresponding to the source database of the second identifier and a corresponding relation between a pre-stored database type and a migration rule. Wherein the first candidate migration rule is stored in at least one of a general rule maintenance library, a solution maintenance library, a general rule library, and a special rule library.
All selectable source data which can be migrated are also displayed on a display page of the electronic device, a subsequent docking staff selects the source data to be migrated based on all the source data which can be migrated and displayed on the display page of the electronic device, at this time, the electronic device receives an input first identifier of the source data to be migrated, the specifically displayed identifier may be an identifier of the source data, and the identifier may be information which uniquely identifies the source data, such as a field name of the source data.
The dockee selects a migration rule corresponding to the source data based on the first candidate migration rule displayed on the display page of the electronic device, and the electronic device receives the migration rule corresponding to the selected source data at this time.
The electronic equipment generates a first configuration file containing a first identifier of source data and a corresponding relation between the identifier of the source data to be migrated and a migration rule corresponding to the identifier.
The electronic equipment writes the first configuration file into a corresponding parameter position of a pre-stored data migration program to generate a first target program, finally runs the first target program, processes source data of the first identification by adopting the first migration rule, and migrates the processed first target data into a target database.
And if the source data are not migrated to the target database when the first target program is operated, determining that the first target program is a non-executable program, outputting prompt information for prompting a developer to perform secondary development, and performing secondary development by a subsequent developer based on the first target program to obtain an executable second target program, wherein a second configuration file is written in the second target program, and the second configuration file carries a first identifier of the source data to be migrated and a corresponding relation between the identifier of the source data to be migrated and a second migration rule corresponding to the identifier.
And executing the second target program, processing the source data of the first identifier by adopting a second migration rule, and migrating the second target data obtained after processing to a target database.
Example 8:
in the following, a detailed comparison is made between the method for data migration in the related art and the method for data migration in the present application.
In the related art, data migration is generally performed based on an Extract-Transform-Load (ETL) tool, an Oracle database-owned data migration tool, and the like, wherein the data extraction refers to selectively acquiring partial information from a certain data source, the data management refers to screening out data which do not meet the requirements of the specification by some methods according to a certain specification, and submitting the data to a data entry party, and after supplementing information, the data quality meets the specification by incorporating the supplementing information. The ETL is a tool adopting a data extraction and conversion technology, the core of the ETL is to convert data in one format into data in another format, the ETL is widely applied to report systems, data warehouses and the like, some ETL products are on the market, and part of ETL products also comprise a data management function; wherein, the data migration tool of database self-bring such as Oracle includes: spool, sqlldr and the like, but these data migration tools are generally only used for migration of fixed format data, are simple in function, need to be additionally developed when relating to data structure change, belong to technical tools, and have no capability of migration scheme aided design.
However, the tools and methods used in data migration in the related art often have certain problems, which are as follows:
the ETL is generally a tool for providing a full flow, such a tool can generally configure a database, support graphical interface operations, support setting of a data conversion flow, and the like, is friendly to the experience of a human, but can be deployed only when certain environmental requirements are met, for example, the environmental requirements on a storage space, a permission, a network, and the like exist, functions are not easily changed after deployment, and the ETL can not be developed for the second time generally, and can only be maintained by a company developing an ETL product, and is only suitable for being used by non-developers or small company data integration components. .
Migration tools provided by databases such as Oracle and the like only support simple import and export functions, the import and export formats are fixed, the conversion of data structures cannot be completed, and the migration tools are the simplest and most basic tools and can become specific practices for importing and exporting data in any data migration engineering.
In addition, in the construction of a large-scale system of a financial institution, data migration work has complexity, which is not difficult to develop, but caused by the strict requirement of the financial institution on data quality, and particularly for the large-scale financial institution, the difficulty of data migration in the related art mainly lies in: large financial institutions have multiple services and complex scenes, so that multiple service systems and multiple system interaction are provided, the data quality problem of historical accumulation is solved, the data is multiple and complex in meaning, the data association relationship is complex, and the meaning and the rule of combing the data need to consume considerable manpower; when a company carrying new software system and a company carrying existing system maintenance are not the same, the function design of the new software system usually considers the existing system function less, the difference between the functions of the new software system and the old software system is large, so the requirements on data format and data quality are also greatly different, when the business meaning of some data needs to be clear, the existing system maintainer, the person using the business system and the new software system developer are often needed, trust and clear responsibility are established among three parties, and as the business systems are numerous, each party needs to communicate with multiple parties, the communication cost is greatly increased; some institutions, especially financial companies, have extremely high data value, and meanwhile, some data irrelevant to funds can also relate to supervision requirements or internal management specifications and must be accurate, otherwise, serious loss can be brought, so that when a new software system is built by the financial institutions, repeated verification is needed, a large amount of time and energy are consumed in the aspect of testing, meanwhile, high value of the data enables partial detailed schemes to also need high-level decisions of customers, and communication cost is further improved; the switching core system of a large financial company is a rare scene, and the data migration tools for the scene are not too many.
Compared with the method for data migration in the related art, the method has many priorities, specifically as follows:
firstly, a reasonable decomposition and design are carried out on a data migration project, most of design and development work is isolated, namely, the work of face-to-face butt joint of developers and butt joint personnel is isolated, the developers and the butt joint personnel are supported to concentrate on the work content of the developers and the butt joint personnel, the working range of the butt joint personnel and the developers is reduced, the developers and the butt joint personnel are supported to better exert the self speciality, all flows (face-to-face butt joint of the butt joint personnel and the developers, scheme design, document management, development, test verification, scheme maintenance and the like) of data injection migration work of the butt joint personnel and the developers are not needed any more, and the data migration work is streamlined to a certain extent to improve the speciality and the efficiency.
In addition, the dockee does not need to pay attention to specific development details, because in the present application, the dockee may select a source database where source data to be migrated is located based on a display page of the electronic device, the electronic device receives a second identifier of the source database where the selected source data to be migrated is located, determines and displays a first candidate migration rule corresponding to a target database type according to the target database type corresponding to the source database of the second identifier and a correspondence between a pre-stored database type and a migration rule, and the dockee selects source data to be migrated based on the display page of the electronic device, and then the electronic device receives the input first identifier of the source data to be migrated, and the dockee selects a migration rule corresponding to the source data based on the first candidate migration rule displayed on the display page of the electronic device, at this time, the electronic device receives the migration rule corresponding to the selected source data, then generates a first configuration file containing a first identifier of the source data and a corresponding relation between the identifier of the source data to be migrated and the migration rule corresponding to the identifier, writes the first configuration file into a pre-stored data migration program by the electronic device, obtains a first target program, performs data migration work based on the first target program, and even if the first target program is a non-executable program, the first target program translates the data migration requirement of the dockee into the non-executable program understood by the developer, and reduces unnecessary communication between the requirers and the developer and reduces the work load of the developer in secondary development when the subsequent developer performs secondary development based on the first target program. For checking rules, reports, solutions and the like, the method for maintaining the assets is also provided, developers can conveniently maintain project assets which have reusability to a certain extent, and repeated development is reduced.
In addition, in essence, the application provides a development auxiliary tool capable of performing data migration, and the development auxiliary tool does not even depend on source data except for testing, does not depend on any authority except for the source data, does not have any invasion on a system corresponding to the source data, can meet the strictest data security requirement, and can be used in any environment.
In addition, the method is suitable for various application scenes, is optimized aiming at the system switching scene of a large financial company, is also suitable for data migration during switching of any complex system or new software system, serves the whole process of receiving test verification in a butt joint mode, avoids function bloat and ensures that the method is completely cross-platform.
In addition, in the construction of a large-scale system of a financial institution, data migration work has complexity, which is not high in development difficulty but caused by strict requirements of the financial institution on data quality, the requirements of the data migration relate to complex business rules, and meanwhile, development and testing of a data migration program also depend on clear business rules, but often face the requirement of continuous adjustment of a client, so that the data migration work is very dependent on experience of project members, and the high requirement on the experience of the project members increases project cost and risk. The method and the system improve the development difficulty through reasonable design, and often relate to the success or failure of a system construction project because the result of data migration work is very important, and the method and the system also reduce the risk of errors of the data migration work and solve the practical problem faced in the financial industry system. In addition, the method reduces the complexity of data migration work in the construction of a new software system, supports the asset library maintenance of data migration development resources, and provides a method for pipelining the data migration work, so that the requirement of each work on the capability of a project implementer is reduced, and the development period is shortened.
In addition, a method for decomposing data migration work is provided, the data migration work is divided into typical modules, project members are supported to concentrate on the modules, the requirements and the working range of developers are reduced, and the project members are supported to better exert the speciality of the project members. The development efficiency is improved by adopting a pipeline form. The invention provides a method for managing data migration development resources, which is beneficial to the accumulation of resources of developers and the reduction of repeated work. The process of the invention can better meet the implementation requirements of the data migration tool and can produce products with controllable quality. The invention belongs to a development tool, does not depend on a specific platform, does not depend on any authority, is flexible to deploy, can develop and test in any self-test environment, and quickly outputs middleware for secondary development, which is an essential difference from tools such as ETL and the like.
Example 9:
fig. 8 is a schematic structural diagram of a data migration apparatus according to some embodiments of the present application, where the apparatus includes:
a receiving module 801, configured to receive a generated first configuration file, where the first configuration file carries a first identifier of source data to be migrated and a correspondence between the first identifier of the source data to be migrated and a first migration rule corresponding to the first identifier;
a migration module 802, configured to determine a parameter location corresponding to the first configuration file from a pre-stored data migration program, and write the first configuration file into the corresponding parameter location to generate a first target program; and running the first target program, processing the source data of the first identifier by adopting the first migration rule, and migrating the processed first target data to a target database.
In one possible embodiment, the apparatus further comprises:
the processing module 803 is configured to receive a second identifier of the source database where the selected source data to be migrated is located; identifying a target database type corresponding to the source database of the second identifier, and displaying a first candidate migration rule corresponding to the target database type according to a corresponding relation between a prestored database type and a migration rule; receiving an input first identifier of source data to be migrated; and receiving a migration rule corresponding to the source data selected from the first candidate migration rules, and generating a first configuration file containing a first identifier of the source data and a corresponding relation between the identifier of the source data to be migrated and the migration rule corresponding to the identifier.
In a possible implementation, the processing module 803 is further configured to determine whether the first identified source data is primary key data; if so, displaying a second candidate migration rule corresponding to the primary key data according to the corresponding relation between the pre-stored primary key data and the migration rule;
the migration module 802 is specifically configured to receive a migration rule corresponding to the source data selected from the first candidate migration rule and/or the second candidate migration rule.
In a possible implementation manner, the processing module 803 is further configured to determine that the first target program is a non-executable program if the first target program is executed without migrating the source data to the target database.
In a possible implementation manner, the processing module 803 is further configured to output a prompt message for prompting a developer to perform secondary development.
In a possible implementation manner, the processing module 803 is further configured to receive a second target program input after the second development; a second configuration file is written in the second target program, and the second configuration file carries a first identifier of the source data to be migrated and a corresponding relation between the identifier of the source data to be migrated and a second migration rule corresponding to the identifier; and executing the second target program, processing the source data of the first identifier by adopting the second migration rule, and migrating the second target data obtained after processing to a target database.
In a possible implementation, the configuration file also carries a target data table identifier in a target database to be migrated;
the migration module 802 is specifically configured to migrate the processed target data to a target data table corresponding to the target data table identifier in the target database.
Example 10:
on the basis of the foregoing embodiments, an embodiment of the present application further provides an electronic device, and fig. 9 is a schematic structural diagram of the electronic device provided in the embodiment of the present application, as shown in fig. 9, including: the system comprises a processor 901, a communication interface 902, a memory 903 and a communication bus 904, wherein the processor 901, the communication interface 902 and the memory 903 are communicated with each other through the communication bus 904;
the memory 903 has stored therein a computer program that, when executed by the processor 901, causes the processor 901 to perform the steps of:
receiving a generated first configuration file, wherein the first configuration file carries a first identifier of source data to be migrated and a corresponding relation between the first identifier of the source data to be migrated and a corresponding first migration rule;
determining a parameter position corresponding to the first configuration file from a pre-stored data migration program, and writing the first configuration file into the corresponding parameter position to generate a first target program; and running the first target program, processing the source data of the first identifier by adopting the first migration rule, and migrating the processed first target data to a target database.
In a possible implementation manner, the processor 901 is further configured to receive a second identifier of a source database where the selected source data to be migrated is located; identifying a target database type corresponding to the source database of the second identifier, and displaying a first candidate migration rule corresponding to the target database type according to a corresponding relation between a prestored database type and a migration rule; receiving an input first identifier of source data to be migrated; and receiving a migration rule corresponding to the source data selected from the first candidate migration rules, and generating a first configuration file containing a first identifier of the source data and a corresponding relation between the identifier of the source data to be migrated and the migration rule corresponding to the identifier.
In a possible implementation, the processor 901 is further configured to determine whether the first identified source data is primary key data; if so, displaying a second candidate migration rule corresponding to the primary key data according to the corresponding relation between the pre-stored primary key data and the migration rule;
the processor 901 is specifically configured to receive a migration rule corresponding to the source data selected from the first candidate migration rule and/or the second candidate migration rule.
In a possible implementation manner, the processor 901 is further configured to determine that the first target program is a non-executable program if the first target program is executed without migrating the source data to the target database.
In a possible implementation manner, the processor 901 is further configured to output a prompt message for prompting the developer to perform secondary development.
In a possible implementation, the processor 901 is further configured to receive a second target program input after the second development; a second configuration file is written in the second target program, and the second configuration file carries a first identifier of source data to be migrated and a corresponding relation between the identifier of the source data to be migrated and a second migration rule corresponding to the identifier; executing the second target program, processing the source data of the first identifier by adopting the second migration rule, and migrating the second target data obtained after processing to a target database.
In a possible implementation manner, the processor 901 is specifically configured to migrate the processed target data to a target data table corresponding to the target data table identifier in the target database.
The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
The communication interface 902 is used for communication between the electronic apparatus and other apparatuses. The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Alternatively, the memory may be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, including a central processing unit, a Network Processor (NP), and the like; but may also be a Digital instruction processor (DSP), an application specific integrated circuit, a field programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or the like.
Example 11:
on the basis of the foregoing embodiments, the present invention further provides a computer-readable storage medium, in which a computer program executable by a processor is stored, and when the program is run on the processor, the processor is caused to execute the following steps:
receiving a generated first configuration file, wherein the first configuration file carries a first identifier of source data to be migrated and a corresponding relation between the first identifier of the source data to be migrated and a first migration rule corresponding to the first identifier;
determining a parameter position corresponding to the first configuration file from a pre-stored data migration program, writing the first configuration file into the corresponding parameter position, and generating a first target program; and running the first target program, processing the source data of the first identifier by adopting the first migration rule, and migrating the processed first target data to a target database.
Further, the process of generating the first configuration file includes:
receiving a second identifier of a source database where the selected source data to be migrated is located;
identifying a target database type corresponding to the source database of the second identifier, and displaying a first candidate migration rule corresponding to the target database type according to a corresponding relation between a prestored database type and a migration rule;
receiving an input first identifier of source data to be migrated;
and receiving a migration rule corresponding to the source data selected from the first candidate migration rules, and generating a first configuration file containing a first identifier of the source data and a corresponding relation between the identifier of the source data to be migrated and the migration rule corresponding to the identifier.
Further, after receiving the input first identifier of the source data to be migrated and before receiving the migration rule corresponding to the source data selected from the candidate migration rules, the method further includes:
determining whether the first identified source data is primary key data;
if yes, displaying a second candidate migration rule corresponding to the primary key data according to a corresponding relation between the pre-stored primary key data and the migration rule;
the receiving a migration rule corresponding to the source data selected from the first candidate migration rule includes:
receiving a migration rule corresponding to the source data selected from the first candidate migration rule and/or the second candidate migration rule.
Further, the method further comprises:
and if the first target program is operated and the source data is not migrated to the target database, determining that the first target program is a non-executable program.
Further, the method further comprises:
and outputting prompt information for prompting the developer to develop for the second time.
Further, the method further comprises:
receiving a second target program input after secondary development; a second configuration file is written in the second target program, and the second configuration file carries a first identifier of the source data to be migrated and a corresponding relation between the identifier of the source data to be migrated and a second migration rule corresponding to the identifier;
executing the second target program, processing the source data of the first identifier by adopting the second migration rule, and migrating the second target data obtained after processing to a target database.
Further, the configuration file also carries a target data table identifier in a target database to be migrated;
the migrating the processed target data to the target database comprises:
and migrating the processed target data to a target data table corresponding to the target data table identification in the target database.
Example 12:
the embodiments of the present application further provide a computer program product, and when executed by a computer, the computer program product implements the data migration method described in any method embodiment applied to an electronic device.
In the above embodiments, the implementation may be realized in whole or in part by software, hardware, firmware, or any combination thereof, and may be realized in whole or in part in the form of a computer program product. The computer program product comprises one or more computer instructions which, when loaded and executed on a computer, cause the processes or functions described in the embodiments of the application to be performed in whole or in part.
In the embodiment of the application, the first configuration file carrying the first identifier of the source data to be migrated and the corresponding relationship between the first identifier to be migrated and the corresponding first migration rule can be written into the pre-stored data migration program to obtain the first target program, and then the first target program is operated to realize data migration, so that face-to-face butting of a butting person and a developer can be avoided, the efficiency of data migration is improved, the error rate of data migration is reduced, and the labor cost of development of the developer is reduced.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.

Claims (17)

1. A method of data migration, comprising:
receiving a generated first configuration file, wherein the first configuration file carries a first identifier of source data to be migrated and a corresponding relation between the first identifier of the source data to be migrated and a corresponding first migration rule;
determining a parameter position corresponding to the first configuration file from a pre-stored data migration program, writing the first configuration file into the corresponding parameter position, and generating a first target program; and running the first target program, processing the source data of the first identifier by adopting the first migration rule, and migrating the processed first target data to a target database.
2. The method of claim 1, wherein generating the first configuration file comprises:
receiving a second identifier of a source database where the selected source data to be migrated is located;
identifying a target database type corresponding to the source database of the second identifier, and displaying a first candidate migration rule corresponding to the target database type according to a corresponding relation between a prestored database type and a migration rule;
receiving an input first identifier of source data to be migrated;
and receiving a migration rule corresponding to the source data selected from the first candidate migration rules, and generating a first configuration file containing a first identifier of the source data and a corresponding relation between the identifier of the source data to be migrated and the migration rule corresponding to the identifier.
3. The method according to claim 2, wherein after receiving the input first identifier of the source data to be migrated and before receiving the migration rule corresponding to the source data selected from the candidate migration rules, the method further comprises:
determining whether the source data of the first identifier is primary key data;
if so, displaying a second candidate migration rule corresponding to the primary key data according to the corresponding relation between the pre-stored primary key data and the migration rule;
the receiving a migration rule corresponding to the source data selected from the first candidate migration rule includes:
receiving a migration rule corresponding to the source data selected from the first candidate migration rule and/or the second candidate migration rule.
4. The method of claim 1, further comprising:
and if the first target program is operated and the source data is not migrated to the target database, determining that the first target program is a non-executable program.
5. The method of claim 4, further comprising:
and outputting prompt information for prompting the developer to develop for the second time.
6. The method of claim 5, further comprising:
receiving a second target program input after secondary development; a second configuration file is written in the second target program, and the second configuration file carries a first identifier of source data to be migrated and a corresponding relation between the identifier of the source data to be migrated and a second migration rule corresponding to the identifier;
executing the second target program, processing the source data of the first identifier by adopting the second migration rule, and migrating the second target data obtained after processing to a target database.
7. The method according to claim 1 or 6, characterized in that the configuration file also carries a target data table identifier in the target database to be migrated;
the migrating the processed target data to the target database comprises:
and migrating the processed target data to a target data table corresponding to the target data table identification in the target database.
8. A data migration apparatus, comprising:
the receiving module is used for receiving the generated first configuration file, wherein the first configuration file carries a first identifier of the source data to be migrated and a corresponding relation between the first identifier of the source data to be migrated and a corresponding first migration rule;
the migration module is used for determining a parameter position corresponding to the first configuration file from a pre-stored data migration program, writing the first configuration file into the corresponding parameter position, and generating a first target program; and running the first target program, processing the source data of the first identifier by adopting the first migration rule, and migrating the processed first target data to a target database.
9. The apparatus of claim 8, further comprising:
the processing module is used for receiving a second identifier of a source database where the selected source data to be migrated is located; identifying a target database type corresponding to the source database of the second identifier, and displaying a first candidate migration rule corresponding to the target database type according to a corresponding relation between a prestored database type and a migration rule; receiving an input first identifier of source data to be migrated; receiving a migration rule from the first candidate
And generating a first configuration file containing the first identifier of the source data and the corresponding relation between the identifier of the source data to be migrated and the migration rule corresponding to the identifier.
10. The apparatus of claim 9, wherein the processing module is further configured to determine whether the first identified source data is primary key data; if yes, displaying a second candidate migration rule corresponding to the primary key data according to a corresponding relation between the pre-stored primary key data and the migration rule;
the migration module is specifically configured to receive a migration rule corresponding to the source data selected from the first candidate migration rule and/or the second candidate migration rule.
11. The apparatus of claim 8, wherein the processing module is further configured to determine that the first target program is a non-executable program if running the first target program does not migrate the source data into a target database.
12. The apparatus of claim 11, wherein the processing module is further configured to output a prompt message prompting a developer to perform secondary development.
13. The apparatus of claim 12, wherein the processing module is further configured to receive a second target program input after the second development; a second configuration file is written in the second target program, and the second configuration file carries a first identifier of the source data to be migrated and a corresponding relation between the identifier of the source data to be migrated and a second migration rule corresponding to the identifier; and executing the second target program, processing the source data of the first identifier by adopting the second migration rule, and migrating the second target data obtained after processing to a target database.
14. The apparatus according to claim 13, wherein the configuration file further carries a target data table identifier in the target database to be migrated;
the migration module is specifically configured to migrate the processed target data to a target data table corresponding to the target data table identifier in the target database.
15. An electronic device, characterized in that the electronic device comprises a processor for implementing the steps of the data migration method according to any one of claims 1 to 7 when executing a computer program stored in a memory.
16. A computer-readable storage medium, characterized in that it stores a computer program which, when being executed by a processor, carries out the steps of the data migration method according to any one of claims 1 to 7.
17. A computer program product, the computer program product comprising: computer program code which, when run on a computer, causes the computer to perform the steps of the data migration method as claimed in any one of claims 1 to 7.
CN202211495843.4A 2022-11-28 2022-11-28 Data migration method, device, equipment and medium Active CN115543969B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211495843.4A CN115543969B (en) 2022-11-28 2022-11-28 Data migration method, device, equipment and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211495843.4A CN115543969B (en) 2022-11-28 2022-11-28 Data migration method, device, equipment and medium

Publications (2)

Publication Number Publication Date
CN115543969A true CN115543969A (en) 2022-12-30
CN115543969B CN115543969B (en) 2023-03-10

Family

ID=84722574

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211495843.4A Active CN115543969B (en) 2022-11-28 2022-11-28 Data migration method, device, equipment and medium

Country Status (1)

Country Link
CN (1) CN115543969B (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070299975A1 (en) * 2006-05-16 2007-12-27 Klaus Daschakowsky Systems and methods for migrating data
US20140208290A1 (en) * 2013-01-22 2014-07-24 Oracle International Corporation Application source code scanning for database migration
CN111177113A (en) * 2019-12-06 2020-05-19 中国平安财产保险股份有限公司 Data migration method and device, computer equipment and storage medium
CN113312336A (en) * 2021-06-22 2021-08-27 中国农业银行股份有限公司 Data processing method, data processing apparatus, electronic device, storage medium, and program product
CN114185873A (en) * 2021-12-17 2022-03-15 平安付科技服务有限公司 Data migration method, device, server and storage medium

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070299975A1 (en) * 2006-05-16 2007-12-27 Klaus Daschakowsky Systems and methods for migrating data
US20140208290A1 (en) * 2013-01-22 2014-07-24 Oracle International Corporation Application source code scanning for database migration
CN111177113A (en) * 2019-12-06 2020-05-19 中国平安财产保险股份有限公司 Data migration method and device, computer equipment and storage medium
CN113312336A (en) * 2021-06-22 2021-08-27 中国农业银行股份有限公司 Data processing method, data processing apparatus, electronic device, storage medium, and program product
CN114185873A (en) * 2021-12-17 2022-03-15 平安付科技服务有限公司 Data migration method, device, server and storage medium

Also Published As

Publication number Publication date
CN115543969B (en) 2023-03-10

Similar Documents

Publication Publication Date Title
US8127237B2 (en) Active business client
US10127218B2 (en) Object templates for data-driven applications
CN106445536B (en) Automatic business design management system
CN110750654A (en) Knowledge graph acquisition method, device, equipment and medium
CN107578217B (en) Working electronic flow autonomous generation method and device and office management system
CN110837356B (en) Data processing method and device
CN111813804A (en) Data query method and device, electronic equipment and storage medium
CN109885290B (en) Application program service description information generation and release method, device and storage medium
CN109299074A (en) A kind of data verification method and system based on templating data base view
CN111784108A (en) Modeling method and device for main data management platform
US20060047723A1 (en) Custom database system and method of building the same
CN111090976B (en) Method and device for generating inspection document based on dynamic data
CN110865806B (en) Code processing method, device, server and storage medium
CN112988576A (en) Method and device for determining software project research and development process
CN115167785B (en) Label-based network disk file management method and device, network disk and storage medium
CN115543969B (en) Data migration method, device, equipment and medium
US20230195792A1 (en) Database management methods and associated apparatus
CN113537954B (en) Project milestone processing method and device, storage medium and electronic equipment
CN111914002B (en) Machine room resource information processing method and device and electronic equipment
CN114896164A (en) Interface optimization method and device, electronic equipment and storage medium
CN104881455B (en) A kind of architectural difference processing method and system based on MYSQL
US20080022258A1 (en) Custom database system and method of building and operating the same
US8631393B2 (en) Custom database system and method of building and operating the same
WO2005089350A2 (en) Custom database system and method of building the same
CN113190463B (en) Code testing method, node and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant