CN111651238A

CN111651238A - System and method for acquiring effective data in virtual machine protection

Info

Publication number: CN111651238A
Application number: CN202010494646.5A
Authority: CN
Inventors: 倪国军; 姚国军; 徐磊; 陈勇铨; 胡军擎
Original assignee: Shanghai Yingfang Software Co ltd
Current assignee: Shanghai Yingfang Software Co ltd
Priority date: 2020-06-03
Filing date: 2020-06-03
Publication date: 2020-09-11

Abstract

The invention discloses a system and a method for acquiring effective data in virtual machine protection, wherein the system comprises: the data processing module is connected with the backup server, acquires configuration and disk data information paths corresponding to the virtual machine snapshots from the backup server, reads all configuration information files and transmits the configuration information files to the backup server, reads disk data blocks by taking the size of a default value as a unit during full or incremental protection, judges whether the read disk data blocks are valid or changed data blocks, and transmits the valid or changed data blocks and offset of the valid or changed data blocks in a disk to the backup server; and the backup server is used for snapshooting the current state of the virtual machine, sending a starting instruction to the data processing module to establish connection, sending the configuration and disk data information path to the data processing module, recording the offset of the effective or changed data block in the disk and storing the effective data block when receiving the effective or changed data block and the offset of the effective or changed data block in the disk sent by the data processing module.

Description

System and method for acquiring effective data in virtual machine protection

Technical Field

The invention relates to the field of computer data backup disaster tolerance, in particular to a system and a method for acquiring effective data in virtual machine protection.

Background

With the rapid development of information technology, in the information-based construction process of each enterprise, the data volume is more and more, the data is more and more important, the requirements for disaster recovery backup are gradually increased, and the disaster recovery backup needs to be protected more rapidly, transmitted less and stored less.

As a platform carried by a Windows server, the Hyper-V virtual platform has the advantages of convenience in deployment, low cost, friendly interface, and the like, but as a result, the Hyper-V virtual platform also has some disadvantages, and most importantly, the Hyper-V virtual platform lacks interface support for querying valid data blocks, where the valid data refers to non-null data in full protection and change data in incremental or differential protection, as shown in fig. 5.

In the prior art, a virtual machine configuration version 5.0 (the highest configuration version supported by a Hyper-V platform on a windows server 2012R 2) is taken as a watershed, and a virtual machine with a version equal to or lower than 5.0 is used, so that an interface cannot acquire an effective data block; above this version of the virtual machine, the interface cannot acquire valid data blocks of the full amount of data, but can acquire changed block data based on snapshots (reference points) (as shown in FIG. 2). With the improvement of interfaces, products of security manufacturers increasingly support Hyper-V virtual platforms, so that Hyper-V is promoted to be favored by users more and more, the use is more and more extensive, more and more virtual machines are deployed, and the security requirement on data security, namely the virtual machines, is more and more urgent.

The security of the virtual machine is performed by the protection of the virtual machine. Generally, protection of a virtual machine can be accomplished in terms of virtual machine backup and restore, virtual machine migration, and virtual machine replication, as shown in fig. 3. The virtual machine backup needs to acquire the configuration and data of the virtual machine and transmit the configuration and data to the storage of another machine for storage; the virtual machine recovery is to transmit the configuration and data of the backed-up virtual machine to a virtual platform and create the same virtual machine as the backup; the virtual machine migration is to migrate a virtual machine from a current virtual platform to the same or another virtual platform, and is a one-time task; the virtual machine replication is to set a plan on the basis of virtual machine migration, periodically replicate data of a source virtual machine to a target virtual machine, keep the source virtual machine and the target virtual machine as consistent as possible, and perform switching to provide uninterrupted service to the outside.

Generally, the data size of the virtual machine configuration information is small, usually in units of Kilobytes (KB). The protection of the virtual machine is mainly the protection of the disk data, the disk data is at least 10GB, and the TB level is more and more common. Currently, Hyper-V disks are divided into three types: as shown in fig. 4, when a fixed-size disk is created, a designated space is allocated and all data is emptied; allocating a small amount of space when the dynamic expansion disk is created, and expanding the space until the size is specified according to the use requirement; the difference disk is a child disk generated by taking a fixed-size disk or a dynamic expansion disk as a parent disk, the most common scenario is virtual machine snapshot, and a disk newly generated in the virtual machine snapshot process is the difference disk and is directed to a snapshot disk (which can be any one of three types of disks, but the root disk must be a fixed-size or dynamic expansion disk). Of these three types of disks, fixed size disks perform best, but there is a large amount of empty data occupying the storage. Null data is one of the data that needs to be de-emphasized.

Virtual machine protection can be divided into full protection, incremental protection and differential protection (as shown in fig. 1) according to different acquired data volumes, wherein the full protection needs to transmit and store all valid data, the incremental protection is performed based on changed data after previous protection (full or another increment), and the differential protection is performed based on changed data after previous full protection, so that, for virtual machine protection, the most important is to acquire valid data blocks, including data blocks in full and changed data blocks in increment or difference,

in the prior art, many of the methods are to install an agent and open a service on a virtual platform host, monitor and record the disk I/O change of the virtual machine, so as to obtain effective data information, but this method has an influence on the performance of the virtual platform host.

Therefore, how to acquire valid data in virtual machine protection so as to perform virtual machine protection in a faster time and with less transmission and less storage is an urgent problem to be solved.

Disclosure of Invention

In order to overcome the defects in the prior art, the present invention provides a system and a method for obtaining valid data in virtual machine protection, so as to obtain valid data for protection in virtual machine protection, so as to perform virtual machine protection in a faster time, with less transmission and less storage.

In order to achieve the above object, the present invention provides a valid data acquisition system in virtual machine protection, which is applied to a hyper-v virtual machine configuration system equal to or lower than version 5.0, and includes:

the data processing module is arranged on the virtual platform host machine and used for disk operation and data transmission on the virtual platform host machine, the data processing module is connected with the backup server, reads all configuration information files according to paths of all files where the configuration information and the disk information which are pointed by the snapshot acquired from the backup server are located and transmits the configuration information files to the backup server, reads disk data blocks by taking the size of a default value as a unit during full protection, judges whether the read disk data blocks are effective data blocks or not, transmits the effective data blocks and offsets of the effective data blocks in a disk to the backup server, calculates hash values of the effective data blocks, and transmits all recorded hash values and the offsets to the backup server after all disk data are read;

the backup server is used for carrying out snapshot on the current state of the virtual machine needing to be protected, obtaining the configuration information pointed by the snapshot and the paths of all files where the disk information is located, sending a starting instruction to the data processing module to start the data processing module, establishing connection with the data processing module, sending the obtained paths to the data processing module after the connection is established, recording the offset of the effective data block in the disk and storing the effective data block into the corresponding storage area when receiving the effective data block sent by the data processing module and the offset of the effective data block in the disk, and receiving and storing the offset information and the hash value information of all the effective data blocks transmitted by the data processing module.

Preferably, the data processing module further comprises:

the connection unit is used for receiving a starting instruction of the backup server, connecting to the backup server according to the starting instruction and establishing connection with the backup server;

the configuration information acquisition and transmission unit is used for reading all configuration information files by using a reading interface of a common file according to the path of the configuration information file pointed by the snapshot acquired from the backup server and transmitting the configuration information files to the backup server;

the system comprises a full-protection effective data acquisition unit, a special disk reading interface, a backup server and a data storage unit, wherein the full-protection effective data acquisition unit is used for reading a disk data block of a disk file by taking the size of a default value as a unit according to a path of the disk file pointed by a snapshot acquired from the backup server, judging whether the read disk data block is an effective data block or not, transmitting the effective data block and an offset of the effective data block in a disk to the backup server if the read disk data block is the effective data block, calculating a hash value of the effective data block, and transmitting all recorded hash values and the offset thereof to the backup server when all disk data are read.

Preferably, the full-protection valid data acquisition unit further comprises:

the disk file opening module is used for opening the disk file according to the path of the file where the disk information is located so as to read corresponding disk data;

the disk data block reading module is used for reading the disk data in the disk file by taking a default value as a unit to obtain a disk data block with the size of the default value;

the effective data block judging module is used for calculating a hash value by dividing the obtained disk data block into a plurality of smaller data blocks, comparing the hash value with the hash value of an empty data block, and determining whether the disk data block is an effective data block according to a comparison result;

the hash value and offset recording module is used for calculating the hash value of the effective data block and recording the hash value of the effective data block and the offset of the effective data block in a disk;

the data transmission module is used for recording the offset of the effective data block in the disk, transmitting the determined effective data block and the offset of the effective data block in the disk to the backup server for data storage, and returning the effective data block and the offset to the disk data block reading module after the storage is finished until all disk data in the disk file are read;

and the hash value and offset transmission module is used for transmitting all the hash values and the offsets recorded by the hash value and offset recording module to the backup server when the disk data block reading module has read all the disk data of the disk file.

Preferably, the valid data block determining module segments the disk data block into a plurality of smaller small data blocks, sequentially compares the hash value of each small data block with the hash value of a null data block of a corresponding size, and if the hash values are equal, continues to compare the next small data block until unequal small data blocks exist, and determines that the corresponding disk data block is a valid data block.

Preferably, the system further includes an incremental protection valid data obtaining unit, configured to receive a hash value of the disk data stored at a reference backup point sent by the backup server, read a disk data block of the disk file in units of the size of the disk data corresponding to the hash value, calculate and record the hash value of the obtained disk data block, compare the hash value of the disk data block with a hash value at the same position in the reference backup point, determine whether the disk data block data is changed data according to a comparison result, transmit the disk data block and an offset thereof in the disk to the backup server when the comparison result is changed data, and transmit all the recorded hash values and offsets thereof to the backup server after all the data of the disk file are completely read.

Preferably, the effective data obtaining unit of incremental protection further comprises:

the reference point acquisition module is used for receiving the disk data hash value which is sent by the backup server and stored by the reference backup point;

the reading module is used for reading the disk data block of the disk file by taking the size of the disk data corresponding to the hash value as a unit according to the path acquired by the configuration information acquisition and transmission unit;

the hash comparison module is used for calculating and recording the obtained hash value of the disk data block and the offset of the disk data block in the disk, comparing the hash value of the disk data block with the hash value at the same position in the reference backup point, and determining whether the data of the disk data block is changed data or not according to the comparison result;

and the comparison processing module is used for transmitting the disk data blocks and the offsets of the disk data blocks in the disk to a backup server when the comparison result of the hash comparison module is changed data, transmitting all recorded hash values and offsets to the backup server after all data of the disk file are read, and returning the read module to read the next disk data block until all data are read when the comparison result is unchanged data.

Preferably, the backup server further comprises:

the snapshot unit is used for carrying out snapshot on the current state of the virtual machine to be protected and obtaining the configuration information pointed by the snapshot and the paths of all files where the disk information is located;

the connection establishing unit is used for sending a starting instruction to the data processing module to start the data processing module when the virtual machine protection is carried out, and establishing connection with the data processing module;

the configuration information sending unit is used for sending the acquired path of all files where the configuration information and the disk information which are pointed by the snapshot are located to the data processing module after the connection is established with the data processing module;

and the storage unit is used for receiving and storing the configuration information file sent by the data processing module, recording the offset of the effective data block in the disk and storing the effective data block into a corresponding storage area when receiving the effective data block sent by the data processing module and the offset of the effective data block in the disk, and receiving and storing the offset information and the hash value information of all the effective data blocks transmitted by the data processing module.

In order to achieve the above object, the present invention further provides a method for obtaining valid data in virtual machine protection, which is applied to a hyper-v virtual machine configuration system equal to or lower than version 5.0, and includes the following steps:

step S1, the data processing module arranged on the virtual platform host computer is started according to the starting instruction of the backup server, and establishes connection with the backup server, after establishing connection, reads all the configuration information files and transmits the configuration information files to the backup server according to the path of the configuration information files pointed by the snapshot acquired by the backup server;

step S2, during full protection, according to the path of the disk file pointed by the snapshot acquired from the backup server, reading the disk data block of the disk file with the size of the default value as a unit, and judging whether the read disk data block is an effective data block, if so, transmitting the effective data block and the offset of the effective data block in the disk to the backup server, calculating the hash value of the effective data block, and when all disk data are read, transmitting all recorded hash values and the offset thereof to the backup server;

step S3, during incremental protection, the data processing module receives the hash value of the disk data saved at the reference backup point sent by the backup server, reads the disk data block in units of the size of the disk data corresponding to the hash value, calculates and records the obtained hash value of the disk data block, compares the hash value of the disk data block with the hash value at the same position in the reference backup point, determines whether the disk data block data is changed data according to the comparison result, transmits the disk data block and its offset in the disk to the backup server when the comparison result is changed data, and transmits all the recorded hash values and their offsets to the backup server when all the disk data are read.

In order to achieve the above object, the present invention further provides a system for obtaining valid data in virtual machine protection, which is applied to a hyper-v virtual machine configuration system with a version greater than 5.0, and includes:

the data processing module is arranged on a virtual platform host machine and used for disk operation and data transmission on the virtual platform host machine, starting according to a starting instruction of a backup server, establishing connection with the backup server, reading all configuration information files according to configuration information pointed by a snapshot acquired from the backup server and paths of all files where the disk information is located, transmitting the configuration information files to the backup server, reading disk data blocks by taking the size of a default value as a unit during full protection, judging whether the read disk data blocks are effective data blocks or not, and transmitting the effective data blocks and offset of the data blocks in a disk to the backup server; during incremental protection, receiving incremental change block information sent by a backup server, acquiring change data of a corresponding position of a disk based on the incremental change block information, and sending the change data to the backup server for backup;

the backup server is used for carrying out snapshot on the current state of the virtual machine needing to be protected, obtaining paths of all files where the configuration information and the disk information which are pointed by the snapshot are located, sending a starting instruction to the data processing module to start the data processing module, establishing connection with the data processing module, sending the obtained paths of all files where the configuration information and the disk information which are pointed by the snapshot are located to the data processing module after the connection is established, recording the offset of the effective data block in the disk and storing the effective data block into a corresponding storage area when the effective data block and the offset of the effective data block in the disk which are sent by the data processing module are received, recording the snapshot ID of the backup, and obtaining incremental change block information according to the snapshot ID of the last backup and transmitting the incremental change block information to the data processing module during incremental protection.

In order to achieve the above object, the present invention further provides a method for obtaining valid data in virtual machine protection, which is applied to a hyper-v virtual machine configuration system with a version greater than 5.0, and includes the following steps:

step S1, the data processing module arranged on the virtual platform host computer is started according to the starting instruction of the backup server, and establishes connection with the backup server, after establishing connection, reads all the configuration information files and transmits to the backup server according to the path of all the files where the configuration information and the disk information which are pointed by the snapshot obtained by the backup server are located;

step S2, during full protection, the data processing module reads the disk data block of the disk file by using the size of the default value as a unit, and determines whether the read disk data block is an effective data block, and if the read disk data block is an effective data block, transmits the effective data block and the offset of the data block in the disk to the backup server;

step S3, during incremental protection, receiving incremental change block information obtained by the backup server according to the last backup snapshot ID, obtaining the change data of the corresponding position of the disk based on the change block information, and sending the change data to the backup server for backup;

step S4, when receiving the valid data block and the offset thereof in the disk sent by the data processing module, the backup server records the offset of the valid data block in the disk, stores the valid data block in the corresponding storage area, and records the snapshot ID of the backup.

Compared with the prior art, the effective data acquisition system and the method in the virtual machine protection are started by the data processing module arranged on the host machine of the virtual platform according to the starting instruction of the backup server, and establishing connection with the backup server, acquiring configuration information and disk data information backed up by the backup server for the virtual machine snapshot after establishing connection, reading all configuration information files, transmitting to the backup server, during the full protection, the data processing module reads the disk data block of the disk file by taking the size of the default value as a unit, judges whether the read disk data block is a valid data block or not, if the read disk data block is the valid data block, the valid data block is transmitted to the backup server along with the offset of the block in disk, the invention can carry out virtual machine protection in a faster time and with less transmission and less storage.

Drawings

FIG. 1 is a schematic diagram illustrating classification by data amount in virtual machine protection;

FIG. 2 is a schematic diagram of interface support for different virtual machine configuration versions in the prior art;

FIG. 3 is a schematic diagram of types in virtual machine protection;

FIG. 4 is a schematic diagram of disk classification in Hyper-V;

FIG. 5 is a diagram illustrating the definition of valid data blocks;

FIG. 6 is a system architecture diagram of an effective data acquisition system in virtual machine protection according to the present invention;

FIG. 7 is an architecture diagram of a valid data acquisition system in virtual machine protection according to a first embodiment of the present invention;

fig. 8 is a detailed structure diagram of a full protection valid data acquisition unit according to a first embodiment of the present invention;

fig. 9 is a detailed structure diagram of an effective data obtaining unit of the incremental protection according to the first embodiment of the present invention;

fig. 10 is a flowchart illustrating steps of a method for obtaining valid data in virtual machine protection according to a second embodiment of the present invention;

fig. 11 is a detailed structural diagram of a full-protection valid data acquisition unit according to a third embodiment of the present invention;

fig. 12 is a flowchart illustrating steps of a method for obtaining valid data in virtual machine protection according to a fourth embodiment of the present invention;

FIG. 13 is a flowchart of full protection for obtaining valid data according to an embodiment of the present invention;

fig. 14 is a flowchart of obtaining valid data by incremental protection according to an embodiment of the present invention.

Detailed Description

Other advantages and capabilities of the present invention will be readily apparent to those skilled in the art from the present disclosure by describing the embodiments of the present invention with specific embodiments thereof in conjunction with the accompanying drawings. The invention is capable of other and different embodiments and its several details are capable of modification in various other respects, all without departing from the spirit and scope of the present invention.

For clarity of the acquisition and transmission of the payload data blocks, the following convention is first followed for the following description:

1. "backup" is used in place of "protection" in the following description. Because the backup, migration, and copy in protection all use the same technique to obtain valid blocks and the backup is more understandable, using only the backup is easier to teach to make clear the invention.

2. Because the incremental protection and the differential protection are both protection by taking the changed block data, and the key point is to take the changed block data, the incremental protection and the differential protection can be put together.

3. Because the data volume of the configuration information of the virtual machine is very small, the method can directly call an interface for reading the common binary file to read data.

4. Generally, a virtual machine is provided with a plurality of disks, and the processing of the plurality of disks can be performed simultaneously by using multiple threads, so that the performance of virtual machine protection is accelerated. The multi-thread concurrent processing is not much related to the invention, so the invention assumes that only one disk exists in the virtual machine.

5. The effective data block is obtained under two conditions, namely, under the condition that the configuration version of the virtual machine is more than 5.0, the virtual machine does not provide a query interface of the full effective data block but provides a query interface of the incremental change data block; secondly, the configuration version of the virtual machine is equal to or less than 5.0, no interface for the virtual machine to inquire the data block is provided under the condition, and the second condition can cover the first condition.

Fig. 6 is a system architecture diagram of an effective data acquisition system in virtual machine protection according to the present invention. As shown in fig. 6, in a first embodiment of the present invention, a system for obtaining valid data in virtual machine protection, which is suitable for a case where a virtual machine configuration version is equal to or less than 5.0, includes:

a data processing module 60, which is disposed on the virtual platform host, and is used for disk operation and data transmission on the virtual platform host, starting according to a start instruction of the backup server, establishing connection with the backup server, after establishing connection, reading all configuration information files and transmitting to the backup server by using a reading interface of a general file according to the configuration information pointed by the snapshot acquired from the backup server and the path of all files where the disk information is located, reading the disk data block with the size of a default value as a unit during full protection, and judging whether the read disk data block is an effective data block, if the read disk data block is an effective data block, transmitting the effective data block and the offset of the data block in the disk to the backup server, and calculating the hash (hash) value of the effective data block, when all disk data are read, transmitting all recorded hash values and offsets thereof to a backup server, and storing the offset information and hash value information of all effective data blocks of the backup by the backup server for use in incremental backup and virtual machine recovery; during incremental protection, receiving a disk data hash value stored by a reference backup point and sent by a backup server, reading a disk data block by taking the size of disk data corresponding to the hash value as a unit, calculating and recording the hash value of the acquired disk data block, comparing the hash value of the disk data block with the hash value at the same position in the reference backup point, determining whether the disk data block is changed data or not according to the comparison result, transmitting the disk data block and the offset (position) thereof in the disk to the backup server when the comparison result is changed data, and transmitting all recorded hash values and the offsets thereof to the backup server when all disk data are read. In the embodiment of the present invention, the first backup is generally a full backup, and the subsequent backups are all incremental backups based on the previous backup. Of course, the backup type may be specified by a rule, for example, timing execution is performed, such as performing full backup at 0 o 'clock every Monday, and performing incremental backup at 0 o' clock every Monday and Saturday.

Specifically, as shown in fig. 7, the data processing module 60 further includes:

the connection unit 601 is configured to receive a start instruction of the backup server, connect to the backup server according to the start instruction, and establish a connection with the backup server. In the embodiment of the present invention, the backup server starts the data processing module through WinRM, the start instruction includes connection information (for example, an ip address and a port of the connected backup server, and the connection unit 601 may be connected to the backup server through the ip address and the port of the backup server), and after receiving the start instruction, the connection unit is connected to the backup server according to the connection information to establish a connection with the backup server.

The configuration information acquisition and transmission unit 602 is configured to read all configuration information files using a reading interface of a general file according to a path of the configuration information file pointed by the snapshot acquired from the backup server and transmit the configuration information files to the backup server. That is to say, when performing backup, the backup server first snapshots a virtual machine to be protected through WinRM, and fixes a current state of the virtual machine for backup, where the current state includes configuration information (a virtual machine configuration file, a runtime state file, a virtual machine memory file, etc.) of the virtual machine and disk data information, and after starting the data processing module, the backup server obtains paths of all files where the configuration information and the disk information that the snapshots point to are located, and sends the paths to the data processing module 60, and the configuration information obtaining and transmitting unit 602 uses a reading interface of a general file after obtaining paths of all files where the configuration information and the disk information that the snapshots point to are sent by the backup server are located, and reads all configuration information files and transmits the configuration information files to the backup server for storage.

A full-protection valid data obtaining unit 603, configured to use a dedicated disk reading interface according to a path of a disk file pointed by a snapshot obtained from a backup server, read a disk data block of the disk file in units of a default value, and determine whether the read disk data block is a valid data block, if the read disk data block is a valid data block, transmit the valid data block and an offset of the valid data block in the disk to the backup server, and calculate a hash (hash) value of the valid data block, and when all disk data are read, transmit all recorded hash values and offsets thereof to the backup server.

Specifically, as shown in fig. 8, the full-protection valid data acquiring unit 603 further includes:

the disk file opening module 603a is configured to open a disk file according to a path of a file where the disk information is located to read corresponding disk data. In an embodiment of the present invention, the disk file opening module 603a calls a windows library virtdisk.dll to open a disk file and prepare to start reading disk data.

The disk data block reading module 603b is configured to read the disk data in the disk file in units of default values (for example, in units of 1 MB), and obtain a disk data block of a default value size.

The valid data block determining module 603c is configured to determine whether the disk data block is a valid data block according to a comparison result by dividing the obtained disk data block into a plurality of smaller data blocks and comparing the smaller data blocks with an empty data block. Specifically, the valid data block determining module 603c divides the disk data block into a plurality of smaller small blocks (for example, a disk data block with a size of 1MB is taken as an example, the disk data block is divided into 128 smaller data blocks with 8 KB) and calculates hash values, compares the hash value of each small data block with the hash values of the empty data blocks (8K data of all 0) with corresponding sizes in sequence, if the hash values are equal, continues to compare the next small data blocks until the hash values are not equal, and determines that the disk data block (with a size of 1 MB) is a valid data block and needs to be backed up; and if all the small data blocks are equal to the empty data block, indicating that the disk data block is not a valid data block, and discarding the disk data block. In the embodiment of the present invention, the valid data block determining module 603c compares a small data block of 8KB with a null data (8K data of all 0 s) block, if they are equal, continues to compare the next 8KB until they are not equal, determines that the disk data block is a valid data block and needs to be backed up, and if all 128 data blocks of 8KB are equal, discards the disk data block. In the embodiment of the present invention, system performance and storage are integrated, 1MB of data is selected as the compared disk data block, and if the small data blocks contained therein are all empty, the small data blocks are invalid, and if there are non-empty data blocks, the small data blocks are all considered valid, and of course, data of other sizes may be selected as the compared disk data block, such as 512KB, which is not limited by the present invention.

A hash value and offset recording module 603d, configured to calculate a hash (hash) value of the valid data block, and record the hash value of the valid data block and an offset of the valid data block in the disk. Because the obtained hash values of the same segment of data are the same through a fixed hash (hash) algorithm, while the obtained hash values of different segments of data are different (the hash collision probability is extremely small to be negligible for 256-bit hash values), the hash values of the effective data blocks are calculated, and the method can be used for incremental backup and virtual machine recovery.

The data transmission module 603e is configured to transmit the valid data block and the offset (location) of the valid data block in the disk to the backup server for data storage, and return the valid data block and the offset (location) to the disk data block reading module 603b after the storage is completed until all disk data in the disk file has been read.

The hash value and offset transmission module 603f is configured to, when the disk data block reading module 603b has completely read all disk data of the disk file, transmit all hash values and offsets thereof recorded by the hash value and offset recording module 603d to the backup server, and the backup server stores offset information and hash value information of all valid data blocks of the backup at this time for use in incremental backup and virtual machine recovery.

Therefore, when the backup is finished, all the configuration files, all the effective data blocks and the hash values thereof, and the offset of each effective data block in the disk are stored in the backup server, and meanwhile, the backup server also records other related information of the backup for the next backup to use, including the time point and the unique identifier of the backup.

An incremental protection valid data obtaining unit 604, configured to receive a hash value of disk data saved at a reference backup point sent by the backup server 61, read a disk data block of a disk file in units of the size of the disk data corresponding to the hash value, calculate and record the hash value of the obtained disk data block, compare the hash value of the disk data block with a hash value at the same position in the reference backup point, determine whether the disk data block is changed data according to the comparison result, transmit the disk data block and an offset (position) thereof in the disk to the backup server when the comparison result is changed data, and transmit all recorded hash values and offsets thereof to the backup server after all data of the disk file are read.

Specifically, as shown in fig. 9, the incrementally-protected valid data acquiring unit 604 further includes:

the reference point obtaining module 604a is configured to receive the disk data hash value stored at the reference backup point sent by the backup server 61. That is, the backup server 61 needs to specify a reference backup point, which may be a full backup point or an incremental backup point, and after determining the reference backup point, obtains the hash value of the disk data stored in the reference backup point, and sends the hash value to the reference point obtaining module 604a of the data processing module 60, where the reference backup point generally refers to a previous backup result, and there is a data chain in the backup data, and the data chain is linked from the first backup to the subsequent backup.

The reading module 604b is configured to read the disk data block of the disk file according to the path acquired by the configuration information acquisition and transmission unit 602 by using the size of the disk data corresponding to the hash value as a unit, and in a specific embodiment of the present invention, assuming that the size of the disk data corresponding to the disk data hash value of the reference backup point is 1MB, the reading module 604b reads the disk data by using 1MB as a unit, and acquires a disk data block of 1 MB. The configuration information acquisition transmission unit 602.

The hash comparison module 604c is configured to calculate and record the hash value of the acquired disk data block and the offset of the disk data block in the disk, compare the hash value of the disk data block with the hash value of the same position in the reference backup point (that is, the same offset as the disk data block), and determine whether the data of the disk data block is changed according to the comparison result.

A comparison processing module 604d, configured to transmit the disk data block and the offset (position) of the disk data block in the disk to the backup server when the comparison result of the hash comparison module 604c is changed data, and transmit all recorded hash values and offsets to the backup server after all data of the disk file are completely read, and return to the reading module 604b to read the next disk data block until all data are completely read if the comparison result is unchanged data. That is, when a disk data block of 1MB size is obtained, the hash comparison module 604c calculates and records a 256-bit hash value of the disk data block, obtains a data hash value of a reference backup point at the same offset, if the obtained data hash value is successful and is the same as the hash value calculated this time, it indicates that the current disk data block is an unchanged block, discards the disk data block, and returns to the reading module 604b to read the next data block, otherwise, it indicates that the disk data block is changed data, and transmits the disk data block and the offset (position) of the disk data block in the backup server. Accordingly, the storage unit 613 of the backup server 61 also records the offset of the valid data in the disk and stores the data in the corresponding storage area, and also the format of the stored data can be specified in the configuration of the backup server.

When all disk data are read, the comparing module 604d further transmits all recorded hash values and offsets to the backup server. Accordingly, the storage unit 613 of the backup server stores the offset information and the hash value information of all valid data blocks backed up this time for use in incremental backup and virtual machine recovery.

Therefore, in the invention, the data processing module on the host machine of the virtual platform only performs the operation of the disk of the virtual machine and the data transmission of the disk, and the backup server performs remote management on the virtual platform through WinRM instead of managing the virtual platform. If the virtual platform is managed through the data processing module, a service needs to be started to communicate with the backup server, and the complexity of the system is increased.

The backup server 61 is configured to snapshot a current state of the virtual machine that needs to be protected, obtain configuration information pointed by the snapshot and paths of all files where the disk information is located, send a start instruction to the data processing module 60 to start the data processing module 60 when the virtual machine is protected, establish a connection with the data processing module, send the obtained configuration information pointed by the snapshot and paths of all files where the disk information is located to the data processing module 60 after the connection is established, record an offset of an effective data block in a disk and store the effective data block into a corresponding storage area when the effective data block and an offset (position) thereof in the disk sent by the data processing module 60 are received, receive and store the offset information and hash value information of all effective data blocks transmitted by the data processing module 60, for use in incremental backup and virtual machine recovery.

That is, all the configuration files, all the valid data blocks and their hash values, and the offset in the disk for each backup are saved in the backup server, and the backup server 61 records the information of this backup for the next backup, including the time point and the unique identifier of this backup.

Specifically, the backup server 61 includes:

the snapshot unit 610 is configured to snapshot a current state of the virtual machine that needs to be protected, where the current state includes configuration information (a virtual machine configuration file, a runtime state file, a virtual machine memory file, and the like) of the virtual machine and disk data information, and even if the virtual machine is still running, continuous reading and writing of data does not change the information in the snapshot. In the embodiment of the present invention, the snapshot unit 610 snapshots a virtual machine that needs to be protected through WinRM, fixes the current state of the virtual machine for backup, and obtains paths of all files where configuration information and disk information that are pointed by the snapshots are located.

The connection establishing unit 611 is configured to send a start instruction to the data processing module 60 to start the data processing module 60 when performing virtual machine protection, and establish a connection with the data processing module 60. The connection establishing unit 611 starts the data processing module 60 through WinRM, and the start instruction includes connection information (ip address, port, etc.) as parameters, so that the data processing module 60 can connect to the backup server according to the connection information to establish a connection with the backup server.

The path information sending unit 612 is configured to send paths of all files where the configuration information and the disk information pointed by the snapshot are located to the data processing module 60 after establishing connection with the data processing module 60. That is to say, when the snapshot unit 610 takes a snapshot of the current state of the virtual machine that needs to be protected, the path information of all files where the configuration information and the disk information that are pointed by the snapshot are located is obtained, and after the connection is established with the data processing module 60, the path information sending unit 612 sends the path information of all files where the configuration information and the disk information that are pointed by the obtained snapshot are located to the data processing module 60.

The storage unit 613 is configured to receive and store the configuration information file sent by the data processing module 60, and when receiving the valid data block sent by the data processing module 60 and its offset (location) in the disk, record the offset of the valid data block in the disk and store the valid data block into a corresponding storage area, where the format of the stored data may be specified in the configuration of the backup server, for example, if the backup server also has a hyper-v virtual platform, the stored data may be stored in a virtual disk format, so as to create a virtual machine by directly using the data, which is also a basic principle of instant recovery. Meanwhile, when the data processing module 60 finishes reading all disk data, and transmits all recorded hash values and offsets thereof to the backup server, the storage unit 613 stores the offset information and hash value information of all valid data blocks of the current backup for use in incremental backup and virtual machine recovery. Similarly, all configuration files, all changed data blocks and their hash values, and the offset in disk for each backup are saved to the backup server. The backup server records the information of the backup for the next backup, including the time point and the unique identifier of the backup.

Fig. 10 is a flowchart illustrating steps of a method for obtaining valid data in virtual machine protection according to the present invention. As shown in fig. 10, in a second embodiment of the present invention, a method for obtaining valid data in virtual machine protection is applicable to a case that a virtual machine configuration version is less than 5.0, and includes the following steps:

step S1, the data processing module disposed on the virtual platform host is started according to the start instruction of the backup server, and establishes a connection with the backup server, and after establishing the connection, reads all the configuration information files and transmits them to the backup server according to the path of all the files where the configuration information and the disk information that the snapshot obtained by the backup server points to.

Specifically, step S1 further includes:

step S100, the backup server performs snapshot on the current state of the virtual machine to be protected, and obtains the path of all files where the configuration information and the disk information pointed by the snapshot are located, where the current state includes the configuration information of the virtual machine (virtual machine configuration file, running state file, virtual machine memory file, etc.) and the disk data information, and these information in the snapshot will not be changed even if the virtual machine is still running and there is continuous reading and writing of data. In the embodiment of the invention, the backup server takes a snapshot of the virtual machine to be protected through WinRM, and fixes the current state of the virtual machine for backup.

Step S101, the backup server sends a starting instruction to the data processing module to start the data processing module to establish connection with the data processing module. Specifically, the backup server starts the data processing module through WinRM, and the start instruction includes connection information (ip address, port, and the like) as parameters, so that the data processing module can be connected to the backup server according to the connection information to establish a connection with the backup server.

Step S102, after establishing connection with the data processing module, the backup server sends the obtained configuration information pointed by the snapshot and the paths of all files where the disk information is located to the data processing module.

Step S103, the data processing module reads all configuration information files using a general file interface according to the path of the configuration information file pointed by the snapshot obtained from the backup server and transmits the configuration information files to the backup server.

Step S2, during full protection, the data processing module reads the disk data block of the disk file using the dedicated disk reading interface according to the path of the disk file pointed by the snapshot acquired from the backup server, using the default value as a unit, and determines whether the read disk data block is a valid data block, if the read disk data block is a valid data block, transmits the valid data block to the backup server together with the offset of the data block in the disk, and calculates the hash (hash) value of the valid data block, when all disk data are read, transmits all recorded hash values and their offsets to the backup server, and the backup server stores the offset information and hash value information of all valid data blocks of the backup for incremental backup and virtual machine recovery.

Specifically, step S2 further includes:

step S200, the data processing module opens the disk file to read the corresponding disk data. In the embodiment of the invention, the data processing module calls a windows library virtdisk.

Step S201, reading the disk data in the disk file by taking a default value (for example, 1MB as a unit) as a unit, and obtaining a disk data block with a size of the default value;

step S202, dividing the obtained disk data block into a plurality of smaller data blocks and comparing the smaller data blocks with the empty data block, and determining whether the disk data block is a valid data block according to the comparison result. In the specific embodiment of the present invention, a disk data block is divided into a plurality of smaller small blocks (for example, a disk data block with a size of 1MB is taken as an example, the disk data block is divided into 128 smaller data blocks with 8 KB) and hash values are calculated, the hash values of the small data blocks are sequentially compared with the hash values of empty data blocks (8K data of all 0) with corresponding sizes, if the hash values are equal, the next small data block is continuously compared until the hash values are not equal, and the disk data block (with a size of 1 MB) is determined to be a valid data block and needs to be backed up; and if the hash values of all the small data blocks are equal to the hash value of the empty data block, indicating that the disk data block is not a valid data block, and discarding the disk data block. In the embodiment of the present invention, the valid data block determining module 603c compares the hash values of 8KB as a small data block with a null data (8K data of all 0 s), if the hash values are equal, continues to compare the next 8KB until the hash values are not equal, determines that the disk data block is a valid data block and needs to be backed up, and if all 128 data blocks of 8KB are equal, discards the disk data block.

Step S203, calculating a hash (hash) value of the valid data block, and recording the hash value of the valid data block and an offset of the valid data block in the disk. Because the obtained hash values of the same segment of data are the same through a fixed hash (hash) algorithm, while the obtained hash values of different segments of data are different (the hash collision probability is extremely small to be negligible for 256-bit hash values), the hash values of the effective data blocks are calculated, and the method can be used for incremental backup and virtual machine recovery.

Step S204, the valid data block and the offset (position) of the valid data block in the disk are transmitted to a backup server for data storage, and the step S201 is returned after the storage is completed until all the disk data in the disk file are read.

Step S205, when the data processing module finishes reading all the disk data of the disk file, all the recorded hash values and offsets thereof are transmitted to the backup server, and the backup server stores the offset information and hash value information of all the valid data blocks of the current backup for use in incremental backup and virtual machine recovery.

Step S3, during incremental protection, receiving a hash value of the disk data stored at a reference backup point sent by a backup server, reading a disk data block in units of the size of the disk data corresponding to the hash value, calculating and recording the hash value of the acquired disk data block, comparing the hash value of the disk data block with the hash value at the same position in the reference backup point, determining whether the disk data block data is changed data according to the comparison result, transmitting the disk data block and its offset (position) in the disk to the backup server when the comparison result is changed data, and transmitting all recorded hash values and their offsets to the backup server when all disk data are read.

Specifically, step S3 further includes:

step S300, receiving the hash value of the disk data saved at the reference backup point sent by the backup server. That is, a reference backup point, which may be a full backup point or an incremental backup point, needs to be specified at the backup server, and after the reference backup point is determined, the hash value of the disk data stored at the reference backup point is obtained and sent to the data processing module.

Step S301, reading the disk data block of the disk file by using the size of the disk data corresponding to the hash value as a unit according to the path obtained in step S1, in a specific embodiment of the present invention, assuming that the size of the disk data corresponding to the disk data hash value of the reference backup point is 1MB, reading the disk data by using 1MB as a unit, and obtaining the disk data block of 1 MB.

Step S302, calculating and recording the hash value of the acquired disk data block and the offset of the disk data block in the disk, comparing the hash value of the disk data block with the hash value of the same position in the reference backup point (i.e. the same offset as the disk data block), and determining whether the data of the disk data block is changed according to the comparison result.

Step S303, when the comparison result in step S302 is changed data, transmitting the disk data block and the offset (position) thereof in the disk to the backup server, and after all data of the disk file are completely read, transmitting all recorded hash values and offsets to the backup server, if the comparison result is unchanged data, returning to step S301 to read the next disk data block until all data are completely read. That is, when a disk data block of 1MB size is obtained, a 256-bit hash value of the disk data block is calculated and recorded, a data hash value of a reference backup point at the same offset is obtained, if the obtaining is successful and the hash value calculated this time is the same, it indicates that the current disk data block is an unchanged block, the disk data block is discarded, and the step S301 is returned to read the next data block, otherwise, it indicates that the disk data block is changed data, and the disk data block and the offset (position) of the disk data block in the disk are transmitted to the backup server.

In the invention, for the system with the virtual machine configuration version more than 5.0 in the Hyper-V virtual platform, because the interface for acquiring the incremental change block is provided, the method can be slightly modified and realized based on the method. For a system with a virtual machine configuration version larger than 5.0 in a Hyper-V virtual platform, incremental data are not obtained by comparing hash values of previous backups but by inquiring snapshots of the previous backups, so that parts to be modified mainly remove relevant part contents of the hash values of effective data blocks, including calculation, comparison, transmission and storage of the hash values; the snapshot ID of the last backup is saved.

Therefore, specifically, in the third embodiment of the present invention, for a system in which a virtual machine configuration version in a Hyper-V virtual platform is greater than 5.0, the system for acquiring valid data in virtual machine protection includes:

the data processing module is arranged on the virtual platform host machine and used for disk operation and data transmission on the virtual platform host machine, the data processing module is started according to a starting instruction of the backup server and is connected with the backup server, after the connection is established, all configuration information files are read by using a reading interface of a general file and transmitted to the backup server according to configuration information pointed by a snapshot acquired from the backup server and paths of all files where the disk information is located, during full protection, a disk data block is read by taking the size of a default value as a unit, whether the read disk data block is an effective data block or not is judged, and if the read disk data block is the effective data block, the effective data block and the offset of the data block in a disk are transmitted to the backup server; during incremental protection, receiving incremental change block information (the incremental change block information includes offset and length of change data and does not include the change data) acquired by a backup server according to a snapshot ID of the last backup, acquiring the change data at the corresponding position of a disk based on the incremental change block information, and sending the change data to the backup server for backup.

Specifically, the data processing module includes:

and the connecting unit is used for receiving a starting instruction of the backup server, connecting to the backup server according to the starting instruction and establishing connection with the backup server. In the embodiment of the invention, the backup server starts the data processing module through WinRM. Since the connection unit is the same as that of the first embodiment, it is not described herein.

And the configuration information acquisition and transmission unit is used for reading all configuration information files and transmitting the configuration information files to the backup server according to the paths of all files where the configuration information and the disk information which are pointed by the snapshot acquired from the backup server are located. Since the configuration information acquisition and transmission unit is the same as the connection unit of the first embodiment, it is not described herein again

And the full-protection effective data acquisition unit is used for reading the disk data block of the disk file by taking the size of the default value as a unit, judging whether the read disk data block is an effective data block, and if the read disk data block is the effective data block, transmitting the effective data block and the offset of the effective data block in the disk to the backup server.

Specifically, as shown in fig. 11, in this embodiment, the full-protection valid data acquiring unit further includes:

and the disk file opening module 103a is configured to open a disk file according to a path of a file where the disk information is located to read corresponding disk data. In an embodiment of the present invention, the disk file opening module 103a calls a windows library virtdisk.dll to open a disk file and prepare to start reading disk data.

The disk data block reading module 103b is configured to read the disk data in the disk file in units of default values (for example, in units of 1 MB), and obtain a disk data block of a default value size.

And the valid data block judging module 103c is configured to calculate a hash value by dividing the obtained disk data block into a plurality of smaller data blocks, compare the hash value with the hash value of an empty data block, and determine whether the disk data block is a valid data block according to a comparison result. Since the valid data block determining module 103c is the same as the valid data block determining module of the first embodiment, it is not described herein again.

The data transmission module 103d is configured to transmit the valid data block and the offset (position) of the valid data block in the disk to the backup server for data storage, and return the valid data block and the offset (position) to the disk data block reading module 103b after the data storage is completed until all disk data in the disk file has been read.

And an incremental protection valid data acquiring unit 104, configured to receive change block information (the change block information includes an offset and a length of the change data and does not include the change data itself) sent by the backup server, and acquire, based on the change block information, the change data in the corresponding position of the disk and send the change data to the backup server for backup.

The backup server is used for carrying out snapshot on the current state of the virtual machine needing to be protected, obtaining the paths of all files where the configuration information and the disk information which are pointed by the snapshot are located, sending a starting instruction to the data processing module to start the data processing module, establishing connection with the data processing module, sending the obtained configuration information pointed by the snapshot and the path of all files where the disk information is located to the data processing module after establishing connection, when receiving the effective data block sent by the data processing module and the offset (position) of the effective data block in the disk, recording the offset of the effective data block in the disk, storing the effective data block into the corresponding storage area, recording the snapshot ID of the backup, and during the incremental protection, and sending the incremental change block information acquired according to the snapshot ID of the last backup to the data processing module.

Specifically, the backup server includes:

the snapshot unit is configured to snapshot a current state of the virtual machine that needs to be protected, where the current state includes configuration information (a virtual machine configuration file, a running state file, a virtual machine memory file, and the like) of the virtual machine and disk data information, and the information in the snapshot is not changed even if the virtual machine is still running and data is continuously read and written. In the specific embodiment of the present invention, the snapshot unit snapshots a virtual machine that needs to be protected through WinRM, fixes the current state of the virtual machine for backup, and obtains the path of all files where configuration information and disk information that are pointed by the snapshot are located.

And the connection establishing unit is used for sending a starting instruction to the data processing module to start the data processing module when the virtual machine protection is carried out, and establishing connection with the data processing module. The connection establishing unit starts the data processing module through WinRM, and the starting instruction comprises connection information (IP address, port and the like) as parameters, so that the data processing module can be connected to the backup server according to the connection information to establish connection with the backup server.

And the path information sending unit is used for sending paths of all files where the configuration information and the disk information which are pointed by the snapshot are located to the data processing module after the connection is established with the data processing module. That is to say, when the snapshot unit takes a snapshot of the current state of the virtual machine that needs to be protected, the path information sending unit will obtain the path of all files where the configuration information and the disk information that the snapshot points to are located.

And the storage recording unit is used for receiving and storing the configuration information file sent by the data processing module, recording the offset of the effective data block in the disk and storing the effective data block into a corresponding storage area when the effective data block sent by the data processing module and the offset (position) of the effective data block in the disk are received, wherein the format of the storage data can be specified in the configuration of the backup server, for example, if the backup server is also provided with a hyper-v virtual platform, the storage data can be stored in a virtual disk format, so that the virtual machine can be created by directly using the data, which is also the basic principle of instant recovery. Meanwhile, the storage recording unit also records the snapshot ID of the current backup for use in incremental backup and virtual machine recovery;

and the incremental change block information acquisition and sending unit is used for sending the incremental change block information acquired according to the snapshot ID backed up last time to the data processing module during incremental protection.

In the present invention, the snapshot ID at the time of full protection is stored in the backup server at the time of full protection. But if the user selects the incremental protection and the full protection is not done before, the protection will automatically be changed to the full protection.

Fig. 12 is a flowchart illustrating steps of a method for obtaining valid data in virtual machine protection according to a fourth embodiment of the present invention. As shown in fig. 12, in a fourth embodiment of the present invention, a method for obtaining valid data in virtual machine protection, which is applied to a case that a virtual machine configuration version is greater than 5.0, includes the following steps:

step S1, the data processing module disposed on the virtual platform host is started according to the start instruction of the backup server, and establishes a connection with the backup server, and after establishing the connection, reads all the configuration information files and transmits them to the backup server according to the path of all the files where the configuration information and the disk information that the snapshot obtained by the backup server points to. Since step S1 is the same as step S1 of the second embodiment, it is not repeated here.

Step S2, during full protection, the data processing module reads the disk data block of the disk file by using the default size as a unit, and determines whether the read disk data block is an effective data block, and if the read disk data block is an effective data block, transmits the effective data block and the offset of the data block in the disk to the backup server.

Specifically, step S2 further includes:

step S202, dividing the obtained disk data block into a plurality of smaller data blocks and comparing the smaller data blocks with the empty data block, and determining whether the disk data block is a valid data block according to the comparison result.

Step S203, the valid data block and the offset (position) of the valid data block in the disk are transmitted to the backup server for data storage, and the step S201 is returned after the storage is completed until all the disk data in the disk file are read.

Step S3, during incremental protection, receiving change block information sent by the backup server (the change block information includes offset and length of change data, and does not include the change data itself), and obtaining change data at a corresponding location of the disk based on the change block information and sending the change data to the backup server for backup.

In step S4, when receiving the valid data block and the offset (location) of the valid data block in the disk sent by the data processing module, the backup server records the offset of the valid data block in the disk, stores the valid data block in the corresponding storage area, and records the snapshot ID of the backup.

Example 1

In this embodiment, for example, a system in which the configuration version of a virtual machine in a Hyper-V virtual platform is less than 5.0 is taken as an example, to protect the virtual machine in the Hyper-V virtual platform, a remote management (WinRM) service of a Windows system needs to be started. And requires the use of Hyper-V's snapshot functionality and the library virtdisk.dll provided by Windows systems to operate virtual disks.

For full protection, as shown in fig. 13, valid data acquisition is performed according to the following steps:

step 1, the backup server takes a snapshot of the virtual machine to be protected through WinRM, and fixes the current state of the virtual machine for backup. The current state includes configuration information of the virtual machine (virtual machine configuration file, runtime state file, virtual machine memory file, etc.) and disk data information. The information in the snapshot is not changed even if the virtual machine is still in operation and the data is continuously read and written.

And 2, starting a data processing module by the backup server through WinRM, and taking connection information (IP addresses, ports and the like) as parameters.

And 3, connecting the data processing module to the backup server according to the connection information, and establishing connection with the backup server.

And 4, the backup server obtains the paths of all files where the configuration information and the disk information which are pointed by the snapshot are located and sends the paths to the data processing module.

And 5, the data processing module transmits a reading interface using a common file, reads all the configuration information files and transmits the configuration information files to the backup server for storage.

And 6, calling the windows library virtdisk.dll by the data processing module to open the disk file and prepare to start reading the disk data.

And 7, reading the disk data by the data processing module by taking the default value 1MB as a unit. The size of the once read block can be adjusted according to the configuration, and if the data has been read, step 12 is entered.

And 8, the data processing module obtains a block of data with the size of 1MB, compares the block of data with a null data (8K data of all 0) by taking 8KB as a small block, continuously compares the next 8KB until the data is not equal to the null data, judges that the data block is an effective data block and needs to be backed up, and enters step 9. If all 128 8KB data blocks are equal, then the block of data is discarded. Proceed to step 7.

Step 9, the data processing module calculates the 256-bit hash value of the 1MB valid data block and records the offset of the data in the disk. The role of the hash value is explained: the same segment of data is subjected to a fixed hash algorithm, so that the obtained hash values are the same, and the obtained hash values are different for different data (for 256-bit hash values, the hash collision probability is negligibly small).

Step 10, the data processing module transmits the 1MB valid data block to the backup server along with the offset (location) of the data block in the disk.

And step 11, recording the offset of the effective data in the disk by the backup server and storing the data in a storage. The format in which the data is stored may be specified in the backup server configuration. If the backup server also carries a hyper-v virtual platform, it can be stored in a virtual disk format to facilitate creating a virtual machine directly using this data. This is also the basic principle of transient recovery. Jump back to step 7.

In step 12, the data processing module has read the disk data, and now transmits all the hash values and their offsets recorded in step 9 to the backup server.

And step 13, the backup server stores the offset information and the hash value information of all the valid data blocks of the backup for use in incremental backup and virtual machine recovery.

Step 14, all configuration files, all valid data blocks and their hash values, and the offset in the disk are saved to the backup server. The backup server records the information of the backup for the next backup, including the time point and the unique identifier of the backup.

And step 15, disconnecting the data processing module from the backup server and quitting, and ending the backup.

In this embodiment, as shown in fig. 14, for incremental protection (a reference backup point needs to be specified, which may be a full backup point or an incremental backup point), effective data acquisition is performed according to the following steps:

the steps 1-5 are the same as the full protection, and are skipped here.

And 6, the backup server finds the reference backup point, obtains the disk data hash value stored by the backup point and sends the disk data hash value to the data processing module.

And 7, the data processing module receives the disk data hash value of the reference backup point, calls a windows library virtdisk.

And 8, reading the disk data by the data processing module by taking 1MB as a unit, wherein the size of the once read block can be adjusted according to the configuration. If the data has been read, step 12 is entered.

And 9, the data processing module obtains a block of data with the size of 1MB, calculates and records the 256-bit hash value of the data block, acquires the hash value of the data backed up last time on the same offset, if the data is successfully acquired and is the same as the hash value calculated this time, the data is an unchanged block, discards the block of data, and returns to the step 8. Otherwise, the block data is changed data, and the next step 10 is performed.

Step 10, the data processing module transmits the 1MB of changed data to the backup server along with the offset (location) of the block of data in the disk.

And 11, recording the offset of the effective data in the disk by the backup server and storing the data in a storage, wherein the format of the storage data can be specified in the configuration of the backup server. Jump back to step 8.

And step 12, if the data processing module has read the disk data at this time, transmitting all the hash values and the offsets thereof recorded in the step 9 to the backup server.

Step 14, all configuration files, all changed data blocks and their hash values, and the offset in the disk are saved to the backup server. The backup server records the information of the backup for the next backup, including the time point and the unique identifier of the backup.

Examples2

In this embodiment, for a system in which the virtual machine configuration version in the Hyper-V virtual platform is greater than 5.0, because an interface for acquiring the incremental change block is provided, the implementation may be slightly modified based on the above embodiments. Because the incremental data is not obtained by comparing based on the hash value of the last backup but is obtained by inquiring based on the snapshot of the last backup, the part to be modified is to remove the content of the relevant part of the hash value, including the calculation, comparison, transmission and storage of the hash value; the snapshot ID of the last backup is saved.

In the full backup, basically consistent with the above full backup, the calculation, transmission and storage of the hash value of the data block which needs to be removed are needed, and what needs to be added is to record the snapshot ID of the backup, so as to inquire the changed data block based on the ID when the snapshot ID is increased.

In the incremental backup, the backup server needs to obtain the snapshot ID of the last backup, then calls a WinRM interface to query incremental change block information (the offset and the length of the change data, and the change data itself is not included), and then sends the change block information to the data processing module. And the data processing module acquires the changed data from the virtual machine disk for backup based on the changed block information.

Therefore, the effective protection of the virtual machine in most Hyper-V virtual platforms can be realized by obtaining the effective data blocks, and no matter what configuration version, what software and hardware configuration and how many disks the virtual machine is, the method can be used for backup, recovery, migration and copying. If the backup server is also a Hyper-V virtual platform, the disk data can be backed up into a disk format, and if necessary, the backup virtual machine can be pulled up in the backup server immediately (at the level of seconds), and the integrity of the backup data can be checked or the pulled virtual machine can be directly used.

In summary, the system and method for obtaining valid data in virtual machine protection according to the present invention is started by a data processing module disposed on a host of a virtual platform according to a start instruction of a backup server, and establishing connection with the backup server, acquiring configuration information and disk data information backed up by the backup server for the virtual machine snapshot after establishing connection, reading all configuration information files, transmitting to the backup server, during the full protection, the data processing module reads the disk data block of the disk file by taking the size of the default value as a unit, judges whether the read disk data block is a valid data block or not, if the read disk data block is the valid data block, the valid data block is transmitted to the backup server along with the offset of the block in disk, the invention can carry out virtual machine protection in a faster time and with less transmission and less storage.

The foregoing embodiments are merely illustrative of the principles and utilities of the present invention and are not intended to limit the invention. Modifications and variations can be made to the above-described embodiments by those skilled in the art without departing from the spirit and scope of the present invention. Therefore, the scope of the invention should be determined from the following claims.

Claims

1. A valid data acquisition system in virtual machine protection is applied to a hyper-v virtual machine configuration system equal to or lower than 5.0 version, and comprises:

the data processing module is arranged on the virtual platform host machine and used for processing disk operation and data transmission on the virtual platform host machine, the data processing module is connected with the backup server, reads all configuration information files according to paths of all files where configuration information and disk information which are pointed by snapshots acquired from the backup server are located and transmits the configuration information files to the backup server, reads disk data blocks by taking the size of a default value as a unit during full protection, judges whether the read disk data blocks are effective data blocks or not, transmits the effective data blocks and offsets of the effective data blocks in a disk to the backup server, calculates hash values of the effective data blocks, and transmits all recorded hash values and the offsets to the backup server when all disk data are read;

2. The system for acquiring valid data under virtual machine protection according to claim 1, wherein the data processing module further includes:

3. The system for acquiring valid data under virtual machine protection according to claim 2, wherein the full-protection valid data acquiring unit further includes:

4. The system for acquiring valid data under virtual machine protection according to claim 3, wherein: the effective data block judging module divides the disk data block into a plurality of smaller small data blocks, compares the hash value of each small data block with the hash value of the empty data block with the corresponding size in sequence, if the hash values are equal, continuously compares the next small data block until the unequal small data blocks exist, and judges that the corresponding disk data block is the effective data block.

5. The system for acquiring valid data under virtual machine protection according to claim 2, wherein said system further comprises an incrementally protected valid data acquisition unit, the backup server is used for receiving the disk data hash value stored by the reference backup point and sent by the backup server, reading the disk data block of the disk file by taking the size of the disk data corresponding to the hash value as a unit, calculating and recording the hash value of the acquired disk data block, comparing the hash value of the disk data block with the hash value at the same position in the reference backup point, determining whether the data block of the disk data block is changed data according to the comparison result, transmitting the data block of the disk and the offset of the data block in the disk to a backup server when the comparison result is changed data, and after all the data of the disk file are read, transmitting all the recorded hash values and the offsets thereof to a backup server.

6. The system for acquiring valid data under virtual machine protection according to claim 5, wherein the incrementally-protected valid data acquiring unit further includes:

7. The system for acquiring valid data under virtual machine protection according to claim 5, wherein the backup server further comprises:

8. A method for obtaining effective data in virtual machine protection is applied to a hyper-v virtual machine configuration system with version 5.0 or below, and comprises the following steps:

9. A valid data acquisition system in virtual machine protection is applied to a hyper-v virtual machine configuration system with version more than 5.0, and comprises:

the backup server is used for carrying out snapshot on the current state of the virtual machine needing to be protected, obtaining the paths of all files where the configuration information and the disk information which are pointed by the snapshot are located, sending a starting instruction to the data processing module to start the data processing module, establishing connection with the data processing module, sending the obtained configuration information pointed by the snapshot and the path of all files where the disk information is located to the data processing module after the connection is established, and when receiving the effective data block sent by the data processing module and the offset of the effective data block in the disk, recording the offset of the effective data block in the disk, storing the effective data block into the corresponding storage area, recording the snapshot ID of the backup, and during the incremental protection, and obtaining incremental change block information according to the snapshot ID of the last backup and transmitting the incremental change block information to the data processing module.

10. A method for obtaining effective data in virtual machine protection is applied to a hyper-v virtual machine configuration system with version more than 5.0, and comprises the following steps: