CN107885616A - A kind of mass small documents back-up restoring method based on file system parsing - Google Patents
A kind of mass small documents back-up restoring method based on file system parsing Download PDFInfo
- Publication number
- CN107885616A CN107885616A CN201710904206.0A CN201710904206A CN107885616A CN 107885616 A CN107885616 A CN 107885616A CN 201710904206 A CN201710904206 A CN 201710904206A CN 107885616 A CN107885616 A CN 107885616A
- Authority
- CN
- China
- Prior art keywords
- backup
- file system
- data block
- backed
- recovery
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1448—Management of the data involved in backup or backup restore
- G06F11/1451—Management of the data involved in backup or backup restore by selection of backup contents
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1461—Backup scheduling policy
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1469—Backup restoration techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/1734—Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention relates to a kind of mass small documents back-up restoring method based on file system parsing, including backup submethod and recovery submethod, wherein, the backup submethod includes：Backup tasks are obtained, selection uses complete backup mode or incremental backup mode, realizes and backs up based on CDP monitoring technologies；The recovery submethod includes：Recovery tasks are obtained, select fine granularity reset mode or whole roll recovery mode, by the parsing to file system, realize the recovery for specifying data.Compared with prior art, the present invention can back up to whole file system, remove from and large amount of small documents either on or off is operated, and reduce the IOPS of disk, improve backup performance；File system can be parsed when recovering simultaneously, meet the needs of specified file or catalogue are recovered；And incremental backup is carried out by CDP technologies, reduces backup data quantity, saves backup space.
Description
Technical field
The present invention relates to computer information storage technology field, more particularly, to a kind of magnanimity based on file system parsing
Small documents back-up restoring method.
Background technology
Today's society is in the epoch of information explosion, and data are rapidly increasing.For many business units, daily all
Heap file generation is had, and these file datas are extremely fragile, it is therefore desirable to file that is newly-increased or changing is carried out in time
Protection, to ensure that last state can be returned in the case of file corruption.
The file operation interface that traditional file backup is provided by call operation system enters to the file in file system
Row traversal, then read file data and backed up.The opening of file, reading, closing can be related to for each file
Operation, every kind of operation can all take certain calculating time.When backup target is the small documents of magnanimity, because read operation accounts for
When it is shorter, open and close the time-consuming significant proportion that can take total time-consuming of file operation, the IOPS (Input/Output of disk
Operations Per Second, i.e., the number per second for being written and read (I/O) operation) will be very high, so can greatly it influence
Backup performance.The small documents of backup millions often take the time of even more than half month several days, and this performance can not meet
Instant protection requirement of the user for file.
Some modes that Backup and Restore is carried out to file system on current market be present, such a mode can meet to magnanimity
Small documents carry out the demand of quick backup.But in actual restoration scenario, generally require specified file or catalogue progress is extensive
It is multiple, and this mode can only be recovered to whole file system, can not accomplish that fine granularity is recovered.
Based on above mentioned problem, it is necessary to one kind can either quick backup large amount of small documents, while and can meet specified file or
The method that catalogue is recovered.
The content of the invention
It is an object of the present invention to overcome the above-mentioned drawbacks of the prior art and provide one kind is based on file system
The mass small documents back-up restoring method of parsing.
The purpose of the present invention can be achieved through the following technical solutions：
A kind of mass small documents back-up restoring method based on file system parsing, it is characterised in that including backing up son side
Method and recovery submethod, wherein,
The backup submethod includes：Backup tasks are obtained, selection uses complete backup mode or incremental backup mode, base
Realize and back up in CDP monitoring technologies；
The recovery submethod includes：Recovery tasks are obtained, fine granularity reset mode or whole roll recovery mode is selected, passes through
Parsing to file system, realize the recovery for specifying data.
Preferably, in the backup submethod, specifically included using the step of complete backup mode：
A1) judge whether to have loaded CDP drivings, if so, then performing step a2), if it is not, being performed after then loading CDP drivings
Step a2)；
A3) initialization files system change data block record information；
A3) according to file system format, the data block assignment record in file system is read；
A4) analyze data block assignment record, all allocated data blocks is read, are backed up, and in backup procedure
More new file system delta data block record information；
A5) by the file system delta data block record information, to step a4) number that changes in backup procedure
Backed up according to block；
A6) data block no longer changes, and completes backup.
Preferably, in the backup submethod, specifically included using the step of incremental backup mode：
B1) judge whether to have loaded CDP drivings, if so, then performing step b2), if it is not, being used after then loading CDP drivings
Complete backup mode is backed up；
Whether the volume for b2) judging to need to back up is backed up completely excessively, if so, then performing step b3), if it is not, then using
Complete backup mode is backed up；
B3 the data block to change after last Backup end) is obtained from CDP drivings to be backed up.
Preferably, the step a4) in, when reading all allocated data blocks, enter data block has continuously been distributed
Row, which merges, to be read.
In the file system, the distribution condition of data block in log file system is carried out usually using bitmap table, when position is 1
Represent that the block that this represents has distributed, position represents that the block that this represents is unallocated when being 0, for the continuous position institute table for 1 of state
These blocks shown, are merged into and are once read out, so as to reduce the number of reading.
Preferably, the CDP, which drives, safeguards a dynamic bit chart and a static bit chart, and in every chart, one
Position represents a data block, and position represents that the block that this represents changes when being 1, position represents that the block that this represents is not sent out when being 0
Changing, wherein,
The dynamic bit chart is used for the information for recording the data block that changes,
When the static bit chart is used to obtain delta data block message, swapped with dynamic bit chart, before return
The data block information to change, do not influence the record of data block information changed after exchanging.
CDP driving can with monitor operating system to the I/O operation of specified file system volume.The minimum of the I/O operation is single
Position is block, and the default size of block is generally 512B integral multiple, and the block of write operation is considered as the data block to change.
Preferably, the step a2) in, initialization files system change data block record information is specially：By dynamic bit
All positions of chart and static bit chart are set to 0.
Preferably, the data block to change is backed up specially：
Dynamic of exchange bitmap table and static bit chart, the data block to change is obtained from new static bit chart and is carried out
Backup, and the data block to continuously changing merges reading.
Preferably, after the completion of the backup of all changes data block, the static bit chart is initialized.
If preferably, being made a mistake before backup is completed, the static bit chart during CDP is driven is merged into dynamic bit
Chart, then initialized static bitmap table.
Preferably, the fine granularity is recovered to specifically include following steps：
C1 the time point successfully backed up) is obtained；
C2 after) time point of recovery is wanted in selection, the volume backed up in the time point is obtained；
C3) volume is parsed according to file system format, travels through the attribute of the catalogue and file under specified path
And data, obtain the catalogue to be recovered or file；
C4) by the catalogue or file access pattern to specified location.
Preferably, the whole roll recovery specifically includes following steps：
D1 the time point successfully backed up) is obtained；
D2 after) time point of recovery is wanted in selection, the volume backed up in the time point is obtained；
D3) by the roll recovery to specified location.
Preferably, during whole roll recovery, the size of source book when the target volume size specified can not be less than backup.
The present invention also provides a kind of mass small documents backup/restoration system based on file system parsing, including：
Client, to need the machine where the file system volume backed up or the purpose to be returned to volume；
Service end, for the machine for receiving and storing Backup Data.
Preferably, the client includes：
Back-up device, for the logic scheduling in data backup procedure, read block and sent from file system volume
To service end；
Recovery device, for the logic scheduling in data recovery procedure, the Backup Data that service end is sent is received, and recover
To the assigned catalogue of client；
CDP supervising devices, for monitoring the data block situation of change in file system, the data block that will be changed in real time
Position recorded in increment bitmap, be obtained for reading during delta data block.
Preferably, the service end includes：
Storage medium, the data of backup are deposited according to certain form, the storage medium can be disk, tape, light
Disk or cloud storage；
Memory management unit, for being managed to the data backed up in storage medium, including reading to data storage,
Write-in, inquiry or deletion action；
File system resolver, according to the form of target file system, the Backup Data in storage medium is solved
Analysis, obtain the structure and data of file system.
Preferably, the file system resolver can support a series of file system formats, including but not limited to NTFS,
JFS2, EXT2, EXT3 and EXT4.
Preferably, the file system resolver provides different file system unified interface, for storage tube
Reason device is managed collectively to different file.
Compared with prior art, the invention has the advantages that：
1st, by being backed up to whole file system volume, the I/O operation to large amount of small documents is eliminated, so as to greatly subtract
Lack the IOPS of disk, improve backup performance；In actual applications, using traditional approach backup mass small documents, its speed
Often it is no more than 10MB/s, and by the way of entire volume backup, in the case where network bottleneck is unrestricted, backup performance can be with
Close to the reading performance of disk；
2nd, the situation of change of file system data block is monitored by CDP technologies, can effectively obtain delta data block, to volume
Incremental backup is carried out, so as to improve backup efficiency, reduces backup data quantity, has saved backup space；
3rd, reading is merged to the data block continuously to change, so as to reduce the number of reading；
4th, parsed, the volume of backup can be parsed by file system, select the catalogue specified or file to be recovered,
Meet the scene that partial document is lost or damaged, avoid the recovery of unnecessary data；
5th, by way of supporting whole roll recovery, meet the scene being damaged when whole file system, greatly improve
The efficiency of file access pattern.
Brief description of the drawings
Fig. 1 is the structured flowchart of apparatus of the present invention；
Fig. 2 is overall volume Backup and Restore schematic flow sheet；
Fig. 3 is volume backup schematic flow sheet completely；
Fig. 4 is volume incremental backup schematic flow sheet；
Fig. 5 recovers schematic flow sheet for volume fine granularity；
Fig. 6 is whole roll recovery schematic flow sheet；
Fig. 7 is the general structure schematic diagram of EXT3 file system；
Fig. 8 is block group descriptor structural representation；
Fig. 9 is inode data structure schematic diagrames；
Figure 10 is bitmap table operating diagram.
Embodiment
With reference to the accompanying drawing in the present invention, example description is carried out to the particular technique and step used in invention, to enter
Capable explanation complete in further detail.Described embodiment is only part of the embodiment of the present invention, rather than whole implementation
Example.Based on the embodiment in the present invention, what those of ordinary skill in the art were obtained under the premise of creative work is not made
Other all embodiments, belong to protection scope of the present invention.
In the user environment of reality, field when needing to back up millions or even more than hundred million grades of file is frequently present of
Scape, and this class file is often the small documents that some sizes are tens KB.In such a scenario, traditional file backup often needs
The time of even tens days several days is expended, backup performance is extremely low, can not meet ageing requirement of the user to file backup.This
When just need to use a kind of mass small documents back-up restoring method and system based on file system parsing provided by the invention.
As shown in figure 1, it illustrates the structured flowchart of present system device.The system includes client and service end.
Wherein client monitors dress for the machine where the needs file system volume backed up or the purpose to be returned to volume, including CDP
Put (CDP drivings), back-up device, recovery device and file system；Service end is used to receive and store the machine of Backup Data, wraps
Include memory management unit, file system resolver and storage medium.
Specifically, the logic scheduling that back-up device is used in data backup procedure, the read block from file system volume
And it is sent to service end.
Recovery device, for the logic scheduling in data recovery procedure, the Backup Data that service end is sent is received, and recover
To the assigned catalogue of client.
CDP supervising devices, for monitoring the data block situation of change in file system, the data block that will be changed in real time
Position recorded in increment bitmap, be obtained for reading during delta data block.
Storage medium, deposits the data of backup according to certain form, its medium can be disk, tape, CD or
Cloud storage.
Memory management unit, for being managed to the data backed up in storage medium, including reading to data storage,
Write-in, inquiry or deletion action.
File system resolver, according to the form of target file system, the Backup Data in storage medium is solved
Analysis, obtain the structure and data of file system；File system resolver can support a series of file system formats, including but not
It is limited to NTFS, JFS2, EXT2, EXT3 and EXT4；File system resolver provides different file system unified interface,
So that memory management unit is managed collectively to different file.
This is discussed in detail by taking the Backup and Restore of the mass small documents in the EXT3 file system under Linux environment as an example below
The embodiment of invention.
As shown in Fig. 2 it represents overall volume Backup and Restore schematic flow sheet, comprise the following steps that：
Step 101：Volume on browsing client, the curly hair to be backed up is chosen to play backup.
In the present embodiment, client is rolled up where screening the file system that all the system are supported on linuxn client,
The EXT3 file system volumes to be backed up are selected, one or more volumes can be selected.After confirming selection, a backup can be generated and appointed
Business is placed in task list.
Being backed up for task is selected to initiate backup from backup list.Backup mode supports backup and incremental backup two completely
Kind mode.
Step 102：Back-up device in client reads the data block to be backed up and is sent to service end.
If the backup mode that step 102 selects is as shown in Figure 3 for backup, its flow completely.
Step 301：The record information of initialization files system change data block.
Data block change information drives (CDP supervising devices) monitoring record by CDP.When backing up initiation, back-up device is first
It can first detect whether driving is already loaded into system kernel, if CDP drivings can first be loaded by not loading.
Disk I/O operation under Linux can all call make_request_fn functions, by realizing certainly in CDP drivings
Oneself make_request_fn functions, and the make_request_fn functions in replacement system kernel reach monitoring disk I/O
Purpose.
The device number of the I/O operation object can be attached in each disk I/O, initiates to begin in task, back-up device can obtain
The device number of the volume to be backed up of task choosing is taken to be sent to CDP drivings, driving will record these device numbers.
The least unit of I/O operation is block, and EXT3 file system is typically defaulted as 4096Bytes.Meeting in each I/O operation
The skew of device number and data including the operation object and length.IO for action type for write-in, its number operated
It is considered as delta data block according to block.CDP drivings monitor all I/O operations, and filter the I/O operation of designated equipment number, to changing
Data block, record its positional information.
CDP drivings use position chart recording change information, and a position represents a data block, and position represents this generation when being 1
The block of table changes, and position represents that the block that this represents does not change when being 0.
As shown in Figure 10, two bitmap tables are safeguarded in CDP drivings, one is dynamic bit chart, and one is static bit chart.
Dynamic bit chart is used for the information for recording the data block that changes；When static bit chart is used to obtain delta data block message, with
Dynamic bit chart swaps, and returns to the data block information to change before, does not influence the data block information changed after exchanging
Record.
Before initiating backup completely, obtain and want the device number of duplicate volume to issue CDP drivings, then initialize in CDP drivings
Dynamic bit chart and static bit chart, by all positions be 0, complete initialization.
Step 302：According to file system format, the data block assignment record in file system is read.
In the present embodiment, the general structure of EXT3 file system is as shown in Figure 7.One EXT3 file system is drawn by one
Guide block and several block groups form, and each block group corresponds to a block group descriptor, and these block group descriptors are uniformly placed on file
Before system, block group is managed, block group descriptor structure is as shown in Figure 8.
Shown in block group descriptor structure as shown in Figure 8, preceding 4 byte representations block chunk bitmap of block group descriptor
Position, the block assignment bit map of block group can be found by the information, can be obtained in all pieces of groups by this document system architecture
Block assignment bit map, its middle position represent that the block that this represents has distributed when being 1, position represents that the block that this represents is unallocated when being 0.
Step 303：Analysis block assignment record, read all allocated data blocks and backed up.
In the present embodiment, the block assignment bit map table obtained in step 302 is analyzed, obtains allocated data block
Backed up, transmit data to service end.It is continuously these blocks represented by 1 position for state, is merged into and once carries out
Read, so as to reduce the number of reading.
Step 304：Changed by data block and recorded, the data block to be changed in the backup procedure of obtaining step 303 is carried out
Backup.
In the present embodiment, the backup procedure time of step 303 is longer, may there is more data block in backup procedure
Change, in order to reduce the inconsistency of Backup Data, need to be backed up to the data block of the change occurred in backup procedure.
, it is necessary to exchange the static bit chart and dynamic bit chart in CDP drivings before delta data block is obtained, original is quiet
State bitmap table switchs to dynamic bit chart, and original dynamic bit chart switchs to static bit chart, the data record hereafter to change
Into new dynamic bit chart.
The data block to be changed from new static bit chart in the backup procedure of obtaining step 303 is backed up, and is sent
To service end.It is continuously these blocks represented by 1 position for state, is merged into and is once read out, so as to reduce reading
Number.
Step 305：The record information of delta data block is arranged, completes backup.
In the present embodiment, it is necessary to initialize the static bitmap in CDP drivings after all data blocks to be backed up have backed up
Table.
If occur that the situation of this backup failure will be caused extremely the static state, it is necessary to during CDP is driven before step 305
Record information in bitmap table is merged into dynamic bit chart, loses data when preventing backup next time from initiating.
If the backup mode that step 102 selects is incremental backup, its flow is as shown in Figure 4.
Step 401：The data block to change after last Backup end is obtained from CDP drivings to be backed up.
, it is necessary to judge whether CDP drivings have loaded before step 401, if driving does not load, load driver, and this
Secondary backup switchs to back up completely.
Before step 401, it need to judge whether the volume to be backed up did complete backup before this, if not doing complete backup,
Then the backup for this volume switchs to back up completely.
In the present embodiment, the volume to be backed up had done complete backup, it is necessary to exchange before delta data block is obtained
Dynamic bit chart and static bit chart in CDP drivings, the data block to be changed after last backup is obtained after exchange and is carried out
Backup, and it is sent to service end.It is continuously these blocks represented by 1 position for state, is merged into and is once read out, so as to
Reduce the number read.
Step 402：The record information of delta data block is arranged, initialized static bitmap table, completes backup.
In the present embodiment, it is necessary to initialize the static bit chart in CDP drivings after all changes data block has backed up.
If occur that the situation of this backup failure will be caused extremely the static state, it is necessary to during CDP is driven before step 402
Record information in bitmap table is merged into dynamic bit chart, loses data when preventing backup next time from initiating.
Step 103：The data Cun Chudao storage mediums that service end will receive.
Storage medium carries out storage management to data using customized a set of storage organization, and service end is received from visitor
After the data at family end, by memory management unit, data are stored in medium according to customized storage organization.
The medium of storage can be disk, CD, tape or cloud storage.
Step 104：The data backed up are browsed from service end.
Step 105：The data to be recovered are selected, restore it the specified location of client.
Before browsing Backup Data, the system provides whole roll recovery and fine granularity is recovered two ways and selected.
If selecting fine granularity to recover, the flow for browsing recovery is as shown in Figure 5.
Step 501：Obtain the time point successfully backed up.
In the present embodiment, choosing being recovered for task to initiate to browse, Backup Data operation, service end pass through storage tube
Reason device obtains all time points that success is backed up in this task from storage medium.
Step 502：After the time point of recovery is wanted in selection, the volume backed up in the time point is obtained.
After selecting the time point to be recovered, the time point information chosen will be sent to memory management unit, storage management
Device obtains all volumes backed up under the time point from storage medium again.
Step 503：The selection catalogue to be browsed obtains subdirectory and subfile under it.
After selecting the catalogue to be browsed, the path for the catalogue chosen will be sent to memory management unit, memory management unit
The interface of file system resolver offer is provided, this document system is parsed, travels through the catalogue and text under the path
Part.
In the present embodiment, Ext3 file system represents file and catalogue by the way of inode (index node),
Inode data structure is as shown in Figure 9.The inode number of each file and catalogue on a file partition is only
The size that one, the inode of the root of Ext3 file system are fixed as 2, inode is 128bytes；In superblock, record
Each how many inode of block group, and this document system how many block group；It can be found by these information specified
The inode that inode number represents.
I_block in inode points to the data of the inode, if what the inode table showed is file, i_block refers to
To position deposit content be this document data；If what the inode table showed is file, the position of i_block sensings
The content of storage is the directory entry of the subdirectory and subfile under the catalogue.Have recorded in directory entry file type (catalogue or
File), the inode number and filename of file.
In the present embodiment, the path that memory management unit will travel through is transmitted to file system resolver, due to root mesh
The inode number of record is fixed as 2, and file system resolver can find the inode of root, then finds the specific item of root
Record item, obtain with root subordinate file name identical directory entry in path, pass through what is recorded in the directory entry that gets
Inode number continually looks for subordinate path, gets the subdirectory and subfile in passed path always according to this process.
Step 504：Repeat step 503 is until browse to the catalogue to be recovered or file.
In the present embodiment, as procedure described above, EXT3 file system is parsed, until browsing to the mesh to be recovered
Record or file.
Step 505：The catalogue to be recovered or file access pattern are selected to the client directory specified.
In the present embodiment, after having selected the catalogue to be recovered or file and having specified the client directory to be returned to, hair
Rise and recover.After recovery tasks are initiated, the file or directory path at the time point that the recovery device of client will recover send to
Service end, file system resolver is called by storage managing server, according to process described in step 503, to file system
System is parsed, and the data of the attribute and file that get catalogue and file are back to client, and client recovery device connects
The data of service end transmission are received, file or catalogue are returned in the catalogue specified, complete to recover.
If selecting whole roll recovery, the flow for browsing recovery is as shown in Figure 6.
Step 601：Obtain the time point successfully backed up.
In the present embodiment, choosing being recovered for task to initiate to browse, Backup Data operation, service end pass through storage tube
Reason device obtains all time points that success is backed up in this task from storage medium.
Step 602：After the time point of recovery is wanted in selection, the volume backed up in the time point is obtained.
After selecting the time point to be recovered, the time point information chosen will be sent to memory management unit, storage management
Device obtains all volumes backed up under the time point from storage medium again.
Step 603：The roll recovery to be recovered is selected into specified client volume.
In the present embodiment, after having selected the volume to be recovered and having specified the client to be recovered volume, initiate to recover.Recover to appoint
After business is initiated, the volume path at the time point that the recovery device of client will recover is sent to service end, the storage tube of service end
The volume Backup Data that reason device was read from storage medium under the time point is back to client, and the recovery device of client receives
After the data sent to service end, write data on the volume specified.Source book when the target volume size of recovery can not be less than backup
Size.
Preferred embodiment of the invention described in detail above.It should be appreciated that one of ordinary skill in the art without
Creative work can is needed to make many modifications and variations according to the design of the present invention.Therefore, all technologies in the art
Personnel are available by logical analysis, reasoning, or a limited experiment on the basis of existing technology under this invention's idea
Technical scheme, all should be in the protection domain being defined in the patent claims.
Claims (10)
1. a kind of mass small documents back-up restoring method based on file system parsing, it is characterised in that including backup submethod
With recover submethod, wherein,
The backup submethod includes：Backup tasks are obtained, selection uses complete backup mode or incremental backup mode, is based on
CDP monitoring technologies realize backup；
The recovery submethod includes：Recovery tasks are obtained, fine granularity reset mode or whole roll recovery mode are selected, by text
The parsing of part system, realize the recovery for specifying data.
2. the mass small documents back-up restoring method according to claim 1 based on file system parsing, it is characterised in that
In the backup submethod, specifically included using the step of complete backup mode：
A1) judge whether to have loaded CDP drivings, if so, then performing step a2), if it is not, performing step after then loading CDP drivings
a2)；
A3) initialization files system change data block record information；
A3) according to file system format, the data block assignment record in file system is read；
A4) analyze data block assignment record, all allocated data blocks is read, are backed up, and updated in backup procedure
File system delta data block record information；
A5) by the file system delta data block record information, to step a4) data block that changes in backup procedure
Backed up；
A6) data block no longer changes, and completes backup.
3. the mass small documents back-up restoring method according to claim 1 based on file system parsing, it is characterised in that
In the backup submethod, specifically included using the step of incremental backup mode：
B1) judge whether to have loaded CDP drivings, if so, then performing step b2), if it is not, using complete after then loading CDP drivings
Backup mode is backed up；
Whether the volume for b2) judging to need to back up is backed up completely excessively, if so, then performing step b3), if it is not, then using complete
Backup mode is backed up；
B3 the data block to change after last Backup end) is obtained from CDP drivings to be backed up.
4. the mass small documents back-up restoring method according to claim 2 based on file system parsing, it is characterised in that
The step a4) in, when reading all allocated data blocks, it will continuously distribute data block and merged reading.
5. the mass small documents back-up restoring method based on file system parsing according to Claims 2 or 3, its feature exist
In the CDP, which drives, safeguards a dynamic bit chart and a static bit chart, and in every chart, a position represents a number
According to block, position represents that the block that this represents changes when being 1, and position represents that the block that this represents does not change when being 0, wherein,
The dynamic bit chart is used for the information for recording the data block that changes,
When the static bit chart is used to obtain delta data block message, swap with dynamic bit chart, occur before returning
The data block information of change.
6. the mass small documents back-up restoring method according to claim 5 based on file system parsing, it is characterised in that
The step a2) in, initialization files system change data block record information is specially：By dynamic bit chart and static bit chart
All positions be set to 0.
7. the mass small documents back-up restoring method according to claim 5 based on file system parsing, it is characterised in that
The data block to change is backed up specially：
Dynamic of exchange bitmap table and static bit chart, it is standby that the data block progress to change is obtained from new static bit chart
Part, and the data block to continuously changing merges reading.
8. the mass small documents back-up restoring method according to claim 5 based on file system parsing, it is characterised in that
After the completion of the backup of all changes data block, the static bit chart is initialized.
9. the mass small documents back-up restoring method according to claim 1 based on file system parsing, it is characterised in that
The fine granularity is recovered to specifically include following steps：
C1 the time point successfully backed up) is obtained；
C2 after) time point of recovery is wanted in selection, the volume backed up in the time point is obtained；
C3) volume is parsed according to file system format, traveling through the catalogue under specified path and file, acquisition will recover
Catalogue or file；
C4) by the catalogue or file access pattern to specified location.
10. the mass small documents back-up restoring method according to claim 1 based on file system parsing, its feature exist
In the whole roll recovery specifically includes following steps：
D1 the time point successfully backed up) is obtained；
D2 after) time point of recovery is wanted in selection, the volume backed up in the time point is obtained；
D3) by the roll recovery to specified location.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710904206.0A CN107885616A (en) | 2017-09-29 | 2017-09-29 | A kind of mass small documents back-up restoring method based on file system parsing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710904206.0A CN107885616A (en) | 2017-09-29 | 2017-09-29 | A kind of mass small documents back-up restoring method based on file system parsing |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107885616A true CN107885616A (en) | 2018-04-06 |
Family
ID=61780991
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710904206.0A Pending CN107885616A (en) | 2017-09-29 | 2017-09-29 | A kind of mass small documents back-up restoring method based on file system parsing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107885616A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110209530A (en) * | 2019-05-09 | 2019-09-06 | 上海爱数信息技术股份有限公司 | A kind of efficient method and system for restoring CDP system I/O data |
| CN110673977A (en) * | 2019-09-27 | 2020-01-10 | 浪潮电子信息产业股份有限公司 | Data recovery optimization method, device, equipment and medium |
| CN110851302A (en) * | 2019-10-31 | 2020-02-28 | 上海爱数信息技术股份有限公司 | Database information backup method and database information recovery method |
| CN112286733A (en) * | 2020-12-23 | 2021-01-29 | 深圳市科力锐科技有限公司 | Method, device and equipment for determining backup data recovery time and storage medium |
| CN112346907A (en) * | 2019-08-09 | 2021-02-09 | 上海爱数信息技术股份有限公司 | Data backup recovery method and system based on heterogeneous object storage |
| CN112464044A (en) * | 2020-12-09 | 2021-03-09 | 上海爱数信息技术股份有限公司 | File data block change information monitoring and management system and method thereof |
| CN113064763A (en) * | 2021-04-16 | 2021-07-02 | 上海英方软件股份有限公司 | Encryption and decryption-based continuous data protection method and device |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2008257364A (en) * | 2007-04-02 | 2008-10-23 | Hitachi Ltd | Storage unit, and volume restoration method |
| CN101777017A (en) * | 2010-02-08 | 2010-07-14 | 北京同有飞骥科技有限公司 | Rapid recovery method of continuous data protection system |
| US20110088027A1 (en) * | 2009-10-14 | 2011-04-14 | Vmware, Inc. | Tracking block-level writes |
| CN102331955A (en) * | 2011-09-14 | 2012-01-25 | 天津火星科技有限公司 | Multiple time granularity data backup method |
| US8224781B2 (en) * | 2010-05-14 | 2012-07-17 | Lsi Corporation | Data protection in a data storage system |
| CN103019890A (en) * | 2012-12-24 | 2013-04-03 | 清华大学 | Block-level disk data protection system and method thereof |
| CN103336728A (en) * | 2013-05-08 | 2013-10-02 | 上海爱数软件有限公司 | Disk data recovery method |
| CN103530204A (en) * | 2013-10-15 | 2014-01-22 | 上海爱数软件有限公司 | Method and system for real-time data backup |
| CN104407940A (en) * | 2014-11-26 | 2015-03-11 | 上海爱数软件有限公司 | Method for quickly recovering CDP system |
| CN104461776A (en) * | 2014-11-26 | 2015-03-25 | 上海爱数软件有限公司 | Application disaster tolerance method based on CDP and iSCSI virtual disk technology |
| CN104714859A (en) * | 2013-12-17 | 2015-06-17 | 南京壹进制信息技术有限公司 | Rapid backup and recovery method of mass files |
| CN105302668A (en) * | 2015-10-21 | 2016-02-03 | 上海爱数信息技术股份有限公司 | Method for optimizing block device data access performance in object-based storage system |
| CN105843704A (en) * | 2016-03-15 | 2016-08-10 | 上海爱数信息技术股份有限公司 | Data protection method and system capable of combining with snapshot function based on distributed block storage |
-
2017
- 2017-09-29 CN CN201710904206.0A patent/CN107885616A/en active Pending
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2008257364A (en) * | 2007-04-02 | 2008-10-23 | Hitachi Ltd | Storage unit, and volume restoration method |
| US20110088027A1 (en) * | 2009-10-14 | 2011-04-14 | Vmware, Inc. | Tracking block-level writes |
| CN101777017A (en) * | 2010-02-08 | 2010-07-14 | 北京同有飞骥科技有限公司 | Rapid recovery method of continuous data protection system |
| US8224781B2 (en) * | 2010-05-14 | 2012-07-17 | Lsi Corporation | Data protection in a data storage system |
| CN102331955A (en) * | 2011-09-14 | 2012-01-25 | 天津火星科技有限公司 | Multiple time granularity data backup method |
| CN103019890A (en) * | 2012-12-24 | 2013-04-03 | 清华大学 | Block-level disk data protection system and method thereof |
| CN103336728A (en) * | 2013-05-08 | 2013-10-02 | 上海爱数软件有限公司 | Disk data recovery method |
| CN103530204A (en) * | 2013-10-15 | 2014-01-22 | 上海爱数软件有限公司 | Method and system for real-time data backup |
| CN104714859A (en) * | 2013-12-17 | 2015-06-17 | 南京壹进制信息技术有限公司 | Rapid backup and recovery method of mass files |
| CN104407940A (en) * | 2014-11-26 | 2015-03-11 | 上海爱数软件有限公司 | Method for quickly recovering CDP system |
| CN104461776A (en) * | 2014-11-26 | 2015-03-25 | 上海爱数软件有限公司 | Application disaster tolerance method based on CDP and iSCSI virtual disk technology |
| CN105302668A (en) * | 2015-10-21 | 2016-02-03 | 上海爱数信息技术股份有限公司 | Method for optimizing block device data access performance in object-based storage system |
| CN105843704A (en) * | 2016-03-15 | 2016-08-10 | 上海爱数信息技术股份有限公司 | Data protection method and system capable of combining with snapshot function based on distributed block storage |
Non-Patent Citations (4)
| Title |
|---|
| 张也 等: "一种远程备份数据虚拟重构方法", 《四川大学学报（自然科学版）》 * |
| 张权 等: "一种基于Linux 标准分区的快照方法", 《现代计算机》 * |
| 文莎: "动态镜像系统及其容灾扩展的设计与实现", 《中国优秀硕士学位论文全文数据库信息科技辑》 * |
| 郭京 等: "基于Linux 的异步网络备份系统的设计与实现", 《计算机工程》 * |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110209530A (en) * | 2019-05-09 | 2019-09-06 | 上海爱数信息技术股份有限公司 | A kind of efficient method and system for restoring CDP system I/O data |
| CN110209530B (en) * | 2019-05-09 | 2024-02-09 | 上海爱数信息技术股份有限公司 | Method and system for recovering IO data of CDP system |
| CN112346907A (en) * | 2019-08-09 | 2021-02-09 | 上海爱数信息技术股份有限公司 | Data backup recovery method and system based on heterogeneous object storage |
| CN112346907B (en) * | 2019-08-09 | 2022-12-30 | 上海爱数信息技术股份有限公司 | Data backup recovery method and system based on heterogeneous object storage |
| CN110673977A (en) * | 2019-09-27 | 2020-01-10 | 浪潮电子信息产业股份有限公司 | Data recovery optimization method, device, equipment and medium |
| CN110673977B (en) * | 2019-09-27 | 2022-06-07 | 浪潮电子信息产业股份有限公司 | Data recovery optimization method, device, equipment and medium |
| CN110851302A (en) * | 2019-10-31 | 2020-02-28 | 上海爱数信息技术股份有限公司 | Database information backup method and database information recovery method |
| CN110851302B (en) * | 2019-10-31 | 2021-06-18 | 上海爱数信息技术股份有限公司 | Database information backup method and database information recovery method |
| CN112464044A (en) * | 2020-12-09 | 2021-03-09 | 上海爱数信息技术股份有限公司 | File data block change information monitoring and management system and method thereof |
| CN112286733A (en) * | 2020-12-23 | 2021-01-29 | 深圳市科力锐科技有限公司 | Method, device and equipment for determining backup data recovery time and storage medium |
| CN112286733B (en) * | 2020-12-23 | 2021-04-06 | 深圳市科力锐科技有限公司 | Method, device and equipment for determining backup data recovery time and storage medium |
| CN113064763A (en) * | 2021-04-16 | 2021-07-02 | 上海英方软件股份有限公司 | Encryption and decryption-based continuous data protection method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107885616A (en) | A kind of mass small documents back-up restoring method based on file system parsing | |
| US9703640B2 (en) | Method and system of performing incremental SQL server database backups | |
| US9002785B2 (en) | Stubbing systems and methods in a data replication environment | |
| CN104040481B (en) | Method and system for merging, storing and retrieving incremental backup data | |
| US8352422B2 (en) | Data restore systems and methods in a replication environment | |
| CN102662992B (en) | Method and device for storing and accessing massive small files | |
| US8725698B2 (en) | Stub file prioritization in a data replication system | |
| CN102521072B (en) | Virtual tape library equipment and data recovery method | |
| CN106055431B (en) | The method and device of backup and recovery based on the online rollback of snapshot | |
| CN101777017B (en) | Quick recovery method of continuous data protection system | |
| US20110106768A1 (en) | Backup using metadata virtual hard drive and differential virtual hard drive | |
| US11397749B2 (en) | Asynchronous replication of in-scope table data | |
| CN107256182A (en) | A kind of method and apparatus of database restoration | |
| US10628298B1 (en) | Resumable garbage collection | |
| EP2825967A1 (en) | Accessing and replicating backup data objects | |
| CN102750317A (en) | Method and device for data persistence processing and data base system | |
| CN103023996A (en) | Cloud data storage system | |
| CN103473258A (en) | Cloud storage file system | |
| CN110633261A (en) | Picture storage method, picture query method and device | |
| CN102867029B (en) | A kind of method managing distributive catalogue of document system and distributed file system | |
| Rao | Data duplication using Amazon Web Services cloud storage | |
| CN110457163A (en) | A kind of data reconstruction method, device and the storage medium of distributed block storage | |
| US11645333B1 (en) | Garbage collection integrated with physical file verification | |
| CN108241557A (en) | The method of data backup in HDFS | |
| CN111339037B (en) | Efficient parallel replication method for parallel distributed file system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180406 |