CN118626439B - Data archiving method and electronic equipment - Google Patents
Data archiving method and electronic equipment Download PDFInfo
- Publication number
- CN118626439B CN118626439B CN202411095926.3A CN202411095926A CN118626439B CN 118626439 B CN118626439 B CN 118626439B CN 202411095926 A CN202411095926 A CN 202411095926A CN 118626439 B CN118626439 B CN 118626439B
- Authority
- CN
- China
- Prior art keywords
- data
- tape
- target
- archiving
- index area
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 66
- 230000002159 abnormal effect Effects 0.000 claims abstract description 7
- 238000010257 thawing Methods 0.000 claims description 27
- 238000004590 computer program Methods 0.000 claims description 7
- 238000013480 data collection Methods 0.000 abstract description 18
- 238000007726 management method Methods 0.000 description 23
- 230000008569 process Effects 0.000 description 16
- 238000013500 data storage Methods 0.000 description 10
- 230000005856 abnormality Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 238000011084 recovery Methods 0.000 description 4
- 238000004140 cleaning Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000009467 reduction Effects 0.000 description 3
- 239000007787 solid Substances 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 239000008358 core component Substances 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the application provides a data archiving method and electronic equipment. According to the embodiment of the application, the archiving plan time period and the archiving execution time period are pre-configured based on actual requirements, so that the data to be archived actually can be screened from the data collected by the data collection equipment, the flexibility of archiving data selection is improved, the execution time of archiving operation can be controlled, the flexibility of archiving operation execution is improved, and the problem that the service life of a magnetic head is influenced due to continuous work of the magnetic head can be effectively avoided. The information recorded in the first index area is synchronously written into the magnetic tape after the archiving task is finished, so that when the first index area is abnormal and unavailable, the first index area can be restored by utilizing the information of the first index area written into the magnetic tape, the problem that the archiving and inquiring of long-period data are influenced due to the unavailable first index area is avoided, and the reliability of the archiving and inquiring of the long-period data is improved.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a data archiving method and an electronic device.
Background
With the rapid development of information technology, network storage devices are widely applied in the fields of data storage and the like, and along with the rapid demand for long-period data storage; by long period data is meant data that requires a longer holding time than a set period of time (e.g., one year, ten years, or several decades, etc.).
In practical applications, magnetic tape has become the best choice for long-period data archiving due to its low cost, large capacity, and long durability, where archiving refers to the process of reading data from a short-period storage device (e.g., hard disk, etc.) and writing it to a long-period storage device (e.g., magnetic tape, etc.) for long-term storage. However, in the current tape archiving scheme, the archiving operation is generally continuously performed on the data collected by the data collecting device (such as the network camera, the photo taking device and the like), which causes the magnetic head in the tape drive to continuously work, and no time is required to perform maintenance, cleaning and other operations on the magnetic head of the tape drive, so that the service life of the magnetic head is reduced, and the reduction of the service life of the magnetic head affects the archiving operation of the subsequent data.
Disclosure of Invention
In view of the above, the present application provides a data archiving method and an electronic device, so as to effectively avoid the problem that the service life of a magnetic head is affected due to continuous operation of the magnetic head, thereby avoiding the problem that the archiving operation of subsequent data is affected due to the reduced service life of the magnetic head.
The embodiment of the application provides a data archiving method, which comprises the following steps:
Obtaining a current archiving task; the current archiving task is used for indicating to conduct archiving operation on target data acquired by data acquisition equipment associated with the current archiving task in a set archiving execution time period; the target data is data acquired by the data acquisition device in the configured set archiving schedule period; setting an archive execution period after setting an archive plan period; the target data is stored in a designated storage space after being obtained from the data acquisition equipment before being archived;
checking whether the target tape is used based on the designated first index area; the target tape is used for storing the archived target data; the first index area records at least the occupied information of the magnetic tape;
if the target tape is not used, writing the information recorded in the first index area into the target tape;
After writing the information recorded in the first index area into the target tape, or after determining that the target tape is used, checking whether the available space of the target tape meets the requirement of accommodating the target data and the information recorded in the first index area, if so, writing the target data into the target tape, if not, writing the information recorded in the first index area into the target tape, and writing the target data into other available tapes;
and recording information of the target data occupying the magnetic tape in the first index area.
The embodiment of the application also provides a data archiving device, which comprises:
The obtaining module is used for obtaining the current archiving task; the current archiving task is used for indicating to conduct archiving operation on target data acquired by data acquisition equipment associated with the current archiving task in a set archiving execution time period; the target data is data acquired by the data acquisition device in the configured set archiving schedule period; setting an archive execution period after setting an archive plan period; the target data is stored in a designated storage space after being obtained from the data acquisition equipment before being archived;
A checking module for checking whether the target tape is used based on the designated first index area; the target tape is used for storing the archived target data; the first index area records at least the occupied information of the magnetic tape;
the first archiving module is used for writing the information recorded in the first index area into the target tape if the target tape is not used;
The second archiving module is used for checking whether the available space of the target tape meets the requirement of containing the target data and the information recorded in the first index area after the information recorded in the first index area is written in the target tape or after the target tape is determined to be used, if so, writing the target data in the target tape, if not, writing the information recorded in the first index area in the target tape, and writing the target data in other available tapes;
And the recording module is used for recording the information of the target data occupying the magnetic tape in the first index area.
The embodiment of the application also provides electronic equipment, which comprises:
A processor; and
A memory in which computer program instructions are stored which, when executed by a processor, cause the processor to perform the steps of the method as above.
Embodiments of the present application also provide a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, cause the processor to perform the steps of the method as above.
According to the technical scheme, the archiving plan time period and the archiving execution time period for archiving the data acquired by the data acquisition equipment are pre-configured based on actual requirements, so that the data to be archived actually can be screened from the data acquired by the data acquisition equipment based on the archiving plan time period, and the flexibility of archiving data selection is improved; the execution time of the archiving operation can be controlled, so that the working time of the magnetic head can be controlled while the execution flexibility of the archiving operation is improved, the problem that the service life of the magnetic head is reduced due to continuous working of the magnetic head is effectively avoided, and the problem that the archiving operation of subsequent data is influenced due to the reduction of the service life of the magnetic head is avoided;
Furthermore, after the archiving task is finished, the information recorded in the first index area is synchronously written into the magnetic tape, so that when the first index area is abnormal and unavailable, the first index area can be restored by using the information of the first index area written into the magnetic tape, the problem that the archiving and inquiring of long-period data are influenced due to the unavailable first index area is avoided, and the reliability of the archiving and inquiring of the long-period data is improved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application.
Fig. 1 is a flowchart of a method according to an embodiment of the present application.
Fig. 2 is a schematic diagram of a system structure according to an embodiment of the present application.
Fig. 3 is a schematic implementation diagram of a storage device according to an embodiment of the present application.
FIG. 4 is a flow chart of a method for detecting the number of available tapes according to an embodiment of the present application.
Fig. 5 is a flowchart of a method for generating an archiving task according to an embodiment of the present application.
Fig. 6 is a flowchart of a method for deleting an archiving task according to an embodiment of the present application.
Fig. 7 is a flowchart of a method for performing an archiving task according to an embodiment of the present application.
FIG. 8 is a flowchart of a method for performing another archiving task provided by an embodiment of the present application.
Fig. 9 is a flowchart of a method for performing another archiving task according to an embodiment of the present application.
Fig. 10 is a flowchart of a method for processing a first index area according to an embodiment of the present application.
FIG. 11 is a flowchart of a method for querying data according to an embodiment of the present application.
FIG. 12 is a flowchart of a method for recovering from a failure of a first index area according to an embodiment of the present application.
Fig. 13 is a block diagram of a device according to an embodiment of the present application.
Fig. 14 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to better understand the technical solution provided by the embodiments of the present application and make the above objects, features and advantages of the embodiments of the present application more obvious, the technical solution in the embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
Referring to fig. 1, fig. 1 is a flowchart of a method provided in an embodiment of the present application. The method is applied to the electronic equipment. In this embodiment, the electronic device may be, for example, a microcomputer, a server, or the like, and the embodiment is not particularly limited.
As shown in fig. 1, the process may include the steps of:
Step 101, obtaining a current archiving task.
In this embodiment, the current archiving task is a task for archiving data collected by a certain data collecting device, that is, the current archiving task is associated with a corresponding data collecting device.
In this embodiment, one archive execution period (i.e., set archive execution period) and one archive plan period (i.e., set archive plan period) are configured for the above-described data collection apparatus. Based on this, in this embodiment, the present electronic device may store the data obtained from the above-described data collection device in the specified storage space first; then, when the current archiving task associated with the data acquisition device is obtained, the target data can be read from the designated storage space and archiving operation can be performed in the set archiving execution time period; here, the target data refers to data acquired by the data acquisition device in the set filing plan period.
In this embodiment, the data acquisition device refers to a device having data acquisition and storage functions; as an example, the data acquisition device may include, but is not limited to, a webcam, a photo snapshot, a digital video recorder, a webcam, and the like, for example.
Alternatively, as an embodiment, the present embodiment may be configured in advance for the same data collection device with at least one set archiving plan period and at least one set archiving execution period for the data collection device. Here, the set archiving schedule period refers to a data acquisition period in which data to be archived are set in advance; the set archive execution period refers to a period in which archive operations are performed on data to be archived, which is set in advance. Wherein the set archiving execution period is after the set archiving schedule period; for example, the archiving schedule period is set to 9:00 to 12:00 (9:00-12:00) on a day, and the archiving execution period is set to be later than the archiving schedule period, for example, 16:00-18:00 on the day.
In this embodiment, all data collected by the data collection device is not archived, but an archiving plan time period and an archiving execution time period for archiving data are preconfigured for the data collection device, so that data archiving can be selectively performed based on actual requirements, and thus the flexibility of data archiving can be improved.
Alternatively, as an embodiment, the above specified storage space may be a storage space preconfigured on the present electronic device. As another embodiment, a storage device for storing data in a short period (such as one month, half year, etc.) may be separately provided, where the storage device may directly obtain the data collected by the data collecting device, or receive the data collected by the data collecting device transmitted by the present electronic device, etc., and the embodiment is not specifically limited; based on this, the above-described designated storage space may be a storage space that is preconfigured on the storage device.
As to how the current archiving task is obtained in this step, it will be described by way of example below, and this is not repeated here.
Step 102, checking whether the target tape is used or not based on the designated first index area; if not, executing step 103; if yes, go to step 104.
In this embodiment, the present electronic device is externally connected to a tape library; tape libraries refer to tape-based backup systems, such as tape libraries may be comprised of at least one tape drive, a robotic arm, a plurality of tapes, and a plurality of tape slots for carrying tapes. Wherein the tape library is configured to enable removal and loading of tape in a tape drive by a robotic arm. Magnetic tape refers to a magnetically-loaded strip of material used to record sound, images, numbers or other signals. A magnetic tape drive refers to an assembly for reading and writing magnetic tape, the core component of which is a magnetic head, and in particular implementations, for example, by the magnetic head contacting the magnetic tape to perform data reading and writing operations.
The target tape refers to a tape currently in use in a tape library, such as a tape currently located in a tape drive. Here, the target tape is used to store the archived target data.
The first index area records at least information that the tape is occupied. Specific content of the occupied information of the magnetic tape is not limited, and for example, information such as a tape identifier of the occupied magnetic tape can be included; the tape identifier here may be, for example, a tape number (e.g., 1, 2,3, etc.), or the like. Based on this, as an embodiment, the checking in step 102 based on the specified first index area whether the target tape is used or not may be, for example, when it is specifically implemented: firstly, obtaining a tape identifier of a tape currently positioned in a tape drive in a tape library as a tape identifier of a target tape; then, checking whether a tape identifier matched with the tape identifier of the target tape exists in the first index area, if so, determining that the target tape is used, otherwise, determining that the target tape is not used.
Alternatively, the first index area in this step may be an area preconfigured on the present electronic device, similar to the configuration of the above-described designated storage space; the area may be a pre-configured area on the storage device, and the embodiment is not particularly limited. The embodiment does not need to additionally introduce a hardware device such as a metadata server for recording the information recorded in the first index area, but configures the first index area in the electronic device or the storage device, so that not only can the hardware cost be reduced, but also the problem that the usability of the information of the first index area contained in the metadata server is influenced due to the hardware stability and the service life of the metadata server can be effectively avoided.
Step 103, writing the information recorded in the first index area into a target tape; and proceeds to step 104.
In this embodiment, when it is determined that the target tape is not used, the information recorded in the first index area is written to the target tape. As an embodiment, in this step, the information recorded in the first index area is written into the target tape, and may be, for example, when the specific implementation is: the information recorded in the first index area is converted into a file, and then the file is written into the target tape. The storage location where the file is written to the target tape is not particularly limited herein, and may be, for example, the tape head position of the target tape.
In this embodiment, since the magnetic tape in the magnetic tape library may be used before entering the magnetic tape library, and old data may remain on the magnetic tape, in order to avoid that the old data remaining on the magnetic tape affects the data archived in this embodiment, as an embodiment, before writing the information recorded in the first index area into the target magnetic tape, it may be checked whether the old data remaining on the target magnetic tape exists, and if so, it is necessary to delete the old data remaining on the target magnetic tape.
Step 104, checking whether the available space of the target tape meets the requirement of accommodating the target data and the information recorded in the first index area; if yes, go to step 105; otherwise, step 106 is performed.
In this embodiment, after writing the information recorded in the first index area to the target tape, or after determining that the target tape has been used, it is checked whether the available space of the target tape satisfies the requirement of accommodating the target data and the information recorded in the first index area, if so, step 105 is performed, and if not, step 106 is performed.
As an embodiment, after the target data in the step 101 is stored in the designated storage space, the occupation condition of the target data in the designated storage space is further recorded in the designated second index area; the specific content of the occupancy is not limited herein, and the occupancy includes at least the size of the target data, the storage location of the target data, and the like. As for the configuration of the second index area, similar to the configuration of the first index area in step 102, the description thereof is omitted.
Based on the above description, as an embodiment, the checking in the present step 104 whether the available space of the target tape meets the requirement of accommodating the above target data and the information recorded in the first index area may be, for example, when the specific implementation is: firstly, obtaining the size of target data from a second index area; then, checking whether the size of the available space of the target tape is larger than or equal to the sum of the size of the target data and the size of the information recorded in the first index area, if so, determining that the available space of the target tape meets the requirement of accommodating the target data and the information recorded in the first index area, and if not, determining that the available space of the target tape does not meet the requirement of accommodating the target data and the information recorded in the first index area.
Step 105, writing the target data to a target tape; and proceeds to step 107.
In this embodiment, after it is determined that the available space of the target tape satisfies the requirement of accommodating the target data and the information recorded in the first index area, the target data is written to the target tape.
As one example, there are many specific implementations of writing the target data to the target tape in step 105, for example, based on the above description, the storage location of the target data in the specified storage space may be obtained from the second index area first; thereafter, the target data may be read from the designated storage space and written to the target tape based on the storage location.
In this embodiment, for the same data acquisition device, the data type of the data to be acquired may be configured for the data acquisition device in advance, for example, the data type of the data to be acquired configured for the network camera is a video type, the data type of the data to be acquired configured for the image capturing machine is an image type, and so on. That is, the data type of the target data may include a video type, a picture type, and the like. Wherein, since video data is usually stored in a streaming storage manner in a designated storage space, the video data is not stored in a file form; in this regard, as an embodiment, in the case where the target data is video data, after the target data is read from the designated storage space based on the storage location, the target data is converted into a file, and then the file is written to the target tape.
Step 106, writing the information recorded in the first index area into a target tape, and writing target data into other available tapes; and proceeds to step 107.
In this embodiment, after determining that the available space of the target tape does not meet the requirement of accommodating the target data and the information recorded in the first index area, the information recorded in the first index area is written to the target tape, and then one other available tape is determined from the tape library to write the target data to the other available tape. As to how to write the target data to other available magnetic tapes, similar to how to write the target data to the target magnetic tapes in step 105 described above, a description thereof will be omitted.
In this embodiment, as may be appreciated from the above description, the information of the occupied magnetic tape in the first index area may include a tape identifier of the occupied magnetic tape; based on this, as an embodiment, the determination manners of other available magnetic tapes in the present step 106 may include: the tape identification of the occupied tape can be determined based on the occupied information of the tape recorded in the first index area; and then, determining a currently unoccupied tape from the tape library according to the determined tape identifications of the occupied tape and the tape identifications of all the tapes in the tape library, for example, searching one tape corresponding to one of the tape identifications except the tape identifications recorded in the first index area from the tape library to serve as the currently unoccupied tape, and taking the unoccupied tape as other available tapes.
In this embodiment, after determining the other available magnetic tape, the other available magnetic tape may be updated to the magnetic tape currently being used, such as notifying the magnetic tape library to load the other available magnetic tape into the magnetic tape drive; thereafter, the step of writing the target data to other available tapes is performed.
In step 107, information of the target data occupying the magnetic tape is recorded in the first index area.
In this embodiment, after the target data is written into the corresponding magnetic tape, it may be considered that the archiving operation of the target data is completed, and at this time, the information that the target data occupies the magnetic tape may be recorded in the first index area.
Alternatively, as an embodiment, after the information recorded in the first index area is written to the target tape in this embodiment, the information occupied by the information recorded in the first index area by the target tape may also be recorded in the first index area.
Thus, the flow shown in fig. 1 is completed.
As can be seen from the flow shown in fig. 1, in the embodiment of the present application, by pre-configuring the archiving plan time period and the archiving execution time period for archiving the data collected by the data collection device based on the actual requirement, the data to be archived actually can be screened from the data collected by the data collection device based on the archiving plan time period, so that the flexibility of archiving data selection is improved; the execution time of the archiving operation can be controlled, so that the working time of the magnetic head can be controlled while the execution flexibility of the archiving operation is improved, the problem that the service life of the magnetic head is reduced due to continuous working of the magnetic head is effectively avoided, and the problem that the archiving operation of subsequent data is influenced due to the reduction of the service life of the magnetic head is avoided;
furthermore, in the process of executing the archiving task, the embodiment of the application synchronously writes the information recorded in the first index area into the magnetic tape, so that when the first index area is abnormal and unavailable, the first index area is restored by using the information of the first index area written into the magnetic tape, thereby avoiding the problem that the archiving and inquiring of long-period data are influenced due to the unavailable first index area and improving the reliability of the archiving and inquiring of the long-period data.
As an embodiment, the embodiment of the application also provides a data archiving system. The data archiving system is described below:
Referring to fig. 2, the system 200 includes: a data acquisition device 201, a management device 202, a storage device 203, and a tape library 204.
The above-described storage device is described below:
in this embodiment, the data acquisition device is connected to a storage device for obtaining data acquired by the data acquisition device and storing the obtained data for a short period. As an embodiment, referring to fig. 3, the storage device 300 includes at least a storage space 301, a thawing space 302, a first index area 303, and a second index area 304.
The storage space refers to a data storage area preconfigured in the storage device, that is, the designated storage space in the step 101. The storage device may store the obtained data into the storage space after obtaining the data collected by the data collection device.
The defreezing space refers to another data storage area preconfigured in the storage device, and the defreezing space may be used to store data read from a tape of the tape library.
Alternatively, the storage space and the thawing space configured in the storage device may be data storage areas that are merged together, or may be two independent data storage areas, which is not limited in detail in this embodiment.
The first index area refers to yet another data storage area preconfigured in the storage device. The information recorded in the first index area is not particularly limited, and may include, for example, a tape identifier of an occupied tape, a device identifier (such as a device name and a device number) of a data acquisition device from which each data on the occupied tape comes, an acquisition time period in which each data is located, and a data identifier (such as a file name to which each data corresponds). It follows that if the first index area is damaged, information of the tape occupied by data is lost, so that the data on the tape cannot be queried.
The second index area refers to yet another data storage area preconfigured in the storage device. The information recorded in the second index area is not particularly limited, and may include, for example, a device identifier of a data acquisition device from which each data in the storage space is derived, a data acquisition time stamp, a data size, and a storage location in the storage space.
Alternatively, the first index area and the second index area configured in the storage device may be data storage areas that are merged together, or may be two independent data storage areas, which is not specifically limited in this embodiment.
Optionally, the implementation manners of the storage space, the thawing space, the first index area, and the second index area may be, for example, a hard disk, a flash memory chip, a Solid state disk (Solid STATE DISK, SSD), an internet protocol-based storage local area network (Internet Protocol Storage Area Network, IPSAN), and a network attached storage (Network Attached Storage, NAS), which are not limited in this embodiment.
The management apparatus described above is described below:
the management apparatus may be regarded as an electronic apparatus for executing the data archiving method provided by the present embodiment described above. For example, the management device may be used for access management of the data collection device, management of an operation mode of the data collection device, writing and reading management of data, archiving management of data, and the like. The operation mode may include modes such as acquisition of video type data and acquisition of picture type data.
Alternatively, the management device and the storage device may be combined into one electronic device, or may be two independent electronic devices, where the two devices are connected through a network, and this embodiment is not limited specifically. In the case where the management device and the storage device are one electronic device, the management device may be a microprocessor or a processing chip or the like in the electronic device, and the storage device may be a storage medium such as a hard disk or the like in the electronic device.
The tape library described above is described below:
a plurality of magnetic tapes may be included in a tape library; the tape library may control tape access to tape drives in the tape library via its internally configured robotic arm to effect switching of the tape currently in use. In the case where the tape library needs to be replaced, it is necessary to manually assist in the operations of entering and exiting the tape library.
The data archiving system provided by the present embodiment is described above. The following describes a data archiving method provided by an embodiment of the present application from the viewpoint of management equipment based on the system described above:
Step one, initializing the configuration of a storage space, a thawing space, a first index area and a second index area in a storage device.
In this embodiment, the initialization configuration may be configured by the management device automatically or manually, and the embodiment is not limited specifically. Accordingly, the specific content of the initialization configuration is not limited, and may be set based on actual requirements, for example, the size and the location of the occupied storage space may be included.
And step two, configuring the working mode, the data acquisition time period and the archiving strategy for indicating the archiving of the data of each data acquisition device.
In this embodiment, a corresponding data acquisition time period and an archiving policy may be preconfigured for each data acquisition device based on actual requirements; the archiving policy may include at least one set archiving schedule period and at least one set archiving execution period, and the present embodiment is not particularly limited.
The data acquisition time period refers to an acquisition time period when the data acquisition equipment acquires data, and can comprise acquisition starting time and acquisition ending time; for example, may be 8:00-20:00.
The set archiving schedule period refers to a preset collection period in which data to be archived is located, and may include, for example, an archiving schedule start time and an archiving schedule end time. The archiving schedule period may be a portion or all of a period selected from the data collection period, such as 9:00-12:00.
The set archive execution period refers to a period of time in which an archive operation is performed on data to be archived, which may include an archive execution start time and an archive execution end time, for example. The archive execution period is set after the archive plan period, and may be 21:00-23:00, for example.
Optionally, when the types of data collected by different data collecting devices are the same, the set archiving execution time periods configured corresponding to the data collecting devices may be the same or different. When the types of data acquired by different data acquisition devices are different, the configured set archiving execution time periods of the different data acquisition devices are not intersected, so that cross archiving among different types of data (such as video data, picture data and the like) can be avoided, and user experience is improved.
And thirdly, controlling the storage device to acquire data acquired by each data acquisition device based on the data acquisition time period configured by each data acquisition device, and storing the acquired data into the storage space.
Here, after the obtained data is stored in the storage space, the information recorded in the second index area is updated synchronously.
The working mode configured by the management device for the data acquisition device a is an acquisition mode of video type data, the configured data acquisition time period is 8:00-20:00, the archiving plan time period is 9:00-12:00, and the archiving execution time period is 21:00-23:00. That is, the storage device acquires video data from the data acquisition device a for 12 hours every day in the range of 8:00 to 20:00, and saves the acquired video data into the storage space; wherein video data acquired by the data acquisition device a in 3 hours of 9:00-12:00 is taken as data to be archived (i.e., the target data described above), the archiving operation to be performed in the magnetic tape written from the storage space to the magnetic tape library is started in 21:00, and the archiving operation is ended in 23:00.
Still further exemplary, the operation mode configured by the management device for the data acquisition device B is an acquisition mode of data of a picture type, the configured data acquisition time period is 10:00-18:00, the archiving plan time period is set to 10:00-15:00, and the archiving execution time period is set to 23:10-23:30. That is, the storage device acquires the snap shot picture data from the data acquisition device B for 8 hours 10:00-18:00 a day, and saves the acquired picture data into the storage space; wherein picture data acquired by the data acquisition device B in 5 hours of 10:00-15:00 is taken as data to be archived, for example, 1000 pictures are taken by the data acquisition device B in the 5 hours, the 1000 pictures are taken as data to be archived, an archiving operation of writing the data to a tape of the tape library from a storage space is performed at 23:10, and the archiving operation is ended at 23:30.
And step four, detecting the number of the remaining available magnetic tapes in the magnetic tape library.
In this embodiment, referring to fig. 4, taking the tape identifier of each tape in the tape library as the tape number as an example, the above-mentioned method flow of detecting the number of remaining available tapes in the tape library may include the following steps:
Step 401, determining the tape number of the occupied tape based on the information recorded in the first index area, and determining the number of unoccupied tapes in the tape library according to the tape number of the occupied tape and the tape numbers of all tapes in the tape library.
In this embodiment, the number of unoccupied tapes in the tape library is determined according to the tape number of the occupied tape and the tape numbers of all tapes in the tape library, and may be: the number of unoccupied tapes in the tape library can be determined based on determining the tape having a tape number different from the tape number of the occupied tape as the unoccupied tape. Unoccupied tape is herein considered to mean the remaining available tape.
Step 402, judging whether the number of unoccupied magnetic tapes is smaller than a preset value; if yes, go to step 403; if not, go back to execute step 401.
The preset value can be flexibly set based on actual requirements, for example, can be 0,1 or 2.
Step 403, sending out an early warning to indicate that the tape of the tape library is insufficient and needs to be replaced, and returning to step 401 after the replacement is completed.
Here, after the warning is issued, it is necessary to manually replace the tape library with a new one, and return to step 401 after the replacement is completed.
Illustratively, the management device detects, through the storage device, a total of 4 tapes of tape numbers 001, 002, 003, 004 of all tapes in the tape library, and retrieves the tape numbers of all the tapes in the first index area; if the 3 magnetic tapes with the numbers 001, 002 and 003 are occupied, it can be calculated that 1 magnetic tape 004 is unoccupied in the magnetic tape library, and the number of the remaining available magnetic tapes (i.e. 1) is larger than the preset value 0, and no early warning prompt is needed. If the 4 magnetic tapes with the numbers of 001, 002, 003 and 004 are already occupied, the number of the remaining 0 magnetic tapes in the magnetic tape library can be calculated to be unoccupied, the number of the remaining available magnetic tapes (namely 0) is equal to the preset value of 0, and the management equipment sends out early warning to prompt that the magnetic tapes in the magnetic tape library are insufficient and the magnetic tapes need to be replaced.
And fifthly, archiving the detection flow of the execution time period.
As one embodiment, the archive task may be generated based on a start time setting an archive execution period, which may be, for example, when embodied: checking, for each data acquisition device, whether a start time of a set archive execution period configured by the data acquisition device matches a current time; if so, checking whether the archiving task associated with the data acquisition device exists in the archiving task list, if not, generating the archiving task associated with the data acquisition device and adding the archiving task to the archiving task list, and if so, returning to the step of checking whether the starting time of the set archiving execution time period configured by the data acquisition device is matched with the current time for each data acquisition device. And if not, skipping the data acquisition device.
For example, referring to fig. 5, the method flow for generating an archiving task based on the start time of the archiving execution period may include the following steps:
step 501, for each data acquisition device, checking whether the start time of the set archive execution period configured by the data acquisition device matches the current time, and if so, executing step 502.
Here, if there is no match, the data acquisition device may be skipped.
Step 502, judging whether the archiving task list contains the archiving task associated with the data acquisition device, if not, executing step 503; if yes, go back to step 501.
Here, the archive task associated with the data collection device refers to an archive task including a device identification of the data collection device and setting an archive execution period.
Step 503, generating and adding the archiving task associated with the data acquisition device to the archiving task list, and returning to execute step 501.
In the present embodiment, the number and the specific content of the generated archiving tasks associated with the data acquisition device are not limited; for example, where the data acquisition device is configured with a plurality of set archive execution periods and a plurality of set archive plan periods, the generated archive tasks associated with the data acquisition device may be a plurality of, where each archive task may include a device identification of the data acquisition device, one set archive execution period of the data acquisition device, and one or more set archive plan periods.
For another example, where the data acquisition device is configured with a set archive execution period and a plurality of set archive plan periods, the generated archive task associated with the data acquisition device may be one, which may include the device identification of the data acquisition device, the set archive execution period of the data acquisition device, and all of the set archive plan periods of the data acquisition device; etc.
Optionally, since the set archiving execution periods between different data acquisition devices are the same, in the process of checking whether the start time of the set archiving execution period configured by each data acquisition device matches the current time, if the start time of the set archiving execution period of the plurality of data acquisition devices matches the current time, the archiving tasks generated by the data acquisition devices may be sequentially added to the archiving task list according to the set rule. The setting rule herein may be, for example, the sequence of the device numbers of the data acquisition device, or the preset priority of the data acquisition device, and the embodiment is not specifically limited.
Based on the above description, as an embodiment, there are many specific implementations of obtaining the current archiving task in the step 101, for example, one archiving task may be obtained from the archiving task list as the current archiving task according to the chronological order of adding to the archiving task list, and, specifically, for example, one archiving task with the earliest time of adding to the archiving task list may be selected from the archiving task list as the current archiving task.
As an embodiment, deletion processing may be performed on the archive tasks in the archive task list based on the end time of the archive execution period, which may be, for example, when specifically implemented: firstly checking whether an archiving task exists in an archiving task list, and if so, acquiring the earliest archiving task added to the archiving task list from the archiving task list as a target archiving task; then, it is checked whether or not the end time of the set archive execution period indicated by the target archive task matches the current time. If so, checking whether the execution state of the target archive task is a first state for representing the execution, if so, stopping executing the target archive task, deleting the target archive task from the archive task list, and if not, deleting the target archive task from the archive task list. If the files do not match, or after deleting the target archive task from the archive task list, the process returns to the step of checking whether the archive task exists in the archive task list.
For example, referring to fig. 6, the method flow for deleting an archive task in the archive task list based on the end time of the archive execution period may include the following steps:
step 601, checking whether an archiving task exists in the archiving task list; if yes, go to step 602; if not, the process continues to step 601.
Step 602, obtaining an archive task with the earliest time added to the archive task list from the archive task list as a target archive task, and checking whether the end time of the set archive execution time period indicated by the target archive task is matched with the current time; if yes, go to step 603; if not, return to execute step 601.
Step 603, checking whether the execution state of the target archiving task is a first state for representing execution; if yes, go to step 604; if not, step 605 is executed.
Step 604, execution of the target archiving task is stopped and step 605 is continued.
Step 605, the target archive task is deleted from the archive task list, and step 601 is executed back.
And step six, archiving task execution flow.
In this embodiment, it should be noted that, the archive tasks in the archive task list are executed according to the chronological order of adding to the archive task list, and the next archive task is usually triggered to be executed after one archive task is executed.
The archiving tasks can be classified into the following two types according to the data type of the data to be archived (i.e., target data): an archiving task of video data type, an archiving task of picture data type. Different types of archiving tasks correspond to different archiving task execution flows.
As an embodiment, referring to fig. 7, the method flow of the archiving task execution in the sixth step includes the following steps:
step 701, judging whether an archiving task exists in the archiving task list; if yes, go to step 702; if not, the process continues to step 701.
Step 702, obtaining a current archiving task from the archiving task list according to the time sequence of adding to the archiving task list.
In step 703, if the current archiving task is a video data type archiving task, corresponding video data is obtained from the storage space as target data according to the set archiving schedule period of the current archiving task, the target data is written into the magnetic tape, and step 705 is continuously performed.
If the current archiving task is an archiving task of a picture data type, step 704, the corresponding picture is obtained from the storage space as target data according to the set archiving schedule period of the current archiving task, the target data is written into the tape, and step 705 is continuously performed.
Step 705, after the current archiving task is executed, deleting the current archiving task in the archiving task list, and returning to execute step 701.
In this embodiment, as an embodiment, when the above-mentioned target data is archived, in the case where the data amount of the target data is large, the target data may be divided into a plurality of batches to obtain a plurality of sub-target data, and then, a corresponding archiving operation is performed on each sub-target data, so that the situations of excessively high resource occupancy, reduced device performance, and the like due to the large data amount of the data to be archived that is archived at one time can be avoided. The sub-target data under each batch can be archived according to the sequence of the collection time periods.
Based on the above description, the following describes the specific implementation procedure of step 703 above:
for example, in an archiving operation after dividing the target data into a plurality of batches, referring to fig. 8, the specific implementation process of step 703 is as follows:
Step 801, for the sub-target data in the current batch, obtain the storage location and size of the sub-target data in the current batch in the storage space from the second index area.
Step 802, inquiring whether the sub-target data in the current batch has completed archiving operation in the first index area, if yes, executing step 803; if not, step 804 is performed.
Here, if index information corresponding to the sub-target data in the current batch can be queried in the first index area, it may be determined that the sub-target data in the current batch has completed the archiving operation, otherwise, it is determined that the sub-target data in the current batch has not completed the archiving operation.
Step 803, when the current lot is not the last lot, one lot of the lots that are not archived is taken as the current lot, and step 801 is returned.
Step 804, writing the information recorded in the first index area into the target tape in the form of a file; and proceeds to step 805.
As to how to write the information recorded in the first index area to the target tape in the form of a file, the following description will be given by way of example, and will not be repeated here.
Step 805, after the sub-target data in the current lot is read from the storage space, the read data is written into the target tape in the form of a file.
Here, the name of the file may be determined based on the device identification (e.g., device name and device number, etc.) of the data collection device, and the data collection time period (e.g., including the start time and the end time) in which the data is located.
Step 806, updating the information recorded in the first index area, and returning to step 801 by taking one of the non-archived batches as the current batch when the current batch is not the last batch.
When the current batch is the last batch, the target data corresponding to the current archiving task is completely archived, and the current archiving task can be considered to be executed.
Here, the information recorded how to update the first index area is not particularly limited; for example, as shown in table 1, the correspondence relationship among the archiving order corresponding to the sub-target data in the current lot, the set archiving plan time period (including the start time and the end time of the sub-target data), the file name (including the device identifier of the data acquisition device and the set archiving plan time period), and the tape number of the occupied tape may be recorded in the first index area.
Table 1: first index region
For example, the data acquisition device a performs video recording of 8:00-20:00 a day, the storage device obtains video data from the data acquisition device a for 12 hours a day of 8:00-20:00 a day, and stores all obtained video data in the storage space, the management device configures a set archiving plan time period for the data acquisition device a to be 9:00-12:00, and sets archiving execution time period to be 21:00-23:00. When the management device executes the corresponding archiving task, the management device reads the video data acquired by the data acquisition device A for 3 hours, namely 9:00-12:00, and reads the data for 1 hour each time, so that the management device can be divided into 3 batches to read the video data from the storage space and then write the video data into the magnetic tape in the form of files; the archiving task execution ends at 23:00.
The following describes a specific implementation procedure of the step 704:
For example, in an archiving operation after dividing the target data into a plurality of batches (for example, taking a picture as a batch), referring to fig. 9, the specific implementation process of the step 704 is as follows:
step 901, for the current picture, obtaining the storage position and size of the current picture in the storage space from the second index area.
Step 902, inquiring whether the current picture has completed archiving operation in the first index area, if yes, executing step 903; if not, step 904 is performed.
Here, the current picture may be understood as sub-target data under the current lot.
Inquiring whether the current picture has completed archiving operation in the first index area, for example, when the archiving operation is specifically implemented: if the index information corresponding to the current picture can be queried in the first index area, determining that the current picture has completed the archiving operation, otherwise, determining that the current picture has not completed the archiving operation.
In step 903, when the current picture is not the last picture that is not archived in the target data, one picture in the pictures that are not archived in the target data is taken as the current picture, and step 901 is returned.
In step 904, the information recorded in the first index area is written to the target tape in the form of a file.
In step 905, after the current picture is read from the storage space, the current picture is written into the target tape.
Here, since the picture itself is stored in the storage space in the form of a file, that is, the picture itself is a file, there is no need to perform a process of converting into a file. The name of the picture may be determined based on the device identification (e.g., device name and device number, etc.) of the data acquisition device, the acquisition timestamp of the picture, etc., and is not particularly limited herein.
Step 906, updating the information recorded in the first index area, and returning to step 901 by taking one of the non-archived pictures in the target data as the current picture when the current picture is not the last non-archived picture in the target data.
When the current picture is the last picture which is not archived in the target data, the current picture indicates that the target data corresponding to the current archiving task has all completed archiving operation, and at this time, the current archiving task can be considered to have been executed and completed.
Here, the information recorded how to update the first index area is not particularly limited; for example, as shown in table 2, the correspondence between the archiving order corresponding to the current picture, the acquisition time stamp of the picture, the file name, and the tape number of the occupied tape may be recorded in the first index area.
Table 2: first index region
For example, the data acquisition device B performs a snapshot of a picture 10:00-18:00 a day, the storage device acquires snapshot picture data from the data acquisition device B10:00-18:00 a day for 8 hours, and stores the acquired picture data in the storage space, and the management device configures a set archiving plan time period for the data acquisition device B to be 10:00-15:00 and a set archiving execution time period to be 23:10-23:30. If data acquisition device B takes 1000 pictures within the 5 hours of 10:00-15:00, the 1000 pictures will begin archiving operations from storage space into tape of the tape library at 23:10.
When the management device executes the corresponding archiving task, 1000 pictures acquired by the data acquisition device B in the time of 10:00-15:00 for 5 hours are read from the storage space each time according to the sequence of the acquisition time and written into the magnetic tape of the magnetic tape library, and the archiving task is completed after the 1000 times of repeated execution; the archiving task execution ends at 23:30.
The following describes how the information recorded in the second index area is written to the target tape in the form of a file:
as an embodiment, referring to fig. 10, the writing of the information recorded in the second index area to the target tape in the form of a file may include the following steps when in specific implementation:
Step 1001, inquiring whether the target tape is used in the first index area; if not, go to step 1002; if yes, go to step 1005.
Step 1002, checking whether there is old data remaining on the target tape; if yes, go to step 1003; if not, go to step 1004.
Step 1003, deleting old data remained in the target tape; and proceeds to step 1004.
Step 1004, writing the information recorded in the first index area into the tape head of the target tape in the form of a file; and proceeds to step 1005.
Step 1005, checking whether the available space size of the target tape is greater than or equal to the sum of the size of the sub-target data in the current batch and the size of the information recorded in the first index area; if yes, go to step 1006; if not, step 1007 is performed.
Step 1006, the process is exited.
Step 1007, writing the information recorded in the first index area to the tail of the target tape in the form of a file; execution continues with step 1008.
Here, if the size of the available space of the target tape is greater than or equal to the sum of the size of the sub-target data in the current batch and the size of the information recorded in the first index area, it indicates that the available space of the target tape is sufficient to accommodate the sub-target data in the current batch and the information recorded in the first index area, and at this time, the process may be exited, and the above-mentioned step of writing the sub-target data in the current batch into the target tape (i.e. step 804 or step 904) may be performed.
If the size of the available space of the target tape is smaller than the sum of the size of the sub-target data in the current batch and the size of the information recorded in the first index area, the target tape cannot accommodate the sub-target data in the current batch and the information recorded in the first index area, at this time, the information recorded in the first index area can be written into the tail of the target tape in the form of a file, and then one other available tape is determined from the tape library to be used for storing the sub-target data in the current batch.
Step 1008, other available tapes are determined from the tape library and returned to step 1001 with the other available tapes as target tapes.
As to how to determine other available tapes from the tape library, reference is made to the description of step 106 above, which is not repeated here.
In this embodiment, in order to avoid the problem that the first index area in the storage device cannot be queried due to damage to the first index area, the information recorded in the first index area is backed up in the tape after the archiving task is completed, so that it can be ensured that when the first index area is abnormally damaged, the data recovery can be performed on the first index area in the storage device through the backed-up first index area information in the tape.
Step seven: and (5) inquiring the data.
In this embodiment, as an embodiment, when an external data query instruction is received, whether an archive task exists in the current archive task list is checked; the data query instruction at least indicates the equipment identifier of the data acquisition equipment from which the query data to be queried is located and the query time period in which the query data is located;
Then, if no archiving task exists in the current archiving task list, checking whether query data corresponding to the data query instruction exists in the designated storage space according to the data query instruction and the second index area; if so, acquiring a storage position corresponding to the query data from the second index area based on the query time period indicated by the data query instruction and the equipment identifier, acquiring and outputting the query data based on the storage position; if not, based on the query time period and the equipment identifier indicated by the data query instruction, acquiring the magnetic tape occupied by the query data and the data identifier corresponding to the query data from the first index area, acquiring the query data from the magnetic tape occupied by the query data based on the data identifier corresponding to the query data, and outputting the query data;
if the current archiving task list has the archiving task, outputting the indication information of the failure of the data query.
In this embodiment, as an embodiment, the obtaining and outputting query data from a magnetic tape occupied by the query data based on the data identifier corresponding to the query data may be, for example, when the specific implementation is: firstly, checking whether the magnetic tape occupied by query data exists in the magnetic tape library or not based on the magnetic tape identifications of all the magnetic tapes in the magnetic tape library; then, if the magnetic tape occupied by the query data exists in the magnetic tape library, checking whether the available space size of the designated defrosting space (namely the defrosting space) is larger than or equal to a preset threshold value, if so, reading the query data from the magnetic tape occupied by the query data based on the data identifier corresponding to the query data and writing the query data into the designated defrosting space, if not, cleaning the data with earliest time written into the designated defrosting space in the designated defrosting space, and returning to the step of checking whether the available space size of the designated defrosting space is larger than or equal to the preset threshold value; and finally, acquiring and outputting the query data from the designated thawing space based on the data identification corresponding to the query data. The data identification here may be a file name or the like.
For example, referring to fig. 11, the method flow of the data query in the seventh step may include the following steps:
step 1101, receiving an external data query instruction.
Here, the data query instruction indicates at least a device identification of the data acquisition device from which the query data to be queried is coming, and a query period in which the query data is located. For example, the data query instruction may be "find data collected by the data collection device a between 8 and 10 points on 29 th year 2024, 4 month", or the like; based on this, the device identification may be "data acquisition device a", and the query period may be "2024, 4, 29, 8 to 10 points.
Step 1102, checking whether an archiving task exists in the current archiving task list; if yes, go to step 1103; if not, go to step 1104.
In step 1103, an indication of the failure of the data query is returned.
Here, in order to avoid interaction between data query and archive task execution, the archive task is preferentially executed when there is an archive task in the current archive task list.
Step 1104, inquiring whether the inquiry data to be inquired exists in the storage space in the second index area; if yes, go to step 1105; if not, then step 1106 is performed.
Step 1105, based on the query time period indicated by the data query instruction and the device identifier, obtaining a storage location corresponding to the query data from the second index area, and obtaining and outputting the query data based on the storage location.
Step 1106, based on the query time period indicated by the data query instruction and the device identifier, obtaining the magnetic tape occupied by the query data, the number of files, the file names and the file sizes corresponding to the query data from the first index area as a query result; and proceeds to step 1107.
Step 1107, when the magnetic tape indicated by the query result exists in the magnetic tape library, judging whether the available space size of the unfreezing space is larger than or equal to a preset threshold value; if yes, go to step 1108; if not, step 1109 is performed.
Here, for each tape indicated by the above query result, if it is found that the tape does not exist in the tape library, a prompt is issued to indicate that the tape does not exist in the tape library, and at this time, the tape is manually placed in the tape library, and then the following steps are continued.
Step 1108, reading query data from the magnetic tape indicated by the query result based on the data identifier corresponding to the query data, and writing the read query data into the thawing space; step 1110 continues.
Step 1109, the file written to the thawing space with the earliest time in the thawing space is cleaned up, and the process returns to step 1107.
Step 1110, reading a file matching the file name from the thawing space based on the file name indicated by the query result as query data and outputting the query data.
And step seven, a fault recovery flow of the first index area.
As one embodiment, when an abnormality is found in the first index area, checking whether at least one candidate data exists in each tape of the tape library based on the data identifier corresponding to the data recorded in each tape of the tape library; here, the candidate data refers to data indicating the first index area information; and if at least one candidate data exists, determining the reference data from each magnetic tape of the magnetic tape library based on the time of writing into the magnetic tape corresponding to the recorded data in each magnetic tape of the magnetic tape library, and clearing the first index area, and reading the reference data from the magnetic tape library and writing into the first index area so as to realize the recovery of the information of the first index area. Here, the reference data refers to candidate data written to the magnetic tape at the latest time.
As to how to check whether the first index area has an abnormality, the following description will be given by way of example, and will not be repeated here.
For example, referring to fig. 12, the fault recovery procedure of the first index area in the seventh step may include the following steps:
Step 1201, checking whether an abnormality occurs in a first index area of the storage device; if yes, go to step 1202; if not, the process continues to step 1201.
In this embodiment, there are many specific implementations for checking whether an abnormality occurs in the first index area of the storage device, for example, the implementation may be: and determining that the first index area is abnormal when the first index area is not accessible or when the inaccessible times of the first index area are larger than a set value. The inaccessible situation here may be, for example, that data cannot be written and/or read from the first index area, which is not particularly limited here.
Step 1202, checking whether a candidate file exists in each tape of the tape library based on the file name corresponding to the file recorded in each tape of the tape library and the file writing time; if yes, go to step 1203.
Here, the file writing time refers to the time at which the file is written to the magnetic tape. The candidate file refers to a file in which the first index area information is recorded (i.e., the candidate data described above). If no candidate file exists in each tape of the tape library, the tape which has been filed recently is selected from the tapes outside the tape library manually and put into the tape library, and then the step is continued.
Step 1203, selecting a candidate file with the latest file writing time from the candidate files as a reference file, and clearing the first index area, reading the reference data from the tape library, and writing the reference data into the first index area.
Here, the reference document is also the above-mentioned reference data.
According to the embodiment, the information recorded in the first index area is backed up to the magnetic tape after the archiving task is finished, so that when the first index area in the storage device is abnormal, the first index area can be rebuilt again through the information of the first index area recorded in the magnetic tape, the first index area can be continuously used for recording the information that the archiving data occupies the magnetic tape, and the data in the magnetic tape can be retrieved again based on the restored first index area, so that the reliability of data archiving and inquiring is improved.
Note that, the present embodiment is not particularly limited to the sequence of the steps of the data archiving method.
Thus, the description of the method provided in this embodiment is completed, and the following describes the device provided in this embodiment of the present application:
As an embodiment, the present embodiment also provides a data archiving device. Referring to fig. 13, fig. 13 is a schematic structural diagram of a device according to an embodiment of the present application. As shown in fig. 13, the apparatus 1300 includes:
An obtaining module 1301, configured to obtain a current archiving task; the current archiving task is used for indicating to conduct archiving operation on target data acquired by data acquisition equipment associated with the current archiving task in a set archiving execution time period; the target data is data acquired by the data acquisition device in the configured set archiving schedule period; setting an archive execution period after setting an archive plan period; the target data is stored in a designated storage space after being obtained from the data acquisition equipment before being archived;
A checking module 1302 for checking whether the target tape is used based on the designated first index area; the target tape is used for storing the archived target data; the first index area records at least the occupied information of the magnetic tape;
a first archiving module 1303, configured to write, if the target tape is not used, information recorded in the first index area to the target tape;
A second archiving module 1304, configured to, after writing the information recorded in the first index area to the target tape, or after determining that the target tape is used, check whether the available space of the target tape meets a requirement for accommodating the target data and the information recorded in the first index area, if so, write the target data to the target tape, if not, write the information recorded in the first index area to the target tape, and write the target data to other available tapes;
A recording module 1305, configured to record information of the target data occupying the magnetic tape in the first index area.
As an embodiment, the apparatus further comprises: the generation module is used for checking whether the start time of the set archiving execution time period configured by the data acquisition device is matched with the current time or not according to each data acquisition device; if so, checking whether an archiving task associated with the data acquisition device exists in the archiving task list, if not, generating the archiving task associated with the data acquisition device and adding the archiving task to the archiving task list, and if so, returning to the step of checking whether the starting time of the set archiving execution time period configured by the data acquisition device is matched with the current time for each data acquisition device;
the current archive task is obtained from the archive task list in chronological order of addition to the archive task list.
The recording module 1305 is further configured to, as one embodiment: after the target data are stored in the appointed storage space, recording the occupation condition of the target data in the appointed storage space into the appointed second index area; the occupation condition at least comprises the size of the target data and the storage position of the target data;
Checking whether the available space of the target tape meets the requirement of accommodating the target data and the information recorded in the first index area, includes: obtaining a size of the target data from the second index area; checking whether the size of the available space of the target tape is larger than or equal to the sum of the size of the target data and the size of the information recorded in the first index area, if so, determining that the available space of the target tape meets the requirement of accommodating the target data and the information recorded in the first index area, and if not, determining that the available space of the target tape does not meet the requirement of accommodating the target data and the information recorded in the first index area;
Writing target data to a target tape or other available tape includes: a storage location for the target data is obtained from the second index area, and the target data is read from the designated storage space and written to the target tape or other available tape based on the storage location.
As one embodiment, the information of the target data occupying the magnetic tape recorded in the first index area includes at least: tape identification of occupied tape;
Other available tapes are determined by the following steps: determining a tape identification of the occupied tape based on the information recorded in the first index region; determining a current unoccupied tape from the tape library according to the determined tape identifications of the occupied tape and the tape identifications of all the tapes in the tape library, and taking the unoccupied tape as other available tapes;
The target data is video data; writing target data to a target tape or other available tape includes: convert the target data to a file and write the file to the target tape or other available tape.
As one embodiment, the occupation condition of the target data recorded in the second index area in the designated storage space further includes: the method comprises the steps of setting an equipment identifier of data acquisition equipment and target data into an archiving plan time period; the information of the target data occupying the magnetic tape recorded in the first index area further comprises: the method comprises the steps of setting equipment identification of data acquisition equipment, setting an archiving plan time period and data identification corresponding to target data;
the apparatus further comprises: the query module is used for checking whether query data corresponding to the data query instruction exists in the designated storage space according to the data query instruction and the second index area if no archiving task exists in the current archiving task list when an external data query instruction is received; the data query instruction at least indicates the equipment identification of the data acquisition equipment from which the query data is received and the query time period; if so, acquiring a storage position corresponding to the query data from the second index area based on the query time period indicated by the data query instruction and the equipment identifier, acquiring and outputting the query data based on the storage position; if not, based on the query time period indicated by the data query instruction and the equipment identifier, acquiring the magnetic tape occupied by the query data and the data identifier corresponding to the query data from the first index area, acquiring the query data from the magnetic tape occupied by the query data based on the data identifier corresponding to the query data, and outputting the query data.
As one embodiment, obtaining and outputting query data from a magnetic tape occupied by the query data based on a data identifier corresponding to the query data, includes: checking whether the magnetic tape occupied by the query data exists in the magnetic tape library based on the magnetic tape identifications of all the magnetic tapes in the magnetic tape library; if the magnetic tape occupied by the query data exists in the magnetic tape library, checking whether the available space size of the designated defrosting space is larger than or equal to a preset threshold value, if so, reading the query data from the magnetic tape occupied by the query data based on the data identifier corresponding to the query data and writing the query data into the designated defrosting space, if not, cleaning the data with earliest time written into the designated defrosting space in the designated defrosting space, and returning to the step of checking whether the available space size of the designated defrosting space is larger than or equal to the preset threshold value; and obtaining the query data from the designated thawing space based on the data identification corresponding to the query data and outputting the query data.
As an embodiment, the apparatus further comprises: the abnormality processing module is used for checking whether at least one candidate data exists in each magnetic tape of the magnetic tape library or not based on the data identifier corresponding to the data recorded in each magnetic tape of the magnetic tape library when the first index area is found to be abnormal; the candidate data refers to data indicating the first index area information; if at least one candidate data exists, determining reference data from each tape of the tape library based on the time of writing to the tape corresponding to the recorded data in each tape of the tape library, and clearing the first index area, reading the reference data from the tape library, and writing the reference data into the first index area; the reference data refers to the latest candidate data written to the magnetic tape.
As an embodiment, the apparatus further comprises: the deleting module is used for checking whether an archiving task exists in the archiving task list, and if so, acquiring the archiving task with the earliest time added to the archiving task list from the archiving task list as a target archiving task; checking whether the end time of the set archiving execution period indicated by the target archiving task matches the current time; if so, checking whether the execution state of the target archive task is a first state used for representing the execution, if so, stopping executing the target archive task, deleting the target archive task from an archive task list, and if not, deleting the target archive task from the archive task list; if not, or after deleting the target archive task from the archive task list, a step of checking whether the archive task exists in the archive task list is returned.
As one embodiment, the set archive execution time periods for which the different data acquisition devices are configured are disjoint when the types of data acquired by the different data acquisition devices are different.
The device description shown in fig. 13 is thus completed. The implementation process of the functions and roles of each module in the above device is specifically shown in the implementation process of the corresponding steps in the above method, and will not be described herein again.
Referring to fig. 14, a schematic hardware structure of an electronic device according to an exemplary embodiment of the application is shown. The electronic device may include a processor 1401, a communication interface 1402, a memory 1403, and a communication bus 1404. The processor 1401, the communication interface 1402, and the memory 1403 perform communication with each other through the communication bus 1404. Wherein the memory 1403 has stored thereon computer program instructions; the processor 1401 may perform the steps of the method described in the above embodiments by executing computer program instructions stored on the memory 1403. The electronic device may further include other hardware according to the actual function of the electronic device, which will not be described in detail.
Correspondingly, the embodiment of the application also provides a computer readable storage medium, and a plurality of computer program instructions are stored on the computer readable storage medium, and when the computer program instructions are executed by a processor, the method disclosed by the above example of the application can be realized.
By way of example, the above-described computer-readable storage media may be any electronic, magnetic, optical, or other physical storage device that can contain or store information, such as executable instructions, data, and the like. For example, the computer readable storage medium may be: RAM (Radom Access Memory, random access memory), volatile memory, non-volatile memory, flash memory, a storage drive (e.g., hard drive), a solid state disk, any type of storage disk (e.g., optical disk, dvd, etc.), or a similar storage medium, or a combination thereof. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.
The foregoing description of the preferred embodiments of the application is not intended to be limiting, but rather is intended to cover all modifications, equivalents, alternatives, and improvements that fall within the spirit and scope of the application.
Claims (9)
1. A method of archiving data, the method comprising:
Obtaining a current archiving task; the current archiving task is used for indicating to conduct archiving operation on target data acquired by data acquisition equipment associated with the current archiving task in a set archiving execution time period; the target data are data acquired by the data acquisition equipment in a configured set archiving plan time period; setting an archive execution period after setting an archive plan period; the target data are stored in a designated storage space after being obtained from the data acquisition equipment before being archived;
Checking whether the target tape is used based on the designated first index area; the target tape is used for storing the archived target data; the first index area records at least the occupied information of the magnetic tape;
if the target tape is not used, writing the information recorded in the first index area into the target tape;
After writing the information recorded in the first index area into a target tape, or after determining that the target tape is used, checking whether the available space of the target tape meets the requirement of accommodating the target data and the information recorded in the first index area, if so, writing the target data into the target tape, if not, writing the information recorded in the first index area into the target tape, and writing the target data into other available tapes;
Recording information of the target data occupying the magnetic tape in the first index area; the information of the target data occupying the magnetic tape recorded in the first index area at least comprises: a data identifier corresponding to the target data;
when the first index area is found to be abnormal, checking whether at least one candidate data exists in each magnetic tape of the magnetic tape library or not based on the data identifier corresponding to the data recorded in each magnetic tape of the magnetic tape library; the candidate data refers to data indicating first index area information; if at least one candidate data exists, determining reference data from each magnetic tape of the magnetic tape library based on the time of writing to the magnetic tape corresponding to the recorded data in each magnetic tape of the magnetic tape library, and clearing a first index area, and reading the reference data from the magnetic tape library and writing into the first index area; the reference data refers to the latest candidate data written to the magnetic tape.
2. The method according to claim 1, characterized in that the method is preceded by the further step of:
Checking, for each data acquisition device, whether a start time of a set archive execution period configured by the data acquisition device matches a current time;
If so, checking whether an archiving task associated with the data acquisition device exists in the archiving task list, if not, generating the archiving task associated with the data acquisition device and adding the archiving task to the archiving task list, and if so, returning to the step of checking whether the starting time of the set archiving execution time period configured by the data acquisition device is matched with the current time for each data acquisition device;
the current archiving task is obtained from the archiving task list according to the time sequence of adding to the archiving task list.
3. The method of claim 1, wherein the target data, after being stored to the designated storage space, further comprises: recording the occupation condition of the target data in the appointed storage space to an appointed second index area; the occupation condition at least comprises the size of target data and the storage position of the target data;
Said checking whether the available space of said target tape meets the requirements for accommodating said target data and said first index area recorded information comprises: obtaining a size of the target data from the second index area; checking whether the available space size of the target tape is greater than or equal to the sum of the size of the target data and the size of the information recorded in the first index area, if so, determining that the available space of the target tape meets the requirement of accommodating the target data and the information recorded in the first index area, and if not, determining that the available space of the target tape does not meet the requirement of accommodating the target data and the information recorded in the first index area;
Writing the target data to a target tape or other available tape includes: a storage location for the target data is obtained from the second index area, and the target data is read from the designated storage space based on the storage location and written to a target tape or other available tape.
4. The method of claim 3, wherein the information recorded in the first index area that the target data occupies the tape further comprises: tape identification of occupied tape;
The other available tapes are determined by: determining a tape identification of the occupied tape based on the information recorded in the first index region; determining a currently unoccupied tape from the tape library according to the determined tape identifications of the occupied tape and the tape identifications of all tapes in the tape library, and taking the unoccupied tape as the other available tapes;
the target data is video data; writing the target data to a target tape or other available tape includes: converting the target data into a file and writing the file to a target tape or other available tape.
5. The method of claim 4, wherein the occupancy of the target data recorded in the second index area in the designated storage space further comprises: the equipment identification of the data acquisition equipment and the set archiving plan time period of the target data;
the information of the target data occupying the magnetic tape recorded in the first index area further comprises: the equipment identification of the data acquisition equipment and the set archiving plan time period;
The method further comprises the steps of: when an external data query instruction is received, if no archiving task exists in the current archiving task list, checking whether query data corresponding to the data query instruction exists in the designated storage space according to the data query instruction and the second index area; the data query instruction at least indicates the equipment identification of the data acquisition equipment from which the query data is received and the query time period;
If so, acquiring a storage position corresponding to the query data from the second index area based on a query time period indicated by the data query instruction and the equipment identifier, acquiring the query data based on the storage position and outputting the query data;
If not, acquiring the magnetic tape occupied by the query data and the data identifier corresponding to the query data from the first index area based on the query time period and the equipment identifier indicated by the data query instruction, acquiring and outputting the query data from the magnetic tape occupied by the query data based on the data identifier corresponding to the query data.
6. The method of claim 5, wherein the obtaining and outputting the query data from the magnetic tape occupied by the query data based on the data identification corresponding to the query data comprises:
checking whether the magnetic tape occupied by the query data exists in the magnetic tape library based on the magnetic tape identifications of all the magnetic tapes in the magnetic tape library;
If the magnetic tape occupied by the query data exists in the magnetic tape library, checking whether the available space size of the designated defrosting space is larger than or equal to a preset threshold value, if so, reading the query data from the magnetic tape occupied by the query data based on a data identifier corresponding to the query data and writing the query data into the designated defrosting space, if not, clearing the data with earliest time written into the designated defrosting space in the designated defrosting space, and returning to the step of checking whether the available space size of the designated defrosting space is larger than or equal to the preset threshold value;
And obtaining the query data from the designated defrosting space based on the data identifier corresponding to the query data and outputting the query data.
7. The method according to claim 2, characterized in that the method further comprises:
checking whether an archiving task exists in the archiving task list, and if so, acquiring the earliest archiving task added to the archiving task list from the archiving task list as a target archiving task;
checking whether the end time of the set archiving execution period indicated by the target archiving task matches the current time;
If so, checking whether the execution state of the target archive task is a first state used for representing the execution, if so, stopping executing the target archive task, deleting the target archive task from an archive task list, and if not, deleting the target archive task from the archive task list;
If not, or after deleting the target archive task from the archive task list, a step of checking whether the archive task exists in the archive task list is returned.
8. The method of claim 1, wherein the set archive execution time periods for which different data acquisition devices are configured are disjoint when the types of data acquired by the different data acquisition devices are different.
9. An electronic device, characterized in that, the electronic device includes:
A processor; and
A memory in which computer program instructions are stored which, when executed by the processor, cause the processor to perform the steps of the method of any one of claims 1 to 8.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202411095926.3A CN118626439B (en) | 2024-08-09 | 2024-08-09 | Data archiving method and electronic equipment |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202411095926.3A CN118626439B (en) | 2024-08-09 | 2024-08-09 | Data archiving method and electronic equipment |
Publications (2)
Publication Number | Publication Date |
---|---|
CN118626439A CN118626439A (en) | 2024-09-10 |
CN118626439B true CN118626439B (en) | 2024-10-29 |
Family
ID=92605356
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202411095926.3A Active CN118626439B (en) | 2024-08-09 | 2024-08-09 | Data archiving method and electronic equipment |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN118626439B (en) |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114860558A (en) * | 2022-04-21 | 2022-08-05 | 中国资源卫星应用中心 | Method convenient for remote sensing satellite data archiving and recovery performance testing |
CN117539392A (en) * | 2023-11-13 | 2024-02-09 | 湖南快乐阳光互动娱乐传媒有限公司 | Tape library data processing method and system, electronic equipment and storage medium |
Family Cites Families (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH07121410A (en) * | 1993-10-22 | 1995-05-12 | Hitachi Ltd | Archiving method |
JP2000250845A (en) * | 1999-03-02 | 2000-09-14 | Hitachi Ltd | Method and device for controlling recording medium competition and medium recorded with the processing program therefor |
US8954663B1 (en) * | 2012-06-25 | 2015-02-10 | Kip Cr P1 Lp | System, method and computer program product for synchronizing data written to tape including writing an index into a data partition so that data can be recovered in case of failure |
CN103605587A (en) * | 2013-11-20 | 2014-02-26 | 上海爱数软件有限公司 | Tape library data backup and filing method |
CN105573863A (en) * | 2015-12-14 | 2016-05-11 | 北京尚易德科技有限公司 | Index file recovery method and apparatus and video monitoring system |
CN107590030A (en) * | 2017-09-29 | 2018-01-16 | 郑州云海信息技术有限公司 | A kind of method, apparatus and system of data recording on tape backup |
US11152018B1 (en) * | 2020-10-29 | 2021-10-19 | International Business Machines Corporation | Optimized tape unmount operation |
CN113190520A (en) * | 2021-05-26 | 2021-07-30 | 中国建设银行股份有限公司 | Method and system for processing filing log |
CN115421983A (en) * | 2022-09-29 | 2022-12-02 | 中国农业银行股份有限公司 | Data backup method and device based on mainframe |
-
2024
- 2024-08-09 CN CN202411095926.3A patent/CN118626439B/en active Active
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114860558A (en) * | 2022-04-21 | 2022-08-05 | 中国资源卫星应用中心 | Method convenient for remote sensing satellite data archiving and recovery performance testing |
CN117539392A (en) * | 2023-11-13 | 2024-02-09 | 湖南快乐阳光互动娱乐传媒有限公司 | Tape library data processing method and system, electronic equipment and storage medium |
Also Published As
Publication number | Publication date |
---|---|
CN118626439A (en) | 2024-09-10 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US6557073B1 (en) | Storage apparatus having a virtual storage area | |
US8250033B1 (en) | Replication of a data set using differential snapshots | |
US7334083B2 (en) | Library system, virtual library apparatus, cache restoring method, and computer readable recording medium recorded thereon a cache restoring program | |
US20050262316A1 (en) | Backup acquisition method and disk array apparatus | |
US20070043903A1 (en) | Data Processing Apparatus and Method, Control Program Therefor, and Recording Medium Having Program Recorded Thereon | |
CN102314383A (en) | Failure recovery method and device for data index | |
CN112269681A (en) | Method, device and equipment for continuously protecting virtual machine data | |
CN107315659B (en) | Metadata redundancy backup method and device | |
CN103761165A (en) | Log backup method and log backup device | |
CN118626439B (en) | Data archiving method and electronic equipment | |
CN102355568A (en) | Method and device for carrying out charged uninstallation and installation of hard disk for digital video recorder | |
CN103605587A (en) | Tape library data backup and filing method | |
WO2018010766A1 (en) | Apparatus and method for backing up data of computing devices in a computer network | |
US7600151B2 (en) | RAID capacity expansion interruption recovery handling method and system | |
US9002788B2 (en) | System for configurable reporting of network data and related method | |
CN112181311B (en) | Cyclic recording method, equipment and storage medium | |
CN110019233B (en) | Data storage method and system | |
CN115328696A (en) | Data backup method in database | |
CN112784101B (en) | Video data storage method and device and data storage equipment | |
CN115562582A (en) | Data migration method based on blue-ray disc library, server and blue-ray disc library | |
CN117785070B (en) | Data storage control method and device | |
CN105072372A (en) | Automatic repair method for FAT file system on DVR terminal | |
JP4466192B2 (en) | Information recording apparatus, information recording method, and program | |
JP6287055B2 (en) | Information processing apparatus, information collection method, and information collection program | |
CN114880277B (en) | Block device snapshot method, system and storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant |