JP2007025843A - Data storage device and version management program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a data storage device capable of reading, even if versions are increased, data of the latest version at high speed. <P>SOLUTION: The data storage device 1 comprises a meta data generation means 23b generating metadata by associating the latest data inputted, the version of the latest data, and the address of data preceding by one generation to the latest data stored in a memory 30; a differential data generation means 21b generating differential data that corresponds to the difference between the latest data and the data preceding by one generation to the latest data stored in the memory 30; and a data change means 23c changing the differential data generated by the differential data generation means 21b to data preceding by one generation to the latest data in reference to the metadata. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a data storage device and a version management program for storing and managing data in association with a plurality of versions.

  In general, when creating data such as software (program), content (video data, etc.), storage means (storage device) in which a version (version number) is associated with newly created data as data is added or changed The method of storing and managing the information is used. At this time, a plurality of versions of data are stored in the storage means in order to reuse or refer to old versions of data. In this case, storage means with a large capacity is required as the data version increases.

However, there is usually a lot of overlapping data between versions of data, and storing the data in the storage means for each version as it is is inefficient in use of the storage means, and the cost for the storage means increases accordingly. ing.
Therefore, conventionally, the entire data is stored in the storage means only in the latest version data, and the previous previous version data is only the difference from the latest version data. Thus, every time the latest data is created, the data of the previous version becomes differential data from the latest data, and all the data other than the latest version data becomes differential data (see, for example, Patent Document 1).

Here, with reference to FIG. 13, the outline | summary is demonstrated about the method of managing the data of several versions by the conventional difference data. FIG. 13 is a diagram schematically illustrating a version management method for managing a plurality of versions of data using conventional difference data.
As shown in FIG. 13, in the conventional version management method, data is managed with directory information having a directory structure (for example, Btree [Balanced Tree)), and each data (A and B in this case) is stored. Store in means 30.

At this time, for the old version, only the difference (difference data) between the one new version and the data of the old version is stored in the storage means 30. Here, with respect to the latest data A, the difference data A1 of the previous generation, the difference data A2 of the previous generation, and the like are stored in the storage means 30. Similarly, with respect to the latest data B, the difference data B1 of the previous generation is stored in the storage means 30.
For example, the data one generation before the latest data A can be generated from the data A and the difference data A1. Further, the data two generations before the latest data A can be generated from the data A, the difference data A1, and the difference data A2.

As described above, the conventional version management method reads the latest data A based on the address of each data managed by the directory information, and further reads the difference data A1 and the like to synthesize the past versions. Generate data.
As a result, the conventional version management method stores the entire data as it is only for the latest data, and the past version data is only the difference data, and the capacity of the storage means 30 can be suppressed.
Japanese Patent Laid-Open No. 2000-200018

As described above, in the conventional version management method, when reading a desired version of data, it is necessary to search the address of the data and the difference data from the same directory information. However, as the data version increases, the number of difference data to be generated increases, so the time for searching for the address for both the latest data and the difference data becomes longer, and consequently the time for reading the desired data becomes longer. There is.
In general, access to data tends to decrease rapidly as the version gets older. Therefore, there is a demand for not delaying the reading time of the new version data even if the reading time of the old version data is delayed.

  The present invention has been made in view of the problems as described above, and provides a data storage device and a version management program capable of reading the latest version of data at high speed even when the version is increased. The purpose is to provide.

  The present invention has been created to achieve the above object. First, the data storage device according to claim 1 stores a plurality of data in a storage means for a plurality of generations by a hierarchical structure. In the data storage device, the storage unit and the metadata generation unit are provided.

In such a configuration, the data storage device stores data for a plurality of generations in the storage means. At this time, the data storage device associates the latest data input by the metadata generation unit, the version of the latest data, and the address where the data one generation before the latest data is stored in the storage unit. Generate metadata. This metadata may be added to the data to form a single file together with the data, or may be stored in the storage means separately from the data. However, when data and metadata are stored separately, the metadata is associated with the metadata of the previous generation.
Thus, the data storage device does not need to search past data from the tree structure managed by the directory information, and sequentially accesses past versions of data by referring to the metadata. It becomes possible.

  According to a second aspect of the present invention, there is provided the data storage device according to the first aspect, further comprising a difference data generation unit and a data change unit.

  In such a configuration, the data storage device generates difference data, which is a difference between the latest data and the data one generation before the latest data stored in the storage unit, by the difference data generation unit. This differential data is the amount of data added to the previous generation when data is added, the amount of change when the data is changed, and the amount of deletion when part of the data is deleted. It is the data shown.

Further, the data storage device refers to the metadata by the data changing unit, and changes the data one generation before the latest data to the differential data generated by the differential data generating unit. As a result, the data one generation before is changed to data (difference data) having only the difference from the latest data.
As a result, only the latest version of data is managed as original data that is not difference data, and the previous version of data is only the difference (difference data) from the latest version of data.

  According to a third aspect of the present invention, there is provided the data storage device according to the second aspect, wherein the data storage device according to the second aspect includes data generation means.

  In such a configuration, the data storage device sequentially reflects the older version data in the latest version data stored in the storage unit by using the difference data of one generation or earlier specified by the metadata by the data generation unit. As a result, the specified version of data is generated.

  That is, the data generation means can refer to the difference data that is the data of the previous generation by referring to the metadata corresponding to the latest version of the data. Therefore, the data generation means deletes the added data indicated by the difference data of the previous generation from the latest version of the data, restores the changed data, or adds the deleted data, Generate previous data. In addition, to generate the data of two generations before, the data generation means, the difference data of two generations obtained by referring to the metadata corresponding to the data of the previous generation, and the generated data of the previous generation The data two generations before is generated from the data. Thus, the data generation means can sequentially generate past versions of data based on the difference data specified by the metadata.

  According to a fourth aspect of the present invention, there is provided the data storage device according to the second or third aspect, comprising an access frequency measuring means, a data moving means, and a metadata changing means.

In such a configuration, the data storage device measures the ratio of the access frequency to the difference data, which is the past version data corresponding to the latest version of the data, by the access frequency measuring means. For example, the access frequency measuring means counts the number of accesses to the latest version of data and the number of accesses to past data (difference data) corresponding to the data, and accesses the latest version of data. The ratio of the access frequency to the difference data is calculated based on the number.
The ratio of the access frequency is an index indicating which version of the previous data access has decreased.

  Then, the data storage device transmits, to the other predetermined data storage device, the difference data whose access frequency ratio is equal to or lower than the predetermined ratio by the data moving means. For example, when there are a plurality of other predetermined data storage devices on the network, they are assumed to be logically adjacent data storage devices such as consecutive IP addresses.

Further, the data storage device changes the address described in the metadata corresponding to the difference data to the address of the difference data stored in the storage unit of another data storage device by the metadata changing unit. In this case, the address is obtained by adding information (for example, an IP address) for specifying the data storage device to the physical address of the storage means.
As a result, the difference data whose access frequency ratio is below a certain level is distributed to other data storage devices.

  Further, the data storage device according to claim 5 is the data storage device according to claim 4, wherein the data moving means stores the data capacity stored in the storage means in the other data storage device. The difference data is transmitted to another data storage device based on the ratio of the access frequency when the ratio of the data capacity stored in the means becomes a predetermined value or less.

In such a configuration, the data storage device has a predetermined ratio of the data capacity stored in the storage means of another data storage device to the data capacity stored in the storage means by the data moving means. In the following cases, the data stored in the data storage device is transmitted to another data storage device.
As a result, data is distributed when the difference in data capacity exceeds a predetermined amount.

  According to a sixth aspect of the present invention, there is provided a version management program for managing a plurality of pieces of data for a plurality of generations in a hierarchical structure by using a metadata generation unit, a difference data generation unit, a data change unit, The configuration is made to function as data generation means.

In such a configuration, the version management program corresponds to the latest data input by the metadata generation means, the version of the latest data, and the address at which the data one generation before the latest data is stored in the storage means. Generate the attached metadata.
Then, the version management program generates difference data, which is a difference between the latest data and data one generation before the latest data stored in the storage unit, by the difference data generation unit.

Then, the version management program refers to the metadata by the data changing means and changes the data one generation before the latest data to the difference data generated by the difference data generating means.
As a result, only the latest version of the data is managed by the entire data, and the previous version of the data is only the difference (difference data) from the latest version of the data.

Furthermore, the version management program generates the specified version of data from the latest version of the data stored in the storage unit by using the difference data of one generation or earlier specified by the metadata by the data generation unit.
As a result, the past version data stored as the difference data is generated.

  According to the first aspect of the present invention, since the past version of the data is referred to by the address described in the metadata corresponding to the data after one generation, the difference data has a tree structure. There is no need to manage with directory information. As a result, only the latest version of the data is managed in a tree structure as directory information, the number of nodes managed in the entire tree is reduced, and all the directory information to be managed can be stored in the semiconductor memory and processed. Even if this increases, the access time to the latest version of data is not reduced.

  According to the invention described in claim 2 or claim 6, the difference data of the past version is referred to by the address described in the metadata corresponding to the data after one generation. There is no need to manage directory information with a tree structure. As a result, only the latest version of the data is managed in a tree structure as directory information, the number of nodes managed in the entire tree is reduced, and all the directory information to be managed can be stored in the semiconductor memory and processed. Even if this increases, the access time to the latest version of data is not reduced.

  According to the third or fifth aspect of the invention, it is possible to generate data of a desired version in the past by tracing back from the latest version of data. As a result, the user can acquire the data simply by requesting the desired version of the data, and can save the trouble of reading out the individual data and generating the desired version of the data.

According to the fourth aspect of the present invention, since the data (difference data) whose access frequency has decreased is moved to another data storage device, the data can be distributed to a plurality of data storage devices. As described above, by distributing the data, the storage capacity can be distributed in a plurality of data storage devices. Further, by using a low-priced device such as a tape device as the storage means in the data storage device to which data with low access frequency is moved, the entire system can be kept low.
In addition, the latest data can be searched at high speed by directory management, and the new differential data of the frequently accessed version can be identified with the pointer from the latest data to the old version of the differential data sequentially in the same data storage device. New differential data with a frequently accessed version can be retrieved at high speed.

  According to the fifth aspect of the invention, when the difference in data capacity between the data storage devices becomes large, the data is distributed, and the access load can be distributed in addition to the capacity distribution.

Embodiments of the present invention will be described below with reference to the drawings.
[First Embodiment]
(Outline of data storage device)
First, referring to FIG. 1, the outline of the version management method in the data storage device according to the present invention will be described. FIG. 1 is a diagram schematically showing a version management technique realized by the data storage device according to the first embodiment of the present invention.
As shown in FIG. 1, the version management method implemented in the data storage device 1 according to the present invention manages data with directory information having a directory structure (for example, Btree), and the latest version of data (here, A , B) are stored in the storage means 30.

  At this time, only the difference between the old version data and the new version data (difference data [A1 to A5, B1] in this case) is stored in the storage unit 30. Here, the difference between this method and the conventional method is that the address of the difference data (address on the storage means 30) is indicated as a pointer from the data (or difference data) after one generation. That is, the address of the difference data A1 is stored as a pointer in management information (metadata) associated with the data A after one generation. Further, the address of the difference data A2 is stored as a pointer in management information (metadata) associated with the difference data A1 after one generation.

In this way, by managing the address of the difference data of the old version as a pointer from the data (or difference data) after one generation, it is not necessary to manage the address of the difference data as directory information, and the version increases. Even in this case, the time for searching for the address of the latest data can be shortened.
Hereinafter, the configuration and operation of the data storage device for realizing the present method will be sequentially described.

(Configuration of data storage device)
First, the configuration of the data storage device according to the first embodiment of the present invention will be described with reference to FIG. FIG. 2 is a block diagram showing the configuration of the data storage device according to the first embodiment of the present invention. The data storage device 1 stores a plurality of data for a plurality of generations, and implements the version management method described in FIG.
Here, the data storage device 1 includes a communication transmitting / receiving unit 10, a system management unit 20, and a storage unit 30.

  The communication transmission / reception means 10 transmits / receives data to / from a client computer (not shown) via the network N. For example, the communication transmitting / receiving means 10 is a communication board that transmits and receives data using a TCP / IP (Transmission Control Protocol / Internet Protocol) communication protocol.

  The system management means 20 controls the entire data storage device 1. For example, the system management means 20 is a computer provided with a CPU (Central Processing Unit), a memory, and the like. Here, the system management means 20 includes a data storage / deletion means 21, a data distribution means 22, a data management means 23, a metadata writing / reading means 24, a directory information writing / reading means 25, a control. Information receiving means 26 and control information transmitting means 27 are provided.

  The data storage / deletion unit 21 receives (inputs) the latest data via the communication transmission / reception unit 10 and stores the latest data in the storage unit 30 or deletes the data stored in the storage unit 30. Here, the data storage / deletion unit 21 includes a data reception unit 21a, a difference data generation unit 21b, a data writing unit 21c, and a data deletion unit 21d.

  The data receiving unit 21 a receives the latest data via the communication transmitting / receiving unit 10. The data receiving unit 21 a receives the latest data from the transmission source (for example, client computer) of the latest data notified from the data management unit 23 via the communication transmitting / receiving unit 10. The latest data received by the data receiving means 21a is output to the difference data generating means 21b and the data writing means 21c.

The difference data generating unit 21b generates difference data obtained by taking a difference between the latest data received by the data receiving unit 21a and the data one generation before the latest data stored in the storage unit 30. . This previous generation of data is read from the storage means 30 by the data reading means 22a based on the address notified from the data management means 23.
The difference data generation unit 21b extracts difference data by extracting only newly added data, deleted data, and changed data between the latest data and the data of the previous generation. Generate.
The difference data generated by the difference data generation means 21b is output to the data writing means 21c.
The directory management unit 23a described later deletes the pointer (address) of the difference data from the directory information.

  The data writing unit 21c writes the latest data received by the data receiving unit 21a and the difference data generated by the difference data generating unit 21b to the write address on the storage unit 30 instructed by the data management unit 23. Is.

The data deleting unit 21d deletes data stored in the storage unit 30 based on an instruction from the data management unit 23.
As described above, the data storage / deletion unit 21 stores the latest version data in the storage unit 30, and the old version data is sequentially stored in the storage unit 30 as difference data obtained by taking a difference from the latest version data. Remember.

  The data distribution unit 22 distributes the data stored in the storage unit 30 via the communication transmitting / receiving unit 10 to a request source (for example, a client computer) that has requested the data. Note that if the requested data is not the latest version, the data distribution unit 22 generates and distributes the instructed version data by sequentially reflecting the old version data using the latest version data. Here, the data distribution unit 22 includes a data reading unit 22a, a data generation unit 22b, and a data transmission unit 22c.

  The data reading means 22a reads the latest data or difference data from the storage means 30 based on requests from the data generation means 22b and the difference data generation means 21b.

When there is a data read request, the data generation means 22b reads the latest data and difference data via the data read means 22a based on the file name and version specified by the data management means 23, and is instructed. Generates version data.
For example, when the latest data is generated by adding certain data to data one generation before, the difference data is only the added data. In this case, the data generation unit 22b deletes the data indicated by the difference data from the latest data, thereby generating data one generation before.

  When the instructed version is the latest version, the data generation unit 22b outputs the read latest data to the data transmission unit 22c as it is. If the instructed version is a version one generation earlier than the latest version, the data generation unit 22b reads the difference data one generation before the read latest data and generates data one generation before. If the instructed version is a version two generations before the latest version, the data generation unit 22b generates data two generations before from the data one generation before and the difference data one generation before. . If the instructed version is older, differential data is sequentially read out and a desired version of data is generated from already generated data. The old version data generated by the data generation means 22b is output to the data transmission means 22c.

The data transmission unit 22c sends the latest version data or the old version data output from the data generation unit 22b to the request source (for example, client computer) of the “data distribution request” notified from the data management unit 23. The data is transmitted via the communication transmitting / receiving means 10.
As described above, the data distribution unit 22 transmits the latest data read from the storage unit 30 to the request source as it is for the latest version data, and the old version data is sequentially updated from the latest version data. Generate data from the data and send it to the requester.

  The data management unit 23 manages data based on the control information received by the control information receiving unit 26. Here, the data management unit 23 includes a directory management unit 23a, a metadata generation unit 23b, a data change unit 23c, and a data operation control unit 23d.

  The directory management unit 23a manages the latest data and metadata stored in the storage unit 30 in a hierarchical structure of a tree structure (for example, Btree). The directory management means 23a hierarchizes data with the root node of the highest layer, the index node of the intermediate layer, and the leaf node of the lowest layer, and the root node and the index node are pointers to the nodes of the lower layer. (Address) and actual data such as data are managed as leaf nodes. This tree structure is written as directory information in a predetermined area of the storage means 30.

That is, when data is stored in the data storage device 1, the directory management unit 23 a adds it as a leaf node to the directory information in the storage unit 30. When data is distributed from the data storage device 1, the directory management unit 23 a The data pointer (address) is searched using the information.
The directory management unit 23a does not directly manage the pointer (address) of the difference data. The pointer of the difference data is linked and managed by the pointer described in the metadata corresponding to the latest data.

  When the metadata generation unit 23b receives the latest data storage request ("data storage request") as control information, the metadata generation unit 23b specifies the version of the latest data, the address of the latest data, and the data one generation before. Metadata that at least describes the address of the metadata is generated. The metadata generated by the metadata generation unit 23 b is additionally stored in the storage unit 30 via the metadata writing / reading unit 24.

Here, the metadata structure will be described with reference to FIG. 3 (refer to FIG. 2 as appropriate). 3A and 3B are diagrams for explaining the structure of metadata, where FIG. 3A is a data structure diagram showing an example of the structure of metadata, and FIG. 3B is a diagram showing the relationship between metadata and data. .
As shown in FIG. 3A, here, the metadata M includes a file identifier fd, a version number ver, a data size siz, a data pointer (address) dp, and a metadata pointer (address) mp. It is described.

The file identifier fd is information for identifying data, and is information for specifying data such as a file name and a number.
The version number ver is information indicating the version of the data, and is information for specifying the generation order of the data by a combination of alphanumeric characters such as “1.00” and “ver.2”, for example.
The data size siz is information indicating the length of data, and is, for example, a byte length.
The data pointer (address) dp indicates an address at which the data is stored in the storage unit 30.
The metadata pointer (address) mp indicates an address at which the metadata for specifying the data one generation before the data is stored in the storage unit 30. If the metadata is the oldest version, the value of the metadata pointer mp is set to “NULL”.

Thus, by configuring the metadata M, it is possible to access the data retroactively by the metadata pointer mp of the metadata M. For example, in the example of FIG. 3B, the data D ₃ of the latest version (ver. 3) can be accessed by the data pointer dp3 described in the metadata M ₃ . Then, the metadata pointer mp2 described in the metadata M _3, with reference to the metadata M ₂ for specifying the data (difference data) D ₂ of the previous generation (ver.2), the metadata M ₂ With the data pointer dp2 described, it is possible to access the data (difference data) D2 of the previous generation (ver. ₂ ). Similarly, the metadata pointer mp1 described in the metadata M _2, with reference to the metadata M ₁ for specifying the data (difference data) D ₁ of the previous generation (ver.1), the meta data M ₁ The data pointer dp1 described in ( ₁ ) enables access to data (difference data) D1 of the previous generation (ver. ₁ ).
Returning to FIG. 2, the description of the configuration of the data storage device 1 will be continued.

  The data changing unit 23c changes the previous generation data stored in the storage unit 30 to differential data. That is, the data changing unit 23c deletes the previous generation data by the data deletion unit 21d at the stage when the differential data generation unit 21b generates the difference data, and specifies the data pointer dp of the metadata that identifies the previous generation data. The address of the difference data storage destination is written in (see FIG. 3). As a result, the data one generation before the newly stored data is changed to differential data.

The data operation control unit 23d performs data operation based on the control information received by the control information receiving unit 26.
For example, when the data operation control unit 23d receives the “data storage request” that is control information, the data operation control unit 23d instructs the data storage / deletion unit 21 to transmit the latest data and the address managed by the directory management unit 23a. By notifying (storage destination), the latest data is stored (stored).
Further, when the data operation control unit 23 d receives “data distribution request” which is control information, the data operation control unit 23 d stores the data transmission destination (request source), version, and storage unit 30 with respect to the data distribution unit 22. By notifying the address (storage destination), data is generated as necessary and distributed to the client computer.

The metadata writing / reading unit 24 writes metadata to the storage unit 30 and reads metadata from the storage unit 30.
The metadata writing / reading unit 24 writes the metadata generated by the metadata generation unit 23b of the data management unit 23 to the storage unit 30, and the data changing unit 23c and the data operation control unit 23d refer to the metadata. Therefore, the metadata is read from the storage means 30.

The directory information writing / reading unit 25 writes or reads the directory information stored in the storage unit 30.
The directory information writing / reading means 25 writes or reads directory information based on an instruction from the directory management means 23 a of the data management means 23.

  The control information receiving means 26 receives control information that is information other than data from the client computer or the like via the communication transmitting / receiving means 10. The control information received by the control information receiving unit 26 is output to the data management unit 23.

The control information transmission unit 27 transmits the control information output from the data management unit 23 to the client computer or the like via the communication transmission / reception unit 10.
Here, the control information is a “data storage request” indicating that data is stored (written), a “data distribution request” indicating that data is distributed (read), or a response to these instructions.

The storage means 30 stores data (latest data), difference data, and metadata corresponding to the data and difference data for the number of generations, and is a general storage device such as a hard disk.
The storage means 30 stores all the latest version data, and the older version data stores only the difference data from the next generation version data.
The storage unit 30 stores directory information managed by the directory management unit 23a.
Here, the data (latest data), the difference data, the metadata, and the directory information are stored in the same storage unit 30, but may be individually stored in individual storage units.

As described above, by configuring the data storage device 1, the directory management unit 23a manages only the latest metadata, and can access the latest data at high speed. The data storage device 1 can generate old version data by sequentially referring to the differential data that can be referred to by the metadata.
Here, the data storage device 1 is configured to have a function of generating an old version of data. However, the generation of the old version of data is performed by a client computer or the like that has acquired the latest data and difference data. It is good. In this case, the data storage device 1 omits the data generation unit 22b from the configuration.

(Operation of data storage device)
Next, the operation of the data storage device 1 according to the present invention will be described with reference to FIG. 4 and FIG. FIG. 4 is a flowchart showing an operation (data storage operation) for storing the latest data in the data storage device according to the first embodiment of the present invention. FIG. 5 is a flowchart showing an operation of distributing data (data distribution operation) in the data storage device according to the first embodiment of the present invention.

[Data storage operation]
First, the operation of storing the latest data in the data storage device 1 will be described with reference to FIG. 4 (refer to FIG. 2 as appropriate). Here, an operation after a “data storage request” including a file name and version is transmitted from the client computer (not shown) to the data storage device 1 and the latest data is transmitted from the client computer will be described.

First, the data storage device 1 receives the latest data by the data receiving unit 21a via the communication transmitting / receiving unit 10 (step S1).
Then, the data storage device 1 reads the data one generation before the latest data managed by the directory management unit 23a from the storage unit 30 by the data reading unit 22a (step S2).
Further, the data storage device 1 generates difference data by taking the difference between the latest data received in step S1 and the previous generation data read in step S2 by the difference data generation unit 21b (step S1). S3).

In the data storage device 1, the data writing unit 21c writes the difference data generated in step S3 in the storage unit 30 in accordance with an instruction from the data changing unit 23c (step S4).
Then, the data storage device 1 describes the address of the difference data generated in step S3 in the metadata stored in the storage unit 30 corresponding to the previous generation data by the data changing unit 23c. The metadata is changed (step S5).

  Furthermore, the data storage device 1 generates metadata corresponding to the latest data by the metadata generation unit 23b (step S6). That is, the metadata generation unit 23b stores the file name, version, and data size included in the “data storage request”, the address at which the latest data is stored in the storage unit 30, and the metadata changed in step S5. From the address stored in the means 30, metadata corresponding to the latest data is generated.

Then, the data storage device 1 writes the metadata generated in step S6 into the storage unit 30 by the metadata writing / reading unit 24 (step S7).
Further, in the data storage device 1, the data writing unit 21c writes the latest data received in step S1 in the storage unit 30 in accordance with an instruction from the data changing unit 23c (step S8). The latest data may be written to the storage unit 30 before the difference data is generated, for example, after step S1.

Further, in the data storage device 1, the data deletion unit 21d deletes the data of the previous generation from the storage unit 30 in accordance with an instruction from the data change unit 23c (step S9). At this time, the directory management unit 23a deletes the data information of the previous generation from the directory information stored in the storage unit 30.
Then, the data storage device 1 updates the directory information by the directory management unit 23a using the metadata address written in the storage unit 30 in step S7 and the data address written in the storage unit 30 in step S8. (Step S10).

  By the above operation, only the latest data has all the data, and the old version data is converted to only the difference data. Since the metadata corresponding to the latest data indicates the address of the metadata one generation before, the old version data (difference data) and the metadata are stored in the directory information managed by the directory management unit 23a. Instead, only the latest data and its metadata information are stored. As a result, the latest data can be searched and read at high speed.

[Data distribution operation]
Next, the operation of distributing data in the data storage device 1 will be described with reference to FIG. Here, an operation after a “data distribution request” including a file name and version is sent from the client computer (not shown) to the data storage device 1 will be described.

First, the data storage device 1 searches the latest metadata having the instructed file name by referring to the directory information stored in the storage unit 30 by the directory management unit 23a (step S11). This metadata is notified to the data distribution means 22.
Then, the data storage device 1 reads the latest data from the storage unit 30 based on the data pointer described in the metadata from the storage unit 30 by the data reading unit 22a of the data distribution unit 22 (step S12).

  Then, the data storage device 1 determines whether the latest data or the version of the data generated in step S16 described later is the requested version by the data generation unit 22b (step S13).

Here, when the latest data or the version of the data generated in step S16 is not the requested version (requested version), that is, the requested version is the latest data or the version of the data generated in step S16. If it is older (No in step S13), the data storage device 1 refers to the metadata pointer described in the metadata corresponding to the latest data or the data generated immediately before by the data generation means 22b, The previous metadata is read from the storage means 30 via the metadata writing / reading means 24 (step S14).
Then, the data storage device 1 refers to the data pointer described in the previous generation of metadata by the data generation unit 22b, and stores the difference data of the previous generation through the data reading unit 22a. (Step S15).

Subsequently, the data storage device 1 uses the data generation unit 22b to generate data of the previous generation from the latest data or the data generated immediately before and the difference data read in step S15 (step S16).
Then, the data storage device 1 returns to step S13 and determines whether or not the version of the data generated in step S16 is the requested version.

  On the other hand, when the latest data or the version of the generated data is the requested version (Yes in step S13), the data storage device 1 uses the data transmitting / receiving unit 10 to transmit the request by the data transmitting / receiving unit 22c. The version of the data is transmitted to the “data distribution request” request source (step S17).

  Through the above operation, the requested version of data can be generated by tracing back the metadata pointer described in the metadata managed separately from the directory information. At this time, since only the latest version of metadata is managed by the directory information, the latest data can be accessed at high speed.

[Second Embodiment]
(Configuration of data storage device)
Next, the configuration of the data storage device according to the second embodiment of the present invention will be described with reference to FIG. FIG. 6 is a block diagram showing the configuration of the data storage device according to the second embodiment of the present invention. The data storage device 1B stores a plurality of data for a plurality of generations, and implements the file version management method described with reference to FIG. 1 as with the data storage device 1 (see FIG. 2). is there.

Here, the data storage device 1B adds metadata adding means 21e to the data storage / deleting means 21 of the data storage device 1, and adds metadata removing means 22d to the data distribution means 22 of the data storage device 1. .
Further, the data storage device 1B uses the metadata generation unit 23b and the data generation unit 22b of the data storage device 1 as the metadata generation unit 23Bb and the data generation unit 22Bb whose functions are changed, and writes the metadata in the data storage device 1 -The reading means 24 is deleted.

The data storage device 1B is significantly different from the data storage device 1 in that the metadata described in the data storage device 1 is added to the data so that the metadata and the data are handled as one file. .
Hereinafter, a configuration different from the data storage device 1 will be described. In addition, about the structure same as the data storage device 1, the same code | symbol is attached | subjected and description is abbreviate | omitted.

The metadata adding means 21e adds the metadata generated by the metadata generating means 23Bb to the latest data received by the data receiving means 21a and the difference data generated by the difference data generating means 21b. It is.
Thereby, metadata-added data in which metadata is added to the latest data and difference data with metadata in which metadata is added to the difference data are generated. The data with metadata and the difference data with metadata are stored in the storage means 30.
At this time, the directory management unit 23a deletes the pointer (address) of the difference data with metadata from the directory information.

When there is a data read request, the data generation unit 22Bb sends the data with metadata and the difference data with metadata through the data read unit 22a based on the file name and version specified by the data management unit 23B. Read and generate the requested version of the data.
That is, the data generation means 22Bb generates the instructed version data from the data included in the data with metadata and the difference data included in the difference data with metadata. The data generated by the data generation unit 22Bb is output to the data transmission unit 22c.
If the requested version is the latest version, the latest data with metadata is output to the metadata removing unit 22d as it is.

The metadata removing unit 22d deletes the metadata from the metadata-added data read by the data reading unit 22a and extracts only the data.
The data extracted by the metadata removing unit 22d is output to the data transmitting unit 22c.

  When the metadata generation means 23Bb receives a data storage request (“data storage request”) as control information, the metadata generation means 23Bb attaches metadata including the version of the data and the difference data of the previous generation corresponding to the data. Metadata that describes at least the address of the difference data is generated. The metadata generated by the metadata generation unit 23Bb is notified to the metadata addition unit 21e of the data storage / deletion unit 21B.

Here, the structure of the data with metadata will be described with reference to FIG. 7 (refer to FIG. 6 as appropriate). 7A and 7B are diagrams for explaining data with metadata, where FIG. 7A is a data structure diagram illustrating an example of the structure of data with metadata (difference data with metadata), and FIG. 7B is with data. It is a figure which shows the relationship between data and difference data with metadata.
As shown in FIG. 7A, here, in the data-added data MD, a file identifier fd, a version number ver, a data size siz, and a pointer (address) p are described. The metadata in the differential data with metadata has the same structure as the metadata with metadata MD.

The file identifier fd, version number ver, and data size siz are the same as the contents of the metadata M described in FIG.
The pointer (address) p indicates an address where the difference data with metadata including the difference data of the previous generation of the data or difference data is stored in the storage unit 30. If the metadata-added data is the oldest version, the value of the pointer p is set to “NULL”.

In this way, by configuring the data with metadata (difference data with metadata) MD, as shown in FIG. 7B, the difference data with metadata is traced back to the generation by the pointer p of the data with metadata. Can be accessed. For example, in the example of FIG. 7B, the difference of the previous generation (ver. 2) is determined by the pointer p2 described in the metadata of the data with metadata MD ₃ including the data of the latest version (ver. 3). data can be accessed in the differential data MD ₂ with metadata including. Similarly, the pointer p1 that is described in the metadata of the metadata with the differential data MD _2, it is possible to previous generation (ver.1) access metadata with difference data MD ₁ including the difference data.

  As described above, by configuring the data storage device 1B, the directory management unit 23a manages only the latest data (data with metadata), and can access the latest data at high speed. The data storage device 1B can generate the old version of the data by sequentially referring to the differential data with metadata that can be referred to by the metadata of the data with metadata.

(Operation of data storage device)
Next, the operation of the data storage device according to the present invention will be described with reference to FIGS. FIG. 8 is a flowchart showing an operation of storing data (data storage operation) in the data storage device according to the second embodiment of the present invention. FIG. 9 is a flowchart showing an operation of distributing data (data distribution operation) in the data storage device according to the second embodiment of the present invention.

[Data storage operation]
First, an operation of storing data in the data storage device 1B will be described with reference to FIG. 8 (refer to FIG. 6 as appropriate). Here, the operation after a client computer (not shown) requests a “data storage request” including a file name and version to the data storage device 1B, and data is transmitted from the client computer will be described.

First, the data storage device 1B receives the latest data through the communication transmitting / receiving means 10 by the data receiving means 21a (step S21).
Then, the data storage device 1B reads the data with metadata including the data one generation before the latest data managed by the directory management unit 23a from the storage unit 30 by the data reading unit 22a (step S22).

Further, the data storage device 1B uses the difference data generation means 21b to calculate the difference between the latest data received in step S21 and the data included in the data with metadata one generation before read in step S22. The taken difference data is generated, and difference data with metadata including the difference data is generated (step S23).
In the data storage device 1B, in response to an instruction from the data changing unit 23c, the data writing unit 21c writes the differential data with metadata generated in step S23 into the storage unit 30 (step S24).

Then, the data storage device 1B generates metadata for the latest data by the metadata generation unit 23Bb (step S25). That is, the metadata generation unit 23Bb corresponds to the latest data from the file name, version, and data size included in the “data storage request” and the address at which the latest metadata-added data is stored in the storage unit 30. Generate metadata.
Then, the data storage device 1B adds the metadata generated in step S25 to the latest data received in step S21 by the metadata adding unit 21e to generate data with metadata (step S26).

Further, the data storage device 1B writes the data with metadata generated in step S26 into the storage unit 30 by the data writing unit 21c (step S27).
In the data storage device 1B, the data deletion unit 21d deletes the data with the previous generation metadata from the storage unit 30 in accordance with an instruction from the data change unit 23c (step S28). At this time, the information of the data with metadata one generation before is deleted from the directory information by the directory management unit 23a.

Then, the data storage device 1B updates the directory information with the address of the metadata-added data written in the storage unit 30 in step S27 by the directory management unit 23a (step S29).
With the above operation, the data storage device 1B connects the old version of the difference data from the latest data to each generation by using the pointer described in the metadata.

[Data distribution operation]
Next, the operation of distributing data in the data storage device 1B will be described with reference to FIG. 9 (refer to FIG. 6 as appropriate). Here, an operation after a “data distribution request” including a file name and version is sent from the client computer (not shown) to the data storage device 1B will be described.

First, in the data storage device 1B, the directory management unit 23a refers to the directory information stored in the storage unit 30 and searches for data with metadata having the latest version with the specified file name (step S31). ). The address of the data with metadata is notified to the data distribution means 22.
Then, the data storage device 1B reads the latest data with metadata from the storage unit 30 by the data reading unit 22a of the data distribution unit 22B (step S32).
Then, the data storage device 1B determines whether the version of the latest data or the data generated in step S35 described later is the requested version by the data generation unit 22Bb (step S33).

  Here, when the latest data or the version of the data generated in step S35 is not the requested version (requested version), that is, the requested version is the latest data or the version of the data generated in step S35. If it is older (No in step S33), the data storage device 1B refers to the pointer described in the metadata corresponding to the latest data or the data read immediately before by the data generation means 22Bb, The difference data with metadata one generation before is read from the storage means 30 via the reading means 22a (step S34).

Subsequently, the data storage device 1B uses the data generation unit 22Bb to obtain the previous generation data from the latest data or the data generated immediately before and the difference data in the difference data with metadata read in step S35. Generate (step S35).
Then, the data storage device 1B returns to step S33, and determines whether or not the version of the data generated in step S35 is the requested version.

On the other hand, if the version of the latest data or the generated data is the requested version (Yes in step S37), the data storage device 1B deletes the metadata from the data with metadata by the metadata removing unit 22d. Then, only the data is extracted (step S36), and the data transmission means 22c transmits the requested version of the data to the request source of the “data distribution request” via the communication transmission / reception means 10 (step S37).
Through the above operation, the requested version of data can be generated by tracing back the pointer described in the metadata. At this time, since only the latest version of the data with metadata is managed by the directory information, the latest data (data with metadata) can be accessed at high speed.

The configuration and operation of the data storage device according to the present invention have been described with the first embodiment and the second embodiment, but the present invention is not limited to this. For example, in the second embodiment, only the data is transmitted / received by providing the metadata adding means 21e and the metadata removing means 22d, but the data and metadata are exchanged as in MXF (Material eXchange Format). When writing or reading a file in a format that is wrapped and handled as one file, it is necessary to change the contents of the metadata, but it is not necessary to provide means for adding or deleting the metadata area itself. In addition, a plurality of data storage devices may be connected by a plurality of networks, and data and difference data may be distributed and stored among the data storage devices.
In the data storage devices 1 and 1B, the system management means 20 and 20B may be operated by a version management program that causes a general computer to function as each of the means described above.

  Hereinafter, an embodiment in which a data storage device according to the present invention is applied to an autonomous storage system that stores data, difference data, and the like in a distributed manner will be described. Note that the autonomous storage system may use “data backup method and backup data recovery method, network storage device and network storage program” disclosed in the present applicant (Japanese Patent Laid-Open No. 2004-334739). it can.

[Third Embodiment]
(Outline of autonomous storage system)
First, with reference to FIG. 10, an outline of a version management method when the data storage device according to the present invention is applied to an autonomous storage system will be described. FIG. 10 is a diagram schematically showing a version management method realized in the autonomous storage system according to the third embodiment of the present invention.

As shown in FIG. 10, the autonomous storage system 2 shares and manages directory information among a plurality of data storage devices 1C, and stores data distributed among the data storage devices 1C.
Here, the autonomous storage system 2 stores the respective current version of the data A, B in the storage means 30 _1, 30 ₃ of the data storage device 1C _1, 1C _3.
The old version data stores the difference (difference data [here, A1 to A5, B1]) with the new version data distributed among the data storage devices 1C. Here, the addresses of past difference data (for example, A1 to A5) corresponding to the latest data (for example, A) are each indicated by a pointer.

Here, the autonomous storage system 2 determines the data storage arrangement for storing the data and the difference data according to the ratio of the access frequency to the data. For example, when the access to the latest data B is “1”, the access to the previous generation of data (data B + difference data B1) falls below a predetermined ratio (movement boundary; for example, “0.7”). , the difference data B1, different data storage device 1C and the data B (e.g., adjacent data storage device 1C ₄ at logical address) is moved. Note that a plurality of moving boundaries may be set step by step between the data storage devices 1C.

Thereby, for example, (in this case, A1, A2) frequently difference data access is performed, are stored in the same data storage device 1C ₁ latest data A, the difference data A3 percentage of access frequency is reduced in other It will be stored in the data storage device 1C _2.
In this way, by storing the difference data having a high access frequency ratio in the same storage means as the latest data, it is possible to read the data (and the difference data) having a high access frequency at high speed. Further, by separating the difference data having a low access frequency ratio from the latest data, it is possible to distribute the capacity of data stored among the data storage devices.

(Configuration of data storage device)
Next, the configuration of the data storage device according to the third embodiment of the present invention, which constitutes the autonomous storage system, will be described with reference to FIG. FIG. 11 is a block diagram showing a configuration of a data storage device according to the third embodiment of the present invention. The data storage device 1C stores a plurality of data for a plurality of generations, and implements the version management method described with reference to FIG.

  Here, in the data storage device 1C, an access frequency measurement unit 23e, a data movement unit 23f, and a metadata change unit 23g are added to the data management unit 23 of the data storage device 1 described in FIG. Other configurations are the same as those of the data storage device 1 described with reference to FIG.

The access frequency measuring unit 23e periodically measures the ratio of the access frequency to the difference data corresponding to the data stored in the storage unit 30.
That is, the access frequency measuring unit 23e measures the number of read requests (access) to the latest data and the number of read requests (access) to the old data (difference data), and sets the access ratio to the latest data to “1”. The ratio of the access frequency to the difference data is calculated. The ratio of the access frequency is stored in the storage unit 30. Note that the ratio of the access frequency may be stored in addition to the metadata.

The data moving unit 23f is configured to transfer the difference data to another predetermined data storage device (for example, a data storage device logically adjacent by an IP address or the like) based on the access frequency ratio measured by the access frequency measurement unit 23e. ).
Here, the data moving unit 23f distributes the corresponding difference data to the data distribution when the ratio of the access frequency stored in the storage unit 30 becomes equal to or less than the value of the movement boundary stored in the storage unit 30. The means 22 is notified and transmitted to another data storage device. Here, the movement boundary indicates a boundary between the ratio of the access frequency of the difference data to be left in the data storage device 1C and the ratio of the access frequency of the difference data to be moved. Note that the movement boundary may be added to the metadata and stored, and may have a different value for each data.

  As described above, when the movement boundary is set based on the ratio of the access frequency, the data movement unit 23f determines the access frequency of the data in the data storage device 1C when the new version of data is stored in the storage unit 30. When the ratio of the access frequency to the difference data in another predetermined data storage device (logically adjacent data storage device) that stores the difference data corresponding to the data with respect to the ratio is lower than the predetermined value Then, it is determined whether or not to move the difference data stored in the storage means 30 to another data storage device.

That is, the data moving means 23f indicates that, in each version of data storing a new version, the ratio of the access frequency to the differential data after moving the differential data older than that version to another data storage device If it is below the boundary, it is determined that the difference data is to be moved to another data storage device, and the movement is executed.
Thus, the data storage device 1C can distribute the access load.

Note that the movement boundary is stored not only in the rate of access frequency but also in the capacity of data stored in the storage device 30 and other predetermined data storage devices (logically adjacent data storage devices). It is good also as setting based on ratio with the capacity | capacitance of data.
In this case, when a new version of data is stored in the storage unit 30, the data moving unit 23f has a predetermined ratio between the data capacity of the data storage device 1C and the data capacity of the other data storage device. If the difference data is less than the threshold value, it is determined whether or not the difference data stored in the storage unit 30 is moved to another data storage device.

That is, the data moving means 23f indicates that, in each version of the data storing the new version, the ratio of the access frequency to the differential data after moving the differential data older than that version to another data storage device When it is below the movement boundary set according to the capacity ratio, it is determined that the difference data is to be moved to another data storage device, and the movement is executed.
As a result, the data storage device 1C can perform capacity distribution.

When moving the difference data, the data moving unit 23 f transmits a “difference storage request”, which is a request for storing the difference data, to the other data storage device as control information via the control information transmitting unit 27. Then, by instructing the data distribution means 22 to distribute the difference data, the difference data is transmitted to another data storage device.
The data moving unit 23f receives the “difference storage request” from the other data storage device via the control information receiving unit 26, and when the difference data is received from the other data storage device, stores and deletes the data. By notifying the means 21 of the storage request, the difference data is stored in the storage means 30. Further, the data moving unit 23f sends the address where the differential data is stored in the storage unit 30 to the data storage device that has transmitted the differential data via the control information transmitting unit 27 as a response to the “difference storage request”. Send.

  The metadata changing unit 23g updates the metadata stored in the storage unit 30. Here, the metadata changing unit 23g is included in the response to the “difference storage request” from the other data storage device after the difference data is transmitted to the other data storage device by the data moving unit 23f. Based on the address where the difference data is stored, the data pointer dp (see FIG. 3) of the metadata stored in the storage unit 30 is changed. In this case, information in which the identification information (for example, IP address) of the data storage device storing the difference data is added to the address is written in the data pointer dp.

  As described above, by configuring the data storage device 1C, it is possible to distribute the capacity by moving the difference data whose access frequency has decreased to another data storage device. At this time, the version of data (difference data) with high access frequency is managed by the same data storage device, so that the time for generating and distributing the data is not reduced.

[Fourth Embodiment]
The data storage device according to the present invention may be configured as the data storage device 1D shown in FIG. FIG. 12 is a block diagram showing the configuration of the data storage device according to the fourth embodiment of the present invention.
Similar to the data storage device 1C (see FIG. 11), the data storage device 1D implements the version management method described in FIG.

  The data storage device 1D treats the metadata in the data storage device 1C (see FIG. 11) as one file by adding it to the data. That is, the data storage device 1D adds the access frequency measuring unit 23e described in FIG. 11 to the data storage device 1B described in FIG. 6, and further includes a data moving unit 23Df and a metadata changing unit 23Dg. It is added.

  Hereinafter, the data moving unit 23Df and the metadata changing unit 23Dg having different configurations from the data storage device 1B and the data storage device 1D will be described. Since other configurations are the same as those of the data storage device 1B and the data storage device 1D, the description thereof is omitted by giving the same reference numerals.

  The data moving unit 23Df logically adjoins the differential data with metadata based on the access frequency ratio measured by the access frequency measuring unit 23e by another predetermined data storage device (for example, an IP address or the like). Data storage device). That is, the data moving unit 23Df only changes the data to be moved from the difference data to the differential data with metadata, compared to the data moving unit 23f described in FIG. 11, and the other functions are the same as the data moving unit 23f. It is.

The metadata changing unit 23Dg updates metadata included in the differential data with metadata stored in the storage unit 30. That is, the metadata changing unit 23Dg is included in the response to the “difference storage request” from the other data storage device after the difference data with metadata is transmitted to the other data storage device by the data moving unit 23Df. The pointer p (see FIG. 7) described in the metadata included in the difference data with metadata is changed based on the address where the difference data with metadata is stored.
In this way, by configuring the data storage device 1D, similarly to the data storage device 1C (see FIG. 11), the difference data whose access frequency has decreased is moved to another data storage device, thereby distributing the capacity. In addition, the latest version of data can be accessed at high speed.

It is the figure which showed typically the version management technique implement | achieved with the data storage device which concerns on the 1st Embodiment of this invention. 1 is a block diagram showing a configuration of a data storage device according to a first embodiment of the present invention. It is a figure for demonstrating the structure of metadata, Comprising: (a) is a data structure figure which shows an example of the structure of metadata, (b) is a figure which shows the relationship between metadata and data. It is a flowchart which shows the operation | movement which stores the data in the data storage device which concerns on the 1st Embodiment of this invention. It is a flowchart which shows the operation | movement which distributes the data in the data storage device which concerns on the 1st Embodiment of this invention. It is a block diagram which shows the structure of the data storage device based on the 2nd Embodiment of this invention. It is a figure for demonstrating data with metadata, Comprising: (a) is a data structure figure which shows an example of a structure of data with metadata (difference data with metadata), (b) is data with metadata and metadata. It is a figure which shows the relationship with attached difference data. It is a flowchart which shows the operation | movement which stores the data in the data storage device which concerns on the 2nd Embodiment of this invention. It is a flowchart which shows the operation | movement which distributes the data in the data storage device which concerns on the 2nd Embodiment of this invention. It is the figure which showed typically the version management method implement | achieved by the autonomous storage system which concerns on the 3rd Embodiment of this invention. It is a block diagram which shows the structure of the data storage device which concerns on the 3rd Embodiment of this invention. It is a block diagram which shows the structure of the data storage device based on the 4th Embodiment of this invention. It is the figure which showed typically the version management method which manages the data of several versions by the conventional difference data.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Data storage device 10 Communication transmission / reception means 20 System management means 21 Data storage / deletion means 21a Data reception means 21b Differential data generation means 21c Data writing means 21d Data deletion means 21e Metadata addition means 22 Data distribution means 22a Data reading means 22b Data generation means 22c Data transmission means 22d Metadata deletion means 23 Data management means 23a Directory management means 23b Metadata generation means 23c Data change means 23d Data operation control means 23e Access frequency measurement means 23f Data movement means 24g Metadata change means 24 Meta Data writing / reading means 25 Directory information writing / reading means 26 Control information receiving means 27 Control information transmitting means 30 Storage means

Claims (6)

In a data storage device that stores a plurality of data over a plurality of generations by a hierarchical structure,
Storage means for storing the data for the number of generations;
Metadata generation means for generating metadata in which the latest data input, a version of the latest data, and an address in which data one generation before the latest data is stored in the storage means are associated;
A data storage device comprising: Difference data generation means for generating difference data that is a difference between the latest data and data one generation before the latest data stored in the storage means;
Referring to the metadata, data changing means for changing the data one generation before the latest data to differential data generated by the differential data generating means;
The data storage device according to claim 1, further comprising:   Data generation means for generating a specified version of data from the latest version of data stored in the storage means based on the difference data of one generation or earlier specified by the metadata. The data storage device according to claim 2, wherein the data storage device is a data storage device. An access frequency measuring means for measuring a ratio of an access frequency to the difference data corresponding to the data with respect to the data;
Data moving means for transmitting the difference data in which the ratio of the access frequency measured by the access frequency measuring means is equal to or less than a predetermined value to another predetermined data storage device;
Metadata changing means for changing the metadata based on the address of the difference data stored in the storage means of the other data storage device by the other data storage device;
The data storage device according to claim 2, further comprising:   The data movement means is configured to change the access frequency when the ratio of the data capacity stored in the storage means of the other data storage device to the data capacity stored in the storage means becomes a predetermined value or less. 5. The data storage device according to claim 4, wherein the difference data is transmitted to another data storage device based on a ratio. In order to manage a plurality of data over a plurality of generations by a hierarchical structure,
Metadata generation means for generating metadata in which the latest input data, the version of the latest data, and the address at which the data one generation before the latest data is stored in the storage means are associated with each other;
Differential data generation means for generating difference data that is a difference between the latest data and data one generation before the latest data stored in the storage means;
A data changing unit that refers to the metadata and changes the data one generation before the latest data to the differential data generated by the differential data generating unit;
Data generation means for generating a specified version of data from the latest version of data stored in the storage means based on the difference data of one generation or earlier specified by the metadata;
Version management program characterized by functioning as

Images