JP4451722B2 - Database server and database system - Google Patents

Description

  The present invention relates to a data storage method and a retrieval method of a database system that holds data in a compressed format, and reduces the amount of communication data and distributes the load for decompression processing while maintaining efficient use of the database storage area of the server. It relates to a scheme that can be achieved.

  In the data storage method and retrieval method of a database system that holds data in a compressed format, conventionally, in order to reduce the usage rate of various resources in the entire system and to speed up the search process, the data received in record units A method has been proposed in which data is compressed in columns and stored in a database, decompressed and searched if necessary, and the compressed data and column information position are sent to the client side to decompress the data on the client side. (For example, Patent Document 1).

  However, in such a conventional database system data storage method and retrieval method in which records are compressed in units of columns, the unit of compression is small, and the effect of reducing the database storage area of the server is small. Considering how to use today's database such as storing various data and logs on the Web as well as data necessary for mission-critical work, it is desirable that the reduction by compression of the database storage area is as large as possible. Further, the search result data to be transmitted to the client is compressed data. However, since the column position information is transmitted in addition to the search result data, the communication amount reduction effect is small. Furthermore, even if the data is decompressed as necessary at the time of retrieval, the data to be transmitted to the client is compressed data, so it must be decompressed again by the client, and the processing overhead increases as the amount of data decompressed by the server increases. There was a problem.

  As another conventional technique, a method of storing fixed-length and variable-length character string data by replacing it with a fixed-length internal code is proposed in order to simplify and speed up the database processing and to maintain confidentiality of the database. (For example, Patent Document 2).

As described above, in the method of storing the fixed-length and variable-length character string data by replacing the fixed-length character string data with the fixed-length internal code, since the numerical data is not compressed, the effect of reducing the storage area of the server is small. In addition, the decompression means exists only in the server, and since all decompressed data is transmitted to the client, there is a problem that the amount of communication cannot be reduced.
JP-A-8-314957 JP 63-121942 A

  The present invention has been made to solve the above-described problems, and maintains efficient use of the database storage area of the server in the data storage method and retrieval method of the database system that holds the data in a compressed format. The purpose is to reduce the amount of communication data and to distribute the load on decompression processing.

The database server according to the present invention is:
In a database server that receives a request from a database client, stores record data consisting of one or more columns, and retrieves the stored data.
As a data storage processing unit that performs processing to store data in response to a request from the database client,
A specific data type compression unit that compresses column data of a column of a specific data type alone and stores it in the data storage area;
A batch compression unit that collectively compresses column data of columns other than a specific data type and stores the data in a data storage area;
In response to a request from the database client, as a search processing unit that performs processing to search the data storage area according to the search statement,
Of the columns included in the search statement, identify specific data types that are included in the search results but are not search keys as undecompressed portions, and out of the columns included in the search statement, those portions that are not uncompressed An unzipping identification part for identifying,
A data conversion unit for adding a flag indicating whether or not decompression processing is performed to a column that is a search result is characterized.

  In the present invention, it is possible to reduce the amount of communication data and to distribute the load related to the decompression process while maintaining efficient use of the database storage area of the server.

Embodiment 1 FIG.
Hereinafter, the present invention will be described based on embodiments shown in the drawings. FIG. 1 is a block diagram of a database system according to the present invention.

  In FIG. 1, a database server 1 and a database client 2 constituting a database system are connected via a network 3. The database client 2 is called from the application program 4 as necessary. The database server 1 and the database client 2 are names indicating systems, and are not limited to the case corresponding to one device.

  The database server 1 has a data storage area 108, and the data storage area 108 is accessed via a data storage processing unit 101 and a search processing unit 105, which are database management units.

  The data storage processing unit 101 includes a compression unit 102 that compresses data to be input to a database. The compression unit 102 includes a specific data type compression unit 103 that compresses only a specific data type alone, and a specific data type compression unit 103. And a batch compression unit 104 that collectively compresses uncompressed data (column data other than a specific data type). Furthermore, it has a data type determination unit 110 that determines whether or not each column has a specific data type, and a data storage unit 111 that temporarily stores column data other than the specific data type.

  The search processing unit 105 analyzes the search sentence transmitted from the database client 2 and decompresses the search sentence analysis part 112 that determines the column included in the search key and the column included in the search result, and the columns constituting the record data. A decompression identifying unit 106 that discriminates data to be decompressed and data that is not decompressed, and a decompression unit 107 that performs decompression processing. Further, a data storage unit 113 that temporarily stores decompressed data and data that is not decompressed, a search result acquisition unit 114 that acquires search results, and data that adds a decompression flag and converts the search results into communication data A conversion unit 115.

  The database client 2 determines whether or not the data transmitted from the database server 1 is compressed data by the decompression identifying unit 21 and decompresses the data by the decompression unit 22 if decompression is necessary. Configured to pass to.

  When the application program 4 (an example of the search request processing unit) requests the decompressed data transmitted to the database client 2 without decompressing, the application program 4 informs the database client 2 that decompression is not necessary. specify. The decompression unnecessary designation unit 23 stores that decompression is unnecessary. Although there is one database client 2 in FIG. 1, a plurality of database clients 2 may be connected.

  Next, the operation will be described. In this description, the specific data type compressed by the specific data type compression unit 103 is described as a fixed-length character string type. The specific data type may be another type.

  First, the operation of the data storage processing unit 101 registered in the data storage area 108 in the database server 1 will be described using the flowchart of FIG. 3 and the data flow of FIG. FIG. 3 is a flowchart showing a data storage process in consideration of the data type of this embodiment. FIG. 4 is a diagram illustrating a data flow related to the data storage process.

  First, the database server 1 receives record data to be stored from the database client 2 at the input unit 100, and determines whether the data type determination unit 110 is a fixed-length character string type for each column (step 51). If the data type is not a specific data type (fixed length character string type) (step 52), the data of the column is stored in the data storage unit 111 (step 54).

  In the case of a specific data type (fixed-length character string type) (step 52), the column data group accumulated after the previous batch compression is compressed together by the batch compression unit 104 (step 53). At this time, the accumulated column data is cleared. Then, the specific data type (fixed length character string type) column data is compressed by the specific data type compression unit 103 (step 55).

  Steps 51 to 55 are repeated until the last column (step 56). As a result, data is stored in the data storage area 108 in the format shown in FIG.

  FIG. 2 shows a compression result of stored data when one record is composed of four or more columns and each data type is an integer type, a date type, a fixed-length character string type, and an integer type in order. The first and second columns that are not fixed-length character string type data are compressed together, and the third column is compressed only by this.

  Next, the operation of the search processing unit 105 in the database server 1 will be described using the flowchart of FIG. 5 and the data flow of FIG. FIG. 5 is a flowchart showing search processing according to this embodiment. FIG. 6 is a diagram showing a data flow related to the search process.

  The search sentence received by the input unit 100 is analyzed by the search sentence analysis unit 112 (step 61). As a result, the column used for the search key and the column used for the search result are extracted. The decompression identifying unit 106 stores data including a search key or a column included in the search result, except for column data of a specific data type (fixed-length character string type) that is included in the search result but is not a search key. Is decompressed by the decompression unit 107. Further, the decompressing unit 107 associates the decompressed data with the data of a specific data type (fixed-length character string type) as compressed and stores them in the data storage unit (step 62).

  The search result acquisition unit 114 reads the column data included in the search key for each record from the data storage unit, determines whether the search condition is satisfied, and specifies the record that is the search result (step 63). When the data conversion unit 113 converts the obtained search result into communication data, a decompression flag indicating whether or not each result data is decompressed is added, and the decompression flag is added to the compressed column data. These are collectively transmitted as communication data from the output unit 109 to the database client 2 via the network 3 (step 64).

  Next, the operation in the database client 2 will be described using the flowchart of FIG. FIG. 7 is a flowchart showing a procedure until the application program 4 acquires a search result sent from the database server 1 to the database client 2 according to the present embodiment.

  The database client 2 receives the communication data as a search result from the database server 1 at the input unit 20 (step 71), divides the communication data received at the decompression identification unit 21 into result data, and the result data is received by the decompression flag. It is determined whether the data is compressed data (step 72). If it is compressed data (step 73), if the contents of the decompression unnecessary designation part 23 designated by the application program 4 indicate that decompression is necessary (step 74), the decompression part 22 decompresses the data (step 73). 75). Steps 72 to 75 are repeated until the end of the result data (step 76). The search result restored from the communication data in this way is passed to the application program 4.

  As described above, the compression in the server is not performed in units of one column but in a batch except for a specific data type, so that the database storage area of the server can be used efficiently.

  In addition, since communication is performed in a compressed state in the case of a specific data type, the amount of communication data can be reduced as compared with the case where all decompressed data is communicated.

  Furthermore, since the server is divided into the part to be decompressed by the server and the part to be decompressed by the client, load distribution related to the decompression process can be achieved.

  And since the part thawed by the server is not thawed again by the client, there is no trouble twice.

  Furthermore, since the application program can receive the compressed data as needed, the labor of decompression can be saved.

Embodiment 2. FIG.
In the above embodiment, the operation of the data storage processing unit 102 has been described as being stored in units of data received by the database client 2. However, the received data is accumulated in the database server 1 and reaches a certain amount of data. At this point, the data may be compressed and stored. According to the present embodiment, the effect of reducing the database storage area of the server is increased.

Embodiment 3 FIG.
In the above embodiment, the database client 2 has been described as accessing the database server 1 via the network, but the same effect can be obtained even if the database client 2 is arranged in the same device as the database server 1.

Embodiment 4 FIG.
Even in a form that does not specify that decompression is not required, the database storage area of the server can be efficiently used by compressing data groups other than a specific data type collectively.

  In that case, as shown in FIG. 9, the decompression-unnecessary designation unit 23 is omitted, and the process of determining whether decompression is necessary (S74) is not performed as shown in FIG.

Embodiment 5 FIG.
In addition, the designation of not requiring decompression is effective even when the compressed data and the uncompressed data are mixed in the result data as described above.

  For example, if it is assumed that all the divided result data is compressed, as shown in FIG. 10, it is determined that all the result data need not be decompressed (step 74).

  The database server 1 and the database client 2 are computers, and each element can execute processing by a program. Further, the program can be stored in a storage medium so that the computer can read the program from the storage medium.

  FIG. 11 is a diagram illustrating a hardware configuration example of the database server 1 and the database client 2. An arithmetic device 1101, a data storage device 1102, a memory 1103, and a communication interface 1104 are connected to the bus. The data storage device 1102 is, for example, a ROM (Read Only Memory) or a hard disk. The memory 1103 is a normal RAM (Random Access Memory).

  The program is normally stored in the data storage device 1102, and is loaded into the memory 1103 and sequentially read into the arithmetic device 1101 for processing.

It is a figure which shows a system configuration. It is a figure which shows the structural example of a data storage area. It is a figure which shows the flow of a data storage process. It is a figure which shows the data flow of a data storage process. It is a figure which shows the flow of a search process. It is a figure which shows the data flow of a search process. It is a figure which shows the flow of a client process. FIG. 10 is a diagram illustrating a flow of client processing according to the fourth embodiment. FIG. 10 is a diagram illustrating a configuration of a client according to a fourth embodiment. FIG. 20 is a diagram showing a flow of client processing according to the fifth embodiment. It is a figure which shows the hardware structural example.

Explanation of symbols

  1 database server, 2 database client, 4 application program, 20 input unit, 21 decompression identification unit, 22 decompression unit, 23 decompression unnecessary designation unit, 24 output unit.

Claims (4)

In a database server that receives a request from a database client, stores record data consisting of one or more columns, and retrieves the stored data.
As a data storage processing unit that performs processing to store data in response to a request from the database client,
A specific data type compression unit that compresses column data of a column of a specific data type alone and stores it in the data storage area;
A batch compression unit that collectively compresses column data of columns other than a specific data type and stores the data in a data storage area;
In response to a request from the database client, as a search processing unit that performs processing to search the data storage area according to the search statement,
Of the columns included in the search statement, identify specific data types that are included in the search results but are not search keys as undecompressed portions, and out of the columns included in the search statement, those portions that are not uncompressed An unzipping identification part for identifying,
A database server comprising: a data conversion unit that adds a flag indicating the presence or absence of a decompression process for a column that is a search result. A database client that transmits a request to store data and a request to retrieve data via a network, and receives a request to store data from the database client via the network, and includes one or more columns. In a database system comprising a database server for storing record data , receiving a request for retrieving data from the database client via a network, and retrieving the stored record data,
The database server is
As a data storage processing unit that performs processing to store data in response to a request to store data from a database client,
A specific data type compression unit that compresses column data of a column of a specific data type alone and stores it in the data storage area;
A batch compression unit that collectively compresses column data of columns other than a specific data type and stores the data in a data storage area;
In response to a request to search for data from a database client, as a search processing unit that performs processing to search the data storage area according to the search statement,
A search statement that searches records, and enters a search statement that includes multiple columns, the column used for the search key and the column used for the search result, and the search is performed among the multiple columns included in the input search statement It is a column used for the result , and it is a column other than the column used for the search key , and the column of a specific data type is identified as a part not to be decompressed, and among the plurality of columns included in the search statement, A thawing identification part for identifying a column other than a part not to be thawed as a part to be thawed,
A thawing part for thawing the column identified as the part to be thawed by the thawing identification part;
A database system comprising: a data conversion unit that adds a flag indicating the presence or absence of a decompression process for a column used in a search result. The database client,
An input unit for inputting a search result composed of one or a plurality of column data as a plurality of divided data together with a flag indicating the presence or absence of each decompression process;
A decompression identifying unit for identifying data that has not been decompressed by the database server by a flag ;
The database system according to claim 2, further comprising: a decompression unit that decompresses data identified by the decompression identification unit as not being decompressed by the database server . It said database system further which is connected to a database client includes a search request processing unit that requests the search,
Wherein the input unit, the search request processing unit receives the necessity of thawing,
The database client
It has a thawing unnecessary designation part that memorizes the necessity of thawing,
4. The database system according to claim 3 , wherein the decompression unit decompresses data when the decompression necessity is stored in the decompression-unnecessary designation unit.

Images