JP6381823B2 - System, method and program for speeding up database processing - Google Patents

Description

The present invention relates to a system, method, and program for speeding up database processing, and more particularly to a system, method, and program for realizing speedup using a graphic processor and peer-to-peer DMA.

A database management system (DBMS), in particular, a relational database management system (RDBMS) has become an indispensable component in today's information systems. Therefore, speeding up the processing of the RDBMS is extremely important for improving the efficiency of the information system, and many speeding up techniques have been proposed.

As one of such high-speed technologies, one using a graphic processing unit (GPU) is known. GPUs are a common component in today's personal computers and game consoles and are available at low cost, but are essentially parallel processors with many cores, so they are not graphics processing. In addition, general-purpose applications are possible. Such an application method is generally called GPGPU (General Purpose GPU). Non-Patent Document 1 and Non-Patent Document 2 are known as techniques for speeding up database access by GPGPU.

In the conventional database access speed-up technology using GPGPU, the movement of processing target data from the secondary storage device (storage) to the main storage device (main memory) has become a bottleneck in performance. In order to process data stored in a database on the storage, the central processing unit (CPU) first secures a buffer area on the main storage device, and then loads data from the secondary storage device to the buffer. Only when the loading process is completed can it be possible to access the data stored in the secondary storage device.

In the current general hardware technology, the bandwidth between the CPU and the main memory is 50 GB to 300 GB per second, whereas the bandwidth of the peripheral device bus connecting the storage and the CPU is about 4 GB to 15 GB per second. Inevitably become a bottleneck in performance. In the conventional database processing method, it is necessary to transfer a large amount of data through this bottleneck, and there is a problem that the performance improvement effect of parallel processing by GPGPU is offset.

GPUDirect RDMA (http://docs.nvidia.com/cuda/gpudirect-rdma/index.html) GPGPU Accelerates PostgreSQL (http://www.slideshare.net/kaigai/gpgpu-accelerates-postgresql)

A database query acceleration system, method, and program that can be implemented at low cost are provided.

The present invention is a data processing method executed in a database processing system including a secondary storage device , a main storage device, a central processing device, a parallel processing device, a peripheral device bus, and a relational database stored in the secondary storage device. There,
The central processing unit receiving a query execution plan;
The central processing unit causes the secondary storage device to transfer the data in the table specified in the query execution plan in the relational database to the parallel processing device without going through the main storage device. Issuing a command;
The secondary storage device passes the data in the table specified in the query execution plan in the relational database without passing through the main storage device and without interposing the central processing unit. Transferring to a parallel processing unit;
The parallel processing device processing the data in the table specified in the query execution plan in the relational database in parallel;
The parallel processing unit writing the result of processing the data in the table specified in the query execution plan in the relational database to the main storage without interposing the central processing unit. The problem is solved by providing a data processing method.

Further, according to the present invention, the step of writing to the main storage device targets only column data used in the query execution plan among the results of the step of processing in parallel. The above-mentioned problem is solved by providing a data processing method.

The present invention also provides a data processing executed in a database processing system including a secondary storage device , a main storage device, a central processing unit, a parallel processing device, a peripheral device bus, and a relational database stored in the secondary storage device. A program,
Instructions for causing the central processing unit to receive a query execution plan;
Said central processing unit, to the secondary storage device, to transfer the data in the specified table in the query execution plan in in the relational database to the parallel processing unit without passing through said main memory An instruction to issue an instruction;
In the secondary storage device , the data in the table specified in the query execution plan in the relational database without passing through the main storage device and without interposing the central processing unit, Instructions to be transferred to the parallel processor;
An instruction for causing the parallel processing device to process the data in the table specified in the query execution plan in the relational database in parallel;
An instruction for causing the parallel processing device to write the result of processing the data in the table specified in the query execution plan in the relational database to the main storage without interposing the central processing unit. The above-described problem is solved by providing a data processing program including the above-described data processing program.

Further, according to the present invention, the instruction to be written to the main storage device is intended only for column data used in the query execution plan among the results of the instruction to be processed in parallel. Paragraph 0010 The above problem is solved by providing the data processing program described.

The present invention is a database processing system including a secondary storage device , a main storage device, a central processing unit, a parallel processing device, a peripheral device bus, and a relational database stored in the secondary storage device ,
The central processing unit receives a query execution plan,
The central processing unit causes the secondary storage device to transfer the data in the table specified in the query execution plan in the relational database to the parallel processing device without going through the main storage device. Issue an order,
The secondary storage device passes the data in the table specified in the query execution plan in the relational database without passing through the main storage device and without interposing the central processing unit. Transfer to a parallel processor,
The parallel processing device processes the data in the table specified in the query execution plan in the relational database in parallel,
The parallel processing device provides a database processing system that writes the result of processing the data in the table specified in the query execution plan in the database to the main storage without interposing the central processing unit This solves the problem.

Further, according to the present invention, in the parallel processing device, only column data used in the query execution plan among the results of the instructions to be processed in parallel is written in the main storage device . The problem is solved by providing a database processing system.

A database query acceleration system, method, and program that are not easily affected by the performance bottleneck of the peripheral device bus can be implemented at low cost.

It is an example of a functional structure of the computer which concerns on the Example of this invention. It is an example of a function structure of the relational DBMS software which concerns on the Example of this invention. It is an example of the query execution plan of relational DBMS which concerns on the Example of this invention. It is a 1st example of the data processing method of relational DBMS which concerns on the Example of this invention. It is a 2nd example of the data processing method of relational DBMS which concerns on the Example of this invention.

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 shows an example of a functional configuration of a computer according to an embodiment of the present invention. The central processing unit (101) executes a program stored in the main storage device (102) and moves data between the main storage device (102) and the secondary storage device (103). The central processing unit (101) may be a processor (CPU) used in a general computer. The main storage device (102) may be a DRAM used in a general computer. The parallel processing device (104) is a processor that has a large number of cores and can perform parallel processing, and may be a graphical processing unit (GPU). In addition, the parallel processing device (104) may be a processor with a large number of cores called MIC (Many Integrated Core) or a dedicated design processor implemented with FGPA (Programmable Gate Array). Hereinafter, these are collectively referred to as a coprocessor. The peripheral device bus (106) is a means for connecting the main storage device (102), the secondary storage device (103), and the parallel processing device (104), and supports peer-to-peer DMA (direct memory access). . The peripheral device bus (106) is desirably provided with an NVMe function for connecting an SSD (semiconductor disk) to the industry standard PCI-E. The secondary storage device (103) is preferably an SSD based on flash memory technology. Although not shown, the computer according to the present invention may be provided with a network connection function for transmitting and receiving data and user requests via the network.

FIG. 2 shows an example of the functional configuration of the relational DBMS software according to the present invention. First, the query syntax analysis unit (201) receives a query sentence described in the SQL format, and creates a query syntax tree that is an internal format. Next, the query optimization unit (202) performs an optimization process on the query syntax tree and creates a query execution plan which is an internal format. The query execution plan is executed by the central processing unit (101) and the parallel processing unit (104), and queries (queries), joins (joins), classifications (sorts), aggregations (aggregates), etc. for the data stored in the database. Is performed. The present invention is not limited to a relational DBMS, and can be widely applied to any database management system that performs parallel processing on a large amount of data.

FIG. 3 shows an example of a relational DBMS query execution plan according to an embodiment of the present invention. In general, a query is represented by a scan process and a join process. The scanning process is a process of reading only rows (records, tuples) that match a scanning condition (for example, a specified column (column, field) value is within a specified numerical range) from a table in the relational DBMS. (For example, in the SQL statement, SELECT ORDER_ID FROM ORDER WHERE SUM> 10000)). Join processing (join) is processing that outputs a combination of rows that match the value of a specific column from multiple tables (for example, SELECT CMASTER.CUSTOMER_NAME FROM ORDER, CMASTER WHERE ORDER.CUSTOMER = CMASTER .CUSTOMER_ID). Both are processes typically performed in a relational DBMS, and improving the efficiency of both leads to the efficiency of the entire information system. Since both handle large amounts of data, it is important to minimize the movement of data, especially the movement of data via the peripheral device bus (106), which is a bottleneck in performance.

FIG. 4 shows a first example of the relational DBMS processing method according to the embodiment of the present invention. Here, an example of processing in a part of the scanning processing of FIG. 3 is shown. In FIG. 4, a thick arrow line represents a data flow, and a broken arrow line represents a control flow or a logical relationship. First, the central processing unit (101) dynamically generates an instruction sequence that can be executed by the parallel processing unit (104) according to the scanning condition of the query execution plan, and places it on the main storage unit (102) (S401). Since the table scanning process performs the same process (for example, numerical value range check) on a large number of rows, it is easy to generate an instruction for the parallel processing device (104). Next, the central processing unit (101) transfers the generated instruction sequence from the main storage unit (102) to the parallel processing unit (104) (S402). Next, the central processing unit (101) issues a command for transferring a set of rows in the database to be processed to the parallel processing unit (104) to the secondary storage unit (103) (S403). This command is preferably to initiate data transfer by peer-to-peer DMA. Next, the secondary storage device (103) transfers a set of rows in the database to be processed to the parallel processing device (104) without going through the main storage device (102) in accordance with this command ( It is desirable that the central processing unit (101) is designed not to intervene in this transfer process) (S404). Next, the parallel processing device (104) performs a predetermined scanning process, and only the rows that meet the scanning conditions are transferred to the main memory (102) (S405). Further, only the relevant data in the columns (columns, fields) required for the subsequent processing (for example, data corresponding to the column ORDER_ID in the example of the above SQL statement) is transferred in each row to be transferred. It is desirable. Here, the data to be processed may be each block of the database, and the parallel processing device (104) selects only the row that is finally used (that is, the row that satisfies the scanning condition). It is desirable that the data is transferred from the result buffer to the main storage device (102) by peer-to-peer DMA without using the central processing unit (101). In the method of the present invention, only necessary data is finally placed in the main storage device (102), and a large amount of data is mainly transferred via the peripheral device bus (106) only for temporary work. Since the data is not transferred onto the storage device (102), the processing load of the central processing unit (101), the consumption of the main storage device (memory), and the bandwidth of the peripheral device bus (106) can be reduced. In addition, the time required for data transfer can be shortened, and the efficiency of the entire database processing system can be improved.

FIG. 5 shows a second example of the data processing method of the relational DBMS according to the embodiment of the present invention. In this example, the efficiency of the combination of the combining process and the scanning process is shown. In FIG. 5, as in FIG. 4, a thick arrow line represents a data flow, and a broken arrow line represents a control flow. In FIG. 5, it is assumed that the row (record, tuple) of the table A that matches the scanning condition A has already been loaded on the main storage device (102). It is desirable that this loading process be performed by a highly efficient method as shown in FIG. First, the central processing unit (101) dynamically generates an instruction sequence that can be executed by the parallel processing unit (104) in accordance with the join condition of the query execution plan and the scanning condition B, and places it on the main storage unit (102). (S501). Since the table join process and the scan process perform the same process for many rows, it is easy to generate an instruction for the parallel processing device (104). Next, the central processing unit (101) transfers the generated instruction sequence and the processing target row of the table A from the main storage unit (102) to the parallel processing unit (104) (S502). Next, the central processing unit (101) issues an instruction for transferring a set of rows in the database to be processed to the parallel processing unit (104) to the secondary storage unit (103) (S503). This command is preferably to initiate data transfer by peer-to-peer DMA. Next, the secondary storage device (103) transfers the set of rows of the table B in the database to the parallel processing device (104) without passing through the main storage device (102) according to this command (this It is desirable that the central processing unit (101) is designed not to intervene in the transfer process) (S504). Next, the parallel processing device (104) performs predetermined combining processing and scanning processing, and transfers only the rows that match the combining condition and scanning condition B to the main memory (102) (S505). Furthermore, it is desirable to transfer only relevant data of columns (columns, fields) required for the subsequent processing within each row to be transferred. Here, the data to be processed may be each block of the database, and the parallel processing device (104) parallels only the rows that are finally used (that is, the rows that satisfy the scanning condition B and the combining condition). It is desirable to design the transfer from the result buffer to the main memory (102) by peer-to-peer DMA without using the central processing unit (101) and placing it in the result buffer in the processing unit (104). Similar to the example shown in FIG. 4, according to the method of the present invention, only the necessary data is finally placed in the main storage device (102), and a large amount of data is stored only for temporary work. Since it is not transferred to the main storage device (102) via the device bus (106), the processing load of the central processing unit (101), the amount of memory consumption, and the bandwidth of the peripheral device bus (106) The width can be reduced. In addition, the time required for data transfer can be shortened, and the efficiency of the entire database processing system can be improved.

(Technologically significant effect of the present invention)
According to the present invention, the following technically significant effects can be obtained. First, it is not necessary to secure an area for loading unnecessary data on the main storage device, and the required memory capacity can be reduced. Second, since query processing is processed in parallel and asynchronously by a coprocessor with a high degree of parallelism, the load on the CPU side can be reduced, and the response time of the query can be shortened. Third, it can be realized using only commodity products already on the market, and can be realized at low cost.

Claims (6)

A data processing method executed in a database processing system including a secondary storage device, a main storage device, a central processing device, a parallel processing device, a peripheral device bus, and a relational database stored in the secondary storage device,
The central processing unit receiving a query execution plan;
The central processing unit causes the secondary storage device to transfer the data in the table specified in the query execution plan in the relational database to the parallel processing device without going through the main storage device. Issuing a command;
The secondary storage device passes the data in the table specified in the query execution plan in the relational database without passing through the main storage device and without interposing the central processing unit. Transferring to a parallel processing unit;
The parallel processing device processing in parallel the scan, join or aggregation of the data in the table specified in the query execution plan in the relational database;
The parallel processing unit writing the result of processing the data in the table specified in the query execution plan in the relational database to the main storage without interposing the central processing unit. Data processing method. The data processing method according to claim 1, wherein the step of writing to the main storage device targets only column data used in the query execution plan among the results of the step of processing in parallel. A data processing program executed in a database processing system including a secondary storage device, a main storage device, a central processing device, a parallel processing device, a peripheral device bus, and a relational database stored in the secondary storage device,
Instructions for causing the central processing unit to receive a query execution plan;
Causing the central processing unit to transfer the data in the table specified in the query execution plan in the relational database to the parallel processing unit without passing through the main storage unit. An instruction to issue an instruction;
In the secondary storage device, the data in the table specified in the query execution plan in the relational database without passing through the main storage device and without interposing the central processing unit, Instructions to be transferred to the parallel processor;
An instruction that causes the parallel processing device to process in parallel the scanning, joining, or aggregation of the data in the table specified in the query execution plan in the relational database;
An instruction for causing the parallel processing device to write the result of processing the data in the table specified in the query execution plan in the relational database to the main storage without interposing the central processing unit. Including data processing program. The data processing program according to claim 3, wherein the instruction to be written to the main storage device targets only column data used in the query execution plan among results of the instructions to be processed in parallel. A database processing system including a secondary storage device, a main storage device, a central processing unit, a parallel processing device, a peripheral device bus, and a relational database stored in the secondary storage device;
The central processing unit receives a query execution plan,
The central processing unit causes the secondary storage device to transfer the data in the table specified in the query execution plan in the relational database to the parallel processing device without going through the main storage device. Issue an order,
The secondary storage device passes the data in the table specified in the query execution plan in the relational database without passing through the main storage device and without interposing the central processing unit. Transfer to a parallel processor,
The parallel processing device processes, in parallel , scanning, joining, or aggregation of the data in the table specified in the query execution plan in the relational database,
The database processing system, wherein the parallel processing device writes the result of processing the data in the table specified in the query execution plan in the database to the main storage without interposing the central processing unit. 6. The database processing system according to claim 5, wherein the parallel processing device writes only column data used in the query execution plan among the results of the instructions to be processed in parallel to the main storage device.

Images