CN110658994A

CN110658994A - Data processing method and device based on HDD (hard disk drive) and SSD (solid State disk) hybrid disk array

Info

Publication number: CN110658994A
Application number: CN201910828441.3A
Authority: CN
Inventors: 邸忠辉
Original assignee: Suzhou Wave Intelligent Technology Co Ltd
Current assignee: Suzhou Wave Intelligent Technology Co Ltd
Priority date: 2019-09-03
Filing date: 2019-09-03
Publication date: 2020-01-07
Anticipated expiration: 2039-09-03
Also published as: CN110658994B

Abstract

The embodiment of the application discloses a data processing method and a data processing device based on a HDD (hard disk drive) and SSD (solid state disk) hybrid disk array, which not only improve the data processing rate of the disk array, but also improve the data processing performance of a network transmission and storage system. The method comprises the following steps: the method comprises the steps of firstly receiving a target data writing request, then reading out storage data of a corresponding data disk HDD in a disk array according to the target data writing request, then calculating to obtain second check data by utilizing the target data, the storage data and first check data stored by a check disk SSD in the disk array, and further writing the target data into the corresponding data disk HDD in the disk array and writing the second check data into the check disk SSD in the disk array.

Description

Data processing method and device based on HDD (hard disk drive) and SSD (solid State disk) hybrid disk array

Technical Field

The application relates to the technical field of data processing, in particular to a data processing method and device based on a HDD and SSD hybrid disk array.

Background

With the rapid development of computer technology, the network data processing amount is also increased with storm wind, and nowadays, in order to improve the read-write performance in the data processing process and ensure the security of data, data is stored in a form of a disk array, such as a commonly used RAID-3 disk array, and the data throughput of a storage system is greatly improved by storing and reading data on a plurality of disks in the disk array at the same time.

In general, a RAID-3 Disk array includes a check Disk and a plurality of data disks, and the Disk types of the check Disk and the data disks are the same, and are all Hard Disk drives (Hard Disk drives for short), but in this storage manner, in a scenario of writing data randomly, if it is assumed that data needs to be written into Disk arrays of N different stripes and data is written into only one different data Disk in the Disk array of each stripe, data needs to be read and written once on each data Disk, and data needs to be read and written N times on the check Disk. Therefore, in a scene of randomly writing data with a large load, the read-write speed of the check disk HDD is low, so that the performance bottleneck of the disk array is likely to be formed, and even when the storage system is busy, the check disk HDD becomes the performance bottleneck of the whole storage system. This will seriously affect the data processing performance of the network transmission and storage system.

Therefore, how to increase the data processing rate of the RAID-3 disk array so as to further improve the data processing performance of the network transmission and storage system becomes a problem to be solved.

Disclosure of Invention

In order to solve the above problems, the present application provides a data processing method and apparatus based on a HDD and SSD (Solid State Disk, SSD for short) hybrid Disk array, and the specific technical solution is as follows:

in a first aspect, the present application provides a data processing method based on a hybrid disk array of an HDD and an SSD, where the method is applied to a RADI-3 disk array, a data disk in the disk array is a mechanical hard disk HDD, and a check disk in the disk array is a solid state hard disk SSD, and the method includes:

receiving a target data write request;

reading the storage data of the corresponding data disk HDD in the disk array according to the target data writing request;

calculating to obtain second check data by using the target data, the storage data and the first check data stored by the check disk SSD of the disk array;

and writing the target data into a corresponding data disk HDD in a disk array, and writing the second check data into a check disk SSD of the disk array.

In an optional implementation manner, a capacity difference between the data disk HDD and the check disk SSD in the disk array is less than or equal to a preset threshold.

In an optional implementation manner, the calculating, by using the target data, the storage data, and first parity data stored in a parity disk SSD of the disk array, to obtain second parity data includes:

performing exclusive-or calculation on the stored data and the target data to obtain third check data;

and performing exclusive OR calculation on the third check data and the first check data stored in the SSD in the disk array to obtain second check data.

In an optional implementation manner, when a capacity difference between a data disk HDD and a verification disk SSD in the disk array is greater than a preset threshold and the capacity of the verification disk SSD is smaller than the capacity of the data disk HDD, the verification disk SSD further includes an index table, and the index table stores a corresponding relationship between the disk array stripe number and a compressed data storage location of the first verification data.

acquiring first check data stored in a check disk SSD of the disk array according to the index table;

performing exclusive-or calculation on the third check data and the first check data to obtain fourth check data;

and compressing the fourth check data to obtain second check data.

In a second aspect, the present application provides a data processing apparatus based on a hybrid disk array of an HDD and an SSD, where the apparatus is applied to a raid-3 disk array, a data disk in the disk array is a mechanical hard disk HDD, and a check disk in the disk array is a solid state hard disk SSD, and the apparatus includes:

a receiving unit configured to receive a target data write request;

a reading unit, configured to read, according to the target data write request, storage data of a corresponding data disk HDD in the disk array;

the calculation unit is used for calculating to obtain second check data by utilizing the target data, the storage data and the first check data stored in the check disk SSD of the disk array;

and the writing unit is used for writing the target data into a corresponding data disk HDD in a disk array and writing the second check data into a check disk SSD of the disk array.

In an alternative implementation, the computing unit includes:

the first calculation subunit is used for performing exclusive-or calculation on the storage data and the target data to obtain third check data;

and the second calculating subunit is configured to perform exclusive or calculation on the third check data and the first check data stored in the check disk SSD in the disk array to obtain second check data.

In an alternative implementation, the computing unit includes:

the third calculation subunit is used for performing exclusive-or calculation on the storage data and the target data to obtain third check data;

the obtaining subunit is configured to obtain, according to the index table, first verification data stored in a verification disk SSD of the disk array;

the fourth calculating subunit is configured to perform xor calculation on the third check data and the first check data to obtain fourth check data;

and the compression subunit is used for compressing the fourth check data to obtain second check data.

According to the data processing method based on the HDD and SSD hybrid disk array, a target data write-in request is received, then the storage data of the corresponding data disk HDD in the disk array is read according to the target data write-in request, then the target data, the storage data and the first check data stored by the check disk SSD in the disk array are utilized, the second check data are obtained through calculation, further, the target data can be written into the corresponding data disk HDD in the disk array, and the second check data are written into the check disk SSD in the disk array. Therefore, in the embodiment of the application, the solid state disk SSD is used as the check disk of the disk array, and the read-write performance of the solid state disk SSD is greatly improved compared with the HDD disk, so that the check disk does not become a performance bottleneck of the disk array in a scenario of randomly writing data with a large load, and thus the data processing rate of the disk array can be improved, and the data processing performance of the network transmission and storage system can be further improved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive labor.

Fig. 1 is a flowchart of a data processing method based on a HDD and SSD hybrid disk array according to an embodiment of the present application;

fig. 2 is one of the flow diagrams of data writing based on the HDD and SSD hybrid disk array according to the embodiment of the present application;

fig. 3 is a second schematic flowchart of data writing based on a HDD and SSD hybrid disk array according to an embodiment of the present application;

fig. 4 is a schematic structural diagram of a data processing apparatus based on a HDD and SSD hybrid disk array according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

In order to facilitate understanding of the technical solutions provided in the present application, the following briefly describes the research background of the technical solutions in the present application.

As is well known, with the rapid development of network technology, the processing amount of network data is increasing, and in order to improve the read-write performance during the processing of network data and ensure the security of data, data is stored in the form of a disk array, for example, in a commonly used RAID-3 disk array, but since a check disk and a plurality of data disks included in the disk array are HDDs, in this way, in a scenario where data is randomly written in with a large load, since the read-write speed of the check disk HDD is low, the performance bottleneck of the disk array will be likely to be caused, and even when the storage system is busy, the check disk HDD may become the bottleneck of the performance of the entire storage system, which will seriously affect the data processing performance of the network transmission and storage system.

Based on this, in order to solve the above problems, the present application provides a data processing method and apparatus based on a HDD and SSD hybrid disk array, so as to improve the data processing rate of the disk array, and further improve the data processing performance of the network transmission and storage system.

The data processing method based on the HDD and SSD hybrid disk array according to the embodiment of the present application will be described in detail below with reference to the drawings. Referring to fig. 1, which shows a flowchart of a data processing method based on a HDD and SSD hybrid disk array according to an embodiment of the present application, the embodiment may include the following steps:

s101: a target data write request is received.

In practical application, the data processing method based on the HDD and SSD hybrid disk array provided in this embodiment may be applied to the RADI-3 disk array, and in the disk array, the mechanical hard disk HDD is used as a data disk, and the solid state hard disk SSD is used as a check disk, so that the hybrid disk array based on the HDD and the SSD is configured to perform data processing.

In this embodiment, any data written or read by using this embodiment is defined as target data. Also, the present embodiment does not limit the data type of the target data, for example, the target data may be binary data, hexadecimal data, or the like. Further, the HDD and SSD based hybrid disk array may continue to perform the subsequent step S102 after receiving the target data write request.

The target data write request indicates a write address of the target data, that is, an identifier (such as an address of a corresponding data disk) indicating that the target data needs to be written into the corresponding data disk of the HDD and SSD based hybrid disk array.

S102: and reading the storage data of the corresponding data disk HDD in the disk array according to the target data writing request.

In this embodiment, after receiving the target data write request in step S101, the data stored in the target data write request may be further read from the corresponding data disk HDD in the disk array according to the write address of the target data indicated by the write request (i.e. the identifier of the corresponding data disk in the disk array).

Specifically, referring to fig. 2, assuming that the hybrid disk array based on HDDs and SSDs is composed of 4-block data disks HDDs and one check disk SSD, and the target data write request indicates that the target data B needs to be written in the leftmost data disk HDD of the disk array, the storage data a needs to be read out from the leftmost data disk HDD first for the subsequent step S103.

S103: and calculating to obtain second check data by using the target data, the storage data and the first check data stored in the check disk SSD of the disk array.

In this embodiment, after the storage data corresponding to the data disk HDD in the disk array is read in step S102, the verification data (defined as the first verification data here) stored in the verification disk SSD of the disk array needs to be read, and on this basis, the storage data read from the data disk HDD, the target data, and the first verification data read from the verification disk SSD may be used to calculate the second verification data for performing the subsequent step S104.

In this implementation, the preset threshold refers to a critical value used for distinguishing whether the capacities of the data disk HDD and the verification disk SSD are the same, and if the capacity difference between the data disk HDD and the verification disk SSD is smaller than or equal to the critical value, it may be determined that the capacities between the data disk HDD and the verification disk SSD are the same, and further, the following steps a1-a2 may be continuously performed; otherwise, it can be determined that the capacities between the data disk HDD and the check disk SSD are different. It should be noted that the specific value of the preset threshold may be set according to an actual situation, which is not limited in this embodiment of the application.

Based on this, the specific implementation process of this step S103 may include the following steps a1-a 2:

step A1: and carrying out XOR calculation on the stored data and the target data to obtain third check data.

In this implementation manner, after the storage data of the corresponding data disk HDD in the disk array and the first check data stored in the check disk SSD are read, the storage data and the first check data may be further subjected to xor calculation to obtain third check data.

Specifically, referring to fig. 2, assuming that the hybrid disk array based on HDDs and SSDs is composed of 4 data disks HDD and one check disk SSD, and the target data write request indicates that the target data B needs to be written into the leftmost data disk HDD of the disk array, after the storage data a is read from the leftmost data disk HDD, a and B may be subjected to an xor calculation to obtain a calculation result C, which is defined as third check data, for performing the subsequent step a 2.

For example, the following steps are carried out: assuming that the target data B is 1111 and the storage data a is 0010 read from the leftmost data disk HDD in fig. 2, the calculation result C is 1101 obtained by performing an exclusive or calculation on the target data B and the storage data a, which can be used as third check data for performing the subsequent step a 2.

Step A2: and performing exclusive-or calculation on the third check data and the first check data stored in the SSD in the disk array to obtain second check data.

In this implementation manner, after the third parity data is calculated in step a1, the third parity data may be subjected to xor calculation with the first parity data stored in the read parity disk SSD, so as to obtain the second parity data.

Specifically, referring to fig. 2, assuming that the hybrid disk array based on HDDs and SSDs is composed of 4 data disks HDD and one parity disk SSD, and the target data write request indicates that the target data B needs to be written in the leftmost data disk HDD of the disk array, and at the same time, the first parity data D is read from the parity disk SSD, and the third parity data C is calculated through step a1, then C and D may be further subjected to xor calculation to obtain a calculation result E, which is defined as the second parity data, for performing the subsequent step S104.

For example, the following steps are carried out: assuming that the target data B is 1111 and the storage data a is 0010 read from the leftmost data disk HDD in fig. 2, the xor calculation between the target data B and the storage data a can obtain a calculation result C of 1101, and if the first check data D read from the check disk SSD is 1111, the xor calculation between C and D can be further performed, that is, the xor calculation between 1111 and 1101 can obtain a calculation result E of 0010, which is used as the second check data for performing the subsequent step S104.

In another possible implementation manner of the embodiment of the application, when a capacity difference between the data disk HDD and the check disk SSD in the disk array is greater than a preset threshold and the capacity of the check disk SSD is smaller than the capacity of the data disk HDD, the check disk SSD further includes an index table, and the index table stores a corresponding relationship between a disk array stripe number and a compressed data storage location of the first check data.

In this implementation manner, the capacity difference between the data disk HDD and the check disk SSD in the disk array is greater than the preset threshold, and the capacity of the check disk SSD is smaller than the capacity of the data disk HDD, so as to ensure the smoothness of data processing, the check data may be compressed first, and then the compressed check data is written into the check disk SSD, so as to ensure that the data processing amount of the check disk is consistent with the data processing amount of the multiple data disk HDDs, so as to ensure that the data processing rate of the entire disk array is not reduced.

Specifically, an index table is pre-established in the check disk SSD to store a corresponding relationship between the number of the stripe of the disk array and the storage location of the check data after the first check data is compressed, so that the corresponding compressed data is read from the designated location according to the number of the stripe of the disk array by querying the corresponding relationship stored in the index table, and then the compressed data is decompressed to obtain the corresponding first check data, so as to perform the subsequent steps.

Based on this, the specific implementation process of the step S103 may include the following steps B1-B3:

step B1: and carrying out XOR calculation on the stored data and the target data to obtain third check data.

It should be noted that step B1 is identical to step a1, and please refer to the description of step a1 for details.

Step B2: and acquiring first check data stored in a check disk SSD of the disk array according to the index table.

In this implementation manner, since the capacity of the check disk SSD is smaller than the capacity of the data disk HDD, the corresponding relationship between the stripe number of the current disk array and the storage location of the compressed data of the first check data needs to be queried according to the index table pre-established in the check disk SSD, so as to query the compressed data of the first check data corresponding to the current disk array, and further, the check data needs to be decompressed to obtain the first check data corresponding to the current disk array, so as to execute the subsequent step B3.

Step B3: and carrying out XOR calculation on the third check data and the first check data to obtain fourth check data.

The third check data is obtained through the step B1, and after the first check data is obtained through the step B2, the first check data and the second check data may be further subjected to xor calculation to obtain fourth check data.

Specifically, referring to fig. 3, it is assumed that the HDD and SSD-based hybrid disk array is composed of 4 data disks HDD and a check disk SSD, and the target data write request indicates that the target data B needs to be written into the leftmost data disk HDD of the disk array, and at the same time, the compressed data is read from the check disk SSD as D according to the index table, and after decompressing it, the first check data E is obtained, and the third check data C is calculated through step B1, the calculation result F can be obtained by xoring C with E, and defined as the fourth check data, so as to execute the subsequent step B4.

Step B4: and compressing the fourth check data to obtain second check data.

After the fourth check data is obtained in step B3, in order to reduce the storage space occupied by the fourth check data, it needs to be compressed, and the compressed result is used as the second check data to execute the subsequent step S104. For example, referring to fig. 3, after the xor calculation is performed on the first check data E and the third check data C to obtain the fourth check data F, the fourth check data F may be further compressed to obtain a compression result G, which is used as the second check data to perform the subsequent step S104.

It should be noted that, the volume data compression method and the corresponding decompression method in the above steps may both adopt existing or future data compression and decompression methods, and the method may be selected according to actual situations, which is not limited in the embodiment of the present application.

S104: and writing the target data into a corresponding data disk HDD in the disk array, and writing the second check data into a check disk SSD of the disk array.

In this embodiment, after the second parity data is obtained in step S103, the second parity data may be further written into the parity disk SSD of the current disk array to update the parity data in the parity disk. Meanwhile, the target data can be written into the data disk indicated by the target data writing request, and the writing process of the target data is completed.

In summary, in the data processing method based on the HDD and SSD hybrid disk array provided in this embodiment, a target data write request is received first, then the storage data of the corresponding data disk HDD in the disk array is read according to the target data write request, and then the second check data is calculated by using the target data, the storage data, and the first check data stored by the check disk SSD in the disk array, so that the target data can be written into the corresponding data disk HDD in the disk array, and the second check data can be written into the check disk SSD in the disk array. Therefore, in the embodiment, the solid state disk SSD is used as the check disk of the disk array, and the read-write performance of the solid state disk SSD is greatly improved compared with the HDD disk, so that the check disk does not become a performance bottleneck of the disk array in a scenario of randomly writing data with a large load, thereby improving the data processing rate of the disk array and further improving the data processing performance of the network transmission and storage system.

Based on the above data processing method based on the HDD and SSD hybrid disk array, the present application also provides a data processing apparatus based on the HDD and SSD hybrid disk array, where the apparatus is applied to the RADI-3 disk array, a data disk in the disk array is a mechanical hard disk HDD, and a check disk is a solid state hard disk SSD, as shown in fig. 4, the apparatus includes:

a receiving unit 401 configured to receive a target data write request;

a reading unit 402, configured to read, according to the target data write request, storage data of a corresponding data disk HDD in the disk array;

a calculating unit 403, configured to calculate to obtain second check data by using the target data, the storage data, and first check data stored in a check disk SSD of the disk array;

a writing unit 404, configured to write the target data into a corresponding data disk HDD in a disk array, and write the second parity data into a parity disk SSD of the disk array.

In an implementation manner of this embodiment, a capacity difference between the data disk HDD and the verification disk SSD in the disk array is less than or equal to a preset threshold.

In an implementation manner of this embodiment, the calculating unit 404 includes:

In an implementation manner of this embodiment, when a capacity difference between a data disk HDD and a verification disk SSD in the disk array is greater than a preset threshold and the capacity of the verification disk SSD is smaller than the capacity of the data disk HDD, the verification disk SSD further includes an index table, where the index table stores a corresponding relationship between the disk array stripe number and a compressed data storage location of the first verification data.

In summary, in the data processing apparatus based on a hybrid disk array of an HDD and an SSD provided in this embodiment, a target data write request is first received, then, according to the target data write request, stored data of a corresponding data disk HDD in the disk array is read, and then, second parity data is calculated by using the target data, the stored data, and first parity data stored by a parity disk SSD in the disk array, so that the target data can be written into the corresponding data disk HDD in the disk array, and the second parity data can be written into the parity disk SSD in the disk array. Therefore, in the embodiment, the solid state disk SSD is used as the check disk of the disk array, and the read-write performance of the solid state disk SSD is greatly improved compared with the HDD disk, so that the check disk does not become a performance bottleneck of the disk array in a scenario of randomly writing data with a large load, thereby improving the data processing rate of the disk array and further improving the data processing performance of the network transmission and storage system.

It should be noted that, in the present specification, the embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments may be referred to each other. For the system or the device disclosed by the embodiment, the description is simple because the system or the device corresponds to the method disclosed by the embodiment, and the relevant points can be referred to the method part for description.

It is further noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data processing method based on a HDD and SSD hybrid disk array is characterized in that the method is applied to a RADI-3 disk array, a data disk in the disk array is a mechanical hard disk (HDD), a check disk in the disk array is a Solid State Disk (SSD), and the method comprises the following steps:

receiving a target data write request;

2. The method according to claim 1, wherein the capacity difference between the data disks HDD and the check disks SSD in the disk array is less than or equal to a preset threshold.

3. The method of claim 2, wherein the calculating second parity data using the target data, the storage data, and first parity data stored by a parity disk (SSD) of the disk array comprises:

4. The method according to claim 1, wherein when the capacity difference between the data disk HDD and the check disk SSD in the disk array is greater than a preset threshold and the capacity of the check disk SSD is smaller than the capacity of the data disk HDD, the check disk SSD further includes an index table, and the index table stores a corresponding relationship between the disk array stripe number and the compressed data storage location of the first check data.

5. The method of claim 4, wherein the calculating the second parity data by using the target data, the storage data, and the first parity data stored in the parity disk SSD of the disk array comprises:

and compressing the fourth check data to obtain second check data.

6. A data processing device based on a hybrid disk array of an HDD and an SSD is characterized in that the device is applied to a RADI-3 disk array, a data disk in the disk array is a mechanical hard disk (HDD), a check disk in the disk array is a Solid State Disk (SSD), and the device comprises:

a receiving unit configured to receive a target data write request;

7. The apparatus of claim 6, wherein a difference between capacities of the data disks HDD and the check disks SSD in the disk array is less than or equal to a preset threshold.

8. The apparatus of claim 7, wherein the computing unit comprises:

9. The apparatus according to claim 6, wherein when a capacity difference between the data disk HDD and the verification disk SSD in the disk array is greater than a preset threshold and the capacity of the verification disk SSD is smaller than the capacity of the data disk HDD, the verification disk SSD further includes an index table, and the index table stores a correspondence between the disk array stripe number and a compressed data storage location of the first verification data.

10. The apparatus of claim 9, wherein the computing unit comprises: