CN113590638A - Distributed data storage system based on block chain - Google Patents
Distributed data storage system based on block chain Download PDFInfo
- Publication number
- CN113590638A CN113590638A CN202110816420.7A CN202110816420A CN113590638A CN 113590638 A CN113590638 A CN 113590638A CN 202110816420 A CN202110816420 A CN 202110816420A CN 113590638 A CN113590638 A CN 113590638A
- Authority
- CN
- China
- Prior art keywords
- data
- module
- storage
- host
- block
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013500 data storage Methods 0.000 title claims abstract description 42
- 238000004364 calculation method Methods 0.000 claims abstract description 6
- 238000012423 maintenance Methods 0.000 claims abstract description 6
- 239000000872 buffer Substances 0.000 claims description 18
- 238000000034 method Methods 0.000 claims description 9
- 238000004422 calculation algorithm Methods 0.000 claims description 5
- 230000001360 synchronised effect Effects 0.000 claims 1
- 238000010586 diagram Methods 0.000 description 8
- 230000005540 biological transmission Effects 0.000 description 5
- 230000008569 process Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000011084 recovery Methods 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2365—Ensuring data consistency and integrity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/70—Protecting specific internal or peripheral components, in which the protection of a component leads to protection of the entire computer
- G06F21/78—Protecting specific internal or peripheral components, in which the protection of a component leads to protection of the entire computer to assure secure storage of data
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Computer Hardware Design (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to the technical field of block chain storage, and discloses a distributed data storage system based on a block chain, which comprises a host module, a storage machine module and a calculation module, wherein the host module comprises a storage machine, a computer registration module, a data consistency maintenance module, a block request generation module and a data request query module; the invention can provide enough network expansibility, meet the ever-increasing storage requirement of data, effectively improve the block generation and data storage efficiency by means of a delegation rights and interests certification consensus mechanism, greatly improve the advantage of data storage safety by utilizing the characteristics of a block chain, and solve the problems that the existing data storage mode is easy to disorder due to the abundance of data, the nodes of data storage are not clear enough, and the safety of data storage is not high.
Description
Technical Field
The invention relates to the technical field of block chain storage, in particular to a distributed data storage system based on a block chain.
Background
The occupation of data storage in the internet era is extremely important, and with the arrival of the big data era, more and more data information is derived. In the data storage technology, the disadvantages of hardware storage are more and more prominent, and the disadvantages of the hardware storage are more and more prominent due to the storage capacity of the hardware storage. The technical application of cloud storage also has the disadvantage of being carried out by depending on the server, and the application of a large number of servers can gradually inhibit the development of the internet technology and bring huge consumption to resource use.
The block chain is a chain data structure formed by combining data blocks in a sequential connection mode according to a time sequence, and distributed data of the data blocks are related by using a cryptography method; the block chain technology utilizes a block chain type data structure to verify and store data, utilizes a distributed node consensus algorithm to generate and update data, utilizes a cryptography mode to ensure the safety of data transmission and access, utilizes an intelligent contract consisting of automatic script codes to program and operate a brand new distributed infrastructure and computing paradigm of the data, has the characteristics of non-tamper property, decentralization, anonymity, traceability and the like, is widely applied to the financial industry at present, and shows wide application prospects in numerous fields.
At present, a certain disadvantage still exists in a data storage mode, the data storage mode is easily disordered due to the abundance of the data, and meanwhile, the data storage safety is not high due to the instability of a network and a physical environment and the attack of a network hacker, so that a distributed data storage system based on a block chain is provided for solving the problem, wherein the distributed data storage system has extremely high fault tolerance and greatly improves the data storage safety.
Disclosure of Invention
The invention aims to provide a block chain-based distributed data storage system, which is used for carrying out multi-end distribution based on the distributed characteristic, effectively improving the storage efficiency, carrying out quick self-check and recovery on data by utilizing the block chain characteristic, ensuring the extremely high fault tolerance of the data, greatly improving the data storage safety, and solving the problems that the existing data storage mode is easy to disorder due to the abundance of the data, and simultaneously, the data storage safety is not high due to the instability of a network and a physical environment and the attack of a network hacker.
In order to achieve the purpose, the invention provides the following technical scheme: the utility model provides a distributed data storage system based on block chain, includes host computer module, memory module and calculation module, the host computer module has memory and computer registration module, data uniformity maintenance module, block request generation module, data request inquiry module, the memory module has data storage module, data inquiry module, data self-checking module, the calculation module has block calculation module.
Further, the host module comprises a storage computer and computer registration module, a data consistency maintenance module, a block request generation module and a data request query module, and the specific responsibilities are as follows: receiving the registration of a storage computer and a computer, and maintaining the states of all the storage computers and the computers; receiving an external request for adding data; when the size of the data buffer reaches a threshold value, the data of the buffer is required to be packed into a new block, and the new block enters a storage system after the data is finished; thirdly, receiving a request of external data query; fourthly, regularly requiring storage to check the integrity of data; and selecting one effective memory machine to send information to the ineffective memory machine when the memory machine is unavailable.
Further, the storage machine module comprises a data storage module, a data query module and a data self-checking module, and the specific responsibilities are as follows: firstly, storing data; when the host computer requires data checking, checking the data integrity of the host computer and reporting the checking result to the host computer; thirdly, when the host computer requires to receive/send data, the host computer receives/sends the data; and fourthly, when the host requires to inquire the data, inquiring and returning the data according to the block number and the offset given by the host.
Further, the computing module has a block computing module, and the specific responsibilities are as follows: calculating a hash value of a block according to block information sent by a host; secondly, generating blocks according to different consensus algorithms, and mainly selecting a delegation interest certification algorithm to realize efficient data storage.
Compared with the prior art, the invention has the following beneficial effects:
the invention can provide enough data security, meet the storage requirement that the data is not tampered, have the advantages of being capable of rapidly self-checking all data, synchronously recovering in time, effectively improving the storage efficiency, greatly improving the expansibility of a data storage network by utilizing the cooperation of the traditional distributed network architecture, and solving the problems that the existing data storage mode is easy to disorder due to the abundance of data, and the security of data storage is not high due to the instability of the network and the physical environment and the attack of network hackers.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings used in the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
FIG. 1 is a first schematic diagram of the system of the present invention;
FIG. 2 is a first sequence diagram of data storage according to the present invention;
FIG. 3 is a sequence diagram of a data query according to the present invention;
FIG. 4 is a first sequence diagram illustrating data transmission according to the present invention;
FIG. 5 is a second schematic diagram of the system of the present invention;
FIG. 6 is a sequence diagram of data storage according to the present invention;
FIG. 7 is a sequence diagram of a data query according to the present invention;
FIG. 8 is a sequence diagram of data transmission according to the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1-8, a block chain based distributed data storage system includes a host module, a storage module and a computing module, the host module includes a storage and a computer registration module, a data consistency maintenance module, a block request generation module, a data request query module, the storage module includes a data storage module, a data query module, a data self-check module, and the computing module includes a block computing module, the invention can provide sufficient data security, meet the storage requirement of data not being tampered, it can quickly self-check all data, timely and synchronously recover, effectively improve the storage efficiency, and greatly improve the expansibility of data storage network by using the cooperation of the traditional distributed network architecture, solve the problem that the current data storage mode is easy to be disordered due to the abundance of data, and simultaneously due to the instability of network and physical environment, and the attack of network hackers, resulting in the problem that the security of data storage is not high.
In this embodiment, the access right management is implemented by a MasterToken and an AccessToken, the MasterToken is used for the storage computer or the computer to determine the validity of the request sender, and the request is accepted only if the MasterToken attached to the request is consistent with the MasterToken obtained after the request is registered. The storage or computer should verify all requested tokens. The access token is used for verifying the identity of the storage machine when the storage machine sends a request to the host. In addition to registration, when the host receives the request from the storage device, it needs to verify whether the access token is consistent with the access token of the corresponding storage device in its own registry, and the inconsistent request will be rejected.
In the embodiment, a concise host information synchronization strategy is used, namely, if the state information of any host is changed, a new state is sent to all other hosts, and all other hosts cover the state table of the host.
In this embodiment, both the host and the storage machine are provided with buffers for smooth system performance. For the host, the buffer is used to temporarily store the information that needs to be added from the outside. When the amount of data in the buffer reaches a threshold, the host adds all the data in the buffer as a block to the system. For the storage machine, the new data sent by the host computer is processed according to the state of the storage machine, namely, the storage in the effective state immediately stores the data into a storage system when a new block is received; secondly, the machine in other states (receiving, invalid, verifying and transmitting) stores the new block into the buffer area of the machine, and stores the data in the buffer area into the storage system when the machine is converted into the valid state.
In this embodiment, when the system is initialized and started. The memory or computer should know any host address in advance before starting up, and should register with the host after starting up. The host computer should generate a MasterToken and an AccessToken corresponding to the storage machine for the storage machine, record the address of the registrant and two tokens corresponding to the registrant, return the two tokens to the registrant, and end the registration process.
In this embodiment, referring to fig. 2, the host provides an interface for storing data to the outside of the system. When the external sends data to the host, the host stores the data in a buffer and replies to the external with the number of the added block and the intra-block offset (note that the data is not actually added to the block at this time). When the buffer reaches the threshold, the host will ask the computer to pack the buffer data into a new block. After the packaging is finished, the host sends the new block to all the storage machines (no matter what state they are), and the buffer is emptied, and the current block chain state is updated.
In this embodiment, referring to fig. 3, the host provides an interface for querying data from outside the system. When the outside sends a request for inquiring data to the host, the block number and the offset in the block are required to be given. The host checks whether the block number is a buffer (i.e. the data is still in the buffer) or not, and if so, the data is directly returned according to the offset; otherwise, searching all the storage in the effective state and inquiring data, and performing self-checking and recovery as long as the data are not completely consistent until the data are completely the same; and returning the searched data to the outside after storing the data.
In this embodiment, the host maintains a timer for each storage machine, and each time a storage in a "valid" state is required to check its availability. The host will issue the most recent block number to the storage machine. The storage machine will transition to the "verify" state and begin checking the availability of its own saved chain starting with the first chunk. When one of the following two conditions occurs, entering an invalid state, and numbering the first invalid block to the host computer, wherein (i) the chain starts from one block, the hash is not matched, the check is stopped, the first block number which reports that the hash is not matched is matched with the full-chain hash, but the latest block number is not matched (which means that information is lost), and the first missing block number is reported; and when the conditions are not met, the storage machine informs the host that no abnormity exists after the examination is finished, and the host enters an effective state again. After the detection is finished, the host records the target state of the storage machine and carries out further processing.
In this embodiment, as shown in fig. 4, when the storage data is checked and the check result is invalid (hereinafter referred to as D1), the host selects one storage device in a valid state (hereinafter referred to as D2), transmits the accessoken corresponding to D2 to D1, and transmits all data of D2 starting from the number of the invalid block checked by D1 to D1. During this time, D1 should check whether the token of the request to send data is the accessToken of D2. After the transmission is finished, D2 informs the host computer that the transmission is finished, clears the buffer area and enters the effective state again; d1 deletes the accessToken of D2, clears the buffer area, sends the current latest area code to the host computer, if not matched, re-enters the invalid state and re-synchronizes the data; and if the two are matched, entering a valid state.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
Finally, it should be noted that: although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (4)
1. A distributed data storage system based on a block chain comprises a host module, a storage machine module and a computing module, and is characterized in that: the host module comprises a storage computer, a computer registration module, a data consistency maintenance module, a block request generation module and a data request query module, the storage computer module comprises a data storage module, a data query module and a data self-checking module, and the calculation module comprises a block calculation module.
2. The blockchain-based distributed data storage system of claim 1, wherein: the host module comprises a storage computer and computer registration module, a data consistency maintenance module, a block request generation module and a data request query module, and the specific responsibilities are as follows:
1) receiving the registration of the storage computers and the computers, and maintaining the states of all the storage computers and the computers;
2) receiving an external request for adding data; when the size of the data buffer reaches a threshold value, the data of the buffer is required to be packed into a new block, and the new block enters a storage system after the data is finished;
3) receiving a request of external inquiry data;
4) the storage machine is required to check the data integrity at regular time;
5) when the memory machine is not available, a valid memory machine is selected and is allowed to send data to the invalid memory machine.
The same information is synchronized among the hosts through a mechanism, and the information of all the hosts at all times is guaranteed to be uniform. In a system where each host is the same, the storage machine and the user can initiate requests to any one host.
3. The blockchain-based distributed data storage system of claim 1, wherein: the storage machine module comprises a data storage module, a data query module and a data self-checking module, and the specific responsibilities are as follows:
1) storing the data;
2) when the host computer requires data check, the data integrity of the host computer is checked, and the check result is reported to the host computer;
3) when the host computer requires to accept/send data, the host computer accepts/sends the data;
4) when the host requests to inquire data, the data is inquired and returned according to the block number and the offset given by the host.
4. The blockchain-based distributed data storage system of claim 1, wherein: the computing module comprises a block computing module, and the specific responsibilities are as follows:
1) calculating a hash value of the block according to the block information sent by the host;
2) according to the method, the blocks are generated according to different consensus algorithms, and the delegation interest certification algorithm is mainly selected, so that efficient data storage is realized.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110816420.7A CN113590638A (en) | 2021-07-20 | 2021-07-20 | Distributed data storage system based on block chain |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110816420.7A CN113590638A (en) | 2021-07-20 | 2021-07-20 | Distributed data storage system based on block chain |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113590638A true CN113590638A (en) | 2021-11-02 |
Family
ID=78248310
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110816420.7A Pending CN113590638A (en) | 2021-07-20 | 2021-07-20 | Distributed data storage system based on block chain |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113590638A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018112805A1 (en) * | 2016-12-21 | 2018-06-28 | 深圳前海达闼云端智能科技有限公司 | Blockchain storage method and device, and node device |
| CN110633323A (en) * | 2019-09-16 | 2019-12-31 | 腾讯科技（深圳）有限公司 | Business data storage method, device, storage medium and computer equipment |
| CN111339181A (en) * | 2020-02-10 | 2020-06-26 | 腾讯科技（深圳）有限公司 | Block storage method, block storage device, node equipment and storage medium |
| WO2020161688A1 (en) * | 2019-02-08 | 2020-08-13 | Christopher Lyndon Higgins | Distributed ledger computing platforms and associated methods, systems and devices |
| CN111949672A (en) * | 2020-07-30 | 2020-11-17 | 杭州电子科技大学 | Block chain storage method for supporting incremental update of Internet-of-things data |
| CN112905616A (en) * | 2021-03-19 | 2021-06-04 | 广东科学技术职业学院 | Storage system of block chain account book |
-
2021
- 2021-07-20 CN CN202110816420.7A patent/CN113590638A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018112805A1 (en) * | 2016-12-21 | 2018-06-28 | 深圳前海达闼云端智能科技有限公司 | Blockchain storage method and device, and node device |
| WO2020161688A1 (en) * | 2019-02-08 | 2020-08-13 | Christopher Lyndon Higgins | Distributed ledger computing platforms and associated methods, systems and devices |
| CN110633323A (en) * | 2019-09-16 | 2019-12-31 | 腾讯科技（深圳）有限公司 | Business data storage method, device, storage medium and computer equipment |
| CN111339181A (en) * | 2020-02-10 | 2020-06-26 | 腾讯科技（深圳）有限公司 | Block storage method, block storage device, node equipment and storage medium |
| CN111949672A (en) * | 2020-07-30 | 2020-11-17 | 杭州电子科技大学 | Block chain storage method for supporting incremental update of Internet-of-things data |
| CN112905616A (en) * | 2021-03-19 | 2021-06-04 | 广东科学技术职业学院 | Storage system of block chain account book |
Non-Patent Citations (1)
| Title |
|---|
| 翟社平 等: "区块链关键技术中的数据一致性研究", 《计算机技术与发展》, vol. 28, no. 09, pages 94 - 100 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108268208B (en) | RDMA (remote direct memory Access) -based distributed memory file system | |
| Dragojević et al. | {FaRM}: Fast remote memory | |
| CN113972986B (en) | Block chain-based industrial internet identification information analysis method and related device | |
| CN107493340B (en) | Data distribution verification method, device and system in block chain network | |
| CN101997823B (en) | Distributed file system and data access method thereof | |
| WO2016177285A1 (en) | Data pushing method and device | |
| CN110730225A (en) | Data processing method of Internet of things based on block chain, Internet of things and storage medium | |
| CN109067930A (en) | Domain name cut-in method, domain name analytic method, server, terminal and storage medium | |
| JP2023541298A (en) | Transaction processing methods, systems, devices, equipment, and programs | |
| CN104184832A (en) | Data submitting method and device in network application | |
| EP4207688A1 (en) | Asynchronous bookkeeping method and apparatus for blockchain, medium, and electronic device | |
| CN112738294B (en) | Domain name resolution method and device based on block chain, electronic equipment and storage medium | |
| CN108111325B (en) | Resource allocation method and device | |
| US10664349B2 (en) | Method and device for file storage | |
| CN109684273A (en) | A kind of snapshot management method, apparatus, equipment and readable storage medium storing program for executing | |
| Amiri et al. | Saguaro: An edge computing-enabled hierarchical permissioned blockchain | |
| CN111488626A (en) | Data processing method, device, equipment and medium based on block chain | |
| CN113064764A (en) | Method and apparatus for performing blocks in a blockchain system | |
| CN107276914B (en) | Self-service resource allocation scheduling method based on CMDB | |
| CN109583980A (en) | Method, system, electronic equipment and the storage medium of order processing | |
| US7933962B1 (en) | Reducing reliance on a central data store while maintaining idempotency in a multi-client, multi-server environment | |
| CN107967265B (en) | File access method, data server and file access system | |
| CN117082106A (en) | Multi-level data networking method, system, device and equipment oriented to government cloud environment | |
| CN113590638A (en) | Distributed data storage system based on block chain | |
| CN111241069A (en) | Data flattening method and system based on block chain |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |