Disclosure of Invention
In view of the foregoing problems, it is an object of the present invention to provide a data processing method based on a block chain, which realizes efficient and secure access to data.
According to one aspect of the invention, the invention provides a data processing system and a processing method based on a block chain, which are characterized in that the data processing system comprises a user terminal and a data processing server, wherein the user terminal is provided with a data transmission module and can realize data interaction with the data processing server; when a user initiates a data storage request, the data processing system processes a data packet requested to be stored by the user into two parts, wherein the size of the first part of data is smaller than that of the second part of data.
The first part of data is stored at the end of the block chain after being encrypted.
And the second part of data is stored on the cloud storage and/or the fog storage.
Further, the data processing server processes the data packet requested to be stored by the user to generate m sub-file packets, namely, a second part of data, wherein m is a positive integer greater than or equal to 2.
Further, the processing process of the data processing server on the data packet requested to be stored by the user comprises fragmentation processing, locking processing and encryption processing. The data processing server determines a key pair corresponding to the identity identification according to the identity identification of the file packet, and generates a unique hash value or data with the unique hash value, namely the first part of data, in the processing process of the data packet.
In one embodiment, the second portion of data is processed into a plurality of copies of data, and the plurality of copies of data are respectively stored on a plurality of cloud storage and/or different network nodes of the cloud storage.
In one embodiment, the data processing server is connected to a plurality of storage nodes, and the stored path data information is stored on the data processing server.
Further, the first portion data size is <1MB, preferably <100 KB.
Further, the data processing server stores user authentication information.
In one embodiment, the user terminal is provided with a login authentication module, a data display module, a data encryption and decryption operation module, a data transmission module and a network interface. The user terminal can be APP, computer software or other network login platforms.
Further, the data decryption operation module is used for pairing and verifying a private key corresponding to the data package identity and a hash value in the block chain, and then decrypting to obtain a corresponding processing process of the data package.
In one embodiment, the data storage method based on the block chain comprises the following steps:
(S101) a user terminal requests to store a data file, and a data processing server receives a storage demand instruction and a data packet initiated by a user;
(S102) the data processing server processes the data packet requested to be stored by the user, wherein the processing process comprises fragmentation processing and locking processing, m groups of data fragments are randomly generated from the data packet, m is a positive integer larger than or equal to 2, and then m encrypted sub-file packets, namely the second part of data, are generated through encryption processing;
(S103) the data processing server determines a key pair corresponding to the identity identification according to the identity identification of the file packet, and generates the processing process of the data packet in the step S102 into a unique hash value or data with the unique hash value, namely first part of data;
and (S104) the data processing server uploads the first part of data to a block chain end connected with the data processing server for storage, and transmits the m sub-file packets of the second part of data to a plurality of storage nodes connected with the data processing server for storage respectively.
In one embodiment, the data access method based on the block chain comprises the following steps:
(S201) a user logs in a user terminal, requests to access user data from the user terminal, and a data processing server determines a private key corresponding to a data identity according to the data identity of a data file requested by the user;
(S202) a process of reading a first portion of data from a block chain end and decrypting and restoring the first portion of data into a data packet according to a private key;
(S203) the data processing server reading a plurality of sub-file packages corresponding to the second portion of data from the plurality of storage nodes; and according to the data processing process of decrypting the first part of data in the step S202, decrypting the data of the second part of the plurality of encrypted sub-file packages to generate and display the file originally stored by the user.
In one embodiment, the block chain-based data deletion method comprises the following steps:
(S301) the user terminal requests to delete the user data, and the data processing server determines a private key corresponding to the data identity according to the data identity corresponding to the data file requested by the user;
(S302) a processing process of decrypting the first part of data by the private key to restore the data file, and then sending data deleting requests to a plurality of storage systems corresponding to the data encryption file packet by the data processing server according to the data processing process, namely deleting the second part of data;
(S303) the block chain end deleting the first part of data corresponding to the private key;
(S304) the user terminal displays the deletion status to the user.
In one embodiment, the data modification method based on the block chain comprises the following steps:
(S401) the user terminal requests to modify the stored data and uploads the modified data packet to the data processing server;
(S402) the data processing server determines a private key corresponding to the data file according to the data identity corresponding to the requested data file, and decrypts and restores the data file before modification; meanwhile, the modified data packet is subjected to fragmentation, locking and encryption processing, and the data packet is generated into a plurality of modified sub-file packets, namely modified second part data;
(S403) the data processing server generates the modified data packet processing procedure in step S402 as a new unique hash value, i.e. the modified first partial data, and replaces the original data corresponding to the end of the chunk link;
(S404) the data processing server respectively transmits the plurality of sub-file packages of the modified second part of data to a plurality of storage nodes connected with the data processing server, and simultaneously deletes the sub-file package corresponding to the original data package.
The invention has the beneficial technical effects
(1) The invention provides a safe and efficient data storage method and system based on a block chain aiming at personal or industrial big data.
(2) The block chain is adopted to store the user data, the data anonymity is high, the data exchange can be conveniently completed on the block chain, and the method has a good application prospect in the aspect of Internet of things or Internet of vehicles application.
(3) The invention provides a data storage system based on a block chain, which overcomes the problem of infinite increase of storage capacity requirement aiming at the block chain of big data, and can easily store single <1 TB-level data at the end of a block chain.
Detailed Description
In view of the foregoing problems, it is an object of the present invention to provide a data processing and data storage system based on a block chain, which realizes efficient and secure data access.
As shown in fig. 1, according to an aspect of the present invention, the present invention provides a data processing system and a processing method based on a block chain, and the present invention provides a data processing system and a processing method based on a block chain, wherein the data processing system includes a user terminal and a data processing server, wherein the user terminal has a data transmission module and can implement data interaction with the data processing server; when a user initiates a data storage request, the data processing system processes a data packet requested to be stored by the user into two parts, wherein the size of the first part of data is smaller than that of the second part of data.
In one embodiment, the data processing server provides and determines the key pair corresponding to the identity according to the user identity and/or the file identity contained in the user instruction. And the data processing server generates the processing process of the data packet into a unique hash value or data with the unique hash value, namely the first part of data according to the key pair corresponding to the identity. The first portion of data is stored at a block link end.
As shown in fig. 2 and 3, the second part of data is stored on the cloud storage and/or the fog storage.
Further, the data processing server processes the data packet requested to be stored by the user to generate m sub-file packets, namely, a second part of data, wherein m is a positive integer greater than or equal to 2.
Further, the processing process of the data processing server on the data packet requested to be stored by the user comprises fragmentation processing, encryption processing and locking processing. The fragmentation processing is to effectively divide the data packet into a plurality of data fragments, such as 1 picture or 1 video file, and may be divided into any plurality of fragments, such as 10000 fragments, and perform permutation and combination or other transformation processing on the fragmented data fragments to obtain m groups of data (file) fragments, where the data fragments may be in random size and/or permutation. And locking, namely locking the paths and the recovery modes of the file fragments of the plurality of encrypted file packets. And (4) performing encryption processing, namely performing encryption packaging processing to generate m encrypted file packets, wherein m is a positive integer greater than or equal to 2. The data processing server generates a unique hash value or data with the unique hash value in the processing process of the data packet, wherein the hash value is the identity of the data packet, namely the first part of data.
In one embodiment, the hash value of the data packet may be verified by a local processor or data server pair, so as to decrypt, restore, and process the fragmentation, locking, and encryption of the data.
In one embodiment, the second portion of data is processed into a plurality of copies of data, and the plurality of copies of data are respectively stored on a plurality of cloud storage and/or different network nodes of the cloud storage.
In one embodiment, the data processing server is connected to a plurality of storage nodes, and the stored path data information is stored on the data processing server.
Further, the first portion data size is <1MB, preferably <100 KB.
Further, the data processing server stores user authentication information.
As shown in fig. 4, in one embodiment, the user terminal has a login authentication module, a display module, a data decryption operation module, a data transmission module, and a network interface. The user terminal can be an APP, computer software or other network login platforms; and the login authentication module authenticates the identity information of the user.
Further, the data decryption operation module is used for pairing and verifying a private key corresponding to the data package identity and a hash value in the block chain, and then decrypting to obtain a corresponding processing process of the data package.
According to another aspect of the present invention, as shown in fig. 2 and 3, the present invention provides a blockchain-based data storage system, which is characterized by comprising a data processing server and at least two storage systems, wherein the first storage system is a blockchain storage system, the second storage system comprises a plurality of storage media, and the data processing server is respectively connected to the plurality of storage media of the first storage system and the second storage system through a network.
In one embodiment, the data processing server processes the data packet file from the user into two parts of data, wherein the first part of data Da is smaller in size than the second part of data Db, and stores the first part of data Da in the block chain storage system, wherein the second part of data Db is a plurality of sub-file packets, and the plurality of sub-file packets are stored in a plurality of storage media of the second storage system respectively.
And the data processing server determines a key pair corresponding to the identifier according to the user identity identifier and/or the file identity identifier contained in the user instruction. First partial data is generated from the key pair. The first part of data includes a process of generating the second part of data Db from the data package file.
Further, the data processing server provides a unique key pair for each user's data package file, and generates the first part of data according to the key pair.
Further, the block link end has a plurality of nodes, each block link node containing the complete first partial data.
Further, the second part of data is i sub-file packets Dbi, and different sub-file packets Dbi are respectively stored on the storage medium nodes of the second storage system, wherein i is a positive integer greater than or equal to 2, and i < the number of the storage medium nodes of the second storage system.
In one embodiment, the Dbi size of each data after the second part of data is decomposed is equivalent.
In one embodiment, each network node of the second storage system stores at most 1 piece of decomposed user data Dbi.
In one embodiment, each of the sub-file packages Dbi after the second part of data is decomposed is stored in n storage medium nodes of the second storage system, that is, the second part of data is stored in the storage medium nodes of the second storage system, n is a positive integer greater than 1, and each network node stores at most j decomposed user data, and the multiple data of each node are different from each other.
In one embodiment, n is preferably a positive integer within 2-10, and j is preferably a positive integer < n/2.
Further, the node data information is stored on a data processing server.
In one embodiment, the node data information is stored on a local memory of the user.
Further, the first portion data size is <1MB, preferably <100 KB.
In one embodiment, the data storage system further comprises a data backup memory, and the data backup server performs a compressed backup process on the first part of data and the second part of data.
In one embodiment, the user terminal can customize how many file packets the data is divided into, i.e. can customize the value of i.
In one embodiment, the media for storing the data packet Dbi stored by the user terminal in a customized manner includes a local storage, a cloud storage connected with the data processing server, an internet of things storage, or a distributed network storage.
As shown in fig. 5, the block chain-based data storage method of the present invention includes the following steps:
(S101) a user terminal requests to store a data file, and a data processing server receives a storage demand instruction and a data packet initiated by a user;
(S102) the data processing server processes the data packet requested to be stored by the user, wherein the processing process comprises fragmentation processing and locking processing, m groups of data fragments are randomly generated from the data packet, m is a positive integer larger than or equal to 2, and then m encrypted sub-file packets, namely the second part of data, are generated through encryption processing;
(S103) the data processing server determines a key pair corresponding to the identity identification according to the identity identification of the file packet, and generates the processing process of the data packet in the step S102 into a unique hash value or data with the unique hash value, namely first part of data;
and (S104) the data processing server uploads the first part of data to a block chain end connected with the data processing server for storage, and transmits the m sub-file packets of the second part of data to a plurality of storage nodes connected with the data processing server for storage respectively.
As shown in fig. 6, the block chain-based data access method of the present invention includes the following steps:
(S201) a user logs in a user terminal, requests to access user data from the user terminal, and a data processing server determines a private key corresponding to a data identity according to the data identity of a data file requested by the user;
(S202) a process of reading a first portion of data from a block chain end and decrypting and restoring the first portion of data into a data packet according to a private key;
(S203) the data processing server reading a plurality of sub-file packages corresponding to the second portion of data from the plurality of storage nodes; and according to the data processing process of decrypting the first part of data in the step S202, decrypting the data of the second part of the plurality of encrypted sub-file packages to generate and display the file originally stored by the user.
As shown in fig. 7, the block chain-based data deleting method of the present invention includes the following steps:
(S301) the user terminal requests to delete the user data, and the data processing server determines a private key corresponding to the data identity according to the data identity corresponding to the data file requested by the user;
(S302) a processing process of decrypting the first part of data by the private key to restore the data file, and then sending data deleting requests to a plurality of storage systems corresponding to the data encryption file packet by the data processing server according to the data processing process, namely deleting the second part of data;
(S303) the block chain end deleting the first part of data corresponding to the private key;
(S304) the user terminal displays the deletion status to the user.
As shown in fig. 8, the block chain-based data modification method of the present invention includes the following steps:
(S401) the user terminal requests to modify the stored data and uploads the modified data packet to the data processing server;
(S402) the data processing server determines a private key corresponding to the data file according to the data identity corresponding to the requested data file, and decrypts and restores the data file before modification; meanwhile, the modified data packet is subjected to fragmentation, locking and encryption processing, and the data packet is generated into a plurality of modified sub-file packets, namely modified second part data;
(S403) the data processing server generates the modified data packet processing procedure in step S402 as a new unique hash value, i.e. the modified first partial data, and replaces the original data corresponding to the end of the chunk link;
(S404) the data processing server respectively transmits the plurality of sub-file packages of the modified second part of data to a plurality of storage nodes connected with the data processing server, and simultaneously deletes the sub-file package corresponding to the original data package.
As described above, the present invention can be preferably realized. It will be appreciated by those skilled in the art that changes, modifications, substitutions, integrations and variations may be made in these embodiments without departing from the principles and spirit of the invention, which falls within the scope of the invention. The present invention is not limited to the specific embodiments described above, and can be practiced by any conventional technique.