CN109587185B - Cloud storage system and object processing method in cloud storage system - Google Patents
Cloud storage system and object processing method in cloud storage system Download PDFInfo
- Publication number
- CN109587185B CN109587185B CN201710896038.5A CN201710896038A CN109587185B CN 109587185 B CN109587185 B CN 109587185B CN 201710896038 A CN201710896038 A CN 201710896038A CN 109587185 B CN109587185 B CN 109587185B
- Authority
- CN
- China
- Prior art keywords
- server
- bucket
- instruction
- information
- ktcpvs
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/543—User-generated data transfer, e.g. clipboards, dynamic data exchange [DDE], object linking and embedding [OLE]
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides a cloud storage system and an object processing method in the cloud storage system. The object processing method comprises the following steps: the method comprises the steps that a first server receives a first writing instruction sent by a client, wherein the first writing instruction is used for requesting to write a first object into a first bucket; the first server writes the first object into a first bucket of the first storage node set according to the first writing instruction and sends the first object to the second server; the second server receives a second write instruction sent by the client, wherein the second write instruction is used for requesting to write a second object into the first bucket; and the second server writes the second object into the first bucket of the second storage node set according to the second writing instruction and sends the second object to the first server. The cloud storage system and the object processing method in the cloud storage system are beneficial to saving resources.
Description
Technical Field
The present application relates to the field of cloud storage, and more particularly, to a cloud storage system and an object processing method in the cloud storage system.
Background
In the field of public cloud storage, an active-passive (AP) mode may be adopted to provide storage service for a user.
In the existing AP mode, one area is divided into two Areas (AZ) providing services, one is called a master AZ, and the other is called a slave AZ. A cluster (cluster) is deployed in the master AZ, and a cluster is also deployed in the backup AZ. The clusters in the master AZ and the clusters in the standby AZ form a Cluster Group (CG). The direction of copying of the object (object) is from the cluster in the primary AZ to the cluster in the backup AZ. Requests for user write objects in this area are all handled by the cluster in the master AZ.
Specifically, when the client sends a write object request to the cluster in the master AZ, the cluster in the master AZ stores the object and copies the object to the cluster in the backup AZ; when the client side sends the write object request to the cluster in the standby AZ, the cluster in the standby AZ needs to send the object to the cluster in the main AZ first, and after the cluster in the main AZ stores the object, the cluster in the main AZ copies the object to the cluster in the standby AZ.
The processing method in the existing AP mode can cause large data forwarding amount and consume more resources.
Disclosure of Invention
The application provides a cloud storage system and an object processing method in the cloud storage system, which are beneficial to saving resources.
In a first aspect, the present application provides an object processing method in a cloud storage system. The cloud storage system comprises a first server, a first storage node set, a second server and a second storage node set, wherein the first server and the first storage node set are located in a first service area of the cloud storage system, the second server and the second storage node set are located in a second service area of the cloud storage system, and the first storage node set and the second storage node set both comprise a first bucket.
The object processing method comprises the following steps:
the method comprises the steps that a first server receives a first writing instruction sent by a client, wherein the first writing instruction is used for requesting to write a first object into a first bucket;
the first server writes the first object into a first bucket of the first storage node set according to the first writing instruction, and sends the first object to the second server, so that the second server writes the first object into the first bucket of the second storage node set;
the second server receives a second write instruction sent by the client, wherein the second write instruction is used for requesting to write a second object into the first bucket;
and the second server writes the second object into the first bucket of the second storage node set according to the second writing instruction, and sends the second object to the first server, so that the first server writes the second object into the first bucket of the first storage node set.
In the object processing method, when the first server and the second server receive a writing instruction requesting to write the object, both the first server and the second server can store the writing instruction and copy the object to the other side. Compared with the prior art, the method reduces the forwarding flow of the object, thereby saving resources.
With reference to the first aspect, in a first possible implementation manner, the cloud storage system further includes a first KTCPVS.
In this case, the first server receives a first write instruction sent by the client, and includes:
the method comprises the steps that a first KTCPVS receives a first writing instruction sent by a client, determines to send the first writing instruction to a first server according to the first writing instruction, and sends the first writing instruction to the first server;
the first server receives a first write command sent by a first KTCPVS.
The second server receives a second write instruction sent by the client, and the second write instruction comprises the following steps:
the first KTCPVS receives a second writing instruction sent by the client, determines to send the second writing instruction to the second server according to the second writing instruction, and sends the second writing instruction to the second server;
and the second server receives the second write instruction sent by the first KTCPVS.
With reference to the first possible implementation manner, in a second possible implementation manner, the object processing method further includes:
the method comprises the steps that a first KTCPVS receives a first reading instruction sent by a client, the first reading instruction is used for requesting to read a first object in a first barrel, the first reading instruction is sent to a first server according to the first reading instruction, and the first reading instruction is sent to the first server;
the method comprises the steps that a first server receives a first read instruction sent by a first KTCPVS, and reads a first object in a first bucket of a first storage node set according to the first read instruction;
the first KTCPVS receives a second reading instruction sent by the client, wherein the second reading instruction is used for requesting to read a second object in the first barrel, determining to send the second reading instruction to the second server according to the second reading instruction, and sending the second reading instruction to the second server;
and the second server receives a second reading instruction sent by the first KTCPVS, and reads a second object in the first bucket of the second storage node set according to the second reading instruction.
In the implementation manner, the first KTCPVS sends the first write instruction to the first server, and after the first server stores the first object as a main server, the first server sends the first read instruction to the first server, and the first server reads the first object from the first storage node set, so that the strong consistency of the first object can be maintained.
Similarly, strong consistency of the second object may be maintained.
With reference to the second possible implementation manner, in a third possible implementation manner, there is first information on the first KTCPVS, where the first information includes: the first bucket corresponds to a first server and a second server in sequence.
In this case, the determining, by the first KTCPVS, that the first write command is to be sent to the first server according to the first write command includes: the first KTCPVS calculates a hash value of the first object relative to the number of the servers corresponding to the first bucket according to the first writing instruction to obtain a first hash value; and determining to send a first write instruction to a first server corresponding to the first bucket according to the first information and the first hash value.
The first KTCPVS determines to send the second write command to the second server according to the second write command, and the method comprises the following steps: the first KTCPVS calculates a hash value of the second object relative to the number of the servers corresponding to the first bucket according to the second write instruction to obtain a second hash value; and determining to send a second write instruction to a second server corresponding to the second server of the first bucket according to the first information and the second hash value.
The first KTCPVS determines to send the first read instruction to the first server according to the first read instruction, and the method comprises the following steps: the first KTCPVS calculates a hash value of the first object relative to the number of the servers corresponding to the first bucket according to the first reading instruction to obtain a first hash meeting; and determining to send a first read instruction to a first server corresponding to the first bucket according to the first information and the first hash value.
The first KTCPVS determines to send the second reading instruction to the second server according to the second reading instruction, and the method comprises the following steps: the first KTCPVS calculates a hash value of the number of the servers corresponding to the first bucket relative to the second object according to the second reading instruction to obtain a second hash value; and determining to send a second read instruction to a second corresponding server of the first bucket according to the first information and the second hash value.
With reference to the third possible implementation manner, in a fourth possible implementation manner, the first information includes first sub information, second sub information, and third sub information, and the first sub information is: the first barrel comprises a first sub-barrel and a second sub-barrel in sequence, and the second sub-information is as follows: the main server corresponding to the first sub-bucket is the first server, and the third sub-information is: the main server corresponding to the second sub-bucket is the second server.
In this case, the determining, by the first KTCPVS, that the first write instruction is sent to the first server corresponding to the first bucket according to the first information and the first hash value includes: the first KTCPVS determines to write the first object into a first corresponding first sub-bucket of the first bucket according to the first sub-information and the first hash value; and determining to send the first write instruction to the first server according to the second sub information and the first sub bucket.
The first KTCPVS determines to send a second writing instruction to a second server corresponding to the second barrel of the first barrel according to the first information and the second hash value, and the method comprises the following steps: the first KTCPVS determines to write the second object into a second corresponding sub-bucket of the first bucket according to the first sub-information and the second hash value; and determining to send a second write instruction to the second server according to the third sub information and the second sub bucket.
The first KTCPVS determines, according to the first information and the first hash value, to send a first read instruction to a first server corresponding to a first bucket, including: the first KTCPVS determines to read a first object in a first sub-bucket corresponding to the first sub-bucket according to the first sub-information and the first hash value; and determining to send the first read instruction to the first server according to the second sub information and the first sub bucket.
The first KTCPVS determines to send a second read instruction to a second server corresponding to the second bucket of the first bucket according to the first information and the second hash value, and the method includes: the first KTCPVS determines to read a second object in a second sub-bucket corresponding to the second sub-bucket of the first bucket according to the first sub-information and the second hash value; and determining to send a second reading instruction to the second server according to the third sub information and the second sub bucket.
With reference to the third or fourth possible implementation manner, in a fifth possible implementation manner, the cloud storage system further includes a location server, and the location server is configured with the first information.
In this case, the object processing method further includes: the first KTCPVS acquires first information from the location server.
In combination with the fifth possible implementation manner, in a sixth possible implementation manner, second information is further configured on the location server, the second information includes that the KTCPVS corresponding to the first bucket is the first KTCPVS, and the cloud storage system further includes domain name system equipment.
In this case, the first KTCPVS receives a first write command sent by the client, and includes:
the method comprises the steps that domain name system equipment receives a first uniform resource locator sent by a client, wherein the first uniform resource locator comprises a first writing instruction; sending information of the first bucket to a location server according to the first uniform resource locator;
the method comprises the steps that a position server receives information of a first barrel sent by domain name system equipment; sending first indication information to a domain name system according to the second information and the information of the first barrel, wherein the first information is used for indicating the first KTCPVS;
the domain name system receives first indication information sent by a position server; sending first indication information to a client;
the method comprises the steps that a client receives first indication information sent by domain name system equipment, and sends a first writing instruction to a first KTCPVS according to the first indication information;
and the first KTCPVS receives a first writing instruction sent by the client according to the first indication information.
The first KTCPVS receives a second write instruction sent by the client, and the second write instruction comprises the following steps:
the domain name system equipment receives a second uniform resource locator sent by the client, wherein the second uniform resource locator comprises a second writing instruction; sending information of the first bucket to a location server according to the second uniform resource locator;
the method comprises the steps that a position server receives information of a first barrel sent by domain name system equipment; sending first indication information to a domain name system according to the second information and the information of the first barrel;
the domain name system receives first indication information sent by a position server; sending first indication information to a client;
the client receives first indication information sent by the domain name system equipment, and sends a second writing instruction to the first KTCPVS according to the first indication information;
and the first KTCPVS receives a second writing instruction sent by the client according to the first indication information.
The first KTCPVS receives a first read instruction sent by a client, and the method comprises the following steps:
receiving a third uniform resource locator sent by a client by domain name system equipment, wherein the third uniform resource locator comprises a first reading instruction; sending information of the first bucket to a location server according to the third uniform resource locator;
the method comprises the steps that a position server receives information of a first barrel sent by domain name system equipment; sending first indication information to a domain name system according to the second information and the information of the first barrel;
the domain name system receives first indication information sent by a position server; sending first indication information to a client;
the method comprises the steps that a client receives first indication information sent by domain name system equipment, and sends a first reading instruction to a first KTCPVS according to the first indication information;
and the first KTCPVS receives a first reading instruction sent by the client according to the first indication information.
The first KTCPVS receives a second reading instruction sent by the client, and the second reading instruction comprises the following steps:
receiving a fourth uniform resource locator sent by the client by the domain name system equipment, wherein the fourth uniform resource locator comprises a second reading instruction; sending information of the first bucket to a location server according to the fourth uniform resource locator;
the method comprises the steps that a position server receives information of a first barrel sent by domain name system equipment; sending first indication information to a domain name system according to the second information and the information of the first barrel;
the domain name system receives first indication information sent by a position server; sending first indication information to a client;
the client receives first indication information sent by the domain name system equipment, and sends the second reading instruction to the first KTCPVS according to the first indication information;
and the first KTCPVS receives a second reading instruction sent by the client according to the first indication information.
In this implementation, the client can obtain the instruction information of the KTCPVS corresponding to the first bucket from the location server through the domain name system device, so that a plurality of KTCPVS can be arranged in the cloud storage system, and the load pressure of the KTCPVS can be balanced.
In a second aspect, the present application provides a cloud storage system, where the cloud storage system includes a first server, a first storage node set, a second server, a second storage node set, a first KTCPVS, a location server, and a domain name system device, which are used to implement the object processing method in the first aspect or any one of possible implementation manners of the first aspect.
In a third aspect, the present application provides a computer program product containing instructions. When the computer program product runs in a cloud storage system, the cloud storage system is caused to execute the object processing method in the first aspect or any one of the possible implementation manners of the first aspect.
In a fourth aspect, the present application provides a computer-readable storage medium. The computer readable storage medium has stored therein program code for execution by the cloud storage system. The program code comprises instructions for performing the object processing method of the first aspect or any one of the possible implementations of the first aspect.
Drawings
FIG. 1 is a schematic architecture diagram of an application scenario of one embodiment of the present application;
FIG. 2 is a schematic flow chart diagram of an object processing method 200 according to one embodiment of the present application;
FIG. 3 is a detailed schematic diagram of S210 and S230 in the object processing method 200;
FIG. 4 is a schematic architecture diagram of an application scenario of another embodiment of the present application;
FIG. 5 is a schematic flow chart diagram of steps that may also be included in the object processing method 200;
FIG. 6 is a schematic architecture diagram of an application scenario of another embodiment of the present application;
FIG. 7 is a schematic architecture diagram of an application scenario of another embodiment of the present application;
FIG. 8 is a schematic flow chart diagram of an application scenario of another embodiment of the present application;
fig. 9 is a schematic flow chart of a cloud storage system according to another embodiment of the present application.
Detailed Description
The technical solution in the present application will be described below with reference to the accompanying drawings.
Fig. 1 is a schematic structural diagram of a cloud storage system according to an embodiment of the present application. It should be understood that more or fewer devices may also be included in the cloud storage system of the embodiments of the present application. The apparatus in fig. 1 may be hardware, software functionally divided, or a combination of the two.
The cloud storage system shown in fig. 1 includes a first cluster and a second cluster. The first cluster may be referred to as cluster1 for short and the second cluster may be referred to as cluster2 for short.
The cluster1 includes a physical server and N storage nodes. The storage node may be located in the physical server or may be separate from and in communication with the physical server. The physical server runs a virtual machine, such as a Linux Virtual Server (LVS). The N storage nodes comprise nodes 1 to N, and N is an integer not less than 2. The first server may be in communication with the N storage nodes.
The cluster2 includes a physical server and M storage nodes. The storage node may be located in the physical server or may be separate from and in communication with the physical server. The physical server runs a virtual machine, such as a second server. Wherein M is a positive integer, M storage nodes comprise a node 1 to a node M, and M is an integer not less than 2. The second server may be in communication with the M storage nodes.
The storage node in this embodiment of the present application may be a hard disk, such as a mechanical hard disk, a solid state disk, or an external storage device, such as a storage array, a Direct Access Storage (DAS), and the like, which is not limited in this embodiment of the present application.
It should be understood that the LVS in the embodiment of the present application is only an example, and may be other types of virtual servers.
The physical server in cluster1 may be referred to as a first server, the physical server in cluster2 may be referred to as a second server, the N storage nodes in cluster1 may be referred to as a first set of storage nodes, the M storage nodes in cluster2 may be referred to as a second set of storage nodes,
the first server and the second server may communicate with each other. The first server and the second server may communicate with the client. It should be understood that references herein to mutual communication includes both direct communication and communication through other devices or means.
The region served by the cloud storage system shown in fig. 1 is referred to as region (region) 1. Area 1 is divided into AZ1 and AZ 2. Cluster1 is located in AZ1 and cluster2 is located in AZ 2.
Cluster1 and cluster2 may form two cluster groups, one may be referred to as a first cluster group, abbreviated CG 1; the other may be referred to as a second cluster group, abbreviated CG 2.
In the first cluster group, the replication relationship of the objects is from the first cluster to the second cluster. In the second cluster group, the replication relationship of the objects is from the second cluster to the first cluster.
The object replication relationship from the first cluster to the second cluster means that when the object is written, the first server in the first cluster stores the object into the first storage node set of the first cluster, and sends the object to the second server in the second cluster, so that the second server stores the object into the second storage node set in the second cluster; when reading the object, the first server of the first cluster reads the object from the first storage node set of the first cluster. That is, AZ1 to which the first cluster belongs is a primary AZ, and AZ2 to which the second cluster belongs is a secondary AZ.
The object replication relationship from the second cluster to the first cluster means that when the object is written, the second server in the second cluster stores the object into the second storage node set in the second cluster, and sends the object to the first server in the first cluster, so that the first server stores the object into the first storage node set in the first cluster; when reading the object, the second server of the second cluster reads the object from the second storage node set of the second cluster. That is, AZ2 to which the second cluster belongs is a master AZ, and AZ1 to which the first cluster belongs is a slave AZ.
One schematic flow diagram of an object processing method 200 in the cloud storage system shown in figure 1 is shown in figure 2,
it should be understood that fig. 2 shows steps or operations of the object processing method 200, but these steps or operations are merely examples, and other operations or variations of the operations in fig. 2 may also be performed by the embodiments of the present application. Moreover, the various steps in FIG. 2 may be performed in a different order presented in FIG. 2, and it is possible that not all of the operations in FIG. 2 may be performed.
The object processing method 200 may include S210, S220, S230, and S240.
S210, the first server receives a first write instruction sent by the client, and the first write instruction is used for requesting to write a first object into a first bucket.
This step can be implemented in a number of ways, with possible implementations being described in detail in the following.
S220, the first server writes the first object into a first bucket of the first storage node set according to the first writing instruction, and sends the first object to the second server, so that the second server writes the first object into the first bucket of the second storage node set.
Further, after receiving the first object sent by the first server, the second server may store the first object in the first bucket of the second storage node set.
And S230, the second server receives a second write instruction sent by the client, wherein the second write instruction is used for requesting to write a second object into the first bucket.
This step can be implemented in a number of ways, with possible implementations being described in detail in the following.
The second object is a different object than the first object. For example, the name of the first object is different from the name of the second object.
And S240, the second server writes the second object into the first bucket of the second storage node set according to the second write instruction, and sends the second object to the first server, so that the first server writes the second object into the first bucket of the first storage node set.
Further, after receiving the second object, the first server stores the second object in the first bucket of the first storage node set.
In the object processing method, when the first server and the second server receive a writing instruction requesting to write the object, both the first server and the second server can store the writing instruction and copy the object to the other side. Compared with the prior art, the method reduces the forwarding flow of the object, thereby saving resources.
In addition, after the first server and the second server receive the write instruction, the object may be directly stored, so that when any one cluster of the first server and the second server cannot provide the service and the operation such as the active-standby switching is not performed, the other server may still provide the service, thereby reducing the influence.
From another perspective, the first server stores the first object and copies the first object to the second server, which is equivalent to that a cluster formed by the first server and the first storage node set is a main cluster, and a cluster formed by the second server and the second storage node set is a standby cluster; the second server stores the second object and copies the second object to the first server, which is equivalent to that the cluster formed by the second server and the second storage node set is a main cluster, and the cluster formed by the first server and the first storage node set is a standby cluster.
One possible implementation of S210 and S220 is described below in conjunction with fig. 3. In the case of using this implementation, as shown in fig. 4, the cloud storage system further includes a Kernel Transmission Control Protocol Virtual Server (KTCPVS). For convenience of the following description, this KTCPVS is referred to as a first LTCPVS.
As shown in fig. 3, S210 may include S211 and S212, and S230 may include S231 and S232.
S211, the first KTCPVS receives a first writing instruction sent by the client, determines to send the first writing instruction to the first server according to the first writing instruction, and sends the first writing instruction to the first server.
S212, the first server receives a first writing instruction sent by the first KTCPVS.
That is, the client sends the first write command to the first server through the first KTCPVS. Correspondingly, the first server receives a first writing instruction sent by the client through the first KTCPVS.
And S231, the first KTCPVS receives a second write instruction sent by the client, determines to send the second write instruction to the second server according to the second write instruction, and sends the second write instruction to the second server.
And S232, the second server receives the second writing instruction sent by the first KTCPVS.
That is, the second server receives the second write command sent by the client through the first KTCPVS. Correspondingly, the second server receives a second writing instruction sent by the client through the first KTCPVS.
Optionally, the object processing method 200 may further include S250, S260, S270, and S280 shown in fig. 5.
And S250, the first KTCPVS receives a first read instruction sent by the client, wherein the first read instruction is used for requesting to read a first object in the first barrel, determining to send the first read instruction to the first server according to the first read instruction, and sending the first read instruction to the first server.
S260, the first server receives the first read instruction sent by the first KTCPVS, and reads the first object in the first bucket of the first storage node set according to the first read instruction.
And S270, the first KTCPVS receives a second reading instruction sent by the client, the second reading instruction is used for requesting to read a second object in the first barrel, and the second reading instruction is sent to the second server according to the second reading instruction and is sent to the second server.
And S280, the second server receives a second reading instruction sent by the first KTCPVS, and reads a second object in the first bucket of the second storage node set according to the second reading instruction.
In S260 and S270, the first KTCPVS sends the first read instruction to the first server, so that the strong consistency of the read first object can be maintained.
In S260 and S270, the first KTCPVS sends the second read instruction to the second server, so that the strong consistency of the read second object can be maintained.
In the object processing method 200, the first KTCPVS may store first information, where the first information includes: the first bucket corresponds to a first server and a second server in sequence.
The first bucket corresponds to the first server and the second server in sequence, that is, the first server corresponding to the first bucket is the first server, and the second server corresponding to the first bucket is the second server.
In this case, in S250, one possible implementation manner of determining to send the first read instruction to the first server according to the first read instruction may include:
the first KTCPVS calculates a hash value of the first object relative to the number of the servers corresponding to the first bucket according to the first writing instruction to obtain a first hash value;
and determining to send a first write instruction to a first server corresponding to the first bucket according to the first information and the first hash value.
For example, after the first KTCPVS obtains the name of the first object in the first write instruction, the name of the first object may be quantified. Since the first bucket corresponds to the first server and the second server, the first KTCPVS can calculate a hash value of a numerical value obtained by digitizing the name of the first object with respect to 2.
For example, if the numerical value obtained by digitizing the name of the first object is 143, the hash value 1 can be obtained by modulo 2 with 143.
The first KTCPVS may then determine to send a first write command to a first server (first server) corresponding to the first bucket.
In S260, determining, by the first KTCPVS according to the second write command, that a possible implementation manner of sending the second write command to the second server may include:
the first KTCPVS calculates a hash value of the second object relative to the number of the servers corresponding to the first bucket according to the second write instruction to obtain a second hash value;
and determining to send a second write instruction to a second server corresponding to the second server of the first bucket according to the first information and the second hash value.
For example, after the first KTCPVS obtains the name of the second object in the second write instruction, the name of the second object may be quantified. Since the first bucket corresponds to the first server and the second server, the first KTCPVS can calculate a hash value of a numerical value obtained by digitizing the name of the second object with respect to 2.
For example, if the name of the second object is quantified to obtain a value of 158, then the value of 158 is modulo 2 to obtain a hash value of 2.
In S270, determining, by the first KTCPVS according to the first read instruction, that a possible implementation manner of sending the first read instruction to the first server may include:
the first KTCPVS calculates a hash value of the first object relative to the number of the servers corresponding to the first bucket according to the first reading instruction to obtain a first hash value;
and determining to send a first read instruction to a first server corresponding to the first bucket according to the first information and the first hash value.
The specific implementation is similar to the related content of the first writing instruction, and for the sake of brevity, will not be described herein again.
In S280, determining, by the first KTCPVS according to the second read instruction, that the second read instruction is sent to the second server may include:
the first KTCPVS calculates a hash value of the number of the servers corresponding to the first bucket relative to the second object according to the second reading instruction to obtain a second hash value;
and determining to send a second read instruction to a second corresponding server of the first bucket according to the first information and the second hash value.
The specific implementation is similar to the related content of the first writing instruction, and for the sake of brevity, will not be described herein again.
Alternatively, the first information stored on the first KTCPVS may include first sub information, second sub information, and third sub information.
The first sub-information is: the first barrel sequentially comprises a first sub-barrel and a second sub-barrel; the second sub-information is: the main server corresponding to the first sub-bucket is a first server; the third sub-information is: the main server corresponding to the second sub-bucket is the second server.
Accordingly, a first bucket in the first set of storage nodes includes a first sub-bucket and a second sub-bucket, and a first bucket in the second set of storage nodes also includes the first sub-bucket and the second sub-bucket.
The first information may be understood as: the first bucket is thus divided into a first sub-bucket and a second sub-bucket; the object replication relationship corresponding to the first sub-bucket is from the first cluster to the second cluster, that is, the first cluster and the second cluster form CG 1; the replication relationship corresponding to the second sub-bucket is from the second cluster to the first cluster, i.e. the first cluster and the second cluster form CG 2.
The first bucket is represented by bucket1, the first sub-bucket is represented by bucket1_1, the second sub-bucket is represented by bucket1_2, the first cluster is represented by cluster1, and the second cluster is represented by cluster2, then the first information can be represented as: packet 1{ packet 1_1, packet 1_2}, packet 1_1{ CG 1: cluster1- > cluster2}, packet 1_2{ CG 2: cluster2- > cluster1 }.
In this case, in S250, one possible implementation manner of determining to send the first write instruction to the first server according to the first write instruction may include:
the first KTCPVS calculates a hash value of the first object relative to the number of sub-buckets corresponding to the first bucket according to the first writing instruction to obtain a first hash value; determining to write the first object into a first sub-bucket corresponding to the first sub-bucket according to the first sub-information and the first hash value; and determining to send the first write instruction to the first server according to the second sub information and the first sub bucket.
At this point, the first server writes the first object into a first sub-bucket of the first set of storage nodes. And after receiving the first object sent by the first server, the second server writes the first object into a first sub-bucket of the second storage node set.
In S260, determining, by the first KTCPVS according to the second write command, that a possible implementation manner of sending the second write command to the second server may include:
the first KTCPVS calculates a hash value of the second object relative to the number of the servers corresponding to the first bucket according to the second write instruction to obtain a second hash value; the first KTCPVS determines to write the second object into a second corresponding sub-bucket of the first bucket according to the first sub-information and the second hash value; and determining to send a second write instruction to the second server according to the third sub information and the second sub bucket.
At this point, the second server writes the second object into a second sub-bucket of the second set of storage nodes. And after receiving the second object sent by the second server, the first server writes the second object into a second sub-bucket of the first storage node set.
In S270, determining, by the first KTCPVS according to the first read instruction, that a possible implementation manner of sending the first read instruction to the first server may include:
the first KTCPVS calculates a hash value of the first object relative to the number of the servers corresponding to the first bucket according to the first reading instruction to obtain a first hash value; determining to read a first object in a first sub-bucket corresponding to the first sub-bucket of the first bucket according to the first sub-information and the first hash value; and determining to send the first read instruction to the first server according to the second sub information and the first sub bucket.
At this point, the first server reads the first object in the first sub-bucket of the first set of storage nodes.
In S280, determining, by the first KTCPVS according to the second read instruction, that the second read instruction is sent to the second server may include:
the first KTCPVS calculates a hash value of the number of the servers corresponding to the first bucket relative to the second object according to the second reading instruction to obtain a second hash value; determining to read a second object in a second sub-bucket corresponding to the second sub-bucket of the first bucket according to the first sub-information and the second hash value; and determining to send a second reading instruction to the second server according to the third sub information and the second sub bucket.
At this point, the first server reads the first object in the first sub-bucket of the first set of storage nodes.
Optionally, as shown in fig. 6, a Location Server (LS) may also be included in the cloud storage system. The location server is configured with first information.
In this case, the first information stored on the first KTCPVS may be obtained from a location server.
In other words, in the object processing method 200, before the first KTCPVS uses the first information, the method may include: the first KTCPVS acquires first information from the location server.
Optionally, as shown in fig. 7, the cloud storage system may further include a Domain Name System (DNS) device. In this case, the location server may further be configured with second information, where the second information includes that the KTCPVS corresponding to the first bucket is the first KTCPVS.
Correspondingly, the receiving, by the first KTCPVS, the first write instruction sent by the client may include:
receiving, by a domain name system device, a first Uniform Resource Locator (URL) sent by a client, where the first uniform resource locator includes a first write instruction; sending information of the first bucket to a location server according to the first uniform resource locator;
the method comprises the steps that a position server receives information of a first barrel sent by domain name system equipment; sending first indication information to a domain name system according to the second information and the information of the first barrel, wherein the first information is used for indicating the first KTCPVS;
the domain name system receives first indication information sent by a position server; sending first indication information to a client;
the method comprises the steps that a client receives first indication information sent by domain name system equipment, and sends a first writing instruction to a first KTCPVS according to the first indication information;
and the first KTCPVS receives a first writing instruction sent by the client according to the first indication information.
The receiving, by the first KTCPVS, the second write instruction sent by the client may include:
the domain name system equipment receives a second uniform resource locator sent by the client, wherein the second uniform resource locator comprises a second writing instruction; sending information of the first bucket to a location server according to the second uniform resource locator;
the method comprises the steps that a position server receives information of a first barrel sent by domain name system equipment; sending first indication information to a domain name system according to the second information and the information of the first barrel;
the domain name system receives first indication information sent by a position server; sending first indication information to a client;
the client receives first indication information sent by the domain name system equipment, and sends a second writing instruction to the first KTCPVS according to the first indication information
And the first KTCPVS receives a second writing instruction sent by the client according to the first indication information.
The receiving, by the first KTCPVS, the first read instruction sent by the client may include:
receiving a third uniform resource locator sent by a client by domain name system equipment, wherein the third uniform resource locator comprises a first reading instruction; sending information of the first bucket to a location server according to the third uniform resource locator;
the method comprises the steps that a position server receives information of a first barrel sent by domain name system equipment; sending first indication information to a domain name system according to the second information and the information of the first barrel;
the domain name system receives first indication information sent by a position server; sending first indication information to a client;
the client receives first indication information sent by the domain name system equipment, and sends a first read instruction to the first KTCPVS according to the first indication information
And the first KTCPVS receives a first reading instruction sent by the client according to the first indication information.
The first KTCPVS receiving the second read instruction sent by the client may include:
receiving a fourth uniform resource locator sent by the client by the domain name system equipment, wherein the fourth uniform resource locator comprises a second reading instruction; sending information of the first bucket to a location server according to the fourth uniform resource locator;
the method comprises the steps that a position server receives information of a first barrel sent by domain name system equipment; sending first indication information to a domain name system according to the second information and the information of the first barrel;
the domain name system receives first indication information sent by a position server; sending first indication information to a client;
the client receives first indication information sent by the domain name system equipment, and sends the second reading instruction to the first KTCPVS according to the first indication information;
and the first KTCPVS receives a second reading instruction sent by the client according to the first indication information.
In this implementation, the client can obtain the instruction information of the KTCPVS corresponding to the first bucket from the location server through the domain name system device, so that a plurality of KTCPVS can be arranged in the cloud storage system, and the load pressure of the KTCPVS can be balanced.
In the cloud storage system according to the embodiment of the application, optionally, more clusters may be deployed in each AZ, the clusters may form more cluster groups, and the buckets in the clusters may be divided into more sub-buckets, which correspond to the cluster groups.
For example, the cloud storage system shown in fig. 8, wherein two clusters, cluster1 and cluster 3 respectively, are deployed in AZ 1; two clusters, cluster2 and cluster 4, are deployed in AZ 2.
The cluster 3 comprises a physical server and S storage nodes, and the cluster 4 comprises a physical server and T storage nodes. S and T are positive integers.
The physical server in the cluster 3 may be referred to as a third physical server, and the S storage nodes may be referred to as a third storage node set; the physical server in the cluster 4 may be referred to as a fourth physical server and the T storage nodes may be referred to as a fourth set of storage nodes.
A bucket in the cloud storage system may be divided into 4 sub-buckets, which are sub-bucket 1, sub-bucket 2, sub-bucket 3, and sub-bucket 4. Cluster1, cluster2, cluster 3, and cluster 4 may be divided into 4 logical cluster groups, cluster group 1, cluster group 2, cluster group 3, and cluster group 4, respectively.
The cluster group 1 comprises a cluster1 and a cluster2, and the replication relationship of the object is from the cluster1 to the cluster 2; the cluster group 2 comprises a cluster2 and a cluster1, and the replication relationship of the object is from the cluster2 to the cluster 1; the cluster group 3 comprises a cluster 3 and a cluster 4, and the replication relationship of the object is from the cluster 3 to the cluster 4; the cluster group 4 includes a cluster 4 and a cluster 3, and the replication relationship of the object is from the cluster 4 to the cluster 3.
Sub-bucket 1 maps to cluster group 1, sub-bucket 2 maps to cluster group 2, sub-bucket 3 maps to cluster group 3, and sub-bucket 4 maps to cluster group 4.
Let bucket1 denote the first bucket, bucket1_1 denote the first sub-bucket, bucket1_2 denote the second sub-bucket, bucket1_3 denote the third sub-bucket, bucket1_4 denote the fourth sub-bucket, cluster1 denotes the first cluster, cluster2 denotes the second cluster, cluster1 denotes the third cluster, cluster2 denotes the fourth cluster, CG3 denotes cluster 3, CG4 denotes cluster group 4, then the first information can be expressed as:
bucket1{bucket1_1,bucket1_2,bucket1_3,bucket1_4},bucket1_1{CG1:cluster1->cluster2},bucket1_2{CG2:cluster2->cluster1},bucket1_3{CG3:cluster3->cluster4},bucket1_4{CG4:cluster4->cluster3}。
if the target cluster group determined by the LS as the object is the cluster group 1, the cluster1 processes the object and copies the processing result to the cluster 2; if the target cluster group determined by the LS as the object is the cluster group 2, the cluster group 2 processes the object and copies the processing result to the cluster group 1; if the target cluster group determined by the LS as the object is the cluster group 3, processing the cluster 3 object, and copying a processing result to the cluster 4; if the target cluster group determined by the LS for the object is the cluster group 4, the cluster group 4 processes the object and copies the processing result to the cluster group 3.
It should be understood that the number of sub-buckets, the number of clusters, and the number of cluster groups are all examples, and the cloud storage system according to the embodiment of the present application may further include more clusters, more clusters may be divided into more cluster groups, and a bucket may also be divided into more sub-buckets.
The more clusters in the cloud storage system, the more cluster groups can be divided, and the more clusters can bear loads. This may allow more flexibility in balancing load pressure across the various clusters. In addition, if a cluster fails and cannot provide services normally, the number of affected users can be reduced.
In the case that the cloud storage system includes more cluster groups, the object reading or writing method may refer to the object processing method shown in fig. 1 to 8, and for brevity, details are not described here again.
A schematic architecture diagram of a cloud storage system 900 according to an embodiment of the present application is shown in fig. 9. It should be understood that the cloud storage system 900 shown in fig. 9 is only an example, and the cloud storage system of the embodiment of the present application may further include other modules or units, or include devices having functions similar to those of the respective devices in fig. 9, or not include all the devices in fig. 9.
The cloud storage system 900 comprises a first server, a first storage node set, a second server and a second storage node set, wherein the first server and the first storage node set are located in a first service area of the cloud storage system, the second server and the second storage node are located in a second service area of the cloud storage system, and the first storage node set and the second storage node set both comprise a first bucket.
Wherein the first server is configured to: receiving a first write instruction sent by a client, wherein the first write instruction is used for requesting to write a first object into the first bucket; according to a first write instruction, writing the first object into the first bucket of the first storage node set, and sending the first object to the second server, so that the second server writes the first object into the first bucket of the second storage node set.
The second server is configured to: receiving a second write instruction sent by the client, where the second write instruction is used to request to write a second object into the first bucket, and according to the second write instruction, writing the second object into the first bucket of the second storage node set, and sending the second object to the first server, so that the first server writes the second object into the first bucket of the first storage node set.
Optionally, the cloud storage system 900 may further include a first KTCPVS.
The first KTCPVS is used for: receiving the first writing instruction sent by the client; determining to send the first write instruction to the first server according to the first write instruction; sending the first write instruction to the first server; receiving the second writing instruction sent by the client; determining to send the second write instruction to the second server according to the second write instruction; and sending the second writing instruction to the second server.
The first server is specifically configured to: and receiving the first writing instruction sent by the client through the first KTCPVS.
The second server is specifically configured to: and receiving the second write instruction sent by the client through the first KTCPVS.
Optionally, the first KTCPVS is further configured to: receiving a first read instruction sent by the client, wherein the first read instruction is used for requesting to read the first object in the first bucket; determining to send the first read instruction to the first server according to the first read instruction; sending the first read instruction to the first server; receiving a second reading instruction sent by the client, wherein the second reading instruction is used for requesting to read the second object in the first bucket; determining to send a second reading instruction to the second server according to the second reading instruction; and sending the second reading instruction to the second server.
The first server is further configured to: receiving the first read instruction sent by the first KTCPVS; reading the first object in the first bucket of the first set of storage nodes according to the first read instruction.
The second server is further configured to: receiving the second reading instruction sent by the first KTCPVS; reading the second object in the first bucket of the second set of storage nodes according to the second read instruction.
Optionally, there is first information on the first KTCPVS, where the first information: the first bucket corresponds to the first server and the second server in sequence.
First KTCPVS specifically is used for: according to the first writing instruction, calculating hash values of the first object relative to the number of servers mapped by the first bucket to obtain first hash values; and determining to send the first write instruction to the first server corresponding to the first bucket according to the first information and the first hash value.
First KTCPVS specifically is used for: according to the second writing instruction, calculating hash values of the second object relative to the number of servers mapped by the first bucket to obtain second hash values; and determining to send the second write instruction to the second server corresponding to the second one of the first bucket and the second bucket according to the first information and the second hash value.
First KTCPVS specifically is used for: according to the first reading instruction, calculating hash values of the first object relative to the number of servers mapped by the first bucket to obtain first hash values; and determining to send the first read instruction to the first server corresponding to the first bucket according to the first information and the first hash value.
First KTCPVS specifically is used for: according to the second reading instruction, calculating hash values of the second object relative to the number of servers mapped by the first barrel to obtain second hash values; and determining to send the second read instruction to the second server corresponding to the second one of the first bucket and the second bucket according to the first information and the second hash value.
Optionally, the first information includes first sub information, second sub information, and third sub information, where the first sub information is: the first barrel comprises a first sub-barrel and a second sub-barrel in sequence, and the second sub-information is as follows: the main server corresponding to the first sub-bucket is the first server, and the third sub-information is: and the main server corresponding to the second sub-bucket is the second server.
First KTCPVS specifically is used for: according to the first writing instruction, calculating a hash value of the first object relative to the number of sub-buckets corresponding to the first bucket to obtain a first hash value; according to the first sub information and the first hash value, determining to write the first object into the first sub bucket corresponding to the first sub bucket; and determining to send the first write instruction to the first server according to the second sub-information and the first sub-bucket.
First KTCPVS specifically is used for: according to the second writing instruction, calculating a hash value of the number of sub-buckets corresponding to the first bucket of the second object to obtain a second hash value; determining to write the second object into a second sub-bucket corresponding to the second sub-bucket of the first bucket according to the first sub-information and a second hash value; and determining to send the second write instruction to the second server according to the third sub information and the second sub bucket.
First KTCPVS specifically is used for: according to the first reading instruction, calculating a hash value of the first object relative to the number of sub-buckets corresponding to the first bucket to obtain a first hash value; determining to read the first object in the first sub-bucket corresponding to the first sub-bucket first according to the first sub-information and the first hash value; and determining to send the first read instruction to the first server according to the second sub-information and the first sub-bucket.
First KTCPVS specifically is used for: according to the second reading instruction, calculating a hash value of the second object relative to the number of sub-buckets corresponding to the first bucket to obtain a second hash value; determining to read the second object in the second sub-bucket corresponding to the second sub-bucket of the first bucket according to the first sub-information and the second hash value; and determining to send the second reading instruction to the second server according to the third sub information and the second sub bucket.
Optionally, the cloud storage system 900 may further include a location server, and the first information is configured on the location server.
Correspondingly, the first KTCPVS is further configured to: the first information is obtained from the location server.
Optionally, the cloud storage system 900 is configured with a domain name system device, and correspondingly, the location server is further configured with second information, where the second information includes: the KTCPVS that first bucket corresponds does first KTCPVS.
The domain name system device is configured to: receiving a first uniform resource locator sent by the client, wherein the first uniform resource locator comprises the first writing instruction; sending information of the first bucket to the location server according to the first uniform resource locator; receiving a second uniform resource locator sent by the client, wherein the second uniform resource locator comprises the second write instruction; sending information of the first bucket to the location server according to the second uniform resource locator; receiving a third uniform resource locator sent by the client, wherein the third uniform resource locator comprises the first reading instruction; sending information of the first bucket to the location server according to the third uniform resource locator; receiving a fourth uniform resource locator sent by the client, wherein the fourth uniform resource locator comprises the second reading instruction; sending information of the first bucket to the location server according to the fourth uniform resource locator.
The location server is configured to: receiving information of the first bucket sent by the domain name system device; and sending first indication information to the domain name system according to the second information and the information of the first barrel, wherein the first information is used for indicating the first KTCPVS.
The domain name system is further configured to: receiving the first indication information sent by the location server; and sending the first indication information by the client, so that the client can send the first writing instruction to the client according to the first indication information, so that the client can send the second writing instruction to the client according to the first indication information, and the client can send the first reading instruction to the first KTCPVS according to the first indication information, so that the client can send the second reading instruction to the first KTCPVS according to the first indication information.
Each device in the cloud storage system shown in fig. 9 may be configured to execute the object processing method 200 shown in fig. 2, and details are not repeated here.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (14)
1. An object processing method in a cloud storage system is characterized in that the cloud storage system comprises a first server, a first storage node set, a second server and a second storage node set, the first server and the first storage node set are located in a first service area of the cloud storage system, the second server and the second storage node set are located in a second service area of the cloud storage system, and the first storage node set and the second storage node set both comprise a first bucket;
the object processing method comprises the following steps:
the first server receives a first write instruction sent by a client, wherein the first write instruction is used for requesting to write a first object into the first bucket;
the first server writes the first object into the first bucket of the first storage node set according to a first write instruction, and sends the first object to the second server so that the second server writes the first object into the first bucket of the second storage node set;
the second server receives a second write instruction sent by the client, wherein the second write instruction is used for requesting to write a second object into the first bucket;
the second server writes the second object into the first bucket of the second storage node set according to the second write instruction, and sends the second object to the first server, so that the first server writes the second object into the first bucket of the first storage node set.
2. The object processing method according to claim 1, wherein the cloud storage system further comprises a first kernel transfer control protocol virtual server KTCPVS;
the first server receives a first writing instruction sent by a client, and the first writing instruction comprises the following steps:
the first KTCPVS receives the first writing instruction sent by the client, determines to send the first writing instruction to the first server according to the first writing instruction, and sends the first writing instruction to the first server;
the first server receives the first writing instruction sent by the first KTCPVS;
the second server receives a second write instruction sent by the client, and the second write instruction comprises:
the first KTCPVS receives the second writing instruction sent by the client, determines to send the second writing instruction to the second server according to the second writing instruction, and sends the second writing instruction to the second server;
and the second server receives the second writing instruction sent by the first KTCPVS.
3. The object processing method according to claim 2, characterized in that the object processing method further comprises:
the first KTCPVS receives a first read instruction sent by the client, wherein the first read instruction is used for requesting to read the first object in the first bucket, determining to send the first read instruction to the first server according to the first read instruction, and sending the first read instruction to the first server;
the first server receives the first read instruction sent by the first KTCPVS, and reads the first object in the first bucket of the first storage node set according to the first read instruction;
the first KTCPVS receives a second reading instruction sent by the client, wherein the second reading instruction is used for requesting to read the second object in the first barrel, determining to send the second reading instruction to the second server according to the second reading instruction, and sending the second reading instruction to the second server;
and the second server receives the second reading instruction sent by the first KTCPVS, and reads the second object in the first bucket of the second storage node set according to the second reading instruction.
4. The object processing method of claim 3, wherein said first KTCPVS has first information thereon, said first information comprising: the first bucket corresponds to the first server and the second server in sequence;
the first KTCPVS determining, according to the first write command, to send the first write command to the first server, including:
the first KTCPVS calculates a first hash value of the number of the servers corresponding to the first object relative to the first bucket according to the first write instruction; according to the first information and the first hash value, determining to send the first write instruction to the first server corresponding to the first bucket;
the first KTCPVS determining to send the second write command to the second server according to the second write command, including:
the first KTCPVS calculates a second hash value of the number of the servers corresponding to the second object relative to the first bucket according to the second write instruction; according to the first information and the second hash value, determining to send the second write instruction to the second server corresponding to the second one of the first bucket;
the first KTCPVS determining to send the first read command to the first server according to the first read command, including:
the first KTCPVS calculates the first hash value of the number of the first object relative to the servers corresponding to the first bucket according to the first read instruction; determining to send the first read instruction to the first server corresponding to the first bucket first according to the first information and the first hash value;
the first KTCPVS determines to send the second read instruction to the second server according to the second read instruction, and includes:
the first KTCPVS calculates a second hash value of the number of the servers corresponding to the second object relative to the first bucket according to the second read instruction; and determining to send the second read instruction to the second server corresponding to the second one of the first bucket and the second bucket according to the first information and the second hash value.
5. The object processing method according to claim 4, wherein the first information includes first sub information, second sub information, and third sub information, and the first sub information is: the first barrel comprises a first sub-barrel and a second sub-barrel in sequence, and the second sub-information is as follows: the main server corresponding to the first sub-bucket is the first server, and the third sub-information is: the main server corresponding to the second sub-bucket is the second server;
the first KTCPVS determining, according to the first information and the first hash value, to send the first write command to the first server corresponding to the first bucket, includes:
the first KTCPVS determines to write the first object into the first sub-bucket corresponding to the first sub-bucket according to the first sub-information and the first hash value; determining to send the first write instruction to the first server according to the second sub-information and the first sub-bucket;
the determining, by the first KTCPVS, to send the second write command to the second server corresponding to the second one of the first bucket according to the first information and the second hash value includes:
the first KTCPVS determines to write the second object into the second sub-bucket corresponding to the second one of the first sub-bucket according to the first sub-information and the second hash value; determining to send the second write instruction to the second server according to the third sub information and the second sub bucket;
the first KTCPVS determines, according to the first information and the first hash value, to send the first read instruction to the first server corresponding to the first bucket, where the sending of the first read instruction includes:
the first KTCPVS determines to read the first object in the first sub-bucket corresponding to the first sub-bucket first according to the first sub-information and the first hash value; determining to send the first read instruction to the first server according to the second sub-information and the first sub-bucket;
the determining, by the first KTCPVS, that the second read instruction is sent to the second server corresponding to the second one of the first buckets according to the first information and the second hash value includes:
the first KTCPVS determines to read the second object in the second sub-bucket corresponding to the second sub-bucket of the first bucket according to the first sub-information and the second hash value; and determining to send the second reading instruction to the second server according to the third sub information and the second sub bucket.
6. The object processing method according to claim 4 or 5, wherein the cloud storage system further comprises a location server on which the first information is configured;
the object processing method further includes:
and the first KTCPVS acquires the first information from the position server.
7. The object processing method of claim 6, wherein second information is further configured on the location server, and the second information includes that the KTCPVS corresponding to the first bucket is the first KTCPVS; the cloud storage system further comprises domain name system equipment;
the first writing instruction that first KTCPVS received the client side sends includes:
the domain name system equipment receives a first uniform resource locator sent by the client, wherein the first uniform resource locator comprises the first writing instruction; sending identification information of the first bucket to the location server according to the first uniform resource locator;
the location server receives the identification information of the first bucket sent by the domain name system device; sending first indication information to the domain name system according to the second information and the identification information of the first bucket, wherein the first indication information is used for indicating the first KTCPVS;
the domain name system receives the first indication information sent by the location server; sending the first indication information to the client;
the client receives the first indication information sent by the domain name system device, and sends the first writing instruction to the first KTCPVS according to the first indication information;
the first KTCPVS receives the first writing instruction sent by the client according to the first indication information;
the first KTCPVS receives a second write instruction sent by the client, and the method comprises the following steps:
the domain name system equipment receives a second uniform resource locator sent by the client, wherein the second uniform resource locator comprises the second writing instruction; sending identification information of the first bucket to the location server according to the second uniform resource locator;
the location server receives the identification information of the first bucket sent by the domain name system device; sending the first indication information to the domain name system according to the second information and the identification information of the first bucket;
the domain name system receives the first indication information sent by the location server; sending the first indication information to the client;
the client receives the first indication information sent by the domain name system device, and sends the second write instruction to the first KTCPVS according to the first indication information;
the first KTCPVS receives the second writing instruction sent by the client according to the first indication information;
the first KTCPVS receives a first reading instruction sent by the client, and the method comprises the following steps:
the domain name system equipment receives a third uniform resource locator sent by the client, wherein the third uniform resource locator comprises the first reading instruction; sending identification information of the first bucket to the location server according to the third uniform resource locator;
the location server receives the identification information of the first bucket sent by the domain name system device; sending the first indication information to the domain name system according to the second information and the information of the first bucket;
the domain name system receives the first indication information sent by the location server; sending the first indication information to the client;
the client receives the first indication information sent by the domain name system device, and sends the first reading instruction to the first KTCPVS according to the first indication information;
the first KTCPVS receives the first reading instruction sent by the client according to the first indication information;
the first KTCPVS receives a second reading instruction sent by the client, and the method comprises the following steps:
the domain name system equipment receives a fourth uniform resource locator sent by the client, wherein the fourth uniform resource locator comprises the second reading instruction; sending identification information of the first bucket to the location server according to the fourth uniform resource locator;
the location server receives the identification information of the first bucket sent by the domain name system device; sending first indication information to the domain name system according to the second information and the information of the first bucket;
the domain name system receives the first indication information sent by the location server; sending the first indication information to the client;
the client receives the first indication information sent by the domain name system device, and sends the second reading instruction to the first KTCPVS according to the first indication information;
and the first KTCPVS receives the second reading instruction sent by the client according to the first indication information.
8. A cloud storage system is characterized in that the cloud storage system comprises a first server, a first storage node set, a second server and a second storage node set, the first server and the first storage node set are located in a first service area of the cloud storage system, the second server and the second storage node set are located in a second service area of the cloud storage system, and the first storage node set and the second storage node set both comprise a first bucket;
the first server is configured to: receiving a first write instruction sent by a client, wherein the first write instruction is used for requesting to write a first object into the first bucket; according to a first writing instruction, writing the first object into the first bucket of the first storage node set, and sending the first object to the second server so that the second server can write the first object into the first bucket of the second storage node set;
the second server is configured to: receiving a second write instruction sent by the client, where the second write instruction is used to request to write a second object into the first bucket, and according to the second write instruction, writing the second object into the first bucket of the second storage node set, and sending the second object to the first server, so that the first server writes the second object into the first bucket of the first storage node set.
9. The cloud storage system of claim 8, wherein said cloud storage system further comprises a first Kernel Transmission Control Protocol Virtual Server (KTCPVS);
the first KTCPVS is used for: receiving the first writing instruction sent by the client; determining to send the first write instruction to the first server according to the first write instruction; sending the first write instruction to the first server; receiving the second writing instruction sent by the client; determining to send the second write instruction to the second server according to the second write instruction; sending the second write instruction to the second server;
the first server is specifically configured to: receiving the first writing instruction sent by the client through the first KTCPVS;
the second server is specifically configured to: and receiving the second write instruction sent by the client through the first KTCPVS.
10. The cloud storage system of claim 9, wherein said first KTCPVS is further configured to: receiving a first read instruction sent by the client, wherein the first read instruction is used for requesting to read the first object in the first bucket; determining to send the first read instruction to the first server according to the first read instruction; sending the first read instruction to the first server; receiving a second reading instruction sent by the client, wherein the second reading instruction is used for requesting to read the second object in the first bucket; determining to send a second reading instruction to the second server according to the second reading instruction; sending the second reading instruction to the second server;
the first server is further configured to: receiving the first read instruction sent by the first KTCPVS; reading the first object in the first bucket of the first set of storage nodes according to the first read instruction;
the second server is further configured to: receiving the second reading instruction sent by the first KTCPVS; reading the second object in the first bucket of the second set of storage nodes according to the second read instruction.
11. The cloud storage system of claim 10, wherein said first KTCPVS has first information thereon, said first information comprising: the first bucket corresponds to the first server and the second server in sequence;
first KTCPVS specifically is used for: calculating a first hash value of the first object relative to the number of servers mapped by the first bucket according to the first write instruction; according to the first information and the first hash value, determining to send the first write instruction to the first server corresponding to the first bucket;
first KTCPVS specifically is used for: calculating a second hash value of the second object relative to the number of servers mapped by the first bucket according to the second write instruction; according to the first information and the second hash value, determining to send the second write instruction to the second server corresponding to the second one of the first bucket;
first KTCPVS specifically is used for: calculating the first hash value of the first object relative to the number of servers mapped by the first bucket according to the first read instruction; determining to send the first read instruction to the first server corresponding to the first bucket first according to the first information and the first hash value;
first KTCPVS specifically is used for: calculating the second hash value of the second object relative to the number of servers mapped by the first bucket according to the second read instruction; and determining to send the second read instruction to the second server corresponding to the second one of the first bucket and the second bucket according to the first information and the second hash value.
12. The cloud storage system of claim 11, wherein the first information comprises first sub information, second sub information, and third sub information, and the first sub information is: the first barrel comprises a first sub-barrel and a second sub-barrel in sequence, and the second sub-information is as follows: the main server corresponding to the first sub-bucket is the first server, and the third sub-information is: the main server corresponding to the second sub-bucket is the second server;
first KTCPVS specifically is used for: according to the first writing instruction, calculating the first hash value of the first object relative to the number of sub-buckets corresponding to the first bucket; according to the first sub-information and the first hash value, determining to write the first object into the first sub-bucket corresponding to the first bucket; determining to send the first write instruction to the first server according to the second sub-information and the first sub-bucket;
first KTCPVS specifically is used for: according to the second writing instruction, calculating the second hash value of the number of sub-buckets corresponding to the second object relative to the first bucket; determining to write the second object into the second sub-bucket corresponding to the second one of the first sub-bucket according to the first sub-information and the second hash value; determining to send the second write instruction to the second server according to the third sub information and the second sub bucket;
first KTCPVS specifically is used for: calculating the first hash value of the first object relative to the number of sub-buckets corresponding to the first bucket according to the first read instruction; determining to read the first object in the first sub-bucket corresponding to the first sub-bucket first according to the first sub-information and the first hash value; determining to send the first read instruction to the first server according to the second sub-information and the first sub-bucket;
first KTCPVS specifically is used for: calculating the second hash value of the second object relative to the number of sub-buckets corresponding to the first bucket according to the second read instruction; determining to read the second object in the second sub-bucket corresponding to the second sub-bucket of the first bucket according to the first sub-information and the second hash value; and determining to send the second reading instruction to the second server according to the third sub information and the second sub bucket.
13. The cloud storage system according to claim 11 or 12, wherein the cloud storage server further comprises a location server on which the first information is configured;
the first KTCPVS is also used for: the first information is obtained from the location server.
14. The cloud storage system of claim 13, wherein the location server is further configured with second information, and the second information comprises: the KTCPVS corresponding to the first barrel is the first KTCPVS; the cloud storage system further comprises domain name system equipment;
the domain name system device is configured to: receiving a first uniform resource locator sent by the client, wherein the first uniform resource locator comprises the first writing instruction; sending identification information of the first bucket to the location server according to the first uniform resource locator; receiving a second uniform resource locator sent by the client, wherein the second uniform resource locator comprises the second write instruction; sending identification information of the first bucket to the location server according to the second uniform resource locator; receiving a third uniform resource locator sent by the client, wherein the third uniform resource locator comprises the first reading instruction; sending identification information of the first bucket to the location server according to the third uniform resource locator; receiving a fourth uniform resource locator sent by the client, wherein the fourth uniform resource locator comprises the second reading instruction; sending identification information of the first bucket to the location server according to the fourth uniform resource locator;
the location server is configured to: receiving the identification information of the first bucket sent by the domain name system device; sending first indication information to the domain name system according to the second information and the identification information of the first barrel, wherein the first information is used for indicating the first KTCPVS;
the domain name system is further configured to: receiving the first indication information sent by the location server; and sending the first indication information by the client, so that the client can send the first writing instruction to the client according to the first indication information, so that the client can send the second writing instruction to the client according to the first indication information, and the client can send the first reading instruction to the first KTCPVS according to the first indication information, so that the client can send the second reading instruction to the first KTCPVS according to the first indication information.
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710896038.5A CN109587185B (en) | 2017-09-28 | 2017-09-28 | Cloud storage system and object processing method in cloud storage system |
CN202011127481.4A CN112422630A (en) | 2017-09-28 | 2017-09-28 | Cloud storage system and object processing method in cloud storage system |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710896038.5A CN109587185B (en) | 2017-09-28 | 2017-09-28 | Cloud storage system and object processing method in cloud storage system |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202011127481.4A Division CN112422630A (en) | 2017-09-28 | 2017-09-28 | Cloud storage system and object processing method in cloud storage system |
Publications (2)
Publication Number | Publication Date |
---|---|
CN109587185A CN109587185A (en) | 2019-04-05 |
CN109587185B true CN109587185B (en) | 2020-11-03 |
Family
ID=65912961
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201710896038.5A Active CN109587185B (en) | 2017-09-28 | 2017-09-28 | Cloud storage system and object processing method in cloud storage system |
CN202011127481.4A Pending CN112422630A (en) | 2017-09-28 | 2017-09-28 | Cloud storage system and object processing method in cloud storage system |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202011127481.4A Pending CN112422630A (en) | 2017-09-28 | 2017-09-28 | Cloud storage system and object processing method in cloud storage system |
Country Status (1)
Country | Link |
---|---|
CN (2) | CN109587185B (en) |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111309522A (en) * | 2020-02-14 | 2020-06-19 | 星辰天合(北京)数据科技有限公司 | Object storage data protection method and device based on snapshot technology |
CN114327249A (en) * | 2020-10-10 | 2022-04-12 | 华为云计算技术有限公司 | Data modification method of object storage service, service node, client and equipment |
CN114448781B (en) * | 2021-12-22 | 2024-06-07 | 天翼云科技有限公司 | Data processing system |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102880715A (en) * | 2012-10-09 | 2013-01-16 | 南京市测绘勘察研究院有限公司 | Data pool management method and data pool management system on basis of cloud storage |
CN103067525A (en) * | 2013-01-18 | 2013-04-24 | 广东工业大学 | Cloud storage data backup method based on characteristic codes |
CN103226483A (en) * | 2013-03-25 | 2013-07-31 | 成都瑞科电气有限公司 | DHBS (dual hot-backup system) and method thereof based on SOA (service-oriented architecture) and cloud storage |
CN103763383A (en) * | 2014-01-27 | 2014-04-30 | 西安雷迪维护系统设备有限公司 | Integrated cloud storage system and storage method thereof |
Family Cites Families (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR101638436B1 (en) * | 2010-12-10 | 2016-07-12 | 한국전자통신연구원 | Cloud storage and management method thereof |
CN104202375B (en) * | 2014-08-22 | 2017-10-24 | 广州华多网络科技有限公司 | The method and system of synchrodata |
-
2017
- 2017-09-28 CN CN201710896038.5A patent/CN109587185B/en active Active
- 2017-09-28 CN CN202011127481.4A patent/CN112422630A/en active Pending
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102880715A (en) * | 2012-10-09 | 2013-01-16 | 南京市测绘勘察研究院有限公司 | Data pool management method and data pool management system on basis of cloud storage |
CN103067525A (en) * | 2013-01-18 | 2013-04-24 | 广东工业大学 | Cloud storage data backup method based on characteristic codes |
CN103226483A (en) * | 2013-03-25 | 2013-07-31 | 成都瑞科电气有限公司 | DHBS (dual hot-backup system) and method thereof based on SOA (service-oriented architecture) and cloud storage |
CN103763383A (en) * | 2014-01-27 | 2014-04-30 | 西安雷迪维护系统设备有限公司 | Integrated cloud storage system and storage method thereof |
Also Published As
Publication number | Publication date |
---|---|
CN109587185A (en) | 2019-04-05 |
CN112422630A (en) | 2021-02-26 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN109949111B (en) | Electronic bill identification distribution method, electronic bill generation method, device and system | |
US11249834B2 (en) | Storage system with coordinated recovery across multiple input-output journals of different types | |
RU2501072C2 (en) | Distributed storage of recoverable data | |
CN102594858B (en) | Mirror solution in cloud storage environment | |
CN109995881B (en) | Load balancing method and device of cache server | |
CN109408590B (en) | Method, device and equipment for expanding distributed database and storage medium | |
US20120167088A1 (en) | System for moving individual virtual machine among cloud computing environment and method thereof | |
KR20120018178A (en) | Swarm-based synchronization over a network of object stores | |
JP2010074604A (en) | Data access system, data access method and data access program | |
CN109587185B (en) | Cloud storage system and object processing method in cloud storage system | |
CN110321225B (en) | Load balancing method, metadata server and computer readable storage medium | |
CN104142871B (en) | Data backup method and device and distributed file system | |
US11099767B2 (en) | Storage system with throughput-based timing of synchronous replication recovery | |
CN106855834B (en) | Data backup method, device and system | |
WO2015109483A1 (en) | Data storage method and device | |
CN109597903A (en) | Image file processing apparatus and method, document storage system and storage medium | |
KR101601877B1 (en) | Apparatus and method for client's participating in data storage of distributed file system | |
US10956442B1 (en) | Dedicated source volume pool for accelerated creation of block data volumes from object data snapshots | |
US20230006925A1 (en) | Systems for Session-Based Routing | |
CN108932149B (en) | Data transmission method and device | |
EP4116815A1 (en) | Storage system, data processing method and apparatus, node, and storage medium | |
US11360712B2 (en) | Storage system with continuous data verification for synchronous replication of logical storage volumes | |
KR102697183B1 (en) | Apparatus for managing virtual network function and method for the same | |
JP5690296B2 (en) | Load balancing program and load balancing apparatus | |
JP2019159912A (en) | Image transfer system, server apparatus, client terminal, image transfer method, and program |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |