Nothing Special   »   [go: up one dir, main page]

CN106909557B - Memory cluster storage method and device and memory cluster reading method and device - Google Patents

Memory cluster storage method and device and memory cluster reading method and device Download PDF

Info

Publication number
CN106909557B
CN106909557B CN201510976655.7A CN201510976655A CN106909557B CN 106909557 B CN106909557 B CN 106909557B CN 201510976655 A CN201510976655 A CN 201510976655A CN 106909557 B CN106909557 B CN 106909557B
Authority
CN
China
Prior art keywords
original
key
hash ring
hash
primary key
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201510976655.7A
Other languages
Chinese (zh)
Other versions
CN106909557A (en
Inventor
杨维
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tianyi Cloud Technology Co Ltd
Original Assignee
China Telecom Corp Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Telecom Corp Ltd filed Critical China Telecom Corp Ltd
Priority to CN201510976655.7A priority Critical patent/CN106909557B/en
Publication of CN106909557A publication Critical patent/CN106909557A/en
Application granted granted Critical
Publication of CN106909557B publication Critical patent/CN106909557B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a data storage method and device of a memory cluster and a data reading method and device of the memory cluster, and relates to the field of databases. The data storage method of the memory cluster comprises the following steps: constructing an original hash ring of each node based on consistent hash operation; grouping all nodes in the original hash ring according to an adjacent relation, so that the adjacent nodes have the same server number to construct a logical hash ring; storing each node to a corresponding server according to the logical hash ring; and establishing the mapping relation of each node between the original hash ring and the logical hash ring. Therefore, adjacent nodes are stored in the same server, data moving and recovery in the memory space are more efficient, and the service unavailability time caused by fault recovery is reduced.

Description

Memory cluster storage method and device and memory cluster reading method and device
Technical Field
The present invention relates to the field of databases, and in particular, to a data storage method and apparatus for a memory cluster, and a reading method and apparatus for a memory cluster.
Background
The consistent hash algorithm provides four definitions for judging the quality of the hash algorithm in a dynamically changing cache environment:
balance: the balance means that the result of the hash can be distributed into all buffers as much as possible, so that all buffer spaces can be utilized. Many hash algorithms are able to satisfy this condition.
Monotonicity: monotonicity means that if some content is already assigned to the corresponding buffer by hashing, a new buffer is added to the system. The result of the hash should be able to ensure that the original allocated content can be mapped to the original or new buffer and not to other buffers in the old buffer set.
Dispersibility: in a distributed environment, the terminal may not see all of the buffers, but only a portion thereof. When a terminal wishes to map content onto a buffer through a hashing process, the result of hashing is inconsistent because the buffer ranges seen by different terminals may be different, and the end result is that the same content is mapped into different buffers by different terminals. This situation is obviously to be avoided because it results in the same content being stored in different buffers, reducing the efficiency of the system storage. The definition of dispersibility is the severity of the occurrence of the above. A good hash algorithm should be able to avoid as much as possible the occurrence of inconsistencies, i.e. to reduce the scatter as much as possible.
Loading: the loading problem is actually a dispersion problem from another perspective. Since different terminals may map the same content to different buffers, it is also possible for a particular buffer to be mapped to different content by different users. As well as scatter, this should be avoided, so a good hashing algorithm should minimize the buffering load.
If a single physical machine is taken as an example minimum unit, according to a traditional annular space construction mode of consistent hashing, as shown in fig. 1, data of a certain partition is distributed on all servers more uniformly, and can better play a balancing role in access. However, if the single instance is handled, other nodes host, and then the failed node recovers and moves data, the adjacent instances (i.e. nodes in the graph) may be physically distributed on different servers, and the move recovery in the memory space may require data transmission between different physical machines for recovery through the network, resulting in poor availability of system services.
Disclosure of Invention
The technical problem to be solved by the invention is as follows: how to enable adjacent nodes to be stored in the same server, thereby enabling data migration and recovery in memory space to be more efficient.
According to an aspect of the embodiments of the present invention, a data storage method of a memory cluster is provided, including: constructing an original hash ring of each node based on consistent hash operation; grouping all nodes in the original hash ring according to an adjacent relation, so that the adjacent nodes have the same server number to construct a logical hash ring; storing each node to a corresponding server according to the logical hash ring; and establishing the mapping relation of each node between the original hash ring and the logical hash ring.
According to another aspect of the embodiments of the present invention, a method for reading data of a memory cluster is provided, including: determining nodes of data to be read on an original hash ring based on consistent hash operation; determining the nodes of the data to be read on the logical hash ring according to the mapping relation of each node on the original hash ring and the logical hash ring; and reading data from the corresponding server according to the server number corresponding to the node on the logical hash ring.
According to another aspect of the embodiments of the present invention, there is provided a data storage device of a memory cluster, including: the original hash ring construction module is used for constructing an original hash ring of each node based on consistent hash operation; the logical hash ring construction module is used for grouping all nodes in the original hash ring according to an adjacent relation, so that the adjacent nodes have the same server number to construct the logical hash ring; the storage module is used for storing each node to a corresponding server according to the logical hash ring; and the mapping relation establishing module is used for establishing the mapping relation of each node between the original hash ring and the logical hash ring.
According to another aspect of the embodiments of the present invention, there is provided a data reading apparatus for a memory cluster, including: the node determining module is used for determining the nodes of the data to be read on the original hash ring based on the consistent hash operation and determining the nodes of the data to be read on the logical hash ring according to the mapping relation of each node on the original hash ring and the logical hash ring; and the data reading module is used for reading data from the corresponding server according to the server number corresponding to the node on the logical hash ring.
The invention has at least the following advantages:
by constructing the logical hash ring, establishing the mapping relation between the original hash ring and the logical hash ring, and storing each node to the corresponding server according to the logical hash ring, the adjacent nodes are stored in the same server, so that the data moving and recovery in the memory space are more efficient.
Other features of the present invention and advantages thereof will become apparent from the following detailed description of exemplary embodiments thereof, which proceeds with reference to the accompanying drawings.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 shows a schematic diagram of a ring space construction scheme of a conventional consistent hash.
Fig. 2 is a schematic flow chart illustrating a data storage method of a memory cluster according to an embodiment of the present invention.
Fig. 3 is a schematic flow chart illustrating a data storage method of a memory cluster according to another embodiment of the present invention.
FIG. 4 is a diagram illustrating an embodiment of the present invention for slicing table partitions of the original primary key K1 into slice numbers.
FIG. 5 is a diagram illustrating another embodiment of the present invention for slicing table partitions of the original primary key K1 into tile numbers.
Fig. 6 is a schematic diagram illustrating an embodiment of a data reading method for a memory cluster according to the present invention.
Fig. 7 is a schematic structural diagram of a data storage device of a memory cluster according to an embodiment of the present invention.
Fig. 8 shows a schematic structural diagram of one embodiment of the dicing module of the present invention.
Fig. 9 shows a schematic structural view of another embodiment of the dicing module of the present invention.
Fig. 10 is a schematic diagram illustrating an embodiment of a data reading apparatus of a memory cluster according to the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
A data storage method of a memory cluster according to an embodiment of the present invention is described below with reference to fig. 2.
Fig. 2 is a schematic flow chart illustrating a data storage method of a memory cluster according to an embodiment of the present invention. As shown in fig. 2, the data storage method of the memory cluster of this embodiment includes
Step S202, constructing an original hash ring of each node based on consistent hash operation. The original hash ring may be constructed with reference to the prior art.
Step S204, grouping the nodes in the original hash ring according to the adjacent relation, so that the adjacent nodes have the same server number to construct a logical hash ring.
And step S206, storing each node to a corresponding server according to the logical hash ring.
Step S208, establishing the mapping relation of each node between the original hash ring and the logical hash ring.
According to the method, the logical hash ring is constructed, the mapping relation between the original hash ring and the logical hash ring is established, and each node is stored in the corresponding server according to the logical hash ring, so that adjacent nodes are stored in the same server, and the data moving and recovery in the memory space are more efficient.
For data in the memory space, the stronger the data dispersion, the more complicated the moving and recovery of the data. Therefore, the data storage method of the memory cluster and the storage balancing method of the memory cluster can be combined for use, so that the technical effect of the invention is more obvious.
A data storage method of a memory cluster according to an embodiment of the present invention is described below with reference to fig. 3.
Fig. 3 is a schematic flow chart illustrating a data storage method of a memory cluster according to another embodiment of the present invention. As shown in fig. 3, on the basis of the embodiment shown in fig. 2, the present embodiment further includes:
step S3012, the table subsections of the original primary key K1 are segmented to obtain the subsection numbers. The original primary key K1 is an original logical key of the custom object DO, and the original primary key K1 is composed of a database segment, a data table segment, and a table partition segment.
In step S3014, a balanced primary key K1' is generated according to the original primary key K1 and the fragment number. Wherein, the balance primary key K1' is a balance logic key of the customized object DO.
In step S3016, the key-value data is stored in the data storage format of (K1', DO).
By the method, the table partition of the original primary key K1 is further subdivided, the balance key is reconstructed according to the original key and the partition number, and data is stored based on the balance key, so that the data is uniformly distributed on the data nodes of the memory cluster, and the storage capacity of the memory cluster is fully utilized. Then, the further subdivided data are stored by using the embodiment method shown in fig. 2, so that the adjacent nodes are stored in the same server, and therefore, in the data moving and recovering process, the data only need to be operated in the same server, the data operation between the servers is avoided, and the data operation efficiency is improved.
The method for splitting the table partition of the original primary key K1 into the partition numbers according to an embodiment of the present invention is described below with reference to fig. 4.
FIG. 4 is a diagram illustrating one embodiment of the present invention for slicing table partitions of the original primary key K1 into tile numbers. As shown in fig. 4, (K2, V) is stored in the custom object DO, the secondary key K2 is a data key of the custom object DO, and V is a data value of the custom object DO. A specific implementation method for obtaining the fragment number by segmenting the table fragment of the original primary key K1 comprises the following steps:
in step S402, a hash operation is performed on the original primary key K1 and the secondary key K2.
And S404, performing remainder operation on the counted number M of the divided pieces by using the hash value obtained by the hash operation.
In step S406, the result of the remainder operation is used as the slice number.
For example, the number of cluster nodes is 4, and the maximum number of partitions is designed to be 40. Then the user stores a new value into table1 of db 1: Key-Value, the partition corresponding to this Value is: string slice is String. value of (mat. abs (key. hashcode ())/40). Assume that the result of the calculation is 23. If the full partition key name is db1_ table1_ p1, then the new partition name is db1_ table1_ p1_ 23.
The data is further "sliced" into slices on a partition basis. When the number of slices is larger than the number of nodes, the slices are uniformly distributed on different nodes according to the characteristics of the TwyProxy. In principle, the more the number of slices is, the smaller the data storage granularity is, the more uniform the data distribution is, so that the uniform distribution of data on the data nodes of the memory cluster is realized, and the storage capacity of the memory cluster is fully utilized. However, the slice data is not necessarily too large, and is generally kept at N (N.ltoreq.10) times the number of nodes.
The method for splitting the table partition of the original primary key K1 to obtain the partition number according to another embodiment of the present invention is described below with reference to fig. 5.
FIG. 5 is a diagram illustrating another embodiment of the present invention for slicing table partitions of the original primary key K1 into tile numbers. As shown in fig. 5, another specific implementation method for segmenting the table partition of the original primary key K1 to obtain the fragment number includes:
step S502, md5 operation is carried out on the original primary key K1 and the secondary key K2;
in step S504, the last n bits of the digest value obtained by the md5 operation are taken as the slice number, and the maximum value that the n bits can represent is not greater than the number M of the slice.
The original primary key K1 and the original secondary key K2 are subjected to md5 operation to obtain the fragment number, so that the smaller the data storage granularity is, the more uniform the data distribution is, the uniform distribution of data on the data nodes of the memory cluster is realized, and the storage capacity of the memory cluster is fully utilized. Compared with the embodiment shown in fig. 4, the md5 algorithm can calculate the slice number with a unique value according to the primary key K1 and the secondary key K2, but the md5 algorithm occupies a high CPU and affects the performance under a large concurrent pressure.
Furthermore, the custom object DO may be stored in the form of a MAP.
A data reading method of the memory cluster according to an embodiment of the present invention is described below with reference to fig. 6.
Fig. 6 is a schematic diagram illustrating an embodiment of a data reading method for a memory cluster according to the present invention. As shown in fig. 6, the data reading method of the memory cluster according to the embodiment includes:
step S602, determining a node of the data to be read on the original hash ring based on the consistent hash operation.
Step S604, determining a node on the logical hash ring of the data to be read according to a mapping relationship between the original hash ring and the logical hash ring of each node.
And step S606, reading data from the corresponding server according to the server number corresponding to the node on the logical hash ring.
By the method, the data reading mode adopted by the user is not changed, and at the moment, the adjacent nodes are stored in the same server, so that the consistent hash function is realized, and the balance of data storage is also realized.
A data storage device of a memory cluster according to an embodiment of the present invention is described below with reference to fig. 7.
Fig. 7 is a schematic diagram illustrating another embodiment of a data reading method for a memory cluster according to the present invention. As shown in fig. 7, the data storage device 70 of the memory cluster of this embodiment includes:
an original hash ring construction module 702, configured to construct an original hash ring of each node based on consistent hash operation;
the logical hash ring construction module 704 groups the nodes in the original hash ring according to an adjacent relationship, so that the adjacent nodes have the same server number, so as to construct the logical hash ring.
A storage module 706, configured to store each node to a corresponding server according to the logical hash ring.
A mapping relationship establishing module 708, configured to establish a mapping relationship between the original hash ring and the logical hash ring for each node.
Further, the data storage device 70 may further include:
and the segmentation module 710 is used for segmenting the table segment of the original primary key K1 to obtain a segment number.
And a balanced primary key generation module 712, configured to generate a balanced primary key K1' according to the original primary key K1 and the fragment number.
A key-value data storage module 714 for storing key-value data in nodes in the form of data stores of (K1', DO).
The original primary key K1 is an original logical key of the customized object DO, the original primary key K1 is composed of a database segment, a data table segment and a table partition segment, and the balance primary key K1' is a balance logical key of the customized object DO.
The dicing module of one embodiment of the invention is described below in conjunction with fig. 8.
Fig. 8 shows a schematic structural diagram of one embodiment of the dicing module of the present invention. As shown in fig. 8, the cutting module 810 of this embodiment includes:
a hash operation unit 8102 for performing hash operation on the original primary key K1 and the secondary key K2;
and a remainder operation unit 8104 configured to perform a remainder operation on the counted number of slices M by using the hash value obtained by the hash operation, and use a result of the remainder operation as a slice number.
A dicing module according to another embodiment of the invention is described below with reference to fig. 9.
Fig. 9 shows a schematic structural view of another embodiment of the dicing module of the present invention. As shown in fig. 9, the cutting module 910 of this embodiment includes:
an md5 arithmetic unit 9102 for performing md5 arithmetic on the original primary key K1 and the secondary key K2;
the slice number determining unit 9104 is configured to take the last n bits of the digest value obtained by the operation of md5 as a slice number, where the maximum value that n bits can represent is not greater than the number M of counted slices.
A data reading apparatus of a memory cluster according to an embodiment of the present invention is described below with reference to fig. 10.
Fig. 10 is a schematic structural diagram illustrating a data reading apparatus of a memory cluster according to an embodiment of the present invention. As shown in fig. 10, the data reading apparatus 100 of this embodiment includes:
a node determining module 1002, configured to determine, based on consistent hash operation, a node of data to be read on an original hash ring, and determine, according to a mapping relationship between each node on the original hash ring and a logical hash ring, a node of the data to be read on the logical hash ring;
and the data reading module 1004 is configured to read data from a corresponding server according to the server number corresponding to the node on the logical hash ring.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.

Claims (2)

1. A data storage method of a memory cluster comprises the following steps:
constructing an original hash ring of each node based on consistent hash operation;
grouping all nodes in the original hash ring according to an adjacent relation, so that the adjacent nodes have the same server number to construct a logical hash ring;
storing each node to a corresponding server according to the logical hash ring;
establishing a mapping relation of each node between an original hash ring and a logical hash ring;
segmenting the table subsection of the original primary key K1 to obtain a segment number, which comprises the following steps: carrying out hash operation on the original primary key K1 and the original secondary key K2; carrying out remainder operation on the counted number M of the divided pieces by the hash value obtained by the hash operation; taking the result of the remainder operation as a fragment number; or, performing md5 operation of the fifth version of the message digest algorithm on the original primary key K1 and the secondary key K2; taking the last n bits of the abstract value obtained by the operation of md5 as a slice number, wherein the maximum numerical value which can be represented by the n bits is not more than the number M of the slice;
generating a balanced primary key K1' according to the original primary key K1 and the fragment number;
storing the key-value data into the node in a data storage form of (K1', DO);
the original primary key K1 is an original logical key of the customized object DO, the original primary key K1 is composed of a database segment, a data table segment and a table partition segment, the balanced primary key K1' is a balanced logical key of the customized object DO, the customized object DO stores (K2, V), the secondary key K2 is a data key of the customized object DO, and V is a data value of the customized object DO.
2. A data storage device of a memory cluster, comprising:
the original hash ring construction module is used for constructing an original hash ring of each node based on consistent hash operation;
the logical hash ring building module is used for grouping all nodes in the original hash ring according to an adjacent relation, so that the adjacent nodes have the same server number to build the logical hash ring;
the storage module is used for storing each node to a corresponding server according to the logical hash ring;
the mapping relation establishing module is used for establishing the mapping relation of each node between the original hash ring and the logical hash ring;
the segmentation module is used for segmenting the table subsections of the original primary key K1 to obtain the subsection number, and comprises: the hash operation unit is used for carrying out hash operation on the original primary key K1 and the original secondary key K2; the residue taking operation unit is used for carrying out residue taking operation on the counted number M of the divided pieces by the hash value obtained by the hash operation and taking the result of the residue taking operation as the number of the divided pieces; or, the md5 arithmetic unit is used for carrying out md5 arithmetic on the original primary key K1 and the secondary key K2; the fragment number determining unit is used for taking the last n bits of the digest value obtained by the operation of md5 as a fragment number, and the maximum value which can be expressed by the n bits is not more than the number M of counted fragments;
the balanced primary key generation module is used for generating a balanced primary key K1' according to the original primary key K1 and the fragment number;
a key-value data storage module for storing key-value data into a node in a data storage form of (K1', DO);
the original primary key K1 is an original logical key of the customized object DO, the original primary key K1 is composed of a database segment, a data table segment and a table partition segment, and the balance primary key K1' is a balance logical key of the customized object DO.
CN201510976655.7A 2015-12-23 2015-12-23 Memory cluster storage method and device and memory cluster reading method and device Active CN106909557B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201510976655.7A CN106909557B (en) 2015-12-23 2015-12-23 Memory cluster storage method and device and memory cluster reading method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510976655.7A CN106909557B (en) 2015-12-23 2015-12-23 Memory cluster storage method and device and memory cluster reading method and device

Publications (2)

Publication Number Publication Date
CN106909557A CN106909557A (en) 2017-06-30
CN106909557B true CN106909557B (en) 2020-06-16

Family

ID=59199933

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510976655.7A Active CN106909557B (en) 2015-12-23 2015-12-23 Memory cluster storage method and device and memory cluster reading method and device

Country Status (1)

Country Link
CN (1) CN106909557B (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109600447B (en) * 2018-12-21 2021-08-10 北京百度网讯科技有限公司 Method, device and system for processing data
CN110096227B (en) * 2019-03-28 2023-04-18 北京奇艺世纪科技有限公司 Data storage method, data processing device, electronic equipment and computer readable medium
CN110489059B (en) * 2019-07-11 2022-04-12 平安科技(深圳)有限公司 Data cluster storage method and device and computer equipment
CN112702167B (en) 2020-12-14 2022-11-11 苏州浪潮智能科技有限公司 Cluster feature code obtaining method and device and electronic equipment

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102244685A (en) * 2011-08-11 2011-11-16 中国科学院软件研究所 Distributed type dynamic cache expanding method and system supporting load balancing

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103235820B (en) * 2013-04-27 2016-10-05 北京搜狐新媒体信息技术有限公司 Date storage method and device in a kind of group system
CN104462124B (en) * 2013-09-22 2018-04-06 中国电信股份有限公司 Data storing platform method for organizing and data storing platform based on linear Hash table
CN104301425A (en) * 2014-10-24 2015-01-21 南方电网科学研究院有限责任公司 Object registration center service cluster implementation method based on object type and Hash ring

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102244685A (en) * 2011-08-11 2011-11-16 中国科学院软件研究所 Distributed type dynamic cache expanding method and system supporting load balancing

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
Oracle数据库系统优化调整;刘超 等;《信息安全与技术》;20140710;103-104 *

Also Published As

Publication number Publication date
CN106909557A (en) 2017-06-30

Similar Documents

Publication Publication Date Title
US9984106B2 (en) Systems, methods, and media for managing an in-memory NOSQL database
KR102376713B1 (en) Composite partition functions
US10209908B2 (en) Optimization of in-memory data grid placement
CN112486860B (en) Method, apparatus and computer program product for managing address mapping of a storage system
US8891520B2 (en) Scaling redundancy elimination middleboxes
CN106909557B (en) Memory cluster storage method and device and memory cluster reading method and device
CN110347651B (en) Cloud storage-based data synchronization method, device, equipment and storage medium
US20110071989A1 (en) File aware block level deduplication
CN106161633B (en) Transmission method and system for packed files based on cloud computing environment
JP2021500649A (en) Computer implementation methods, computer program products, and systems for storing records in shard database shard tables, computer implementation methods, computer program products, and systems for retrieving records from shard database shard tables. System, as well as a system for storing shard databases
CN106909556B (en) Memory cluster storage balancing method and device
US11140220B1 (en) Consistent hashing using the power of k choices in server placement
US20200065306A1 (en) Bloom filter partitioning
US20130232177A1 (en) Systems, methods, and media for managing ram resources for in-memory nosql databases
CN110851525A (en) Data fragmentation method, related equipment and computer storage medium
US8903871B2 (en) Dynamic management of log persistence
CN114461407B (en) Data processing method, data processing device, distribution server, data processing system, and storage medium
CN114443658A (en) Data processing method and device, computer storage medium and electronic equipment
WO2019153546A1 (en) Ten-thousand-level dimension data generation method, apparatus and device, and storage medium
CN115525655A (en) Method and system for data query slicing
US11310309B1 (en) Arc jump: per-key selection of an alternative server when implemented bounded loads
CN116431615A (en) Flexible data partition routing method for complex service scene
CN116016508A (en) Storage system based on distributed object and control method thereof
CN114880297A (en) Distributed data deduplication method and system based on fingerprints
US10712959B2 (en) Method, device and computer program product for storing data

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20220126

Address after: 100007 room 205-32, floor 2, building 2, No. 1 and No. 3, qinglonghutong a, Dongcheng District, Beijing

Patentee after: Tianyiyun Technology Co.,Ltd.

Address before: No.31, Financial Street, Xicheng District, Beijing, 100033

Patentee before: CHINA TELECOM Corp.,Ltd.