CN103577440B - A kind of data processing method and device in non-relational database - Google Patents
A kind of data processing method and device in non-relational database Download PDFInfo
- Publication number
- CN103577440B CN103577440B CN201210266418.8A CN201210266418A CN103577440B CN 103577440 B CN103577440 B CN 103577440B CN 201210266418 A CN201210266418 A CN 201210266418A CN 103577440 B CN103577440 B CN 103577440B
- Authority
- CN
- China
- Prior art keywords
- data
- user
- mark
- storage address
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
This application provides the data processing method and device in a kind of non-relational database, the non-relational database includes the distributed system of at least two units, the unit includes at least two data point storehouses, the data point storehouse includes some data forms, the data point storehouse has data point storehouse mark, and the data form has data form mark；Described method includes：User's mark of data to be stored is generated, the data to be stored are to store the data in the distributed system first；In the data form that the data to be stored are write to a certain data point storehouse, and identified according to the corresponding data point storehouse mark in the data point storehouse and the corresponding data form of the data form, generate target storage address；Record the mapping relations of user's mark and target storage address.The application can make lookup extract the more quick stability for accurately improving operation of data, extend the data volume supported, the cost of reduction operation maintenance.
Description
Technical field
The application is related in a kind of technical field of non-relational database, more particularly to a kind of non-relational database
Data processing method, the data processing equipment in a kind of non-relational database, the data in a kind of non-relational database
Querying method, and, the data query arrangement in a kind of non-relational database.
Background technology
In routine duties, sometimes will face substantial amounts of data message, and these substantial amounts of information be it is random, it
Between there is no certain association, be non-relational data.
Such as certain financial product have substantial amounts of user data need analysis, analyze these user data during need from
Multiple dimensions are carried out, and each dimension has many key-value pairs again, store these key-value pairs and need to use non-relational data
Library storage.But various commercial solution costs are higher in the market, and non-relational database scheme of increasing income can not be met
Requirement in terms of performance and capacity.The non-relational database technology increased income of current popular mainly occurs that two categories below is asked
Topic：
1st, the data volume supported is limited, and there is the inconsistent fatal problem of read-write；Or system is extremely unstable, and operation
Machine maintenance cost is too high.
2nd, purely realized with key-value pair storage mode, it is impossible to meet the memory requirement of mass users information.
Therefore, the problem of those skilled in the art are in the urgent need to address be：A kind of number in non-relational database is provided
According to the method and device of processing and inquiry, the systematicness to improve non-relational data storage makes lookup extract data more
It is quick accurate, improve the stability of operation；The data volume supported is extended, the cost of operation maintenance is reduced.
The content of the invention
Technical problems to be solved in this application are to provide data processing and inquiry in a kind of non-relational database
Method and device, the systematicness to improve non-relational data storage improves the stability of operation；Extend the data supported
Amount, reduces the cost of operation maintenance.
It is described this application discloses the data processing method in a kind of non-relational database in order to solve the above problems
Non-relational database includes the distributed system of at least two units, and the unit includes at least two data point storehouses, institute
Stating data point storehouse includes some data forms, and the data point storehouse has data point storehouse mark, and the data form has data
Form is identified；
Described method includes：
User's mark of data to be stored is generated, the data to be stored are stored first in the distributed system
Data；
In the data form that the data to be stored are write to a certain data point storehouse, and it is corresponding according to the data point storehouse
Data point storehouse mark and the corresponding data form mark of the data form, generate target storage address；
Record the mapping relations of user's mark and target storage address.
Preferably, the data to be stored include user's characteristic information；User's mark of the generation data to be stored
Step includes：
Extract user's characteristic information in the data to be stored；
Record the user's characteristic information and the mapping of preset storage sequence number, and the correspondence is stored into sequence number note
Identified for user.
Preferably, it is described to write data to be stored in the data form in a certain data point storehouse, and according to the data point
The step of corresponding data point storehouse mark in storehouse and the corresponding data form mark of the data form, generation target storage address
Including：
The data form of the preset number of users upper limit is not up in selection one data point storehouse, target data form is generated；
The data to be stored are write into the target data form；
The identifier combination in the mark and corresponding data of the target data form point storehouse is generated into target storage address.
Preferably, it is described record user mark and target storage address mapping relations the step of include：
Set up the key-value pair of target storage address described in selection user's identity map；
The key-value pair is recorded in map listing.
Preferably, described method also includes：
User in data to be added is obtained to identify；The data to be added are that the built user of the distributed system is arrived in storage
It is the supplementary data of built user's mark in the corresponding target storage address of mark；
Target storage address is obtained according to user mark inquiry；
The data to be added are write into the target storage address.
Preferably, described method also includes：
Target storage address is obtained according to user's mark inquiry, corresponding data point storehouse and the mesh pointed to from target storage address
The user data in the target storage address is extracted in mark data form.
Preferably, the data to be added include user's characteristic information；It is described to obtain what user in data to be added identified
Step includes：
Extract the user's characteristic information in the data to be added；
Correspondence storage sequence number, correspondence storage Serial No. user's mark are obtained according to user's characteristic information mapping
Know.
Preferably, if legacy data point library storage load reaches the upper limit, the unit of the distributed system is increased, and
New data point storehouse is set up in the unit.
Preferably, the data to be stored or data storage to be added of the same user's mark of the correspondence are in a data form
Among.
The embodiment of the present application also discloses the data processing equipment in a kind of non-relational database, the non-relational number
Include the distributed system of at least two units according to storehouse, the unit includes at least two data point storehouses, the data point storehouse
Including some data forms, the data point storehouse has data point storehouse mark, and the data form has data form mark；
Described device includes：
User's identifier generation module, user's mark for generating data to be stored, the data to be stored are to deposit first
Store up the data in the distributed system；
Target storing address generating module, the data form for the data to be stored to be write to a certain data point storehouse
In, and identified according to the corresponding data point storehouse mark in the data point storehouse and the corresponding data form of the data form, it is raw
Into target storage address；
Mapping relations logging modle, the mapping relations for recording user's mark and target storage address.
Preferably, the data to be stored include user's characteristic information；User's identifier generation module includes：
User's characteristic information extracting sub-module, for extracting user's characteristic information in the data to be stored；
User's mark generation submodule, for recording the user's characteristic information and the mapping of preset storage sequence number,
And the correspondence storage sequence number is designated as user's mark.
Preferably, the target storing address generating module includes：
Target data form generates submodule, for selecting to be not up to the preset number of users upper limit in a data point storehouse
Data form, generates target data form；
Data to be stored write submodule, the data to be stored are write into the target data form；
Identifier combination submodule, the identifier combination generation mesh by the mark and corresponding data of the target data form point storehouse
Mark storage address.
Preferably, the mapping relations logging modle includes：
Key-value pair setting up submodule, the key assignments for setting up target storage address described in selection user's identity map
It is right；
Key-value pair record sub module, the key-value pair is recorded in map listing.
Preferably, described device also includes：
User's identifier acquisition module, identify for obtaining in data to be added user；The data to be added are that storage is arrived
It is the supplementary data of built user's mark in the corresponding target storage address of the built user's mark of distributed system；
Target storage address acquisition module, for according to the user mark inquiry obtain target storage address；
Target storage address writing module, for the data to be added to be write into the target storage address.
Preferably, described device also includes：
Target storage address is obtained according to user's mark inquiry, corresponding data point storehouse and the mesh pointed to from target storage address
The user data in the target storage address is extracted in mark data form.
Preferably, the data to be added include user's characteristic information；User's identifier acquisition module includes：
User's characteristic information extracting sub-module, for extracting the user's characteristic information in the data to be added；
Store sequence number acquisition submodule, correspondence storage sequence number is obtained according to user's characteristic information mapping, it is described
Correspondence storage Serial No. user's mark.
Preferably, if legacy data point library storage load reaches the upper limit, the unit of the distributed system is increased, and
New data point storehouse is set up in the unit.
Preferably, the data to be stored or data storage to be added of the same user's mark of the correspondence are in a data form
Among.
The embodiment of the present application also discloses a kind of data query method in non-relational database, the non-relational number
Include the distributed system of at least two units according to storehouse, the unit includes at least two data point storehouses, the data point storehouse
Including some data forms, the data point storehouse has data point storehouse mark, and the data form has data form mark；
Methods described includes：
The inquiry request of user data is received, the inquiry request includes user's mark；
Target storage address is obtained according to user mark inquiry, the target storage address is by target data form
Mark and the identifier combination generation in corresponding data point storehouse；
User data is extracted from the corresponding data point storehouse of target storage address sensing and target data form；
Return to the user data.
The embodiment of the present application also discloses the data query arrangement in a kind of non-relational database, the non-relational number
Include the distributed system of at least two units according to storehouse, the unit includes at least two data point storehouses, the data point storehouse
Including some data forms, the data point storehouse has data point storehouse mark, and the data form has data form mark；
Described device includes：
User's identifier acquisition module, the inquiry request for receiving user data, the inquiry request include user's mark
Know；
Target storage address enquiry module, target storage address obtained according to user mark inquiry, the target deposits
Address is stored up to be generated by the identifier combination in mark and corresponding data point storehouse of target data form；
Data storage acquisition module, corresponding data point storehouse and target matrix for being pointed to from the target storage address
User data is extracted in lattice；
Data storage returns to module, for returning to the user data.
Compared with prior art, the application has advantages below：
The application, which stores application scenarios by a kind of non-relational database, has substantial amounts of user data to need what is analyzed
Data, while using data point storehouse and tables of data come orderly storage and management user data, and set up user's mark and target
The map listing of storage address, the effective systematicness for improving non-relational data storage makes lookup extract data more quick
Accurately, the stability of operation is improved；The data volume supported is extended, the cost of operation maintenance is reduced.
Brief description of the drawings
Fig. 1 is the flow chart of the data processing method embodiment 1 in a kind of non-relational database of the application；
Fig. 2 is the flow chart of the data processing method embodiment 2 in a kind of non-relational database of the application；
Fig. 3 is the structured flowchart of the data processing equipment embodiment 1 in a kind of non-relational database of the application；
Fig. 4 is the structured flowchart of the data processing equipment embodiment 2 in a kind of non-relational database of the application；
Fig. 5 is a kind of schematic diagram of target storage address numbering of the application；
Fig. 6 is the flow chart of the data query embodiment of the method in a kind of non-relational database of the application；
Fig. 7 is the structured flowchart of the data query arrangement embodiment in a kind of non-relational database of the application.
Embodiment
It is below in conjunction with the accompanying drawings and specific real to enable above-mentioned purpose, the feature and advantage of the application more obvious understandable
Mode is applied to be described in further detail the application.
Relevant database：The database on the basis of relational model is built upon, by means of mathematical concepts such as algebras of sets
The data come with method in processing data storehouse.Relation is used in the various contacts between various entities and entity in real world
Model is represented.Normal data query language SQL is exactly a kind of language based on relational database, and this language is performed to relation
The retrieval and operation of data in database.
Non-relational database：Stored with key-value pair, its structure is not fixed, each tuple there can be different word
Section, each tuple can increase some key-value pairs of oneself as needed, thus be not intended to be limited to fixed structure, can subtract
Some few times and the expense in space.
The application scenarios of the application have substantial amounts of user data to need analysis, the user data of the application application scenarios magnanimity
Can not opening relationships model one by one, therefore the need for relevant database does not adapt to the application Users'Data Analysis.The application
Need to carry out from multiple dimensions during Users'Data Analysis, each dimension there are many k/v key-value pairs again, stores these
Key assignments is the most suitable using non-relational database.But various commercial solution costs are higher in the market, and non-close of increasing income
It is requirement type database scheme can not meet performance and capacity again in terms of.
The non-relational database technology of increasing income of current popular, there is cassandra and HanderSocket.
Cassandra uses local file system data storage, but the data volume supported is limited, and there is read-write not
Consistent fatal problem.Although supporting cluster expansion, system is extremely unstable, and O＆M machine cost is too high.
HanderSocket is that pure k/v key-value pairs are realized, application scenarios are MemoryCache alternative solution.Nothing
Method meets the memory requirement of mass users information.
So the scheme and non-relational database scheme of existing relevant database can not all cover and use the application's
In application scenarios.
The application, which stores application scenarios by a kind of non-relational database, has substantial amounts of user data to need what is analyzed
Data, while using data point storehouse and tables of data come orderly storage and management user data, and set up user's mark and target
The map listing of storage address, the effective systematicness for improving non-relational data storage makes lookup extract data more quick
Accurately, the stability of operation is improved；The data volume supported is extended, the cost of operation maintenance is reduced.
Herein described non-relational database includes the distributed system of at least two units, the unit include to
Few two data point storehouse, the data point storehouse includes some data forms, and the data point storehouse has data point storehouse mark, described
Data form has data form mark.
Reference picture 1, shows the flow of the data processing method embodiment 1 in a kind of non-relational database of the application
Figure, can specifically include：
Step 101, user's mark of generation data to be stored, the data to be stored are that the distribution is arrived in storage first
Data in system；
In a preferred embodiment of the present application, the data to be stored are to need to store the distributed system first
Data in system are, it is necessary to which newly-built user identifies.
In a preferred embodiment of the present application, the data to be stored can include user's characteristic information；The step
Rapid 101 can specifically include following sub-step：
User's characteristic information in sub-step S11, the extraction data to be stored；
In the specific implementation, user's characteristic information can be the network ID of user, the register account number of user in systems, or
Person is information of Association Identity proof of user etc.；As long as can distinguish the information of user and other users can turn into use
Family characteristic information.
Sub-step S12, record the user's characteristic information and the mapping of preset storage sequence number, and the correspondence is deposited
Storage sequence number is designated as user's mark.
In the specific implementation, preset storage sequence number can be directly using the serial number of storage, such as the 10021st storage
Data when one data to be stored, then can be stored sequence number and be set as：001021.
In the specific implementation, map listing can also be set up, by user's characteristic information and preset storage sequence number record
In map listing, mapping relations are formed.
Step 102, the data to be stored are write in the data form in a certain data point storehouse, and according to the data point
The corresponding data point storehouse mark in storehouse and the corresponding data form mark of the data form, generate target storage address；
In a preferred embodiment of the present application, the step 102 can specifically include following sub-step：
The data form of the preset number of users upper limit is not up in sub-step S21, selection one data point storehouse, target is generated
Data form；
In the specific implementation, the table quantity in single storehouse can be difficult excessively management, it is possible to be limited within 1000.So may be used
So that according to the demand of oneself dynamically the purpose of dilatation can be reached in increase data point storehouse.
Sub-step S22, the data to be stored are write into the target data form；
Sub-step S23, by the mark of the target data form and the identifier combination in corresponding data point storehouse generation target deposit
Store up address.
In the specific implementation, the target storage address mark exclusive equivalent to one, can divide according to available data
Storehouse is identified and data form identifies generation, certain data form can be identified into data point storehouse ID association corresponding with its and got up
Generate the target storage address of this data form.As shown in Figure 5.First four can be the volume in the mark of data point storehouse in Fig. 5
Number information, latter three can be the number information in data form mark.
Step 103, the mapping relations for recording user's mark and target storage address；
In the specific implementation, the map listing of user's mark and target storage address can be set up.
In a preferred embodiment of the present application, the step 103 can specifically include following sub-step：
Sub-step S31, the key-value pair for setting up target storage address described in selection user's identity map；
Sub-step S32, the key-value pair is recorded in map listing.
Step 104, according to user mark inquiry obtain target storage address, from target storage address point to corresponding data
The user data in the target storage address is extracted in point storehouse and target data form.
In the specific implementation, when to extract the data of some user, correspondence user can be found according to correspondence user's mark
All data storages target storage address.Then the number the need for calling in the preset interface extraction target storage address
According to.
Reference picture 2, shows the flow of the data processing method embodiment 2 in a kind of non-relational database of the application
Figure, can specifically include：
User identifies in step 201, acquisition data to be added；
In a kind of preferred embodiment of the application, the data to be added, which need to store to the distributed system is built, to be used
It is the supplementary data of built user's mark in the corresponding target storage address of family mark.
In concrete practice, data to be added do not need newly-built user to identify.But find the use existed
The corresponding target storage address of family mark, data to be added write-in wherein.
In a preferred embodiment of the present application, the data to be added include user's characteristic information；The step 201
Following sub-step can specifically be included：
Sub-step S41, the user's characteristic information extracted in the data to be added；
Sub-step S42, according to the user's characteristic information mapping obtain correspondence storage sequence number, it is described correspondence storage sequence
Number for user mark.
Step 202, according to the user mark inquiry obtain target storage address；
Step 203, the data to be added are write into the target storage address；
Step 204, according to user mark inquiry obtain target storage address, from target storage address point to corresponding data
The user data in the target storage address is extracted in point storehouse and target data form.
In a preferred embodiment of the present application, if legacy data point library storage load reaches the upper limit, increase is described
The unit of distributed system, and set up in the unit new data point storehouse.
In the specific implementation, the table quantity in single storehouse can be difficult excessively management, it is possible to be limited within 1000.So may be used
So that according to the demand of oneself, dynamically increase storehouse can dilatation.
In a preferred embodiment of the present application, the same user data is stored among a data form.
In the specific implementation, due to the whole data model of a user, i.e., all data messages of one user all exist
In one table, it is to avoid the query process of multilist or many storehouses, therefore the efficiency and stability of inquiry can also obtain bigger carry
Rise.
It should be noted that for embodiment of the method, in order to be briefly described, therefore it to be all expressed as to a series of action group
Close, but those skilled in the art should know, the application is not limited by described sequence of movement, because according to this Shen
Please, some steps can be carried out sequentially or simultaneously using other.Secondly, those skilled in the art should also know, specification
Described in embodiment belong to necessary to preferred embodiment, involved action not necessarily the application.
Those skilled in the art more fully understand the application for convenience, below in conjunction with table 1 by a complete example more
Further illustrate the application：
Table 1：Shown is the model information of a user：
| ID | Nick | Age | Sex |
| 10000 | Lionbule | 30 | man |
Table 1
A1, extraction user's mark：10000；Other data informations of wherein user also include：Data{nick：
Lionbule；age：30；sex：man}.
When A2, the model information of storage user, memory interface is first called：Api save (long id, Map ＜ String,
Object ＞ data), inquire about target storage address.If user identifies：10000 corresponding mapping address are not present, then apply
New target storage address A, Data { nick：Lionbule；age：30；sex：Man } row of database table is converted to by key
Then record stores new target storage address A in the mapping table in new target storage address A.If user identifies：
10000 corresponding mapping address are existing, then extract corresponding target storage address B.Data { nick：Lionbule；age：
30；sex：Man } the row record of database table is converted in target storage address B by key.
B1, now need to store the newly-added information of user into the target storage address of this user, it is known that this user
Mark：10000, first call query interface：Api query (long id), are identified according to user：The corresponding mapping of 10000 inquiries
Target storage address.
B2, identified by user：10000, which inquire corresponding mapping address, is present, then extracts corresponding target storage
Location C.
B3, the information Data { nick for increasing user newly：Lionbule；age：30；sex：Man } store and arrive target storage
In the C of location.
C1, now need obtain user mark：10001 information.First call query interface：Api query (long id),
Identified according to user：The corresponding mapping objects storage address of 10001 inquiries.
C2, inquiry find user's mark without result：10001 corresponding mapping address are not present, then throw exception information is simultaneously
Exit query process.Return result to client.
Reference picture 3 shows the structural frames of the data processing equipment embodiment 1 in a kind of non-relational database of the application
Figure, can specifically include：
User's identifier generation module 301, user's mark for generating data to be stored；
In a preferred embodiment of the present application, the data to be stored are to need to store the distributed system first
Data in system are, it is necessary to which newly-built user identifies.
In a preferred embodiment of the present application, the data to be stored include user's characteristic information；User's mark
Following submodule can specifically be included by knowing generation module 301：
User's characteristic information extracting sub-module, for extracting user's characteristic information in the data to be stored；
User's mark generation submodule, for recording the user's characteristic information and the mapping of preset storage sequence number,
And the correspondence storage sequence number is denoted as user's mark.
Target storing address generating module 302, the tables of data for the data to be stored to be write to a certain data point storehouse
In lattice, and identified according to the corresponding data point storehouse mark in the data point storehouse and the corresponding data form of the data form,
Generate target storage address；
In a preferred embodiment of the present application, the target storing address generating module 302 specifically can include with
Lower submodule：
Target data form generates submodule, for selecting to be not up to the preset number of users upper limit in a data point storehouse
Data form, generates target data form；
Data to be stored write submodule, for the data to be stored to be write into the target data form；
Identifier combination submodule, the identifier combination life for the mark and corresponding data of the target data form to be divided to storehouse
Into target storage address.
Mapping relations logging modle 303, the mapping relations for recording user's mark and target storage address；
In a preferred embodiment of the present application, the mapping relations logging modle 303 can specifically include following son
Module：
Key-value pair setting up submodule, the key assignments for setting up target storage address described in selection user's identity map
It is right；
Key-value pair record sub module, for recording the key-value pair in map listing.
User data extraction module 304, for according to user mark inquiry obtain target storage address, call preset interface
Extract the user data in the target storage address.
Reference picture 4, shows the structural frames of the data processing equipment embodiment 2 in a kind of non-relational database of the application
Figure, can specifically include：
User's identifier acquisition module 401, identify for obtaining in data to be added user；
In a preferred embodiment of the present application, the data to be added need to store built to the distributed system
It is the supplementary data of built user's mark in the corresponding target storage address of user's mark.
In a preferred embodiment of the present application, the data to be added include user's characteristic information；User's mark
Following sub-step can specifically be included by knowing acquisition module 401：
User's characteristic information extracting sub-module, for extracting the user's characteristic information in the data to be added；
Store sequence number acquisition submodule, for obtaining correspondence storage sequence number according to user's characteristic information mapping,
Correspondence storage Serial No. user's mark.
Target storage address acquisition module 402, for according to the user mark inquiry obtain target storage address；
Data write. module 403 to be added, for the data to be added to be write into the target storage address.
User data extraction module 404, for according to user mark inquiry obtain target storage address, from target storage
The user data in the target storage address is extracted in the corresponding data point storehouse of location sensing and target data form.
In a preferred embodiment of the present application, if legacy data point library storage load reaches the upper limit, increase is described
The unit of distributed system, and set up in the unit new data point storehouse.
In a preferred embodiment of the present application, the same user data is stored among a data form.
Reference picture 6, the step of showing the data query embodiment of the method in a kind of non-relational database of the application is flowed
Cheng Tu, specifically may include steps of：
Step 601, the inquiry request for receiving user data, the inquiry request include user's mark；
Step 602, according to the user mark inquiry obtain target storage address, the target storage address is by number of targets
According to the identifier combination generation in mark and corresponding data point storehouse of form；
Step 603, from the target storage address point to corresponding data point storehouse and target data form in extract user
Data；
Step 604, the return user data.
Those skilled in the art more fully understand the application for convenience, below in conjunction with table 1 by a complete example more
Further illustrate the embodiment of the present application：
D1, now need obtain user mark：10000 information.First call query interface：Api query (long id),
Identified according to user：The corresponding mapping objects storage address of 10000 inquiries.
If D2, user's mark：10000 corresponding target storage address are present, the data information Data of user
{nick：Lionbule；age：30；sex：Man } return to client.
Reference picture 7, shows the structural frames of the data query arrangement embodiment in a kind of non-relational database of the application
Figure, can specifically include：
User's identifier acquisition module 701, the inquiry request for receiving user data, the inquiry request include user
Mark；
Target storage address enquiry module 702, for according to the user mark inquiry obtain target storage address, it is described
Target storage address is generated by the identifier combination in mark and corresponding data point storehouse of target data form；
Data storage acquisition module 703, corresponding data point storehouse and number of targets for being pointed to from the target storage address
According to extracting user data in form；
Data storage returns to module 704, for returning to the user data.
Because described device embodiment essentially corresponds to preceding method embodiment, thus in the description of the present embodiment it is not detailed it
Place, may refer to the related description in previous embodiment, does not just repeat herein.
It should be understood by those skilled in the art that, embodiments herein can be provided as method, system or computer program
Product.Therefore, the application can be using the reality in terms of complete hardware embodiment, complete software embodiment or combination software and hardware
Apply the form of example.Moreover, the application can be used in one or more computers for wherein including computer usable program code
The computer program production that usable storage medium is implemented on (including but is not limited to magnetic disk storage, CD-ROM, optical memory etc.)
The form of product.
The application is the flow with reference to method, equipment (system) and computer program product according to the embodiment of the present application
Figure and/or block diagram are described.It should be understood that can be by every first-class in computer program instructions implementation process figure and/or block diagram
Journey and/or the flow in square frame and flow chart and/or block diagram and/or the combination of square frame.These computer programs can be provided
The processor of all-purpose computer, special-purpose computer, Embedded Processor or other programmable data processing devices is instructed to produce
A raw machine so that produced by the instruction of computer or the computing device of other programmable data processing devices for real
The device for the function of being specified in present one flow of flow chart or one square frame of multiple flows and/or block diagram or multiple square frames.
These computer program instructions, which may be alternatively stored in, can guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works so that the instruction being stored in the computer-readable memory, which is produced, to be included referring to
Make the manufacture of device, the command device realize in one flow of flow chart or multiple flows and/or one square frame of block diagram or
The function of being specified in multiple square frames.
These computer program instructions can be also loaded into computer or other programmable data processing devices so that in meter
Series of operation steps is performed on calculation machine or other programmable devices to produce computer implemented processing, thus in computer or
The instruction performed on other programmable devices is provided for realizing in one flow of flow chart or multiple flows and/or block diagram one
The step of function of being specified in individual square frame or multiple square frames.
Although having been described for the preferred embodiment of the application, those skilled in the art once know basic creation
Property concept, then can make other change and modification to these embodiments.So, appended claims are intended to be construed to include excellent
Select embodiment and fall into having altered and changing for the application scope.
Above to the data processing method and a kind of non-relational in a kind of non-relational database provided herein
Data processing equipment in database is described in detail, used herein principle and implementation of the specific case to the application
Mode is set forth, and the explanation of above example is only intended to help and understands the present processes and its core concept；Meanwhile,
For those of ordinary skill in the art, according to the thought of the application, have change in specific embodiments and applications
Become part, in summary, this specification content should not be construed as the limitation to the application.
Claims (16)
1. the data processing method in a kind of non-relational database, it is characterised in that the non-relational database is included extremely
The distributed system of few two units, the unit includes at least two data point storehouses, and the data point storehouse includes some numbers
According to form, the data point storehouse has data point storehouse mark, and the data form has data form mark, and each divides storehouse
Data form with the preset number of users upper limit, if legacy data point library storage load reaches the upper limit, increases the distribution
The unit of formula system, and set up in the unit new data point storehouse；
Described method includes：
User's mark of data to be stored is generated, the data to be stored are to store the number in the distributed system first
According to；
In the data form that the data to be stored are write to a certain data point storehouse, and according to the data point corresponding data in storehouse
Divide storehouse mark and the corresponding data form mark of the data form, generate target storage address；
Record the mapping relations of user's mark and target storage address；
Wherein, the data to be stored include user's characteristic information；The step of user of the generation data to be stored identifies is wrapped
Include：
Extract user's characteristic information in the data to be stored；
Record the user's characteristic information and the mapping of preset storage sequence number, and the correspondence is stored into sequence number and be designated as using
Family is identified.
2. the method as described in claim 1, it is characterised in that the data that data to be stored are write to a certain data point storehouse
In form, and according to the corresponding data point storehouse mark in the data point storehouse and the corresponding tables of data case marker of the data form
Know, the step of generating target storage address includes：
The data form of the preset number of users upper limit is not up in selection one data point storehouse, target data form is generated；
The data to be stored are write into the target data form；
The identifier combination in the mark and corresponding data of the target data form point storehouse is generated into target storage address.
3. method as claimed in claim 2, it is characterised in that described to record reflecting for user's mark and target storage address
The step of penetrating relation includes：
Set up the key-value pair of target storage address described in selection user's identity map；
The key-value pair is recorded in map listing.
4. method as claimed in claim 3, it is characterised in that also include：
User in data to be added is obtained to identify；The data to be added are that storage is identified to the built user of the distributed system
Corresponding target storage address in, be the supplementary data of built user mark；
Target storage address is obtained according to user mark inquiry；
The data to be added are write into the target storage address.
5. method as claimed in claim 4, it is characterised in that also include：
Target storage address is obtained according to user's mark inquiry, corresponding data point storehouse and the number of targets pointed to from target storage address
According to the user data extracted in form in the target storage address.
6. method as claimed in claim 5, it is characterised in that the data to be added include user's characteristic information；It is described to obtain
The step of user identifies in data to be added is taken to include：
Extract the user's characteristic information in the data to be added；
Correspondence storage sequence number, correspondence storage Serial No. user's mark are obtained according to user's characteristic information mapping.
7. the method as described in claim 1, it is characterised in that the data to be stored of the same user's mark of correspondence are waited to add
Addend evidence is stored among a data form.
8. the data processing equipment in a kind of non-relational database, it is characterised in that the non-relational database is included extremely
The distributed system of few two units, the unit includes at least two data point storehouses, and the data point storehouse includes some numbers
According to form, the data point storehouse has data point storehouse mark, and the data form has data form mark, and each divides storehouse
Data form with the preset number of users upper limit, if legacy data point library storage load reaches the upper limit, increases the distribution
The unit of formula system, and set up in the unit new data point storehouse；
Described device includes：
User's identifier generation module, user's mark for generating data to be stored, the data to be stored are that storage is arrived first
Data in the distributed system；
Target storing address generating module, for the data to be stored to be write in the data form in a certain data point storehouse, and
According to the corresponding data point storehouse mark in the data point storehouse and the corresponding data form mark of the data form, target is generated
Storage address；
Mapping relations logging modle, the mapping relations for recording user's mark and target storage address；
Wherein, the data to be stored include user's characteristic information；User's identifier generation module includes：
User's characteristic information extracting sub-module, for extracting user's characteristic information in the data to be stored；
User's mark generation submodule, for recording the user's characteristic information and the mapping of preset storage sequence number, and will
The correspondence storage sequence number is designated as user's mark.
9. device as claimed in claim 8, it is characterised in that the target storing address generating module includes：
Target data form generation submodule, the data for being not up to the preset number of users upper limit in selection one data point storehouse
Form, generates target data form；
Data to be stored write submodule, the data to be stored are write into the target data form；
Identifier combination submodule, by the mark of the target data form and the identifier combination in corresponding data point storehouse generation target deposit
Store up address.
10. device as claimed in claim 9, it is characterised in that the mapping relations logging modle includes：
Key-value pair setting up submodule, the key-value pair for setting up target storage address described in selection user's identity map；
Key-value pair record sub module, the key-value pair is recorded in map listing.
11. device as claimed in claim 10, it is characterised in that also include：
User's identifier acquisition module, identify for obtaining in data to be added user；The data to be added are storages described in
It is the supplementary data of built user's mark in the corresponding target storage address of the built user's mark of distributed system；
Target storage address acquisition module, for according to the user mark inquiry obtain target storage address；
Target storage address writing module, for the data to be added to be write into the target storage address.
12. device as claimed in claim 11, it is characterised in that also include：
Target storage address is obtained according to user's mark inquiry, corresponding data point storehouse and the number of targets pointed to from target storage address
According to the user data extracted in form in the target storage address.
13. device as claimed in claim 12, it is characterised in that the data to be added include user's characteristic information；It is described
User's identifier acquisition module includes：
User's characteristic information extracting sub-module, for extracting the user's characteristic information in the data to be added；
Store sequence number acquisition submodule, correspondence storage sequence number, the correspondence are obtained according to user's characteristic information mapping
Store Serial No. user mark.
14. device as claimed in claim 8, it is characterised in that the data to be stored of the same user's mark of correspondence are treated
Interpolation data is stored among a data form.
15. a kind of data query method in non-relational database, it is characterised in that the non-relational database is included extremely
The distributed system of few two units, the unit includes at least two data point storehouses, and the data point storehouse includes some numbers
According to form, the data point storehouse has data point storehouse mark, and the data form has data form mark, and each divides storehouse
Data form with the preset number of users upper limit, if legacy data point library storage load reaches the upper limit, increases the distribution
The unit of formula system, and set up in the unit new data point storehouse；
Methods described includes：
The inquiry request of user data is received, the inquiry request includes user's mark；
According to the user mark inquiry obtain target storage address, the target storage address by target data form mark
With the identifier combination generation in corresponding data point storehouse；
User data is extracted from the corresponding data point storehouse of target storage address sensing and target data form；
Return to the user data；
Wherein, the user data includes user's characteristic information；User's mark is generated in the following way：
Extract user's characteristic information in the user data；
Record the user's characteristic information and the mapping of preset storage sequence number, and the correspondence is stored into sequence number and be designated as using
Family is identified.
16. the data query arrangement in a kind of non-relational database, it is characterised in that the non-relational database is included extremely
The distributed system of few two units, the unit includes at least two data point storehouses, and the data point storehouse includes some numbers
According to form, the data point storehouse has data point storehouse mark, and the data form has data form mark, and each divides storehouse
Data form with the preset number of users upper limit, if legacy data point library storage load reaches the upper limit, increases the distribution
The unit of formula system, and set up in the unit new data point storehouse；
Described device includes：
User's identifier acquisition module, the inquiry request for receiving user data, the inquiry request include user's mark；
Target storage address enquiry module, according to the user mark inquiry obtain target storage address, the target storage
Location is generated by the identifier combination in mark and corresponding data point storehouse of target data form；
During data storage acquisition module, the corresponding data for being pointed to from the target storage address divide storehouse and target data form
Extract user data；
Data storage returns to module, for returning to the user data；
Wherein, the user data includes user's characteristic information；User's mark is generated in the following way：
Extract user's characteristic information in the user data；
Record the user's characteristic information and the mapping of preset storage sequence number, and the correspondence is stored into sequence number and be designated as using
Family is identified.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210266418.8A CN103577440B (en) | 2012-07-27 | 2012-07-27 | A kind of data processing method and device in non-relational database |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210266418.8A CN103577440B (en) | 2012-07-27 | 2012-07-27 | A kind of data processing method and device in non-relational database |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103577440A CN103577440A (en) | 2014-02-12 |
| CN103577440B true CN103577440B (en) | 2017-08-11 |
Family
ID=50049245
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210266418.8A Active CN103577440B (en) | 2012-07-27 | 2012-07-27 | A kind of data processing method and device in non-relational database |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103577440B (en) |
Families Citing this family (28)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105005572A (en) * | 2014-04-24 | 2015-10-28 | 中国移动通信集团云南有限公司 | Database mapping method and device |
| CN105302842A (en) * | 2014-07-31 | 2016-02-03 | 北大方正集团有限公司 | Data processing method and device |
| CN105468346A (en) * | 2014-09-05 | 2016-04-06 | 中兴通讯股份有限公司 | Timing event management method, storage method, execution method and corresponding apparatus |
| CN106294421B (en) * | 2015-05-25 | 2020-02-04 | 阿里巴巴集团控股有限公司 | Data writing and reading method and device |
| CN105224677B (en) * | 2015-10-16 | 2018-10-30 | 上海晶赞科技发展有限公司 | A kind of database operation method and device |
| CN105447151A (en) * | 2015-11-27 | 2016-03-30 | 深圳市金蝶友商电子商务服务有限公司 | Method for accessing distributed database, data source proxy apparatus and application server |
| CN106815258B (en) * | 2015-11-27 | 2020-01-17 | 华为技术有限公司 | Data storage method and coordination node |
| CN105653692B (en) * | 2015-12-30 | 2019-06-21 | 北京奇艺世纪科技有限公司 | A kind of file memory method and device based on non-relational database |
| CN105681477B (en) * | 2016-04-08 | 2018-11-16 | 四川师范大学 | A kind of data access method and a kind of server |
| CN105828052A (en) * | 2016-06-02 | 2016-08-03 | 中国联合网络通信集团有限公司 | Video monitoring method and monitoring system based on Storm technology |
| CN107665153A (en) * | 2016-07-27 | 2018-02-06 | 中兴通讯股份有限公司 | Data back up method, restoration methods and device in a kind of big data system |
| CN108399175B (en) * | 2017-02-07 | 2022-04-12 | 阿里巴巴集团控股有限公司 | Data storage and query method and device |
| CN109144404A (en) * | 2017-06-27 | 2019-01-04 | 北京金山安全软件有限公司 | Data storage and query method and device based on multiple DMPs and electronic equipment |
| CN108733787A (en) * | 2018-05-11 | 2018-11-02 | 中国平安人寿保险股份有限公司 | Database operation method, device, electronic equipment and storage medium |
| CN108809952A (en) * | 2018-05-22 | 2018-11-13 | 隆正信息科技有限公司 | Object authentication method and system based on more mark general unique keepsake chains of sectional |
| CN110727739B (en) * | 2018-06-28 | 2024-06-18 | 北京京东尚科信息技术有限公司 | Data storage method and device |
| CN110851435B (en) * | 2018-08-03 | 2022-02-11 | 杭州海康威视数字技术股份有限公司 | Data storage method and device |
| CN109284340B (en) * | 2018-09-14 | 2020-11-20 | 清华大学 | Method for storing and retrieving relational data in resource-constrained intelligent object |
| CN109597834B (en) * | 2018-10-22 | 2024-05-07 | 平安科技（深圳）有限公司 | Massive data storage method, device, medium and equipment based on redis |
| CN110555703B (en) * | 2019-07-31 | 2023-05-23 | 创新先进技术有限公司 | Credit contract multi-view display method and device |
| CN110659258A (en) * | 2019-09-06 | 2020-01-07 | 平安普惠企业管理有限公司 | Document generation method, system, computer device and computer readable storage medium |
| CN111581049B (en) * | 2020-04-08 | 2023-10-10 | 北京百度网讯科技有限公司 | Distributed system running state monitoring method, device, equipment and storage medium |
| CN111680313B (en) * | 2020-06-08 | 2022-06-28 | 南京领行科技股份有限公司 | Data processing method, device, equipment and storage medium |
| CN111651424B (en) * | 2020-06-10 | 2024-05-03 | 中国科学院深圳先进技术研究院 | Data processing method, device, data node and storage medium |
| CN112632117B (en) * | 2020-12-30 | 2024-08-13 | 广州华多网络科技有限公司 | Method and device for processing numbered data, electronic equipment and storage medium |
| CN113111138A (en) * | 2021-05-10 | 2021-07-13 | 网易传媒科技（北京）有限公司 | Data processing method, device, computing equipment and medium |
| CN113420186B (en) * | 2021-06-18 | 2022-10-04 | 自然资源部第三地形测量队 | Data storage method, data storage device, computer readable storage medium and data reading method |
| CN113806405A (en) * | 2021-09-18 | 2021-12-17 | 王剑 | Method for inquiring and storing medical record data and related device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2002063055A (en) * | 2000-08-23 | 2002-02-28 | Ricoh Co Ltd | Writing delay data base managing method and system |
| EP1548630A2 (en) * | 2003-12-24 | 2005-06-29 | Nec Corporation | Product identification data management system and product identification data management method |
| CN101330431A (en) * | 2007-06-19 | 2008-12-24 | 阿里巴巴集团控股有限公司 | Method and system for storing instant information |
| CN101523341A (en) * | 2006-08-07 | 2009-09-02 | 谷歌公司 | Distribution of content document to varying users with security, customization and scalability |
| CN102314480A (en) * | 2011-07-05 | 2012-01-11 | 万达信息股份有限公司 | Distributed data storage method specific to mass data |
-
2012
- 2012-07-27 CN CN201210266418.8A patent/CN103577440B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2002063055A (en) * | 2000-08-23 | 2002-02-28 | Ricoh Co Ltd | Writing delay data base managing method and system |
| EP1548630A2 (en) * | 2003-12-24 | 2005-06-29 | Nec Corporation | Product identification data management system and product identification data management method |
| CN101523341A (en) * | 2006-08-07 | 2009-09-02 | 谷歌公司 | Distribution of content document to varying users with security, customization and scalability |
| CN101330431A (en) * | 2007-06-19 | 2008-12-24 | 阿里巴巴集团控股有限公司 | Method and system for storing instant information |
| CN102314480A (en) * | 2011-07-05 | 2012-01-11 | 万达信息股份有限公司 | Distributed data storage method specific to mass data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103577440A (en) | 2014-02-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103577440B (en) | A kind of data processing method and device in non-relational database | |
| CN103810237B (en) | Data managing method and system | |
| JP6356675B2 (en) | Aggregation / grouping operation: Hardware implementation of hash table method | |
| US7756889B2 (en) | Partitioning of nested tables | |
| CA2906669C (en) | Efficiently performing operations on distinct data values | |
| US8099440B2 (en) | Method for laying out fields in a database in a hybrid of row-wise and column-wise ordering | |
| US9367463B2 (en) | System and method utilizing a shared cache to provide zero copy memory mapped database | |
| CN104281701B (en) | Multiscale Distributed Spatial data query method and system | |
| US10114846B1 (en) | Balanced distribution of sort order values for a multi-column sort order of a relational database | |
| US8135688B2 (en) | Partition/table allocation on demand | |
| JP6434154B2 (en) | Identifying join relationships based on transaction access patterns | |
| US11288287B2 (en) | Methods and apparatus to partition a database | |
| CN103455335A (en) | Multilevel classification Web implementation method | |
| CN104951462B (en) | Method and system for managing database | |
| CN105359142B (en) | Hash connecting method and device | |
| US8589451B1 (en) | Systems and methods for generating a common data model for relational and object oriented databases | |
| CN106528448A (en) | Distributed caching mechanism for multi-source heterogeneous electronic commerce big data | |
| CN114443615A (en) | Database management system, related apparatus, method and medium | |
| US9129001B2 (en) | Character data compression for reducing storage requirements in a database system | |
| US10885157B2 (en) | Determining a database signature | |
| CN115168409B (en) | Data query method and device for database sub-tables and computer equipment | |
| US20150324408A1 (en) | Hybrid storage method and apparatus | |
| CN118733591A (en) | Data processing method, device, equipment and medium | |
| US9009155B2 (en) | Parallel set aggregation | |
| CN116383255A (en) | Aggregation query method, system, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20191211 Address after: P.O. Box 31119, grand exhibition hall, hibiscus street, 802 West Bay Road, Grand Cayman, Cayman Islands Patentee after: Innovative advanced technology Co., Ltd Address before: A four-storey 847 mailbox in Grand Cayman Capital Building, British Cayman Islands Patentee before: Alibaba Group Holding Co., Ltd. |