CN106776790A - Concurrent master-slave synchronisation method and device based on token - Google Patents
Concurrent master-slave synchronisation method and device based on token Download PDFInfo
- Publication number
- CN106776790A CN106776790A CN201611046602.6A CN201611046602A CN106776790A CN 106776790 A CN106776790 A CN 106776790A CN 201611046602 A CN201611046602 A CN 201611046602A CN 106776790 A CN106776790 A CN 106776790A
- Authority
- CN
- China
- Prior art keywords
- token
- data
- data syn
- task
- chronization
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/50—Indexing scheme relating to G06F9/50
- G06F2209/5018—Thread allocation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of concurrent master-slave synchronisation method and device based on token.Methods described includes：Receive the data syn-chronization task for carrying out autonomous database；It is that data syn-chronization task sets token according to the order for receiving data syn-chronization task；The data syn-chronization task of token will be provided with and be distributed to a thread in multiple threads, data syn-chronization task is performed according to token by the thread.Using such scheme of the present invention, there are multiple threads to can be used for executed in parallel partial data synchronous task from database, improve data syn-chronization speed, overcome and perform MDL and the dramatically different step problem of data from database that data syn-chronization task is caused from database single thread, the execution sequence of data syn-chronization task is limited by setting token, and serially performing data syn-chronization task according to token, it is ensured that the data with MDL are consistent.
Description
Technical field
The present invention relates to Internet technical field, and in particular to a kind of concurrent master-slave synchronisation method and dress based on token
Put.
Background technology
Distributed data-storage system is exactly to store in many independent equipment data dispersion, that is to say, that be directed to
A data, store to N number of database, usually store to a MDL and multiple from database, wherein, from data
Data in storehouse are to synchronize acquisition with MDL.
MDL and from the data synchronization process between database specifically, MDL receives visitor in the prior art
After the request of data is write at family end, in writing data into MDL, and data writing operation recorded the daily record text of MDL
In part.Then, MDL sends data syn-chronization task to from database, from database root according to data syn-chronization tasks carrying by number
According to the data writing operation write from database, and data writing operation recorded from the journal file of database.In the number
According to synchronizing process, it is used to process the data syn-chronization task of MDL transmission from database only one of which thread.The thread connects
The data syn-chronization task for carrying out autonomous database is received, it is necessary to serially perform the data writing operation involved by data syn-chronization task and write day
Will is operated.In the case that data syn-chronization task is excessive in a short time, thread cannot processing data synchronous task in time, cause
From the problem of database synchronization inefficiency, cause MDL and the inconsistent situation of the data from database occur.
The content of the invention
In view of the above problems, it is proposed that the present invention so as to provide one kind overcome above mentioned problem or at least in part solve on
State the concurrent master-slave synchronisation method based on token and the corresponding concurrent master-slave synchronisation device based on token of problem.
According to an aspect of the invention, there is provided a kind of concurrent master-slave synchronisation method based on token, it includes：
Receive the data syn-chronization task for carrying out autonomous database；
It is that data syn-chronization task sets token according to the order for receiving data syn-chronization task；
The data syn-chronization task of token will be provided with and be distributed to a thread in multiple threads, held according to token by the thread
Row data synchronous task.
According to another aspect of the present invention, there is provided a kind of concurrent master-slave synchronisation device based on token, it includes：
Receiver module, is suitable to receive the data syn-chronization task of autonomous database；
Setup module, is suitable to according to the order for receiving data syn-chronization task, is that data syn-chronization task sets token；
Distribution module, is suitable to be provided with the data syn-chronization task of token and is distributed in multiple parallel processing modules one simultaneously
Row processing module；
Multiple parallel processing modules, are suitable to perform data syn-chronization task according to token.
According to the scheme that the present invention is provided, the data syn-chronization task for carrying out autonomous database is received, according to reception data syn-chronization
The order of task, is that data syn-chronization task sets token, and the data syn-chronization task that will be provided with token is distributed in multiple threads
One thread, data syn-chronization task is performed by the thread according to token.Using such scheme of the present invention, there are multiple lines from database
Journey can be used for executed in parallel partial data synchronous task, improve data syn-chronization speed, overcomes from database single thread and holds
The MDL that row data synchronous task is caused and the dramatically different step problem of data from database, are limited by setting token
The execution sequence of data syn-chronization task, and serially perform data syn-chronization task according to token, it is ensured that the data one with MDL
Cause.
Described above is only the general introduction of technical solution of the present invention, in order to better understand technological means of the invention,
And can be practiced according to the content of specification, and in order to allow the above and other objects of the present invention, feature and advantage can
Become apparent, below especially exemplified by specific embodiment of the invention.
Brief description of the drawings
By reading the detailed description of hereafter preferred embodiment, various other advantages and benefit is common for this area
Technical staff will be clear understanding.Accompanying drawing is only used for showing the purpose of preferred embodiment, and is not considered as to the present invention
Limitation.And in whole accompanying drawing, identical part is denoted by the same reference numerals.In the accompanying drawings：
Fig. 1 shows that the flow of the concurrent master-slave synchronisation method based on token according to an embodiment of the invention is illustrated
Figure；
Fig. 2 shows that the flow of the concurrent master-slave synchronisation method based on token in accordance with another embodiment of the present invention is illustrated
Figure；
Fig. 3 shows the structural representation of the concurrent master-slave synchronisation device based on token according to an embodiment of the invention
Figure.
Specific embodiment
The exemplary embodiment of the disclosure is more fully described below with reference to accompanying drawings.Although showing the disclosure in accompanying drawing
Exemplary embodiment, it being understood, however, that may be realized in various forms the disclosure without should be by embodiments set forth here
Limited.Conversely, there is provided these embodiments are able to be best understood from the disclosure, and can be by the scope of the present disclosure
Complete conveys to those skilled in the art.
Fig. 1 shows that the flow of the concurrent master-slave synchronisation method based on token according to an embodiment of the invention is illustrated
Figure.As shown in figure 1, the method is comprised the following steps：
Step S100, receives the data syn-chronization task for carrying out autonomous database.
Distributed data-storage system is exactly to store in many independent equipment data dispersion, that is to say, that be directed to
A data, store to N number of database, usually store to a MDL and multiple from database, from database
Data are to synchronize acquisition with MDL, that is to say, that after data write-in MDL, and MDL needs will write-in
Data syn-chronization give from database, therefore, MDL to from database send data syn-chronization task, and from database reception come
The data syn-chronization task of autonomous database.
Data syn-chronization task is generated according to the data of write-in MDL, that is to say, that for write-in MDL
Any data will all generate corresponding data syn-chronization task.
Step S101, is that data syn-chronization task sets token according to the order for receiving data syn-chronization task.
Wherein, token denotes the execution sequence of data syn-chronization task, and the corresponding data of preceding token that only sort are same
After step task is performed, the corresponding data syn-chronization task of the posterior token of sequence can be just performed.For example, according to reception number
According to the order of synchronous task, there is provided the token of data syn-chronization task is respectively 1,2,3,4, it is so, only corresponding in token 1
After the completion of data syn-chronization task is performed, the corresponding data syn-chronization task of token 2 can be just performed, if the corresponding data of token 1
Synchronous task has not been performed, then can not perform the corresponding data syn-chronization task of token 2, similarly, by that analogy, performs token 3
With 4 corresponding data syn-chronization tasks.In embodiments of the present invention, it is that data syn-chronization task sets token, it is ensured that same in data
Be not in be synchronized to from the data of the database defect inconsistent with the data for being written to MDL during step.
Step S102, will be provided with the data syn-chronization task of token and is distributed to a thread in multiple threads, by the thread
Data syn-chronization task is performed according to token.
Specifically, after token is set for data syn-chronization task, the data syn-chronization task distribution of token will can be provided with
To a thread in multiple threads, data syn-chronization task is performed according to token by the thread, for the data syn-chronization of different pieces of information
Task can be distributed to different thread process.
According to the method that the above embodiment of the present invention is provided, the data syn-chronization task for carrying out autonomous database is received, according to connecing
The order of data syn-chronization task is received, is that data syn-chronization task sets token, the data syn-chronization task that will be provided with token is distributed to
A thread in multiple threads, data syn-chronization task is performed by the thread according to token.Using such scheme of the present invention, from data
Storehouse has multiple threads to can be used for executed in parallel partial data synchronous task, improves data syn-chronization speed, overcomes from data
Storehouse single thread performs MDL and the dramatically different step problem of data from database that data syn-chronization task is caused, by setting
Token limits the execution sequence of data syn-chronization task, and serially performs data syn-chronization task according to token, it is ensured that with master data
The data in storehouse are consistent.
Fig. 2 shows that the flow of the concurrent master-slave synchronisation method based on token in accordance with another embodiment of the present invention is illustrated
Figure.As shown in Fig. 2 the method is comprised the following steps：
Step S200, receives the data syn-chronization task for carrying out autonomous database.
Data syn-chronization task is generated according to the data of write-in MDL, that is to say, that for write-in MDL
Any data will all generate corresponding data syn-chronization task.
For example, user writes data Set a=1 by client to MDL；Set b=2；Set c=1；
Set a=4, MDL will send 4 data synchronous task Set a=1 to from database respectively；Set b=2；Set c=
1；Set a=4, i.e. received from database and carry out the data syn-chronization task of autonomous database and be：Set a=1；Set b=2；
Set c=1；Set a=4.
Step S201, is that data syn-chronization task sets token according to the order for receiving data syn-chronization task.
Wherein, can be sent to data syn-chronization task from database in real time by MDL, that is to say, that can from database
Constantly to receive the data syn-chronization task of MDL transmission, MDL is the order that MDL is write according to data
Data syn-chronization task is sent to from database, therefore, when token is set for data syn-chronization task, it is also desirable to according to reception data
The order of synchronous task sets token.For example, MDL order have sent 4 data syn-chronizations times to from database
Business, respectively：Set a=1；Set b=2；Set c=1；Set a=4, then, the data syn-chronization received from database is appointed
Business order be：Set a=1；Set b=2；Set c=1；Set a=4, so, can be same according to data are received from database
The order of step task, is that data syn-chronization task sets token, for example, be respectively provided with 4 tokens of data synchronous task for 1,2,
3、4。
Step S202, according to load balancing, a thread is selected from multiple threads.
Wherein, load balancing refers to that the business load pressure of multiple threads is balanced, without the industry of some threads
Business load pressure is very big, and the situation of the business load pressure very little of some threads, according to the business load situation distribution of thread
Data syn-chronization task.
, it is necessary to data syn-chronization task is distributed into thread after token is set for data syn-chronization task, number is performed by thread
According to synchronous task, there are multiple threads to perform data syn-chronization task from database, need exist for being selected from multiple threads
One suitable thread performs data syn-chronization task, and the foundation for selecting thread is load balancing, in order to balance
Business load pressure between each thread so that the business load pressure basis equalization of each thread.
Step S203, the data syn-chronization task that will be provided with token is distributed to selected thread.
According to load balancing, after selecting a thread from multiple threads, the data syn-chronization that will be arranged at token is appointed
Business is distributed to selected thread, and data syn-chronization task is performed by the thread.
Wherein, data syn-chronization task includes：Write data into the data writing operation from database, and by data writing operation
Recorded in journal file and write journalizing.Journal file is recorded for record data write operation, with the number from database
During according to the problems such as going out active, data writing operation that can easily in journal file records the number recovered from database
According to.It is therefore desirable to the data writing operation record in recorded journal file is performed according to the order for receiving data syn-chronization task
's.
The data syn-chronization task received from database can be multiple, that is to say, that can be selected according to load balancing
Select multiple threads and perform data syn-chronization task, wherein, the data writing operation performed between each thread is parallel work-flow；Each line
What is performed between journey writes journalizing for serial operation.For different pieces of information, each thread can be write data into executed in parallel
From the data writing operation of database, without being had any impact to the data write from database, however, being directed to same data
Data syn-chronization task be distributed to same thread, that is to say, that for same data, write data into from database and write data
Operation should also be what is serially performed by thread, it is therefore an objective to prevents from being distributed to different thread process, causes the number from database
Occur according to the situation inconsistent with the data of master data.
Additionally, each thread respectively defines the function that can perform parallel work-flow and serial operation, for example, performing
The function of parallel work-flow be function1, perform serial operation function be function2, that is to say, that write data into from
The daily record of writing that the data writing operation of database can recorded in journal file by function1 execution, by data writing operation is grasped
Work can be performed by function2, and thread is after data syn-chronization task is received, it will by meeting function1, function2
To perform data syn-chronization task.
Step S204, the data writing operation write data into from database is performed by the thread.
For example, being directed to data syn-chronization task：Set a=1；Set b=2；Set c=1；Set a=4, can be with
According to load balancing by data syn-chronization task Set a=1；Set a=4 are distributed to thread 1, data syn-chronization task Set b
=2 are distributed to thread 2, and data syn-chronization task Set c=1 are distributed to thread 3, wherein, for data syn-chronization task in by number
According to the data writing operation write from database, data writing operation with executed in parallel, and can be recorded daily record by first thread 1,2,3
Journalizing of writing in file then needs first thread 1,2,3 serially to perform.Thread 1,2,3 records data writing operation in execution
To in journal file when writing journalizing, it is necessary to performed according to token, specifically, can be held using following method and step
It is about to data writing operation and recorded in journal file write journalizing.
Step S205, the journalizing of writing of the corresponding data syn-chronization task of token that judgement order comes the previous position of token is
It is no to perform completion, if so, then performing step S206；If it is not, then performing step S207.
Wherein, token put in order and receive data syn-chronization task sequence consensus.
Specifically, function2 perform data writing operation be recorded into journal file in write journalizing before, also
Need to judge that what order came the corresponding data syn-chronization task of token of the previous position of token writes whether journalizing performs completion, letter
The code of number function2 can realize arbitration functions, and the order of judgement here comes the corresponding data of token of the previous position of token
Synchronous task write journalizing whether perform completion primarily to ensure write journalizing be according to receive data syn-chronization appoint
What the order of business was performed, it is ensured that when the problems such as data going out active from database, can easily according in journal file
Data writing operation record recover from the data of database.
For example, from database be data syn-chronization task：Set a=1；Set b=2；Set c=1；Set a=4,
It is 1,2,3,4 to be respectively provided with token.Thread 2 is when execution recorded in journal file Set b=2, it is necessary to judge token 1 pair
Whether the data syn-chronization task answered performs completion, and similarly, thread 3 is needed when execution recorded in journal file Set c=1
Judge whether token 1,2 corresponding data syn-chronization tasks perform completion, similarly, thread 1 is being performed Set a=4 records
, it is necessary to judge whether token 1,2,3 corresponding data syn-chronization tasks perform completion during in journal file.
Step S206, to perform and write journalizing for data writing operation to recorded in journal file.
If judging, the journalizing of writing of the corresponding data syn-chronization task of token that order comes the previous position of token has been performed
Into then thread can be performed and write journalizing for data writing operation to recorded in journal file.
Step S207, continues waiting for.
If judging, the journalizing of writing of the corresponding data syn-chronization task of token that order comes the previous position of token is not carried out
Complete, then need to continue waiting for, the daily record of writing that wait order comes the corresponding data syn-chronization task of token of the previous position of token is grasped
After the completion of performing, just perform corresponding recorded in journal file data writing operation of the token and write journalizing.
According to the method that the above embodiment of the present invention is provided, there are multiple threads to can be used for executed in parallel part from database
Data syn-chronization task, improves data syn-chronization speed, overcomes from database single thread and performs the master that data syn-chronization task is caused
Database and the dramatically different step problem of data from database, are that data syn-chronization is appointed according to the order for receiving data syn-chronization task
Business sets token, and according to load balancing, a thread is selected from multiple threads, and the data syn-chronization that will be provided with token is appointed
Business is distributed to selected thread, ensure that the payload traffic pressure of each thread is balanced, negative without some threads
Carry traffic pressure excessive, the too small situation of some threads load pressure, the token that judgement order comes the previous position of token is corresponding
Data syn-chronization task writes whether journalizing performs completion, if so, then performing for data writing operation to recorded into daily record text
Journalizing is write in part；If it is not, then continuing waiting for so that the journalizing of writing that recorded journal file is according to reception data
What synchronous task order strict sequential was performed, it is ensured that when the problems such as data going out active from database, can be easily
Data writing operation record in journal file recovers the data from database.
Fig. 3 shows the structural representation of the concurrent master-slave synchronisation device based on token according to an embodiment of the invention
Figure.As shown in figure 3, the device includes：Receiver module 300, setup module 310, distribution module 320, multiple parallel processing modules
330。
Receiver module 300, is suitable to receive the data syn-chronization task of autonomous database.
Setup module 310, is suitable to according to the order for receiving data syn-chronization task, is that data syn-chronization task sets token.
Distribution module 320, the data syn-chronization task for being suitable to be provided with token is distributed to one in multiple parallel processing modules
Individual parallel processing module.
Multiple parallel processing modules 330, are suitable to perform data syn-chronization task according to token.
Alternatively, distribution module 320 is further adapted for：According to load balancing, from the selection of multiple parallel processing modules
One parallel processing module；The data syn-chronization task that token will be provided with is distributed to selected parallel processing module.
Alternatively, data syn-chronization task includes：The data writing operation from database is write data into, and data behaviour will be write
Note down and write journalizing in journal file.
Alternatively, the data writing operation for being performed between each parallel processing module is parallel work-flow；Each parallel processing mould
What is performed between block writes journalizing for serial operation.
Alternatively, parallel processing module 330 is further adapted for：Judgement order comes the corresponding number of token of the previous position of token
Write whether journalizing performs completion according to synchronous task；Wherein, token put in order and receive data syn-chronization task it is suitable
Sequence is consistent；If so, then to perform write journalizing for data writing operation to recorded in journal file；If it is not, then continue etc.
Treat.
Alternatively, distribution module is further adapted for：Data syn-chronization task for same data is distributed to same thread.
According to the device that the above embodiment of the present invention is provided, there are multiple threads to can be used for executed in parallel part from database
Data syn-chronization task, improves data syn-chronization speed, overcomes from database single thread and performs the master that data syn-chronization task is caused
Database and the dramatically different step problem of data from database, are that data syn-chronization is appointed according to the order for receiving data syn-chronization task
Business sets token, and according to load balancing, a thread is selected from multiple threads, and the data syn-chronization that will be provided with token is appointed
Business is distributed to selected thread, and the daily record of writing that judgement order comes the corresponding data syn-chronization task of token of the previous position of token is grasped
Whether completion is performed, if so, then to perform write journalizing for data writing operation to recorded in journal file；If it is not,
Then continue waiting for, it is ensured that the journalizing of writing that recorded journal file is according to reception data syn-chronization task order strict sequential
Perform, it is ensured that when the problems such as data going out active from database, easily can write number in journal file
Recover the data from database according to operation note.
Algorithm and display be not inherently related to any certain computer, virtual system or miscellaneous equipment provided herein.
Various general-purpose systems can also be used together with based on teaching in this.As described above, construct required by this kind of system
Structure be obvious.Additionally, the present invention is not also directed to any certain programmed language.It is understood that, it is possible to use it is various
Programming language realizes the content of invention described herein, and the description done to language-specific above is to disclose this hair
Bright preferred forms.
In specification mentioned herein, numerous specific details are set forth.It is to be appreciated, however, that implementation of the invention
Example can be put into practice in the case of without these details.In some instances, known method, structure is not been shown in detail
And technology, so as not to obscure the understanding of this description.
Similarly, it will be appreciated that in order to simplify one or more that the disclosure and helping understands in each inventive aspect, exist
Above to the description of exemplary embodiment of the invention in, each feature of the invention is grouped together into single implementation sometimes
In example, figure or descriptions thereof.However, the method for the disclosure should be construed to reflect following intention：I.e. required guarantor
The application claims of shield features more more than the feature being expressly recited in each claim.More precisely, such as following
Claims reflect as, inventive aspect is all features less than single embodiment disclosed above.Therefore,
Thus the claims for following specific embodiment are expressly incorporated in the specific embodiment, and wherein each claim is in itself
All as separate embodiments of the invention.
Those skilled in the art are appreciated that can be carried out adaptively to the module in the equipment in embodiment
Change and they are arranged in one or more equipment different from the embodiment.Can be the module or list in embodiment
Unit or component be combined into a module or unit or component, and can be divided into addition multiple submodule or subelement or
Sub-component.In addition at least some in such feature and/or process or unit exclude each other, can use any
Combine to all features disclosed in this specification (including adjoint claim, summary and accompanying drawing) and so disclosed appoint
Where all processes or unit of method or equipment are combined.Unless expressly stated otherwise, this specification (including adjoint power
Profit is required, summary and accompanying drawing) disclosed in each feature can the alternative features of or similar purpose identical, equivalent by offer carry out generation
Replace.
Although additionally, it will be appreciated by those of skill in the art that some embodiments described herein include other embodiments
In included some features rather than further feature, but the combination of the feature of different embodiments means in of the invention
Within the scope of and form different embodiments.For example, in the following claims, embodiment required for protection is appointed
One of meaning mode can be used in any combination.
All parts embodiment of the invention can be realized with hardware, or be run with one or more processor
Software module realize, or with combinations thereof realize.It will be understood by those of skill in the art that can use in practice
Microprocessor or digital signal processor (DSP) realize the concurrent master-slave synchronisation based on token according to embodiments of the present invention
The some or all functions of some or all parts in equipment.The present invention is also implemented as being retouched here for execution
Some or all equipment or program of device (for example, computer program and computer program product) of the method stated.
It is such to realize that program of the invention be stored on a computer-readable medium, or can have one or more signal
Form.Such signal can be downloaded from internet website and obtained, or on carrier signal provide, or with it is any its
He provides form.
It should be noted that above-described embodiment the present invention will be described rather than limiting the invention, and ability
Field technique personnel can design alternative embodiment without departing from the scope of the appended claims.In the claims,
Any reference symbol being located between bracket should not be configured to limitations on claims.Word "comprising" is not excluded the presence of not
Element listed in the claims or step.Word "a" or "an" before element is not excluded the presence of as multiple
Element.The present invention can come real by means of the hardware for including some different elements and by means of properly programmed computer
It is existing.If in the unit claim for listing equipment for drying, several in these devices can be by same hardware branch
To embody.The use of word first, second, and third does not indicate that any order.These words can be explained and run after fame
Claim.
The invention discloses：
A1, a kind of concurrent master-slave synchronisation method based on token, it includes：
Receive the data syn-chronization task for carrying out autonomous database；
It is that the data syn-chronization task sets token according to the order for receiving data syn-chronization task；
The data syn-chronization task of token will be provided with and be distributed to a thread in multiple threads, held according to token by the thread
The row data syn-chronization task.
A2, the concurrent master-slave synchronisation method based on token according to A1, wherein, the data that token will be provided with
A thread is further included during synchronous task is distributed to multiple threads：
According to load balancing, a thread is selected from multiple threads；
The data syn-chronization task that token will be provided with is distributed to selected thread.
A3, the concurrent master-slave synchronisation method based on token according to A1, wherein, the data syn-chronization task includes：
The data writing operation from database is write data into, and by write data operation note to journal file
Write journalizing.
A4, the concurrent master-slave synchronisation method based on token according to A3, wherein, what is performed between each thread is described
Data writing operation is parallel work-flow；The journalizing of writing performed between each thread is for serial operation.
A5, the concurrent master-slave synchronisation method based on token according to A1, wherein, it is described to be held according to token by the thread
The row data syn-chronization task is further included：
Judge that order comes the token corresponding data syn-chronization task of the previous position of the token and writes whether journalizing is held
Row is completed；Wherein, the token put in order and receive data syn-chronization task sequence consensus；
If so, then performing for journalizing will to be write in write data operation note to journal file；
If it is not, then continuing waiting for.
A6, the concurrent master-slave synchronisation method based on token according to A1, wherein, for the data syn-chronization of same data
Task is distributed to same thread.
B7, a kind of concurrent master-slave synchronisation device based on token, it includes：
Receiver module, is suitable to receive the data syn-chronization task of autonomous database；
Setup module, is suitable to according to the order for receiving data syn-chronization task, is that the data syn-chronization task sets token；
Distribution module, is suitable to be provided with the data syn-chronization task of token and is distributed in multiple parallel processing modules one simultaneously
Row processing module；
Multiple parallel processing modules, are suitable to perform the data syn-chronization task according to token.
B8, the concurrent master-slave synchronisation device based on token according to B7, wherein, the distribution module is further fitted
In：According to load balancing, a parallel processing module is selected from multiple parallel processing modules；The data of token will be provided with
Synchronous task is distributed to selected parallel processing module.
B9, the concurrent master-slave synchronisation device based on token according to B7, wherein, the data syn-chronization task includes：
The data writing operation from database is write data into, and by write data operation note to journal file
Write journalizing.
B10, the concurrent master-slave synchronisation device based on token according to B9, wherein, between each parallel processing module
The write data operation of execution is parallel work-flow；It is serial to write journalizing described in being performed between each parallel processing module
Operation.
B11, the concurrent master-slave synchronisation device based on token according to B7, wherein, the parallel processing module enters one
Step is suitable to：
Judge that order comes the token corresponding data syn-chronization task of the previous position of the token and writes whether journalizing is held
Row is completed；Wherein, the token put in order and receive data syn-chronization task sequence consensus；
If so, then performing for journalizing will to be write in write data operation note to journal file；
If it is not, then continuing waiting for.
B12, the concurrent master-slave synchronisation device based on token according to B7, wherein, the distribution module is further fitted
In：Data syn-chronization task for same data is distributed to same thread.
Claims (10)
1. a kind of concurrent master-slave synchronisation method based on token, it includes：
Receive the data syn-chronization task for carrying out autonomous database；
It is that the data syn-chronization task sets token according to the order for receiving data syn-chronization task；
The data syn-chronization task of token will be provided with and be distributed to a thread in multiple threads, institute is performed according to token by the thread
State data syn-chronization task.
2. the concurrent master-slave synchronisation method based on token according to claim 1, wherein, the number that token will be provided with
A thread is further included in being distributed to multiple threads according to synchronous task：
According to load balancing, a thread is selected from multiple threads；
The data syn-chronization task that token will be provided with is distributed to selected thread.
3. the concurrent master-slave synchronisation method based on token according to claim 1, wherein, the data syn-chronization task bag
Include：
The data writing operation from database is write data into, and day will be write in write data operation note to journal file
Will is operated.
4. the concurrent master-slave synchronisation method based on token according to claim 3, wherein, the institute performed between each thread
Data writing operation is stated for parallel work-flow；The journalizing of writing performed between each thread is for serial operation.
5. the concurrent master-slave synchronisation method based on token according to claim 1, wherein, it is described by the thread according to token
The data syn-chronization task is performed to further include：
Judge that order comes the token corresponding data syn-chronization task of the previous position of the token and writes whether journalizing has performed
Into；Wherein, the token put in order and receive data syn-chronization task sequence consensus；
If so, then performing for journalizing will to be write in write data operation note to journal file；
If it is not, then continuing waiting for.
6. the concurrent master-slave synchronisation method based on token according to claim 1, wherein, the data for same data are same
Step task is distributed to same thread.
7. a kind of concurrent master-slave synchronisation device based on token, it includes：
Receiver module, is suitable to receive the data syn-chronization task of autonomous database；
Setup module, is suitable to according to the order for receiving data syn-chronization task, is that the data syn-chronization task sets token；
Distribution module, is suitable to be provided with the data syn-chronization task of token and is distributed to a parallel place in multiple parallel processing modules
Reason module；
Multiple parallel processing modules, are suitable to perform the data syn-chronization task according to token.
8. the concurrent master-slave synchronisation device based on token according to claim 7, wherein, the distribution module is further fitted
In：According to load balancing, a parallel processing module is selected from multiple parallel processing modules；The data of token will be provided with
Synchronous task is distributed to selected parallel processing module.
9. the concurrent master-slave synchronisation device based on token according to claim 7, wherein, the data syn-chronization task bag
Include：
The data writing operation from database is write data into, and day will be write in write data operation note to journal file
Will is operated.
10. the concurrent master-slave synchronisation device based on token according to claim 9, wherein, each parallel processing module it
Between perform write data operation be parallel work-flow；The journalizing of writing performed between each parallel processing module is string
Row operation.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611046602.6A CN106776790A (en) | 2016-11-11 | 2016-11-11 | Concurrent master-slave synchronisation method and device based on token |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611046602.6A CN106776790A (en) | 2016-11-11 | 2016-11-11 | Concurrent master-slave synchronisation method and device based on token |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106776790A true CN106776790A (en) | 2017-05-31 |
Family
ID=58974408
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611046602.6A Pending CN106776790A (en) | 2016-11-11 | 2016-11-11 | Concurrent master-slave synchronisation method and device based on token |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106776790A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109284193A (en) * | 2018-09-06 | 2019-01-29 | 平安科技（深圳）有限公司 | A kind of distributed data processing method and server based on multithreading |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050033947A1 (en) * | 2003-08-08 | 2005-02-10 | Morris Dale C. | Multiprocessor system with interactive synchronization of local clocks |
| CN101625655A (en) * | 2009-08-20 | 2010-01-13 | 华中科技大学 | Parallel recovery method of memory database |
| CN103729239A (en) * | 2013-11-18 | 2014-04-16 | 芜湖大学科技园发展有限公司 | Distributed type lock algorithm of mirror-image metadata |
| CN103885986A (en) * | 2012-12-21 | 2014-06-25 | 阿里巴巴集团控股有限公司 | Main and auxiliary database synchronization method and device |
| CN104731956A (en) * | 2015-04-02 | 2015-06-24 | 北京奇虎科技有限公司 | Method and system for synchronizing data and related database |
-
2016
- 2016-11-11 CN CN201611046602.6A patent/CN106776790A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050033947A1 (en) * | 2003-08-08 | 2005-02-10 | Morris Dale C. | Multiprocessor system with interactive synchronization of local clocks |
| CN101625655A (en) * | 2009-08-20 | 2010-01-13 | 华中科技大学 | Parallel recovery method of memory database |
| CN103885986A (en) * | 2012-12-21 | 2014-06-25 | 阿里巴巴集团控股有限公司 | Main and auxiliary database synchronization method and device |
| CN103729239A (en) * | 2013-11-18 | 2014-04-16 | 芜湖大学科技园发展有限公司 | Distributed type lock algorithm of mirror-image metadata |
| CN104731956A (en) * | 2015-04-02 | 2015-06-24 | 北京奇虎科技有限公司 | Method and system for synchronizing data and related database |
Non-Patent Citations (2)
| Title |
|---|
| 何耀钦，薛华成: "《数据组织与管理》", 31 May 1992, 机械工业出版社 * |
| 徐高潮等: "《分布计算系统》", 31 January 2004, 高等教育出版社 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109284193A (en) * | 2018-09-06 | 2019-01-29 | 平安科技（深圳）有限公司 | A kind of distributed data processing method and server based on multithreading |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3698294B1 (en) | Machine learning runtime library for neural network acceleration | |
| CN104407945B (en) | The method, apparatus and system of data backup | |
| CN104317749B (en) | Information write-in method and device | |
| CN102947818A (en) | Neural processing unit | |
| CN106648664A (en) | Self-service terminal business process updating method and apparatus | |
| CN106445951A (en) | File transmission method and apparatus | |
| CN106775970B (en) | Window operation synchronous method, device and the electronic equipment of striding course | |
| CN105700956A (en) | Distributed job processing method and system | |
| CN106776775A (en) | Concurrent master-slave synchronisation method and device | |
| CN106775951A (en) | A kind of operation management method and device of Android application | |
| WO2015186278A1 (en) | Attribute enumeration system, attribute enumeration method, and attribute enumeration program | |
| CN108008959A (en) | A kind of Software Development Kit SDK cut-in methods, system and device | |
| CN104506636B (en) | A kind of method of data synchronization and device | |
| CN106776790A (en) | Concurrent master-slave synchronisation method and device based on token | |
| EP2871799A1 (en) | Method and apparatus for processing shuffle instructions | |
| CN105389394A (en) | Data request processing method and device based on a plurality of database clusters | |
| CN109491807A (en) | Data exchange method, device and system | |
| JP6492604B2 (en) | Grouping support program, grouping support method, and grouping support apparatus | |
| JP2009271572A (en) | Control system development support device | |
| CN103530742B (en) | Improve the method and device of scheduling arithmetic speed | |
| CN107193650A (en) | A kind of method and apparatus that video card resource is dispatched in distributed type assemblies | |
| US20150254288A1 (en) | Sorting database collections for parallel processing | |
| CN109241073A (en) | A method of allow artificial intelligence robot that there is human emotion | |
| CN107203461A (en) | compatibility test method and device | |
| CN107135191A (en) | Check that distributed service handles the method and device of integrity degree |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170531 |