CN106101276B - A kind of cluster load balancing method and device - Google Patents
A kind of cluster load balancing method and device Download PDFInfo
- Publication number
- CN106101276B CN106101276B CN201610654053.4A CN201610654053A CN106101276B CN 106101276 B CN106101276 B CN 106101276B CN 201610654053 A CN201610654053 A CN 201610654053A CN 106101276 B CN106101276 B CN 106101276B
- Authority
- CN
- China
- Prior art keywords
- node
- load value
- current
- load
- value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/104—Peer-to-peer [P2P] networks
- H04L67/1044—Group management mechanisms
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Theoretical Computer Science (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Supply And Distribution Of Alternating Current (AREA)
Abstract
The present invention provides a kind of cluster load balancing method and devices, wherein this method comprises: receiving the task that user sends；It obtains in current period in cluster the current load value of each node and each node and the current steady load value of each node is determined according to current load value and the steady load value of previous cycle in the steady load value of previous cycle；According to current steady load value and the corresponding load changing value of above-mentioned task, the prediction load value of each node is determined；Above-mentioned task is distributed to the smallest node of ratio of prediction load value and maximum acceptable load value.Cluster load balancing method and device provided by the invention are suitable for video field of storage, and the effect that flocks together can be effectively avoided the occurrence of in carry out task distribution, also, are suitable for passing node equipments and configure different clusters.
Description
Technical field
The present invention relates to video storage and load-balancing technique fields, in particular to a kind of cluster load balance side
Method and device.
Background technique
Load balancing is one kind of clustering functionality, i.e., load pressure is reasonably allocated to according to certain algorithm certain of cluster
On machine, the pressure of every server is balanced, so that server resource is fully used, validity determines the entirety of cluster
Performance and resource utilization.
Existing load-balancing algorithm is mostly to be suitable for configuring identical computer, the unconspicuous cluster system of the fluctuation of load
System.When load balancer, which receives, endlessly requests, load balancer forwards a request to phase according to its load strategy
The server answered, load strategy used by load balancer can be random algorithm, round robin algorithm, in average load algorithm
It is a kind of.
But in video field of storage, load observation value is mainly I/O (input/output, the input/defeated of disk
Out), and the I/O of disk fluctuation is very big, and need certain time that could reflect the variation of load value the load of node, because
This, when video instantaneously in high volume arrives, node load value at this time can not timely update, and cause all videos to store and divide
The minimum node of the load value that dispensing is thought at the moment, that is, lead to the effect that flocks together, in video field of storage, the service of each node
Device configuration may be not identical, and therefore, load-balancing algorithm in the prior art is in video field of storage and is not suitable for.
Summary of the invention
In view of this, the embodiment of the present invention is designed to provide a kind of cluster load balancing method and device, to solve
Existing load-balancing method, which is applied, is easy to cause the problem of flocking together effect in video field of storage.
In a first aspect, the embodiment of the invention provides a kind of cluster load balancing methods, wherein the described method includes:
Receive the task that user sends；
The current load value of each node and each node are obtained in current period in cluster in the steady of previous cycle
Fixed load value determines the current steady of each node according to the current load value and the steady load value of previous cycle
Load value；
According to the current steady load value and the corresponding load changing value of the task, the pre- of each node is determined
Survey load value；
The task is distributed to the smallest node of ratio of the prediction load value and maximum acceptable load value.
With reference to first aspect, the embodiment of the invention provides the first possible implementation of above-mentioned first aspect,
In, it is described by the task distribute to it is described prediction load value and maximum acceptable load value the smallest node of ratio it
Afterwards, further includes:
The prediction load value of each node after the distribution of acquisition task, each node is pre- after distribution of computation tasks
Survey the ratio of load value and the maximum acceptable load value；
According to the prediction load of each node after the maximum acceptable load value of each node and task distribution
Value calculates the remaining load value that the ratio is less than or equal to the node of default load threshold；
The task of the smallest node of the remaining load value is transferred to the maximum knot of the remaining load value
Point.
With reference to first aspect, the embodiment of the invention provides second of possible implementation of above-mentioned first aspect,
In, it is described according to the current load value and the steady load value of previous cycle, determine that the current steady of each node is negative
Load value, comprising:
Calculate each node current load value and the node previous cycle steady load value it is first poor
Value；
By the current steady load value of each node and the node the steady load value of previous cycle difference
It is denoted as the second difference, according to the proportionate relationship between first difference and second difference, determines each node
Current steady load value.
With reference to first aspect, the embodiment of the invention provides the third possible implementation of above-mentioned first aspect,
In, it is described according to the current load value and the steady load value of previous cycle, determine that the current steady of each node is negative
Load value, comprising:
According to the current load value of each node, each node passes through in the steady load value of previous cycle
Formula (1) calculates the current steady load value of each node；
Nm=(nm-Nm-1)/d+Nm-1 (1)
Wherein, in formula (1), NmFor the current steady load value of any node in the cluster, Nm-1It is described any
Steady load value of the node in previous cycle, nmFor the current load value of any node, d is pre-set ratio, and m is the period.
With reference to first aspect, the embodiment of the invention provides the 4th kind of possible implementation of above-mentioned first aspect,
In, it is described according to the current steady load value and the corresponding load changing value of the task, determine the pre- of each node
Survey load value, comprising:
According to the current steady load value of each node and the corresponding load changing value of the task, pass through formula
(2) the prediction load value of each node is calculated；
Fm=Dm+Nm (2)
Wherein, in formula (2), FmFor the prediction load value of any node in the cluster, DmIt is corresponding for the task
Load changing value, NmFor the current steady load value of any node, m is the period.
Second aspect, the embodiment of the invention provides a kind of cluster load balance device, described device includes:
Receiving module, for receiving the task of user's transmission；
First determining module, for obtaining in current period in cluster the current load value of each node and each described
Node determines each in the steady load value of previous cycle according to the current load value and the steady load value of previous cycle
The current steady load value of the node；
Second determining module is used for according to the current steady load value and the corresponding load changing value of the task, really
The prediction load value of fixed each node；
Task allocating module, for the task to be distributed to the ratio of the prediction load value and maximum acceptable load value
It is worth the smallest node.
In conjunction with second aspect, the embodiment of the invention provides the first possible implementation of above-mentioned second aspect,
In, described device further include:
First computing module, for obtaining the prediction load value of each node after task is distributed, distribution of computation tasks
The ratio of the prediction load value and the maximum acceptable load value of each node afterwards；
Second computing module, for according to each institute after the distribution of the maximum acceptable load value and task of each node
The prediction load value for stating node calculates the remaining load value that the ratio is less than or equal to the node of default load threshold；
Task shift module, for the task of the smallest node of the remaining load value to be transferred to the residual negative
Load is worth the maximum node.
In conjunction with second aspect, the embodiment of the invention provides second of possible implementation of above-mentioned second aspect,
In, first determining module includes:
First computing unit, for calculating the current load value of each node with the node in the steady of previous cycle
First difference of fixed load value；
Determination unit, for by the current steady load value of each node and the node previous cycle stabilization
The difference of load value is denoted as the second difference, according to the proportionate relationship between first difference and second difference, determines each
The current steady load value of a node.
In conjunction with second aspect, the embodiment of the invention provides the third possible implementation of above-mentioned second aspect,
In, first determining module further include:
Second computing unit, for the current load value according to each node, each node is in previous cycle
Steady load value, pass through the current steady load value that formula (1) calculates each node；
Nm=(nm-Nm-1)/d+Nm-1 (1)
Wherein, in formula (1), NmFor the current steady load value of any node in the cluster, Nm-1It is described any
Steady load value of the node in previous cycle, nmFor the current load value of any node, d is pre-set ratio, and m is the period.
In conjunction with second aspect, the embodiment of the invention provides the 4th kind of possible implementation of above-mentioned second aspect,
In, second determining module includes:
Third computing unit, for according to each node current steady load value and the corresponding load of the task
Changing value calculates the prediction load value of each node by formula (2)；
Fm=Dm+Nm (2)
Wherein, in formula (2), FmFor the prediction load value of any node in the cluster, DmIt is corresponding for the task
Load changing value, NmFor the current steady load value of any node, m is the period.
Cluster load balancing method and device provided in an embodiment of the present invention are suitable for video field of storage, are being appointed
The effect that flocks together can be effectively avoided the occurrence of when business distribution, also, is suitable for passing node equipments and is configured different clusters.
To enable the above objects, features and advantages of the present invention to be clearer and more comprehensible, preferred embodiment is cited below particularly, and cooperate
Appended attached drawing, is described in detail below.
Detailed description of the invention
In order to illustrate the technical solution of the embodiments of the present invention more clearly, below will be to needed in the embodiment attached
Figure is briefly described, it should be understood that the following drawings illustrates only certain embodiments of the present invention, therefore is not construed as pair
The restriction of range for those of ordinary skill in the art without creative efforts, can also be according to this
A little attached drawings obtain other relevant attached drawings.
Fig. 1 shows a kind of flow chart of cluster load balancing method provided by the embodiment of the present invention 1；
Fig. 2 shows a kind of structural schematic diagrams of cluster load balance device provided by the embodiment of the present invention 2.
Specific embodiment
In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with the embodiment of the present invention
Middle attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is only
It is a part of the embodiment of the present invention, instead of all the embodiments.The present invention being usually described and illustrated herein in the accompanying drawings is real
The component for applying example can be arranged and be designed with a variety of different configurations.Therefore, of the invention to what is provided in the accompanying drawings below
The detailed description of embodiment is not intended to limit the range of claimed invention, but is merely representative of selected reality of the invention
Apply example.Based on the embodiment of the present invention, those skilled in the art institute obtained without making creative work
There are other embodiments, shall fall within the protection scope of the present invention.
In view of in the related technology, load balancer is mostly using random algorithm, round robin algorithm or average load algorithm
Corresponding server or equipment are requested assignment to by what is received, still, in video field of storage, load observation value is mainly magnetic
The I/O of disk, and the I/O of disk fluctuation is very big, and needs certain time that could reflect the change of load value the load of node
Change, therefore, when video instantaneously in high volume arrives, node load value at this time can not timely update, and lead to all videos
The minimum node of the load value thought at the moment is distributed in storage, that is, leads to the effect that flocks together, in video field of storage, each node
Server configuration may be not identical, therefore, load-balancing algorithm in the prior art is in video field of storage and is not suitable for.
Based on this, the embodiment of the invention provides a kind of cluster load balancing method and devices, are described below by embodiment.
Embodiment 1
The embodiment of the invention provides a kind of cluster load balancing method, this method is suitable for video field of storage, and
The effect that flocks together can effectively be avoided.
As shown in Figure 1, cluster load balancing method provided in an embodiment of the present invention, including step S110-S140.
S110 receives the task that user sends.
Above-mentioned task can be video, when above-mentioned task is video camera when executing the video of video recording plan acquisition, camera shooting
Machine needs to store the transmission of video of acquisition to storage equipment, which can be disk, hard disk etc., should
Before video is stored, the video task is sent to load balancer first, load balancer receives camera transmissions
Video task, and the video task is saved in set of tasks.Certainly, above-mentioned task can also be other in addition to video
Request etc., the embodiment of the present invention does not limit the concrete type of above-mentioned task.
S120 obtains in current period in cluster the current load value of each node and each node in previous cycle
Steady load value determines the current steady load value of each node according to current load value and the steady load value of previous cycle.
Above-mentioned cluster refers to that more machines cooperate jointly, and the multicomputer system externally provided uniform services is above-mentioned more
Platform machine can be multiple servers, be also possible to multiple computers, multiple disks, hard disk etc., each of above-mentioned cluster
Equipment is exactly a node.
During each node in the cluster is worked, load balancer can the real-time or each node of taken at regular intervals
Current load value, which refers to the instant load value of each node, the current load value of above-mentioned each node
The I/O that can be the node, since load balancer is the current load value of each node in real-time or taken at regular intervals cluster,
It therefore can be using the current load value of each node in a load balancer cluster of every acquisition as a cycle.
It is steady in the first two period to be then to rely on each node in the steady load value of previous cycle for above-mentioned each node
Therefore fixed load value can set the incipient stability load value of each node, can be by the incipient stability load value of each node
Be set as 0, thus, it is possible in the cluster acquired according to incipient stability load value and in the period 1 each node present load
Value, calculates the steady load value of each node in the period 1, and so on, the stabilization of available above-mentioned previous cycle is negative
Load value.
When load balancer acquires the current load value of each node in current period, and each node is obtained preceding
After the steady load value in one period, according to the current load value of each node in above-mentioned current period and each node previous
The steady load value in period determines the current steady load value of each node, which specifically includes: calculating working as each node
The first difference of preceding load value and node in the steady load value of previous cycle；By the current steady load value and knot of each node
Point is denoted as the second difference in the difference of the steady load value of previous cycle, is closed according to the ratio between the first difference and the second difference
System, determines the current steady load value of each node.
Wherein, direct proportionality between above-mentioned first difference and the second difference, the preset ratio coefficient can be 16 or
Person 32, it is, of course, also possible to be other numerical value, the value of above-mentioned preset ratio coefficient depends on the load value of each node in cluster
The degree of fluctuation for reaching the load value of stable time and each node can be according to practical need in concrete application scene
Aforementioned proportion coefficient is chosen, the embodiment of the present invention does not limit the specific value of aforementioned proportion coefficient.
Wherein, in above-mentioned cluster each node current steady load value, can also determine by the following method, it is specific to wrap
Include: according to the current load value of each node, each node is calculated each in the steady load value of previous cycle by formula (1)
The current steady load value of a node；
Nm=(nm-Nm-1)/d+Nm-1 (1)
Wherein, in formula (1), NmFor the current steady load value of node any in cluster, Nm-1For any of the above-described node
In the steady load value of previous cycle, nmFor the current load value of any of the above-described node, d is pre-set ratio, and m is the period.
Above-mentioned d is pre-set ratio, and the value of the pre-set ratio is generally 16 or 32, it is, of course, also possible to be other numerical value, on
The value for stating pre-set ratio d reaches the load of stable time and each node depending on the load value of each node in cluster
The degree of fluctuation of value can choose above-mentioned pre-set ratio, the embodiment of the present invention is simultaneously in concrete application scene according to actual needs
The specific value of above-mentioned pre-set ratio is not limited.
In embodiments of the present invention, when calculating the current steady load value of each node in current period by formula (1),
Dependent on node each in cluster previous cycle steady load value, it is therefore desirable to obtain in cluster each node in the last week
The steady load value of phase, similarly, each node is in the acquisition of the steady load value of previous cycle in cluster, and need to rely on cluster
In steady load value of each node in the first two period, and so on, in order to calculate the current of each node in the current period
Steady load value needs to set the incipient stability load value of each node in cluster, by N0Be denoted as any node in cluster just
Beginning steady load value, and by N0Value be set as 0, after collecting the current load value of each node in the period 1, will
The current load value of any node is denoted as n in cluster in period 11, can be calculated in the period 1 and be appointed according to formula (1)
The current steady load value N of one node1, according to the current load value of each node collected in the period 1 and each knot
The initial load value of point can calculate current steady of each node within the period 1 in cluster by formula (1) and load
Value acquires the current load value of each node in cluster in second round, according to node each in cluster in second round later
Current load value and in the period 1 each node steady load value, can be calculated in second round by formula (1)
The current steady load value of each node in cluster, and so on, it is negative in the stabilization in a upper period according to node each in cluster
The current load value of load value and each node current period node calculates the steady load value of each node in current period,
Using the method for above-mentioned integral, the current load value for recording the acquisition of each period is not needed, a period is depended only on
The current load value of steady load value and current period interior knot, so that it may calculate working as each node in current period cluster
Preceding steady load value, calculation method is simpler, also, more steady using the steady load value of the calculated cluster of integration method
It is fixed.
S130 determines the prediction of each node according to current steady load value and the corresponding load changing value of above-mentioned task
Load value.
Wherein, the corresponding load changing value of above-mentioned task refers to that each node received task corresponds to task amount, than
As said, some node receives the video store tasks of a 1MB in cluster, then the corresponding load changing value of above-mentioned task is exactly
1MB, certainly, above-mentioned to be merely illustrative the corresponding load changing value of task, there is no the specific values for limiting load changing value.
In S120, although having calculated the steady load value of each node in current period by integration method,
But when new task reaches node, node cannot update current load value, therefore calculated current period interior knot at once
Steady load value may not be current node real load value, when the instantaneous task of high-volume arrives, load balancer
According to the load value of the current steady of node each in current period, task may be assigned on the same node, to lead
Cause flocks together effect, therefore, in order to solve the effect that flocks together, needs corresponding negative by current steady load value and received task
Changing value is carried, the prediction load value of each node is determined, specifically includes:
According to the current steady load value of each node and the corresponding load changing value of above-mentioned task, counted by formula (2)
Calculate the prediction load value of each node；
Fm=Dm+Nm (2)
Wherein, in formula (2), FmFor the prediction load value of each node, DmFor the corresponding load changing value of task, Nm
For the current steady load value of each node, m is the period.
Wherein, above-mentioned m indicates m-th of period.
In embodiments of the present invention, when by task load to node, the load variation of node can be estimated, knot will be loaded to
The corresponding task amount of task of point is as the node load variation estimated, i.e., the corresponding load of above-mentioned task changes, according to S120
In current steady load value and the corresponding load changing value of above-mentioned task in the current period that gets, pass through formula (2) and count
Calculate the prediction load value of each node.
When task load any one node into cluster, the corresponding prediction load value of node is current steady load value
The sum of load changing value corresponding with estimated task, in this way prediction load value FmCloser to true value, at this moment, task is corresponding negative
Carry variation DmPredicting function can weaken, DmVariation be Dm=Dm-1× (b-1)/b, wherein in the formula, DmIt is m-th week
The corresponding load changing value of task in phase, Dm-1For the corresponding load changing value of task in the m-1 period, b is present count
Value.
Above-mentioned prediction load value makes that the variation of the load value of each node can be embodied at once to after node distribution task
On, to the major part instantaneously submitted or even whole tasks will not all be distributed because the load value of each node changes slowly
Onto the same node, to solve the effect that flocks together, when task high-volume, instantaneous arrival, system can reach approximate stabilization
State.
Above-mentioned task is distributed to the smallest node of ratio of prediction load value and maximum acceptable load value by S140.
By the calculated prediction load value of S130, the effect that flocks together can solve, but matching due to each passing node equipments
Setting may be different, if only assigning the task to current predictive using the prediction load value of each node as load metric
The minimum node of load value, is inaccurate, and the cluster different for passing node equipments, the maximum acceptable load value of each node is not
Together, it is thus impossible to carry out task distribution only by prediction load value.
In embodiments of the present invention, can calculate each node prediction load value and each node it is maximum acceptable
The ratio of load value, using the ratio as the load metric of each node, when load balancer receives the task of user's submission
When, assign the task to the smallest node of above-mentioned ratio.
Wherein, above-mentioned maximum acceptable load value is the maximum load value that each passing node equipments are able to bear, and works as node
Task amount when being more than the maximum acceptable load value, node is by cisco unity malfunction.
In addition to this it is possible to load value can receive according to the maximum of the prediction load value of each node and each node,
By following formula, the remaining load value of each node is calculated；
Cm=M-Fm
Wherein, in above-mentioned formula, M is the maximum acceptable load value of any node in cluster, FmFor any node
Predict load value, CmFor any node remaining load value, m is the period.
After the remaining load value for calculating each node through the above way, using the maximum node of remaining load value as working as
The minimum node of preceding load, that is, assign the task to the maximum node of remaining load value.
After carrying out above-mentioned task distribution, possible cluster is not yet in equilibrium state, and therefore, it is necessary to the equal of cluster
Weighing apparatus degree is judged, and carries out task transfer to the node in cluster according to judging result.
The above-mentioned remaining load value for calculating each node, according to the variance of following formula computing clusters, according to cluster
Variance judges the balance degree of cluster, and the variance of cluster is smaller, it is believed that the balance degree of cluster is higher, i.e. the stability of cluster is got over
It is high；
S=(Cm1-T)2+(Cm2-T)2+...+(Cmk-T)2
Wherein, in above-mentioned formula, T is the average residual load value of cluster, Cm1For the residue of first node in cluster
Load value, Cm2For the remaining load value of second node in cluster, CmkFor the remaining load value of k-th node in cluster, S is
The variance of cluster, m are the period, and k is the number of node in cluster.
In embodiments of the present invention, variance yields can be preset, by the variance S of above-mentioned calculated cluster and default variance yields
It is compared, when above-mentioned variance S is greater than or equal to default variance yields, then judges that cluster current equalization degree is lower, need pair
Node in cluster carries out task transfer, specifically includes:
The prediction load value of each node after the distribution of acquisition task, the prediction load value of each node after distribution of computation tasks
With the ratio of maximum acceptable load value；According to each node after the distribution of the maximum acceptable load value and task of each node
Predict load value, the ratio that calculates is less than or equal to the remaining load value of the node of default load threshold；Remaining load value is minimum
The task of node be transferred to the maximum node of remaining load value.
In embodiments of the present invention, when the balance degree of cluster is lower, the task to node in cluster is needed to shift
When, it is necessary first to the prediction load value for obtaining each node after task distribution, since load balancer can real-time or taken at regular intervals
When the current load value of each node in cluster, i.e. progress task distribution, load balancer can also acquire each node in cluster
Current load value, the current load value of each node and each node are negative in the stabilization of previous cycle after being distributed according to task
Load value calculates the current steady load value of each node of current period, later, according to the current steady of each node of current period
Load value and the corresponding load changing value of task, the prediction load value of each node after distribution of computation tasks；
Then the maximum acceptable load value of the prediction load value and each node of each node after distribution of computation tasks
The ratio is compared by ratio with default load threshold, obtains the node that aforementioned proportion is less than or equal to default load threshold,
Task transfer is carried out to these nodes, is specifically included: calculating aforementioned proportion and is less than or equal to the surplus of the node for presetting load threshold
Above-mentioned node is ranked up by remaining load value according to remaining load value, and it is minimum that remaining load value is chosen from the node after sequence
With the maximum node of remaining load value, the task of the smallest node of remaining load value is transferred to the maximum knot of remaining load value
Point.
Above-mentioned default load threshold is pre-set numerical value, and it can also be other numerical value which, which can be 80%, above-mentioned
Default load threshold can be configured according to practical application, and it is above-mentioned default load threshold that the embodiment of the present invention, which does not limit,
Specific value.When the ratio of the maximum acceptable load value of the default load value and each node of calculated node is less than or waits
When above-mentioned default load threshold, then the task of the node is not shifted.
After load threshold, it can cross and reduce unnecessary task transfer, in addition, failing in node function still can not
When detecting, the remaining load value of failing node may be to be maximum in cluster at this time, if being not provided with load threshold, load is equal
The task of nodes other in cluster may be transferred to the failing node by weighing apparatus, and so as to cause black-hole effect, black-hole effect refers to
Be exactly node function failure when but can not detect its disabler, the current load value of node is zero at this time, and load is equal
Weighing apparatus thinks that the load of this node is minimum always, and load can all select this node every time.After being provided with load threshold, if collection
When the ratio of the maximum acceptable load value of Non-precondition load value and each node is greater than default load threshold in group, load is equal
The task of other nodes will not be transferred to failing node by weighing apparatus, to solve black-hole effect.
When carrying out the task transfer between node, such as, the highest node of remaining load value is A, and remaining load value is most
Low node is B, and in order to improve the balance degree of cluster, the task of node B can be transferred to node A, but if after transfer,
Node A may become the minimum node of remaining load value, and node B becomes the highest node of remaining load value, at this moment again need by
Task is transferred to node B from node A, and this transfer may infinitely go on, and this phenomenon is known as jolting.
In order to eliminate above-mentioned phenomenon of jolting, if the difference between maximum residual load value and most lower remaining load value does not surpass
The possible load capacity of maximum task received is crossed, then terminates the transfer of this subtask.
Cluster load balancing method provided in an embodiment of the present invention is suitable for video field of storage, is carrying out task distribution
When can effectively avoid the occurrence of the effect that flocks together, also, be suitable for passing node equipments and configure different clusters.
Embodiment 2
A kind of cluster load balance device provided in an embodiment of the present invention, load balancing apparatus provided in an embodiment of the present invention
For executing the cluster load balancing method of the offer of embodiment 1, it is equal which can be a kind of cluster load
Weighing apparatus.
As shown in Fig. 2, load balancing apparatus provided in an embodiment of the present invention, including receiving module 210, the first determining module
220, the second determining module 230 and task allocating module 240；
Above-mentioned receiving module 210, for receiving the task of user's transmission；
Above-mentioned first determining module 220, for obtaining the current load value of each node in cluster in current period, and it is each
A node determines each knot according to current load value and the steady load value of previous cycle in the steady load value of previous cycle
The current steady load value of point；
Above-mentioned second determining module 230 is used for according to current steady load value and the corresponding load changing value of above-mentioned task,
Determine the prediction load value of each node；
Above-mentioned task allocating module 240, for above-mentioned task to be distributed to prediction load value and maximum acceptable load value
The smallest node of ratio.
Above-mentioned cluster refers to that more machines cooperate jointly, and the multicomputer system externally provided uniform services is above-mentioned more
Platform machine can be multiple servers, be also possible to multiple computers, multiple disks, hard disk etc., each of above-mentioned cluster
Equipment is exactly a node.
In embodiments of the present invention, when received task is distributed to prediction load value and maximum acceptable load value
After the smallest node of ratio, in order to improve the balance degree of cluster, it is also necessary to be shifted to the task of node in cluster, be
By the first computing module, the second computing module and task shift module come what is realized, specifically include:
Above-mentioned first computing module, for obtaining the prediction load value of each node after task is distributed, distribution of computation tasks
The ratio of the prediction load value and maximum acceptable load value of each node afterwards；Above-mentioned second computing module, for according to each
The prediction load value of each node, the ratio that calculates are less than or equal to default after maximum acceptable load value and the task distribution of node
The remaining load value of the node of load threshold；Above-mentioned task shift module, for by the task of the smallest node of remaining load value
It is transferred to the maximum node of remaining load value.
Wherein, above-mentioned first determining module 220 determines each according to current load value and the steady load value of previous cycle
The current steady load value of node is to be realized by the first computing unit and determination unit, specifically includes:
Above-mentioned first computing unit, the current load value and node for calculating each node are negative in the stabilization of previous cycle
First difference of load value；Above-mentioned determination unit, for by the current steady load value and node of each node in previous cycle
The difference of steady load value is denoted as the second difference, according to the proportionate relationship between the first difference and the second difference, determines each knot
The current steady load value of point.
Wherein, above-mentioned first determining module 220 determines each according to current load value and the steady load value of previous cycle
The current steady load value of node can also be realized by the second computing unit, be specifically included:
Above-mentioned second determination unit, for the current load value according to each node, each node is in the steady of previous cycle
Fixed load value calculates the current steady load value of each node by formula (1)；
Nm=(nm-Nm-1)/d+Nm-1 (1)
Wherein, in formula (1), NmFor the current steady load value of each node, Nm-1It is each node in previous cycle
Steady load value, nmFor the current load value of each node, d is pre-set ratio, and m is the period.
Above-mentioned d is pre-set ratio, and the value of the pre-set ratio is generally 16 or 32, it is, of course, also possible to be other numerical value, on
The value for stating pre-set ratio d reaches the load of stable time and each node depending on the load value of each node in cluster
The degree of fluctuation of value can choose above-mentioned pre-set ratio, the embodiment of the present invention is simultaneously in concrete application scene according to actual needs
The specific value of above-mentioned pre-set ratio is not limited.
Wherein, above-mentioned second determining module 230 is according to current steady load value and the corresponding load changing value of above-mentioned task,
The prediction load value for determining each node is to be realized by third computing unit, specifically includes:
Above-mentioned third unit, for being changed according to the current steady load value of each node and the corresponding load of above-mentioned task
Value calculates the prediction load value of each node by formula (2)；
Fm=Dm+Nm (2)
Wherein, in formula (2), FmFor the prediction load value of each node, DmFor the corresponding load variation of above-mentioned task
Value, NmFor the current steady load value of each node.
Cluster load balance device provided in an embodiment of the present invention, including receiving module, the first determining module, the second determination
Module and task allocating module are suitable for video field of storage, can effectively avoid the occurrence of the effect that flocks together in carry out task distribution
It answers, also, is suitable for passing node equipments and configures different clusters.
Cluster load balance device provided by the embodiment of the present invention for the specific hardware in equipment or can be installed on
Software or firmware in equipment etc..The technical effect of device provided by the embodiment of the present invention, realization principle and generation is with before
It is identical to state embodiment of the method, to briefly describe, Installation practice part does not refer to place, can refer to phase in preceding method embodiment
Answer content.It is apparent to those skilled in the art that for convenience and simplicity of description, the system of foregoing description,
The specific work process of device and unit, the corresponding process during reference can be made to the above method embodiment, details are not described herein.
In embodiment provided by the present invention, it should be understood that disclosed device and method, it can be by others side
Formula is realized.The apparatus embodiments described above are merely exemplary, for example, the division of the unit, only one kind are patrolled
Function division is collected, there may be another division manner in actual implementation, in another example, multiple units or components can combine or can
To be integrated into another system, or some features can be ignored or not executed.Another point, shown or discussed is mutual
Coupling, direct-coupling or communication connection can be INDIRECT COUPLING or communication link by some communication interfaces, device or unit
It connects, can be electrical property, mechanical or other forms.
The unit as illustrated by the separation member may or may not be physically separated, aobvious as unit
The component shown may or may not be physical unit, it can and it is in one place, or may be distributed over multiple
In network unit.It can select some or all of unit therein according to the actual needs to realize the mesh of this embodiment scheme
's.
In addition, each functional unit in embodiment provided by the invention can integrate in one processing unit, it can also
To be that each unit physically exists alone, can also be integrated in one unit with two or more units.
It, can be with if the function is realized in the form of SFU software functional unit and when sold or used as an independent product
It is stored in a computer readable storage medium.Based on this understanding, technical solution of the present invention is substantially in other words
The part of the part that contributes to existing technology or the technical solution can be embodied in the form of software products, the meter
Calculation machine software product is stored in a storage medium, including some instructions are used so that a computer equipment (can be a
People's computer, server or network equipment etc.) it performs all or part of the steps of the method described in the various embodiments of the present invention.
And storage medium above-mentioned includes: that USB flash disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), arbitrary access are deposited
The various media that can store program code such as reservoir (RAM, Random Access Memory), magnetic or disk.
It should also be noted that similar label and letter indicate similar terms in following attached drawing, therefore, once a certain Xiang Yi
It is defined in a attached drawing, does not then need that it is further defined and explained in subsequent attached drawing, in addition, term " the
One ", " second ", " third " etc. are only used for distinguishing description, are not understood to indicate or imply relative importance.
Finally, it should be noted that embodiment described above, only a specific embodiment of the invention, to illustrate the present invention
Technical solution, rather than its limitations, scope of protection of the present invention is not limited thereto, although with reference to the foregoing embodiments to this hair
It is bright to be described in detail, those skilled in the art should understand that: anyone skilled in the art
In the technical scope disclosed by the present invention, it can still modify to technical solution documented by previous embodiment or can be light
It is readily conceivable that variation or equivalent replacement of some of the technical features；And these modifications, variation or replacement, do not make
The essence of corresponding technical solution is detached from the spirit and scope of technical solution of the embodiment of the present invention.Should all it cover in protection of the invention
Within the scope of.Therefore, protection scope of the present invention should be based on the protection scope of the described claims.
Claims (6)
1. a kind of cluster load balancing method, which is characterized in that the described method includes:
Receive the task that user sends；
The current load value of each node and each node are negative in the stabilization of previous cycle in cluster in acquisition current period
Load value determines the current steady load of each node according to the current load value and the steady load value of previous cycle
Value；
According to the current steady load value and the corresponding load changing value of the task, determine that the prediction of each node is negative
Load value；
The task is distributed to the smallest node of ratio of the prediction load value and maximum acceptable load value；
The prediction load value of each node after the distribution of acquisition task, the prediction of each node is negative after distribution of computation tasks
The ratio of load value and the maximum acceptable load value；
According to the prediction load value of each node after the maximum acceptable load value of each node and task distribution, meter
Calculate the remaining load value that the ratio is less than or equal to the node of default load threshold；
The task of the smallest node of the remaining load value is transferred to the maximum node of the remaining load value；
It is described according to the current load value and the steady load value of previous cycle, determine that the current steady of each node is negative
Load value, comprising:
Calculate each node current load value and the node the steady load value of previous cycle the first difference；
The current steady load value of each node and the node are denoted as in the difference of the steady load value of previous cycle
Second difference determines the current of each node according to the proportionate relationship between first difference and second difference
Steady load value.
2. the method according to claim 1, wherein described according to the steady of the current load value and previous cycle
Fixed load value determines the current steady load value of each node, comprising:
According to the current load value of each node, each node passes through formula in the steady load value of previous cycle
(1) the current steady load value of each node is calculated；
Nm=(nm-Nm-1)/d+Nm-1 (1)
Wherein, in formula (1), NmFor the current steady load value of any node in the cluster, Nm-1For any node
In the steady load value of previous cycle, nmFor the current load value of any node, d is pre-set ratio, and m is the period.
3. the method according to claim 1, wherein described according to the current steady load value and the task
Corresponding load changing value determines the prediction load value of each node, comprising:
According to the current steady load value of each node and the corresponding load changing value of the task, counted by formula (2)
Calculate the prediction load value of each node；
Fm=Dm+Nm (2)
Wherein, in formula (2), FmFor the prediction load value of any node in the cluster, DmFor the corresponding load of the task
Changing value, NmFor the current steady load value of any node, m is the period.
4. a kind of cluster load balance device, which is characterized in that described device includes:
Receiving module, for receiving the task of user's transmission；
First determining module, for obtaining in current period the current load value of each node and each node in cluster
It is determined each described in the steady load value of previous cycle according to the current load value and the steady load value of previous cycle
The current steady load value of node；
Second determining module, for determining each according to the current steady load value and the corresponding load changing value of the task
The prediction load value of a node；
Task allocating module, for by the task distribute to it is described prediction load value and maximum acceptable load value ratio most
The small node；
First computing module, it is each after distribution of computation tasks for obtaining the prediction load value of each node after task distribution
The ratio of the prediction load value and the maximum acceptable load value of a node；
Second computing module, for according to each knot after the distribution of the maximum acceptable load value and task of each node
The prediction load value of point calculates the remaining load value that the ratio is less than or equal to the node of default load threshold；
Task shift module, for the task of the smallest node of the remaining load value to be transferred to the remaining load value
The maximum node；
First determining module includes:
First computing unit, the current load value and the node for calculating each node are negative in the stabilization of previous cycle
First difference of load value；
Determination unit, for by the current steady load value of each node and the node previous cycle steady load
The difference of value is denoted as the second difference, according to the proportionate relationship between first difference and second difference, determines each institute
State the current steady load value of node.
5. device according to claim 4, which is characterized in that first determining module further include:
Second computing unit, for the current load value according to each node, each node is in the steady of previous cycle
Fixed load value calculates the current steady load value of each node by formula (1)；
Nm=(nm-Nm-1)/d+Nm-1 (1)
Wherein, in formula (1), NmFor the current steady load value of any node in the cluster, Nm-1For any node
In the steady load value of previous cycle, nmFor the current load value of any node, d is pre-set ratio, and m is the period.
6. device according to claim 4, which is characterized in that second determining module includes:
Third computing unit, for being changed according to the current steady load value of each node and the corresponding load of the task
Value calculates the prediction load value of each node by formula (2)；
Fm=Dm+Nm (2)
Wherein, in formula (2), FmFor the prediction load value of any node in the cluster, DmFor the corresponding load of the task
Changing value, NmFor the current steady load value of any node, m is the period.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610654053.4A CN106101276B (en) | 2016-08-10 | 2016-08-10 | A kind of cluster load balancing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610654053.4A CN106101276B (en) | 2016-08-10 | 2016-08-10 | A kind of cluster load balancing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106101276A CN106101276A (en) | 2016-11-09 |
| CN106101276B true CN106101276B (en) | 2019-07-09 |
Family
ID=57455778
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610654053.4A Active CN106101276B (en) | 2016-08-10 | 2016-08-10 | A kind of cluster load balancing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106101276B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108712422B (en) * | 2018-05-18 | 2021-05-25 | 网宿科技股份有限公司 | Method and device for creating transcoding task |
| CN108696594A (en) * | 2018-05-27 | 2018-10-23 | 佛山市虚拟现实大数据产业研究院有限公司 | A kind of the big data traffic load equalization methods and device of market surpervision block chain |
| CN110602545A (en) * | 2019-09-26 | 2019-12-20 | 杭州米络星科技(集团)有限公司 | Distributed recording execution method for network live broadcast |
| CN111683132B (en) * | 2020-06-04 | 2022-07-19 | 重庆金窝窝网络科技有限公司 | Business distribution method based on micro-service architecture and related device |
| CN112650582A (en) * | 2020-12-21 | 2021-04-13 | 贝壳技术有限公司 | Distributed task processing method and system and processor |
| CN113342665B (en) * | 2021-06-17 | 2023-10-20 | 北京百度网讯科技有限公司 | Task allocation method and device, electronic equipment and computer readable medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102232282A (en) * | 2010-10-29 | 2011-11-02 | 华为技术有限公司 | Method and apparatus for realizing load balance of resources in data center |
| CN103036979A (en) * | 2012-12-12 | 2013-04-10 | 广州尚融网络科技有限公司 | Server loading balancing method and loading balancer |
| WO2015175671A1 (en) * | 2014-05-13 | 2015-11-19 | Nutanix, Inc. | Mechanism for providing load balancing to an external node utilizing a clustered environment for storage management |
| CN105516369A (en) * | 2016-02-04 | 2016-04-20 | 城云科技（杭州）有限公司 | Video cloud platform load balancing method and video cloud platform load balancing dispatcher |
-
2016
- 2016-08-10 CN CN201610654053.4A patent/CN106101276B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102232282A (en) * | 2010-10-29 | 2011-11-02 | 华为技术有限公司 | Method and apparatus for realizing load balance of resources in data center |
| CN103036979A (en) * | 2012-12-12 | 2013-04-10 | 广州尚融网络科技有限公司 | Server loading balancing method and loading balancer |
| WO2015175671A1 (en) * | 2014-05-13 | 2015-11-19 | Nutanix, Inc. | Mechanism for providing load balancing to an external node utilizing a clustered environment for storage management |
| CN105516369A (en) * | 2016-02-04 | 2016-04-20 | 城云科技（杭州）有限公司 | Video cloud platform load balancing method and video cloud platform load balancing dispatcher |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106101276A (en) | 2016-11-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106101276B (en) | A kind of cluster load balancing method and device | |
| US20230093389A1 (en) | Service request allocation method and apparatus, computer device, and storage medium | |
| CN109218355A (en) | Load equalizing engine, client, distributed computing system and load-balancing method | |
| CN103401947A (en) | Method and device for allocating tasks to multiple servers | |
| CN110365748A (en) | Treating method and apparatus, storage medium and the electronic device of business datum | |
| CN113348651A (en) | Dynamic inter-cloud placement of sliced virtual network functions | |
| US20160019084A1 (en) | Method and system for inter-cloud virtual machines assignment | |
| CN107707612B (en) | Method and device for evaluating resource utilization rate of load balancing cluster | |
| KR20140090242A (en) | Optimization mechanisms for latency reduction and elasticity improvement in geographically distributed datacenters | |
| CN109032800A (en) | A kind of load equilibration scheduling method, load balancer, server and system | |
| KR101448413B1 (en) | Method and apparatus for scheduling communication traffic in atca-based equipment | |
| CN111245924A (en) | Load balancing method and device and computer storage medium | |
| CN116909735A (en) | Calculation power scheduling method and device, server and storage medium | |
| CN104301241B (en) | A kind of SOA dynamic load distributing methods and system | |
| CN109995818A (en) | A kind of method and device of server load balancing | |
| CN108200185B (en) | Method and device for realizing load balance | |
| CN105335376B (en) | A kind of method for stream processing, apparatus and system | |
| CN114168312A (en) | Distributed cluster load balancing method and device and storage medium | |
| CN110389839B (en) | Request-based hierarchical structure load balancing method and system | |
| JP2017215933A5 (en) | ||
| CN114338690B (en) | Server selection method and device and electronic equipment | |
| CN115633041B (en) | Multi-cluster management method and device, electronic equipment and readable storage medium | |
| CN110995802A (en) | Task processing method and device, storage medium and electronic device | |
| CN105187483B (en) | Distribute the method and device of cloud computing resources | |
| CN115168017B (en) | Task scheduling cloud platform and task scheduling method thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PP01 | Preservation of patent right |
Effective date of registration: 20220726 Granted publication date: 20190709 |
|
| PP01 | Preservation of patent right |