CN106708622A - Cluster resource processing method and system, and resource processing cluster - Google Patents
Cluster resource processing method and system, and resource processing cluster Download PDFInfo
- Publication number
- CN106708622A CN106708622A CN201610570094.5A CN201610570094A CN106708622A CN 106708622 A CN106708622 A CN 106708622A CN 201610570094 A CN201610570094 A CN 201610570094A CN 106708622 A CN106708622 A CN 106708622A
- Authority
- CN
- China
- Prior art keywords
- node
- resource
- cluster
- label
- uml
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention relates to a cluster resource processing method and system, and a resource processing cluster. The cluster resource processing method comprises the following steps that: monitoring node calculation resource information, and obtaining current idle calculation resources according to the node calculation resource information; receiving an operation request which carries resource configuration information, wherein the resource configuration information comprises a node label; according to the node label, obtaining a corresponding node queue, and distributing target calculation resources to the operation in the current idle calculation resources of the node in the node queue according to the resource configuration information; and on the node corresponding to the target calculation resources, starting a (User Mode Linux) resource isolation container, and executing the operation in a user space. A cluster use ratio can be improved, and operation throughput in the cluster can be improved.
Description
Technical field
The present invention relates to field of computer technology, more particularly to a kind of cluster resource processing method and system, Energy Resources Service
Reason cluster.
Background technology
Cluster is one group of computer that is separate, being interconnected by express network, and they constitute a group, and with list
The pattern of one system is managed.Server in cluster cooperates, and provides a user with network english teaching, comes for user
Say that cluster is like a triangular web.Special computer is rented in such as enterprise procurement, using linux type operating systems, is
Customer group provides access service.
Traditional cluster resource processing method, when being managed container resource, generally requires to update kernel or application behaviour
Make the administrator right of system, updating kernel can influence the service of enterprise's existing deployment, or need to operation maintenance personnel application pipe
Reason person's authority, is unfavorable for that addition server node carries out related deployment, causes cluster server to be unable to flexible deployment, and average profit
It is not high with rate.
The content of the invention
Based on this, it is necessary to for above-mentioned technical problem, there is provided a kind of cluster resource processing method and system, resource treatment
Cluster, improves cluster utilization rate, improves the handling capacity of operation in cluster.
A kind of cluster resource processing method, methods described includes：
Monitor node computing resource information, obtains cluster current idle and calculates money according to the node computing resource information
Source；
The job request for carrying resource allocation information is received, the resource allocation information includes node label；
Corresponding node queue is obtained according to the node label, according to the resource allocation information in the node queue
In node current idle computing resource in be operation distribution target computational resource；
Start user model Linux UML resource isolation containers in the corresponding node of the target computational resource, in user
Space performs operation.
A kind of cluster resource processing system, the system includes：
Computing resource monitoring management module, for monitor node computing resource information, believes according to the node computing resource
Breath obtains cluster current idle computing resource；
Computational resource allocation module, the job request of resource allocation information, the resource allocation information are carried for receiving
Including node label, corresponding node queue is obtained according to the node label, according to the resource allocation information in the section
It is the operation distribution target computational resource in the current idle computing resource of the node in point queue；
Operation execution module, for starting user model Linux UML moneys in the corresponding node of the target computational resource
Source spacing container, operation is performed in user's space.
Above-mentioned cluster resource processing method and system, by monitor node computing resource information, calculate according to the node
Resource information obtains cluster current idle computing resource, receives the job request for carrying resource allocation information, resource allocation information
Including node label, corresponding node queue, the section according to resource allocation information in node queue are obtained according to node label
For operation distributes target computational resource in the current idle computing resource of point, user is started in the corresponding node of target computational resource
Pattern Linux UML resource isolation containers, operation is performed in user's space, according to the node computing resource letter that monitor in real time is obtained
Breath obtain real-time change idle computing resources so that the distribution of resource is flexible and efficient, and by UML resource isolations container with
Family space performs operation, and on the premise of good isolation is ensured, it is not necessary to administrator right, energy flexible deployment clustered node will
Operation is distributed to the node of available free computing resource, by introducing scattered service node, equivalent in existing machine upper portion
A new computing cluster is affixed one's name to, the utilization rate of server resource can have been improved, so as to improve the handling capacity of operation in cluster.
A kind of resource processes cluster, and the cluster includes：
Calculate node, the calculate node node pre-conditioned for idle computing resources meet, the calculate node
The external loading of node of the quantity in the cluster takes amount of computational resources and updates；
Resource management server, for monitor node computing resource information, obtains according to the node computing resource information
Cluster current idle computing resource, receives the job request for carrying resource allocation information, and the resource allocation information includes node
Label, corresponding calculate node queue is obtained according to the node label, and section is calculated described according to the resource allocation information
It is the operation distribution target computational resource in the current idle computing resource of the target computing nodes in point queue；
The target computing nodes are used to start user model Linux UML resource isolation containers, are performed in user's space
Operation.
Above-mentioned resource processes cluster, calculate node in the external loading occupancy amount of computational resources renewal cluster according to node
Quantity, the idle computing resources of the node computing resource acquisition of information real-time change obtained according to monitor in real time so that resource
Distribution is flexible and efficient；And operation is performed in user's space by UML resource isolations container, on the premise of good isolation is ensured,
Administrator right is not needed, energy flexible deployment clustered node distributes operation to the node of available free computing resource, by introducing
Scattered service node composition cluster, a new computing cluster has been affixed one's name to equivalent in existing machine upper portion, can improve service
The utilization rate of device resource, so as to improve the handling capacity of operation in cluster.
Brief description of the drawings
Fig. 1 is the applied environment figure of cluster resource processing method operation in one embodiment；
Fig. 2 is the flow chart of cluster resource processing method in one embodiment；
Fig. 3 is one week interior cpu occupancy schematic diagram of cluster in one embodiment；
Fig. 4 is the flow chart of cluster resource processing method in a specific embodiment；
Fig. 5 is the structured flowchart of resource treatment cluster in one embodiment；
Fig. 6 is the structured flowchart of resource treatment cluster in another embodiment；
Fig. 7 is the structured flowchart of cluster resource processing system in one embodiment；
Fig. 8 is the structured flowchart of cluster resource processing system in another embodiment；
Fig. 9 is the structured flowchart of cluster resource processing system in further embodiment；
Figure 10 is the structured flowchart of cluster resource processing system in another embodiment；
Figure 11 is the structured flowchart of cluster resource processing system in further embodiment；
Figure 12 is the structured flowchart of operation execution module in one embodiment；
Figure 13 is the structured flowchart of cluster resource processing system in another embodiment；
Figure 14 is the structured flowchart of cluster resource processing system in another embodiment.
Specific embodiment
Fig. 1 is the applied environment figure of cluster resource processing method operation in one embodiment.As shown in figure 1, this applies ring
Border includes that end 110, resource management server 120, calculate node 130 and dedicated node 140, wherein calculate node are submitted in operation to
130 include multiple nodes, including node 131 ... node 13n, dedicated node 140 can be one or more, and wherein operation is carried
End 110, resource management server 120, calculate node 130 and dedicated node 140 composition cluster are handed over, can be led to by network
Letter, can be Hadoop clusters or Spark clusters.
It can be notebook computer, desktop computer etc. that end 110 is submitted in operation to, but be not limited thereto.Calculate node 130
Quantity takes computing resource and updates according to the external loading of node, and each clustered node possesses monitoring module, for prison in real time
Control node computing resource information, obtains cluster current idle computing resource, and resource management server 120 is taken according in job request
The resource allocation information of band is that operation distributes destination node, and starts user model Linux UML resource isolations in destination node
Container, operation is performed in user's space.Job manager, the progress of monitoring resultses can be disposed on dedicated node 140.
In one embodiment, as shown in Figure 2, there is provided a kind of cluster resource processing method, to be applied to above-mentioned application
Come in environment for example, comprising the following steps：
Step S210, monitor node computing resource information obtains cluster current idle meter according to node computing resource information
Calculate resource.
Specifically, money of the computing resource information including cpu occupancies, free memory, disk remaining space etc. for calculating
Source relevant information, wherein can carry out being converted to occupancy cpu check figures according to cpu occupancies, and then obtains available cpu check figures, such as
Current cpu occupancies 60%, the total check figures of cpu are 8, can be 8* (1-0.6), about 3 with cpu check figures.Idle free memory is
By total internal memory Mem of nodetotalSubtract the internal memory Mem of external service occupancyexternalObtain, i.e. Memtotal-
Memexternal.The circular of the current idle computing resource of each node can customize, such as when disk remaining space is small
When predetermined threshold value, node is added into blacklist.Each deployment node calculates money by node manager to the idle of this node
Source is timed monitoring and collects, and is pooled to explorer, obtains the total available resource amount of whole cluster, it is ensured that All Jobs
Resource request be all to be divided away from idle computing resources.Wherein node manager is a clothes for operating in calculate node
Business process, manages, distributes, reporting, reclaiming the computing resource of the calculate node.Explorer is to operate in resource management service
One service processes of device, manage, distribute, reclaiming the computing resource of all calculate nodes.
In one embodiment, the current idle computing resource of node is that remaining disk space presets threshold more than disk space
The available cpu check figures of the node of value, the summation of free memory.
Specifically, if the remaining disk space of node is less than disk space predetermined threshold value, this node does not perform calculating
Task, it is ensured that the data storage of calculating task and external service.
In one embodiment, using platform monitoring software such as ganglia, the load condition of the whole cluster of deployment monitoring,
To investigate node failure reason.According to platform identity, the measurement data that increased ganglia in resource management layer is reported patrols
Volume, the utilization power of idle computing resources is visualized.If Fig. 3 is cpu occupancy schematic diagrames in cluster one week, 240 is outside
The resource occupation ratio of service, 250 is the idling-resource ratio of the actual occupancy of operation, and 260 is the remaining available idle meter of operation
Calculate resource ratio.
Step S220, receives the job request for carrying resource allocation information, and resource allocation information includes node label, according to
Node label obtains corresponding node queue, and the current idle of the node according to resource allocation information in node queue calculates money
For operation distributes target computational resource in source.
Specifically, the resource distribution that resource allocation information is used for needed for characterizing Job execution, such as operation can be with when submitting to
Specify the memory size and node label of application container.Node label is used to characterize the type of node, can area by node label
Divide the node of different computing capabilitys.The setting of node label can be self-defined as needed, such as sets different level distinguishing labels.
In one embodiment, before step S220, also include：Believed by the physical attribute information and load condition of node
Breath, is that node sets label.
Specifically, physical attribute information is the physical attribute of node, and it is build-in attribute, such as big calculate node distribution of internal memory
High_mem is identified.Load state information is the load condition of stabilization in the regular period, such as by counting preset time period internal segment
The resource occupation of the external progress of point, compares with predetermined threshold value, external resource is taken high more than the external loading of predetermined threshold value
Node identified with high_load.One node can be set one or more node labels.
Operation can specify the memory size and node label of application container when submitting to by resource allocation information, pass through
Node label obtains corresponding target labels node queue, and suitable node is only selected in target labels node queue to perform
Calculating task.If it is understood that the node mark of acquiescence can be used without specified node label in resource allocation information
Sign.Operation is usually distributed treatment bulk data, and input data is cut into small data block, gives subtask treatment, is
Suitable target computational resource is distributed in each subtask, and each subtask is assigned to each node for meeting computing resource condition
In.By node label, the destination node for meeting condition can be screened in calculate node, to meet the calculating of different work needs
Resource granularity and reduction Job Failure Rate.
Step S230, user model Linux UML resource isolation containers are started in the corresponding node of target computational resource,
User's space performs operation.
Specifically, user model Linux UML (user mode linux) resource isolation container can be created by domestic consumer
Startup, more versatility are built, user mode linux use mmap mode extended memory address space, therefore have little influence on outer
The physical memory that portion's service is used, substantially allows data to be converted operation via cpu, there is the independent process space, protects
Card container internal process cannot access physical machine external service process, it is ensured that the process of external service is not destroyed.In a reality
Apply in example, Job execution terminates, the computing resource taken during release Job execution, it is ensured that the recurrence of idling-resource.
In one embodiment, user model Linux is started in the corresponding node of target computational resource in step S230
The step of UML resource isolation containers, includes：In perform script, the environmental variance of job dependence is set, and the file that operation is needed
In resource carry to container, created with domestic consumer's identity and start UML resource isolation containers.
The root in container can be mapped as with newly-built private file catalogue, it is ensured that limited file access, it is to avoid not
The unauthorized access of mandate, modification file, while result of calculation can be stored, so as to reach the requirement of resource isolation.Can be by modification
The mixed-media network modules mixed-media code of user mode linux, realize UML resource isolations container do not need administrator right can be with extranets
Network is communicated, and carries out data transmission, and operation is performed in user's space.Do not need applications management person's authority, it is possible to perform operation, it is complete
Into communication, improve the flexibility of each calculate node deployment, can be random increase or decrease calculate node, without extra
Application authority.
In the present embodiment, by monitor node computing resource information, cluster is obtained according to the node computing resource information
Current idle computing resource, receives the job request for carrying resource allocation information, and resource allocation information includes node label, according to
Node label obtains corresponding node queue, and the current idle of the node according to resource allocation information in node queue calculates money
For operation distributes target computational resource in source, user model Linux UML resources are started in the corresponding node of target computational resource
Spacing container, operation is performed in user's space, the node computing resource acquisition of information real-time change obtained according to monitor in real time
Idle computing resources so that the distribution of resource is flexible and efficient, and operation is performed in user's space by UML resource isolations container,
On the premise of good isolation is ensured, it is not necessary to administrator right, can flexible deployment clustered node, operation is distributed to available free
The node of computing resource, by introducing scattered service node, a new calculating has been affixed one's name to equivalent in existing machine upper portion
Cluster, can improve the utilization rate of server resource, so as to improve the handling capacity of operation in cluster.
In one embodiment, in step S230, when user's space performs operation, by using user model network
Slirp network simulators start virtual network device and are bridged with physical network card with domestic consumer's identity, realize network service.
Specifically, by tcp/ip remote transmission input datas during Job execution, virtual network inside container and
The physical network of back end needs foundation connection transceiving data bag to be communicated.User mode linux are generally by opening
Dynamic virtual network device, such as tun equipment is bridged with physical network card, realizes the communication with external network, but start tun and set
It is standby to need root authority, in the present embodiment, the mixed-media network modules mixed-media code of user mode linux is have modified, using slirp network moulds
Intend device scheme, start virtual network device with domestic consumer's identity bridges with physical network card, it is ensured that the transmission of inputoutput data
And the efficient convenient communication between UML resource isolation containers.
In one embodiment, user model Linux is started in the corresponding node of target computational resource in step S230
The step of UML resource isolation containers, includes：New files catalogue, if newly-built success, obtains lock, starts first user pattern
Linux UML resource isolation containers, are a UML resource isolation container allocation IP address, and a UML resource isolations container starts
Success, then delete file directory, release lock.
Specifically, each subtask performs to be required for starting a UML resource isolation container, calculating process may need far
Journey accesses input data, it is necessary to is one sole internal ip of each UML resource isolations container allocation, need to ensure and physical node
The network segment do not conflict, lock is obtained by way of new files catalogue, only obtain lock could start UML resource isolation containers
And distributing IP address, it is ensured that it is serial that individual node inner pressurd vessel starts, so as to ensure that each IP address is unique and different.
In one embodiment, after step S230, also include：If external loading increases the resource for taking more than first
The resource that predetermined threshold value or operation subtask take then is cleared up UML resource isolations container and shifts son more than the second predetermined threshold value
Other nodes in task to node queue.
Specifically, collecting the computing resource that operation subtask takes by container process tree in individual node, work as subtask
The computing resource of occupancy is then cleared up UML resource isolations container and shifts calculating task to node queue more than the second predetermined threshold value
In other nodes.External loading increases the resource for taking more than the first predetermined threshold value so that remaining computing resource in node
During less than the first predetermined threshold value, then clear up UML resource isolations container and shift other nodes in subtask to node queue, from
And ensure that computing resource has cushion space to a certain extent.
In one embodiment, the effective transfer and operation in order to ensure task can finally be smoothly completed, for due to making
The resource that industry subtask takes is not included into unsuccessfully number of retries more than mission failure caused by the second predetermined threshold value, so that
Operation subtask can be assigned to the relatively low node of external service load and perform calculating.
In one embodiment, whether key service node is obtained, current time is judged in preset time period, if it is,
It is 0 then to set the current idle computing resource of the key service node, described if current time is not in preset time period
The node computing resource that the current idle computing resource of key service node is obtained for actual monitored.
Specifically, key service node is the node that critical services are provided for outside, such node general work
Time period service request amount Relatively centralized, load is higher, in order to not influence service, when computing resource collection is carried out, with the addition of
Time period limits, and the scope of preset time period can be self-defined as needed, such as can be arranged on one day words spoken by an actor from offstage by configuration file
The available free resource of its time period key service node is 0, then will not distribute calculating task, to night or morning, load
Take resource to decline, node reports actual idling-resource amount again.
In one embodiment, if the current idle computing resource of first node is less than resource predetermined threshold value, by the
One node is temporarily disengaged from cluster.
If specifically, the current idle computing resource of first node be less than resource predetermined threshold value, first node is temporary
When depart from cluster because the load of external service is uncertain, rise in external loading, current idle computing resource is too small
When, cluster is temporarily disengaged from by by first node, job task of not reallocating temporarily, it is ensured that first node can be carried for external service
For cushion space, while avoiding node uncontrollable because of load too high, it is ensured that the limitation of computing resource.When working as first node
Preceding idle computing resources recover during more than resource predetermined threshold value, can recover to add cluster.
In one embodiment, method also includes：For the node that load condition meets stable node condition sets static
Static labels, in the node initiating task manager of static labels, the node of the static labels is proprietary node, is protected
Hold cluster state.
Specifically, the design parameter of stable node condition can be self-defined as needed, such as load relatively low, computing resource and fill
Foot, can be considered the node for meeting stable node condition, static state static labels be set to this node, in the section of static labels
Point initiating task manager, such node can be the proprietary node of cluster, keep cluster state, and service is provided always, can
Job manager is avoided to be cleaned out because of computing resource deficiency, it is to avoid to lose job scheduling information.
In one embodiment, method also includes：The corresponding node blacklist of operation is obtained, if the in node blacklist
The ratio of the corresponding all node numbers of the first label exceedes label threshold value in the corresponding node number of one label and cluster, then will
The corresponding node of the first label removes node blacklist in node blacklist.
Specifically, explorer can be operation sets a interim blacklist, only to the operation effectively, when certain node
Mission failure number of times it is excessive when, by node add blacklist in, when the node number in blacklist exceed cluster in node
Several proportion threshold value p, then blacklist failure.If ratio of the node number of node label X in whole cluster is no more than p,
Therefore blacklist will not fail, fluctuated under big mal-condition in the corresponding node loads of node label X, it is understood that there may be node label
Each node of X is put on the blacklist because mission failure number of times is excessive, and cluster is added unless there are new X label nodes, no
Then blacklist will not be failed, and operation will be waited infinitely, in order to avoid such case occurs, in the present embodiment, if the black name of node
The ratio of the corresponding all node numbers of the first label exceedes label threshold in the corresponding node number of the first label and cluster in list
Value, then remove node blacklist by the corresponding node of the first label in node blacklist, and when calculating proportionality coefficient, use is not
Node in whole cluster, but the corresponding all nodes of the first label in cluster, it is to avoid the situation that operation is infinitely waited, more
Rationally, fault freedom is good.
As shown in figure 4, being the flow chart of cluster resource processing method in a specific embodiment, detailed process is as follows：
Step S310, monitor node computing resource information obtains cluster current idle meter according to node computing resource information
Calculate resource.
Step S320, receives the job request of the resource allocation information for carrying node label, is existed according to resource allocation information
For operation distributes target computational resource in the current idle computing resource of the node in the corresponding node queue of node label.
Step S330, user model Linux UML resource isolation containers are started in the corresponding node of target computational resource,
User's space performs operation.
Step S340, when external loading increases the resource for taking more than the money that the first predetermined threshold value or operation subtask take
When source is more than the second predetermined threshold value, other sections in cleaning UML resource isolations container and transfer tasks subtask to node queue
Point.
Step S350, Job execution terminates, the computing resource taken during release Job execution.
In one embodiment, as shown in Figure 5, there is provided a kind of resource processes cluster, including：
Calculate node 420, the node pre-conditioned for idle computing resources meet of calculate node 420, calculate node 420
The external loading that quantity processes the node in cluster according to resource takes amount of computational resources renewal.
Specifically, pre-conditioned can be self-defined as needed, idle computing resources meet pre-conditioned node can be at any time
As calculate node, idle computing resources are unsatisfactory for pre-conditioned node and can at any time be temporarily disengaged from cluster, the number of calculate node
The external loading for measuring the node processed according to resource in cluster takes amount of computational resources renewal, it is ensured that effective utilization of resource.
In one embodiment, if resource management server is additionally operable to the current idle computing resource of the first calculate node
Less than resource predetermined threshold value, then the first calculate node is temporarily disengaged from cluster.
If specifically, the current idle computing resource of the first calculate node is less than resource predetermined threshold value, first counted
Operator node is temporarily disengaged from cluster, because the load of external service is uncertain, rises in external loading, and current idle is calculated
When resource is too small, cluster is temporarily disengaged from by by the first calculate node, job task of not reallocating, it is ensured that first node can be outer
Portion's service provides cushion space, while avoiding node uncontrollable because of load too high, it is ensured that the limitation of computing resource.
Resource management server 410, for monitor node computing resource information, is obtained according to node computing resource information
One cluster current idle computing resource, receives the job request for carrying resource allocation information, and resource allocation information includes node mark
Sign, corresponding calculate node queue is obtained according to node label, according to target of the resource allocation information in calculate node queue
For operation distributes target computational resource in the current idle computing resource of calculate node 421.
Target computing nodes 421 are used to start user model Linux UML resource isolation containers, and work is performed in user's space
Industry.
In one embodiment, resource treatment cluster includes key service node, and key service node is used to judge current
Whether the time is in preset time period, if it is, the current idle computing resource for setting key service node is 0, if currently
Time is not in preset time period, then the node calculating money that the current idle computing resource of key service node is obtained for actual monitored
Source..
Specifically, key service node is the node that critical services are provided for outside, such node general work
Time period service request amount Relatively centralized, load is higher, in order to not influence service, when computing resource collection is carried out, with the addition of
Time period limits, and the scope of preset time period can be self-defined as needed, such as can be arranged on one day words spoken by an actor from offstage by configuration file
The available free resource of its time period key service node is 0, then will not distribute calculating task, to night or morning, load
Take resource to decline, node reports actual idling-resource amount again, can again be its corresponding calculating task of distribution operation.
In one embodiment, target computing nodes 421 are additionally operable to when user's space performs operation, by using user
Mode network slirp network simulators start virtual network device and are bridged with physical network card with domestic consumer's identity, realize network
Communication.
Specifically, target computing nodes 421 pass through tcp/ip remote transmission input datas, container during Job execution
Internal virtual network and the physical network of back end need foundation connection transceiving data bag to be communicated.user mode
Linux is bridged generally by tun equipment is started with physical network card, realizes the communication with external network, but start tun
Equipment needs root authority, in the present embodiment, the mixed-media network modules mixed-media code of user mode linux is have modified, using slirp networks
Simulator scheme, starts virtual network device and is bridged with physical network card, it is ensured that the biography of inputoutput data with domestic consumer's identity
Efficient convenient communication between defeated and UML resource isolation containers.
In one embodiment, as shown in fig. 6, cluster also includes：
Dedicated node 430, the node label of dedicated node is static labels, cluster state is kept, for initiating task
Manager.
Specifically, dedicated node refers to the node that service-specific is provided for cluster, external service is not provided, it is such
Node keeps providing the service of stabilization, and cluster is not departed from easily.In the node initiating task manager of static labels, collection is kept
Group's state, can always provide service, job manager can be avoided to be cleaned out because of computing resource deficiency, it is to avoid from losing operation and enter
Degree information.
In one embodiment, as shown in Figure 7, there is provided a kind of cluster resource processing system, including：
Computing resource monitoring management module 510, for monitor node computing resource information, according to node computing resource information
Obtain cluster current idle computing resource.
Computational resource allocation module 520, the job request of resource allocation information, resource allocation information bag are carried for receiving
Node label is included, corresponding node queue, the node according to resource allocation information in node queue are obtained according to node label
Current idle computing resource in for operation distribute target computational resource.
Operation execution module 530, for starting user model Linux UML resources in the corresponding node of target computational resource
Spacing container, operation is performed in user's space.
In one embodiment, as shown in figure 8, system also includes：
First resource limits module 540, if increase the resource for taking more than the first predetermined threshold value for external loading or
The resource that operation subtask takes clear up UML resource isolations container more than the second predetermined threshold value, then and transfer tasks subtask extremely
Other nodes in node queue.
In one embodiment, as shown in figure 8, system also includes：
Whether Secondary resource limits module 550, for obtaining key service node, judge current time in Preset Time
Section, if it is, the current idle computing resource for setting key service node is 0, if current time is not in preset time period,
The node computing resource that then the current idle computing resource of key service node is obtained for actual monitored.It is understood that being
System may include first resource limitation module 540 and Secondary resource limitation any one module therein of module 550.
Although it is understood that included in Fig. 8 first resource limitation module 540 and Secondary resource limitation module 550,
At least one module therein is may include in embodiment.
In one embodiment, as shown in figure 9, system also includes：
Cluster resource management module 560, if presetting threshold less than resource for the current idle computing resource of first node
Value, then be temporarily disengaged from cluster by first node.
In one embodiment, as shown in Figure 10, system also includes：
Label setup module 570, is that node sets mark for physical attribute information and load state information according to node
Sign.
In one embodiment, as shown in figure 11, system also includes：
Job management module 580, the node for meeting stable node condition for load condition sets static state static marks
Sign, in the node initiating task manager of static labels, the node of static labels is proprietary node, keeps cluster state.
In one embodiment, operation execution module 530 is additionally operable to be set in perform script the environment of the job dependence
In variable, and file resource carry to the container that operation is needed, created with domestic consumer's identity and start UML resource isolations appearance
Device.
In one embodiment, as shown in figure 12, operation execution module 530 includes：
Lock acquisition module 531, for new files catalogue, if newly-built success, obtains lock.
Job initiation module 532, is UML moneys for starting first user pattern Linux UML resource isolation containers
Source spacing container distribution IP address.
Lock release module 533, starts successfully for a UML resource isolation containers, then delete file directory, release lock.
In one embodiment, as shown in figure 13, system also includes：
UML network communication modules 590, for when user's space performs operation, by using user model network slirp
Network simulator starts virtual network device and is bridged with physical network card with domestic consumer's identity, realizes network service.
In one embodiment, as shown in figure 14, system also includes：
Fault-tolerant module 600, for obtaining the corresponding node blacklist of operation, if the first label correspondence in node blacklist
Node number and cluster in the corresponding all node numbers of the first label ratio exceed label threshold value, then by node blacklist
In the corresponding node of the first label remove node blacklist.
One of ordinary skill in the art will appreciate that all or part of flow in realizing above-described embodiment method, can be
The hardware of correlation is instructed to complete by computer program, described program can be stored in a computer read/write memory medium
In, such as in the embodiment of the present invention, the program can be stored in the storage medium of computer system, and by the computer system
At least one computing device, to realize including the flow of the embodiment such as above-mentioned each method.Wherein, the storage medium can be
Magnetic disc, CD, read-only memory (Read-Only Memory, ROM) or random access memory (Random Access
Memory, RAM) etc..
Each technical characteristic of embodiment described above can be combined arbitrarily, to make description succinct, not to above-mentioned reality
Apply all possible combination of each technical characteristic in example to be all described, as long as however, the combination of these technical characteristics is not deposited
In contradiction, the scope of this specification record is all considered to be.
Embodiment described above only expresses several embodiments of the invention, and its description is more specific and detailed, but simultaneously
Can not therefore be construed as limiting the scope of the patent.It should be pointed out that coming for one of ordinary skill in the art
Say, without departing from the inventive concept of the premise, various modifications and improvements can be made, these belong to protection of the invention
Scope.Therefore, the protection domain of patent of the present invention should be determined by the appended claims.
Claims (27)
1. a kind of cluster resource processing method, methods described includes：
Monitor node computing resource information, cluster current idle computing resource is obtained according to the node computing resource information；
The job request for carrying resource allocation information is received, the resource allocation information includes node label；
Corresponding node queue is obtained according to the node label, according to the resource allocation information in the node queue
It is the operation distribution target computational resource in the current idle computing resource of node；
Start user model Linux UML resource isolation containers in the corresponding node of the target computational resource, in user's space
Perform operation.
2. method according to claim 1, it is characterised in that described after the step of user's space performs operation, also
Including：
If external loading increases resource of the resource more than the first predetermined threshold value or operation subtask occupancy for taking more than second
Predetermined threshold value, then other nodes cleared up in the UML resource isolations container and transfer tasks subtask to the node queue.
3. method according to claim 1, it is characterised in that methods described also includes：
Key service node is obtained, whether current time is judged in preset time period, if it is, setting the key service section
The current idle computing resource of point is 0, if current time is not in preset time period, the current sky of the key service node
The node computing resource that not busy computing resource is obtained for actual monitored.
4. method according to claim 1, it is characterised in that methods described also includes：
If the current idle computing resource of first node is less than resource predetermined threshold value, the first node is temporarily disengaged from collection
Group.
5. method according to claim 1, it is characterised in that the reception carries the job request of resource allocation information
Before step, also include：
It is that node sets label according to the physical attribute information and load state information of node.
6. method according to claim 1, it is characterised in that the current idle computing resource is that remaining disk space is big
The summation of available cpu check figures, free memory in the node of disk space predetermined threshold value.
7. method according to claim 1, it is characterised in that methods described also includes：
For the node that load condition meets stable node condition sets static state static labels, in the node of the static labels
Initiating task manager, the node of the static labels is proprietary node, keeps cluster state.
8. method according to claim 1, it is characterised in that described to start in the corresponding node of the target computational resource
The step of user model Linux UML resource isolation containers, includes：
In perform script, the environmental variance of the job dependence is set, and in file resource carry to the container that operation is needed,
Created with domestic consumer's identity and start UML resource isolation containers.
9. method according to claim 1, it is characterised in that described to start in the corresponding node of the target computational resource
The step of user model Linux UML resource isolation containers, includes：
New files catalogue, if newly-built success, obtains lock, starts first user pattern Linux UML resource isolation containers,
It is the UML resource isolation container allocation IP address；
The first UML resource isolation containers start successfully, then delete the file directory, release lock.
10. method according to claim 1, it is characterised in that methods described also includes：
When user's space performs operation, opened with domestic consumer's identity by using user model network slirp network simulators
Dynamic virtual network device is bridged with physical network card, realizes network service.
11. methods according to claim 1, it is characterised in that methods described also includes：
Obtain the corresponding node blacklist of the operation, if in the node blacklist the corresponding node number of the first label with
The ratio of the corresponding all node numbers of the first label exceedes label threshold value in cluster, then by the first mark in the node blacklist
Sign corresponding node and remove the node blacklist.
12. methods according to claim 1, it is characterised in that described after the step of user's space performs operation, also
Including：
The Job execution terminates, and discharges the computing resource taken during the Job execution.
13. a kind of cluster resource processing systems, it is characterised in that the system includes：
Computing resource monitoring management module, for monitor node computing resource information, obtains according to the node computing resource information
To cluster current idle computing resource；
Computational resource allocation module, the job request of resource allocation information is carried for receiving, and the resource allocation information includes
Node label, obtains corresponding node queue, according to the resource allocation information in the node team according to the node label
It is the operation distribution target computational resource in the current idle computing resource of the node in row；
Operation execution module, for starting user model LinuxUML resource isolations in the corresponding node of the target computational resource
Container, operation is performed in user's space.
14. systems according to claim 13, it is characterised in that the system also includes：
First resource limits module, if increase the resource for taking for external loading appointed more than the first predetermined threshold value or operation
The resource for taking be engaged in more than the second predetermined threshold value, then clear up the UML resource isolations container and transfer tasks subtask to described
Other nodes in node queue.
15. systems according to claim 13, it is characterised in that the system also includes：
Whether Secondary resource limits module, for obtaining key service node, judges current time in preset time period, if
It is that then to set the current idle computing resource of the key service node be 0, if current time is not in preset time period,
The node computing resource that the current idle computing resource of the key service node is obtained for actual monitored.
16. systems according to claim 13, it is characterised in that the system also includes：
Cluster resource management module, if being less than resource predetermined threshold value for the current idle computing resource of first node, will
The first node is temporarily disengaged from cluster.
17. systems according to claim 13, it is characterised in that the system also includes：
Label setup module, is that node sets label for physical attribute information and load state information according to node.
18. systems according to claim 13, it is characterised in that the system also includes：
Job management module, the node for meeting stable node condition for load condition sets static state static labels, in institute
The node initiating task manager of static labels is stated, the node of the static labels is proprietary node, keeps cluster state.
19. systems according to claim 13, it is characterised in that the operation execution module is additionally operable to be set in perform script
The environmental variance of the job dependence is put, and in file resource carry to the container that operation is needed, is created with domestic consumer's identity
Build startup UML resource isolation containers.
20. systems according to claim 13, it is characterised in that the operation execution module includes：
Lock acquisition module, for new files catalogue, if newly-built success, obtains lock；
Job initiation module, is a UML resources for starting first user pattern Linux UML resource isolation containers
Spacing container distributes IP address；
Lock release module, starts successfully for the UML resource isolation containers, then delete the file directory, release lock.
21. systems according to claim 13, it is characterised in that the system also includes：
UML network communication modules, for when user's space performs operation, by using user model network slirp network moulds
Intend device and virtual network device and physical network card bridge joint are started with domestic consumer's identity, realize network service.
22. systems according to claim 13, it is characterised in that the system also includes：
Fault-tolerant module, for obtaining the corresponding node blacklist of the operation, if the first label pair in the node blacklist
The ratio of the corresponding all node numbers of the first label exceedes label threshold value in the node number and cluster answered, then by the node
The corresponding node of the first label removes the node blacklist in blacklist.
23. a kind of resource treatment clusters, it is characterised in that the cluster includes：
Calculate node, the calculate node node pre-conditioned for idle computing resources meet, the quantity of the calculate node
The external loading of the node in the cluster takes amount of computational resources and updates；
Resource management server, for monitor node computing resource information, cluster is obtained according to the node computing resource information
Current idle computing resource, receives the job request for carrying resource allocation information, and the resource allocation information includes node label,
Corresponding calculate node queue is obtained according to the node label, according to the resource allocation information in the calculate node queue
In target computing nodes current idle computing resource in be operation distribution target computational resource；
The target computing nodes are used to start user model Linux UML resource isolation containers, and operation is performed in user's space.
24. clusters according to claim 23, it is characterised in that the cluster includes key service node, the key
Whether service node is used to judge current time in preset time period, if it is, setting the current of the key service node
Idle computing resources are 0, if current time is not in preset time period, the current idle of the key service node calculates money
The node computing resource that source obtains for actual monitored.
25. clusters according to claim 23, it is characterised in that if the resource management server is additionally operable to the first meter
The current idle computing resource of operator node is less than resource predetermined threshold value, then first calculate node is temporarily disengaged from into the collection
Group.
26. clusters according to claim 23, it is characterised in that the target computing nodes are additionally operable to be held in user's space
During row operation, by use user model network slirp network simulators with domestic consumer identity start virtual network device with
Physical network card is bridged, and realizes network service.
27. clusters according to claim 23, it is characterised in that the cluster also includes：
Dedicated node, the node label of the dedicated node is static labels, keeps cluster state, for initiating task management
Device.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610570094.5A CN106708622B (en) | 2016-07-18 | 2016-07-18 | Cluster resource processing method and system and resource processing cluster |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610570094.5A CN106708622B (en) | 2016-07-18 | 2016-07-18 | Cluster resource processing method and system and resource processing cluster |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106708622A true CN106708622A (en) | 2017-05-24 |
| CN106708622B CN106708622B (en) | 2020-06-02 |
Family
ID=58939713
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610570094.5A Active CN106708622B (en) | 2016-07-18 | 2016-07-18 | Cluster resource processing method and system and resource processing cluster |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106708622B (en) |
Cited By (36)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107273196A (en) * | 2017-05-31 | 2017-10-20 | 中国科学院北京基因组研究所 | Bioinformatics high-performance calculation job scheduling and system administration external member |
| CN108279982A (en) * | 2018-02-27 | 2018-07-13 | 郑州云海信息技术有限公司 | Pbs resources and hadoop method for managing resource, system and equipment |
| CN108509268A (en) * | 2018-02-24 | 2018-09-07 | 宁波诺信睿聚投资有限责任公司 | Cluster resource distribution method, device, equipment and computer readable storage medium |
| CN109086134A (en) * | 2018-07-19 | 2018-12-25 | 郑州云海信息技术有限公司 | A kind of operation method and device of deep learning operation |
| CN109104334A (en) * | 2018-08-23 | 2018-12-28 | 郑州云海信息技术有限公司 | The management method and device of monitoring system interior joint |
| WO2019041206A1 (en) * | 2017-08-31 | 2019-03-07 | Entit Software Llc | Managing containers using attribute/value pairs |
| CN109471727A (en) * | 2018-10-29 | 2019-03-15 | 北京金山云网络技术有限公司 | A kind of task processing method, apparatus and system |
| CN109783237A (en) * | 2019-01-16 | 2019-05-21 | 腾讯科技（深圳）有限公司 | A kind of resource allocation method and device |
| CN110120940A (en) * | 2019-04-12 | 2019-08-13 | 华中科技大学 | A kind of file system resource partition method towards Docker container |
| CN110278257A (en) * | 2019-06-13 | 2019-09-24 | 中信银行股份有限公司 | A kind of method of mobilism configuration distributed type assemblies node label |
| CN110287000A (en) * | 2019-05-29 | 2019-09-27 | 北京达佳互联信息技术有限公司 | Data processing method, device, electronic equipment and storage medium |
| CN110389836A (en) * | 2019-07-17 | 2019-10-29 | 腾讯科技（深圳）有限公司 | A kind of more cluster management methods, device, server and storage medium |
| CN110413382A (en) * | 2019-08-06 | 2019-11-05 | 山东超越数控电子股份有限公司 | A kind of method, equipment and the readable medium of the resource dynamic adjustment of Docker container |
| CN110659127A (en) * | 2018-06-29 | 2020-01-07 | 杭州海康威视数字技术股份有限公司 | Method, device and system for processing task |
| CN110704177A (en) * | 2019-09-04 | 2020-01-17 | 金蝶软件（中国）有限公司 | Computing task processing method and device, computer equipment and storage medium |
| CN110719306A (en) * | 2018-07-11 | 2020-01-21 | 阿里巴巴集团控股有限公司 | Network request limiting method, computer equipment and storage medium |
| CN110928666A (en) * | 2019-12-09 | 2020-03-27 | 湖南大学 | Method and system for optimizing task parallelism based on memory in Spark environment |
| CN111078404A (en) * | 2019-12-09 | 2020-04-28 | 腾讯科技（深圳）有限公司 | Computing resource determination method and device, electronic equipment and medium |
| WO2020119117A1 (en) * | 2018-12-14 | 2020-06-18 | 平安医疗健康管理股份有限公司 | Distributed computing method, apparatus and system, device and readable storage medium |
| CN111340613A (en) * | 2020-02-26 | 2020-06-26 | 中国邮政储蓄银行股份有限公司 | Job processing method, system and storage medium |
| CN111767199A (en) * | 2020-06-24 | 2020-10-13 | 中国工商银行股份有限公司 | Resource management method, device, equipment and system based on batch processing operation |
| CN111813545A (en) * | 2020-06-29 | 2020-10-23 | 北京字节跳动网络技术有限公司 | Resource allocation method, device, medium and equipment |
| CN112148467A (en) * | 2019-06-28 | 2020-12-29 | 微软技术许可有限责任公司 | Dynamic allocation of computing resources |
| CN112445602A (en) * | 2019-08-27 | 2021-03-05 | 阿里巴巴集团控股有限公司 | Resource scheduling method, device and system and electronic equipment |
| CN112463290A (en) * | 2020-11-10 | 2021-03-09 | 中国建设银行股份有限公司 | Method, system, apparatus and storage medium for dynamically adjusting the number of computing containers |
| CN112527454A (en) * | 2020-12-04 | 2021-03-19 | 上海连尚网络科技有限公司 | Container group scheduling method and device, electronic equipment and computer readable medium |
| CN112597502A (en) * | 2020-12-17 | 2021-04-02 | 山东乾云启创信息科技股份有限公司 | Large-scale computing service configuration method and system based on trusted cloud |
| CN112835996A (en) * | 2019-11-22 | 2021-05-25 | 北京初速度科技有限公司 | Map production system and method thereof |
| CN113296929A (en) * | 2020-06-29 | 2021-08-24 | 阿里巴巴集团控股有限公司 | Resource matching method, device and system based on cloud computing |
| CN113703952A (en) * | 2020-05-20 | 2021-11-26 | 山东省计算中心（国家超级计算济南中心） | Resource allocation method for queue resource scheduling based on super computer |
| CN114168354A (en) * | 2022-02-11 | 2022-03-11 | 北京易源兴华软件有限公司 | Data-driven data cluster parallel computing allocation method and device |
| CN114640677A (en) * | 2022-05-17 | 2022-06-17 | 云宏信息科技股份有限公司 | Method for adding working nodes to cluster working nodes and APIServer by client configuration |
| CN114826964A (en) * | 2022-04-11 | 2022-07-29 | 京东科技信息技术有限公司 | Resource monitoring method, device and system |
| US11609837B2 (en) | 2021-06-02 | 2023-03-21 | Kyndryl, Inc. | Calibration technique using computer analysis for ascertaining performance of containers |
| CN115951988A (en) * | 2023-03-03 | 2023-04-11 | 北京并行科技股份有限公司 | Job scheduling method, computing device and storage medium |
| CN117009060A (en) * | 2023-09-27 | 2023-11-07 | 腾讯科技（深圳）有限公司 | Resource scheduling method, device, equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101420458A (en) * | 2008-12-09 | 2009-04-29 | 清华大学 | Multimedia content monitoring system, method and device based on content distributing network |
| US20150095919A1 (en) * | 2010-10-27 | 2015-04-02 | Amazon Technologies, Inc. | Methods and system for swapping memory in a virtual machine environment |
| CN104572306A (en) * | 2015-01-28 | 2015-04-29 | 中国石油集团川庆钻探工程有限公司地球物理勘探公司 | Method for managing resources of computer cluster and resource manager |
| CN105323282A (en) * | 2014-07-28 | 2016-02-10 | 神州数码信息系统有限公司 | Enterprise application deployment and management system for multiple tenants |
| CN105487930A (en) * | 2015-12-01 | 2016-04-13 | 中国电子科技集团公司第二十八研究所 | Task optimization scheduling method based on Hadoop |
| CN105512083A (en) * | 2015-11-30 | 2016-04-20 | 华为技术有限公司 | YARN based resource management method, device and system |
-
2016
- 2016-07-18 CN CN201610570094.5A patent/CN106708622B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101420458A (en) * | 2008-12-09 | 2009-04-29 | 清华大学 | Multimedia content monitoring system, method and device based on content distributing network |
| US20150095919A1 (en) * | 2010-10-27 | 2015-04-02 | Amazon Technologies, Inc. | Methods and system for swapping memory in a virtual machine environment |
| CN105323282A (en) * | 2014-07-28 | 2016-02-10 | 神州数码信息系统有限公司 | Enterprise application deployment and management system for multiple tenants |
| CN104572306A (en) * | 2015-01-28 | 2015-04-29 | 中国石油集团川庆钻探工程有限公司地球物理勘探公司 | Method for managing resources of computer cluster and resource manager |
| CN105512083A (en) * | 2015-11-30 | 2016-04-20 | 华为技术有限公司 | YARN based resource management method, device and system |
| CN105487930A (en) * | 2015-12-01 | 2016-04-13 | 中国电子科技集团公司第二十八研究所 | Task optimization scheduling method based on Hadoop |
Cited By (50)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107273196A (en) * | 2017-05-31 | 2017-10-20 | 中国科学院北京基因组研究所 | Bioinformatics high-performance calculation job scheduling and system administration external member |
| WO2019041206A1 (en) * | 2017-08-31 | 2019-03-07 | Entit Software Llc | Managing containers using attribute/value pairs |
| US11816496B2 (en) | 2017-08-31 | 2023-11-14 | Micro Focus Llc | Managing containers using attribute/value pairs |
| CN108509268A (en) * | 2018-02-24 | 2018-09-07 | 宁波诺信睿聚投资有限责任公司 | Cluster resource distribution method, device, equipment and computer readable storage medium |
| CN108279982A (en) * | 2018-02-27 | 2018-07-13 | 郑州云海信息技术有限公司 | Pbs resources and hadoop method for managing resource, system and equipment |
| CN110659127A (en) * | 2018-06-29 | 2020-01-07 | 杭州海康威视数字技术股份有限公司 | Method, device and system for processing task |
| CN110719306A (en) * | 2018-07-11 | 2020-01-21 | 阿里巴巴集团控股有限公司 | Network request limiting method, computer equipment and storage medium |
| CN110719306B (en) * | 2018-07-11 | 2022-07-05 | 阿里巴巴集团控股有限公司 | Network request limiting method, computer equipment and storage medium |
| CN109086134A (en) * | 2018-07-19 | 2018-12-25 | 郑州云海信息技术有限公司 | A kind of operation method and device of deep learning operation |
| CN109104334A (en) * | 2018-08-23 | 2018-12-28 | 郑州云海信息技术有限公司 | The management method and device of monitoring system interior joint |
| CN109471727A (en) * | 2018-10-29 | 2019-03-15 | 北京金山云网络技术有限公司 | A kind of task processing method, apparatus and system |
| WO2020119117A1 (en) * | 2018-12-14 | 2020-06-18 | 平安医疗健康管理股份有限公司 | Distributed computing method, apparatus and system, device and readable storage medium |
| CN109783237B (en) * | 2019-01-16 | 2023-03-14 | 腾讯科技（深圳）有限公司 | Resource allocation method and device |
| CN109783237A (en) * | 2019-01-16 | 2019-05-21 | 腾讯科技（深圳）有限公司 | A kind of resource allocation method and device |
| US11586468B2 (en) | 2019-04-12 | 2023-02-21 | Huazhong University Of Science And Technology | Docker-container-oriented method for isolation of file system resources |
| CN110120940A (en) * | 2019-04-12 | 2019-08-13 | 华中科技大学 | A kind of file system resource partition method towards Docker container |
| CN110287000A (en) * | 2019-05-29 | 2019-09-27 | 北京达佳互联信息技术有限公司 | Data processing method, device, electronic equipment and storage medium |
| CN110287000B (en) * | 2019-05-29 | 2021-08-17 | 北京达佳互联信息技术有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN110278257A (en) * | 2019-06-13 | 2019-09-24 | 中信银行股份有限公司 | A kind of method of mobilism configuration distributed type assemblies node label |
| CN112148467A (en) * | 2019-06-28 | 2020-12-29 | 微软技术许可有限责任公司 | Dynamic allocation of computing resources |
| CN110389836A (en) * | 2019-07-17 | 2019-10-29 | 腾讯科技（深圳）有限公司 | A kind of more cluster management methods, device, server and storage medium |
| CN110413382A (en) * | 2019-08-06 | 2019-11-05 | 山东超越数控电子股份有限公司 | A kind of method, equipment and the readable medium of the resource dynamic adjustment of Docker container |
| CN112445602A (en) * | 2019-08-27 | 2021-03-05 | 阿里巴巴集团控股有限公司 | Resource scheduling method, device and system and electronic equipment |
| CN110704177A (en) * | 2019-09-04 | 2020-01-17 | 金蝶软件（中国）有限公司 | Computing task processing method and device, computer equipment and storage medium |
| CN110704177B (en) * | 2019-09-04 | 2022-06-10 | 金蝶软件（中国）有限公司 | Computing task processing method and device, computer equipment and storage medium |
| CN112835996A (en) * | 2019-11-22 | 2021-05-25 | 北京初速度科技有限公司 | Map production system and method thereof |
| CN110928666A (en) * | 2019-12-09 | 2020-03-27 | 湖南大学 | Method and system for optimizing task parallelism based on memory in Spark environment |
| CN110928666B (en) * | 2019-12-09 | 2022-03-22 | 湖南大学 | Method and system for optimizing task parallelism based on memory in Spark environment |
| CN111078404A (en) * | 2019-12-09 | 2020-04-28 | 腾讯科技（深圳）有限公司 | Computing resource determination method and device, electronic equipment and medium |
| CN111340613A (en) * | 2020-02-26 | 2020-06-26 | 中国邮政储蓄银行股份有限公司 | Job processing method, system and storage medium |
| CN111340613B (en) * | 2020-02-26 | 2023-10-03 | 中国邮政储蓄银行股份有限公司 | Job processing method, job processing system and storage medium |
| CN113703952B (en) * | 2020-05-20 | 2023-10-10 | 山东省计算中心（国家超级计算济南中心） | Resource allocation method for queue resource scheduling based on supercomputer |
| CN113703952A (en) * | 2020-05-20 | 2021-11-26 | 山东省计算中心（国家超级计算济南中心） | Resource allocation method for queue resource scheduling based on super computer |
| CN111767199B (en) * | 2020-06-24 | 2023-09-19 | 中国工商银行股份有限公司 | Resource management method, device, equipment and system based on batch job |
| CN111767199A (en) * | 2020-06-24 | 2020-10-13 | 中国工商银行股份有限公司 | Resource management method, device, equipment and system based on batch processing operation |
| CN113296929A (en) * | 2020-06-29 | 2021-08-24 | 阿里巴巴集团控股有限公司 | Resource matching method, device and system based on cloud computing |
| CN111813545A (en) * | 2020-06-29 | 2020-10-23 | 北京字节跳动网络技术有限公司 | Resource allocation method, device, medium and equipment |
| CN112463290A (en) * | 2020-11-10 | 2021-03-09 | 中国建设银行股份有限公司 | Method, system, apparatus and storage medium for dynamically adjusting the number of computing containers |
| CN112527454A (en) * | 2020-12-04 | 2021-03-19 | 上海连尚网络科技有限公司 | Container group scheduling method and device, electronic equipment and computer readable medium |
| CN112597502B (en) * | 2020-12-17 | 2023-02-10 | 山东乾云启创信息科技股份有限公司 | Large-scale computing service configuration method and system based on trusted cloud |
| CN112597502A (en) * | 2020-12-17 | 2021-04-02 | 山东乾云启创信息科技股份有限公司 | Large-scale computing service configuration method and system based on trusted cloud |
| US11609837B2 (en) | 2021-06-02 | 2023-03-21 | Kyndryl, Inc. | Calibration technique using computer analysis for ascertaining performance of containers |
| CN114168354B (en) * | 2022-02-11 | 2022-05-03 | 北京易源兴华软件有限公司 | Data-driven data cluster parallel computing allocation method and device |
| CN114168354A (en) * | 2022-02-11 | 2022-03-11 | 北京易源兴华软件有限公司 | Data-driven data cluster parallel computing allocation method and device |
| CN114826964A (en) * | 2022-04-11 | 2022-07-29 | 京东科技信息技术有限公司 | Resource monitoring method, device and system |
| CN114826964B (en) * | 2022-04-11 | 2024-04-05 | 京东科技信息技术有限公司 | Resource monitoring method, device and system |
| CN114640677A (en) * | 2022-05-17 | 2022-06-17 | 云宏信息科技股份有限公司 | Method for adding working nodes to cluster working nodes and APIServer by client configuration |
| CN115951988A (en) * | 2023-03-03 | 2023-04-11 | 北京并行科技股份有限公司 | Job scheduling method, computing device and storage medium |
| CN117009060A (en) * | 2023-09-27 | 2023-11-07 | 腾讯科技（深圳）有限公司 | Resource scheduling method, device, equipment and storage medium |
| CN117009060B (en) * | 2023-09-27 | 2024-01-12 | 腾讯科技（深圳）有限公司 | Resource scheduling method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106708622B (en) | 2020-06-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106708622A (en) | Cluster resource processing method and system, and resource processing cluster | |
| CN110290189B (en) | Container cluster management method, device and system | |
| CN108833197B (en) | Active detection method and detection platform based on cloud | |
| CN108399101A (en) | The methods, devices and systems of scheduling of resource | |
| CN109313564A (en) | For supporting the server computer management system of the highly usable virtual desktop of multiple and different tenants | |
| CN107968802A (en) | The method, apparatus and filtering type scheduler of a kind of scheduling of resource | |
| CN103399781B (en) | Cloud Server and virtual machine management method thereof | |
| CN110209492A (en) | A kind of data processing method and device | |
| CN105516267B (en) | Cloud platform efficient operation method | |
| CN107967175A (en) | A kind of resource scheduling system and method based on multiple-objection optimization | |
| CN112202879B (en) | Middleware management method and device, electronic equipment and storage medium | |
| CN115185697A (en) | Cluster resource scheduling method, system, equipment and storage medium based on kubernets | |
| CN112631680A (en) | Micro-service container scheduling system, method, device and computer equipment | |
| US8042158B2 (en) | Management of user authorizations | |
| CN109165135A (en) | A kind of data managing method, computer readable storage medium and terminal device | |
| CN112685157B (en) | Task processing method, device, computer equipment and storage medium | |
| CN112003931B (en) | Method and system for deploying scheduling controller and related components | |
| CN111866190B (en) | Multi-tenant management method, device and system based on project hierarchical management | |
| CN110225088A (en) | A kind of cloud desktop management method and system | |
| CN109818785A (en) | A kind of data processing method, server cluster and storage medium | |
| CN113760441A (en) | Container creation method and device, electronic equipment and storage medium | |
| US9942083B1 (en) | Capacity pool management | |
| CN110750350B (en) | Large resource scheduling method, system, device and readable storage medium | |
| CN102868594B (en) | Method and device for message processing | |
| CN115310835A (en) | Dispatching method and device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |