CN111694666B - Task distribution management method, device, equipment and medium - Google Patents
Task distribution management method, device, equipment and medium Download PDFInfo
- Publication number
- CN111694666B CN111694666B CN202010522290.1A CN202010522290A CN111694666B CN 111694666 B CN111694666 B CN 111694666B CN 202010522290 A CN202010522290 A CN 202010522290A CN 111694666 B CN111694666 B CN 111694666B
- Authority
- CN
- China
- Prior art keywords
- task
- container group
- group copy
- executed
- copy
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000007726 management method Methods 0.000 title claims abstract description 36
- 230000004044 response Effects 0.000 claims abstract description 36
- 238000000034 method Methods 0.000 claims description 14
- 230000006378 damage Effects 0.000 claims description 10
- 230000007246 mechanism Effects 0.000 claims description 8
- 238000004590 computer program Methods 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 206010063385 Intellectualisation Diseases 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention discloses a task distribution management method, device, equipment and medium. The task distribution management method comprises the following steps: acquiring at least one item of task configuration information matched with a task to be executed, wherein the task configuration information comprises: time of destroying is timed; creating a container group copy in at least one node in the Kubernetes system, and distributing tasks to be executed to each container group copy for execution; and when the task end response fed back by each container group copy is obtained, starting to count the time of each container group copy, and destroying the corresponding container group copy when counting to the time of destroying. According to the technical scheme provided by the embodiment of the invention, the Broadcast Job resource object distributes the task to the container group copy, and destroys the container group copy after the task is completed, so that the Kubernetes resource can be saved.
Description
Technical Field
Embodiments of the present invention relate to computer technologies, and in particular, to a method, an apparatus, a device, and a medium for task distribution management.
Background
Along with the rapid development of internet technology, the financial industry gradually develops to the directions of internetworking, digitalization, mobile, automation and intellectualization, and the traditional financial technology development online mode has a longer verification period, the application architecture is difficult to change, and the traditional manual script operation and maintenance mode is challenged.
In the prior art, the DaemonSet object in the Kubernetes is adopted to ensure that one Pod copy runs on all nodes, so that tasks such as log collection, system monitoring and cluster storage are executed, however, once the Pod copy of the DaemonSet object is generated, the Pod copy runs on a server for a long time, and when some timing container tasks need to be executed, the long-time running may cause a certain loss to resources.
Disclosure of Invention
The embodiment of the invention provides a task distribution management method, a device, equipment and a medium, wherein a Broadcast Job resource object distributes a task to a container group copy, and destroys the container group copy after the task is completed, so that Kubernetes resources can be saved.
In a first aspect, an embodiment of the present invention provides a task distribution management method, where the method includes:
acquiring at least one item of task configuration information matched with a task to be executed, wherein the task configuration information comprises: time of destroying is timed;
creating a container group copy in at least one node in the Kubernetes system, and distributing the task to be executed to each container group copy for execution;
and when a task end response fed back by each container group copy is acquired, starting to count each container group copy, and destroying the corresponding container group copy when counting to the timed destroying time.
In a second aspect, an embodiment of the present invention further provides a task distribution management apparatus, where the apparatus includes:
the task configuration information acquisition module is used for acquiring at least one item of task configuration information matched with a task to be executed, and the task configuration information comprises: time of destroying is timed;
the task to be executed distributing module is used for creating a container group copy in at least one node in the Kubernetes system and distributing the task to be executed to each container group copy for execution;
and the container group copy destroying module is used for starting to time each container group copy when acquiring a task end response fed back by each container group copy, and destroying the corresponding container group copy when the time reaches the time of timed destroying.
In a third aspect, an embodiment of the present invention further provides an electronic device, including:
one or more processors;
a memory for storing one or more programs;
when the one or more programs are executed by the one or more processors, the one or more processors implement the task distribution management method provided by any embodiment of the present invention.
In a fourth aspect, an embodiment of the present invention further provides a computer readable storage medium having stored thereon a computer program, which when executed by a processor, implements the task distribution management method provided by any embodiment of the present invention.
According to the technical scheme, the Broadcast Job acquires at least one item of task configuration information matched with a task to be executed, wherein the task configuration information comprises: and when the time is counted to the time of destroying, the corresponding container group copy is destroyed, the container group copy can be destroyed after the task is completed, the container group copy does not run on a server for a long time, and the Kubernetes resource is saved.
Drawings
FIG. 1a is a flow chart of a task distribution management method in accordance with a first embodiment of the present invention;
FIG. 1b is a schematic diagram of creating a copy of a container group in accordance with a first embodiment of the present invention;
FIG. 2 is a flow chart of a task distribution management method in a second embodiment of the present invention;
FIG. 3 is a flow chart of a task distribution management method in a third embodiment of the present invention;
fig. 4 is a schematic structural view of a task distribution management device in a fourth embodiment of the present invention;
fig. 5 is a schematic structural diagram of an apparatus according to a fifth embodiment of the present invention.
Detailed Description
The invention is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting thereof. It should be further noted that, for convenience of description, only some, but not all of the structures related to the present invention are shown in the drawings.
Example 1
Fig. 1a is a flowchart of a task distribution management method according to a first embodiment of the present invention, where the technical solution of the present embodiment is suitable for a case of destroying a corresponding container group after task execution is completed, and the method may be performed by a task distribution management device, where the device may be implemented by software and/or hardware, and may be integrated in various general purpose computer devices, and specifically includes the following steps:
step 110, obtaining at least one item of task configuration information matched with a task to be executed, wherein the task configuration information comprises: and (5) timing the destruction time.
The task configuration information is specific parameters configured for a core resource predefined in a Broadcast Job for a current task to be executed, and an exemplary core resource predefined in the Broadcast Job includes a parallels field, where the parallels field represents the maximum number of copies of a container group that can be executed simultaneously, and the task configuration information is specific to the task to be executed, and may be assigned, for example, assigned as 2, and indicates that at most 2 copies of the container group can be executed simultaneously, where the numerical value is one item of the task configuration information.
In this embodiment, the Broadcast Job resource object obtains a plurality of task configuration information matched with the current task to be executed, where the task configuration information includes a time to destroy, and when the task to be executed is a time container task, for example, a user inputs in advance the task configuration information matched with the time container task, for example, in order to save resources, the task configuration information may be the time to destroy the task after the task is executed for 30 seconds, and the subsequent Broadcast Job resource object may destroy the container group copy according to the time to destroy input by the user. Of course, the task configuration information may include multiple items, and may also be a response type when the task fails, where the task configuration information may specify that the container group copy is destroyed when the task fails, or at least one retry is performed, where the container group copy is destroyed after the retry fails.
Optionally, the Broadcast Job resource object is obtained by initializing and creating a custom resource type CRD, and defining a resource set in the initialized Broadcast Job resource object;
the resource set comprises the following steps: at least one resource field name and data items to be filled corresponding to the resource field names respectively;
and the data item to be filled is associated with task configuration information in the Broadcast Job resource object.
In this optional embodiment, a declaration manner of a Broadcast Job resource object is provided, specifically, firstly, broadcast Job resource definition is performed through a custom resource type CRD, then, a resource set of the Broadcast Job resource object is defined, where the resource set includes at least one resource field name and to-be-filled data items corresponding to each resource field name, the resource field name is a spec declaration field of Broadcast Job in Kubernetes, and to-be-filled data items are associated with task configuration information in the Broadcast Job resource object, that is, to-be-filled data items can be filled for specific to-be-executed tasks.
Illustratively, at least one resource field name in the resource set includes at least one resource field name in template, paused, parallelism, failurePolicy and complexationpolicy, and meaning and type of corresponding fields of each resource field name are shown in table 1.
TABLE 1
Step 120, creating a container group copy in at least one node in the Kubernetes system, and distributing tasks to be executed to each container group copy for execution.
In this embodiment, after acquiring the task configuration information matched with the task to be executed, the Broadcast Job creates a container set copy in at least one node in the Kubernetes system, as shown in fig. 1b, specifically, one container set copy may be created on each node, or a container set copy matched with the Lable field content may be created according to a preset Lable field, and after creating the container set copy, the task to be executed is distributed to each container set copy for execution. The task to be executed may be a task that needs at least one copy of the container group to be executed, such as database backup, log collection, system monitoring, and the like.
Illustratively, the task currently to be performed is a database backup, and Broadcast Job runs one container group copy on each node in the Kubernetes system, that is, one container group copy on each node to perform a database backup operation. The task execution state of the container group copy may be obtained at each node, for example, by inputting kubectl get broadcastjobs statements, the container group copy states shown in table 2 are obtained:
TABLE 2
| NAME | DESIRED | ACTIVE | SUCCEEDED | FAILED |
| Broadcastjob-sample | 1 | 0 | 1 | 0 |
Where the Desired field indicates the Desired container group copy number, active is the currently running container group copy number, reserved is the completed container group copy number, failed is the container group copy number that Failed to execute the task.
And 130, starting timing for each container group copy when a task end response fed back by each container group copy is acquired, and destroying the corresponding container group copy when timing to the time of destroying.
In this embodiment, after a task end response fed back by at least one container group copy is obtained, the Broadcast Job resource object performs the timed destruction of the container group copy according to the timed destruction time contained in the task configuration information, specifically, when the task ends, timing is started, and when the timed destruction time arrives, the responding container group copy is destroyed.
For example, when the time for destroying is 30 seconds, the Broadcast Job starts to count when acquiring the task end response fed back by a certain container group copy, and destroys the container group copy after counting for 30 seconds, where the container group copy does not occupy Kubernetes resources for a long time.
According to the technical scheme, the Broadcast Job acquires at least one item of task configuration information matched with a task to be executed, wherein the task configuration information comprises: and when the time is counted to the time of destroying, the corresponding container group copy is destroyed, the container group copy can be destroyed after the task is completed, the container group copy does not run on a server for a long time, and the Kubernetes resource is saved.
Example two
Fig. 2 is a flowchart of a task distribution management method in a second embodiment of the present invention, which is further refined on the basis of the foregoing embodiment, and provides specific steps for creating a container group copy in at least one node in the Kubernetes system. The following describes a task distribution management method according to a second embodiment of the present invention with reference to fig. 2, including the following steps:
step 210, obtaining at least one piece of task configuration information matched with a task to be executed, where the task configuration information includes: and (5) timing the destruction time.
Optionally, the task configuration information further includes: template information for the container group copy.
In this alternative embodiment, another item of information included in the task configuration information is provided as template information for the container group copy, which may be used to construct the container group copy.
And 220, in at least one node in the Kubernetes system, creating a container group copy according to the template information of the container group copy, and distributing the task to be executed to each container group copy for execution.
In this embodiment, after the Broadcast Job obtains at least one task configuration information matched with a task to be executed, a container group copy is created in at least one node in the Kubernetes system according to template information of the container group copy, and then the task to be executed is distributed to each container group copy to be executed. Wherein the template information for the container group copy may define a series of rules or the like for creating the container group copy.
In another specific example of the embodiment of the present disclosure, the task configuration information further includes: the maximum number of concurrently running container group copies;
creating a container group copy in at least one node in the Kubernetes system, specifically comprising:
calculating the ideal number of the container group copies according to the task quantity of the task to be executed;
creating one container group copy each in an ideal number of nodes in the Kubernetes system if the ideal number does not exceed the maximum number;
if the ideal number exceeds the maximum number, one container group copy is created in each of the maximum number of nodes in the Kubernetes system.
In this specific example, another item of task configuration information is provided, namely, the maximum number of the container group copies running simultaneously, the data type of the task configuration information can be defined as integer, for example, the maximum number of the container group copies running simultaneously is defined as 3, at this time, the container group copies can be created according to the maximum number, specifically, firstly, the ideal number of the container group copies required is determined according to the task amount of the task to be executed, and if the ideal number does not exceed the maximum number defined by the user, one container group copy is created in each of the ideal number of nodes of the Kubernetes system, namely, the ideal number of container group copies are created; if the ideal number exceeds the maximum number, one container group copy is created in each of the maximum number of nodes of the Kubernetes system, i.e., the maximum number of container group copies is created.
For example, the maximum number of container group copies to be run simultaneously is defined as 3, when the number of container group copies to be run is calculated as 2 according to the task amount of the task to be executed, 2 container group copies can be directly created, and when the number of container group copies to be run is calculated as 4, only 3 container group copies can be created.
In another specific example of the embodiment of the present disclosure, the task configuration information further includes: a container group label;
creating a container group copy in at least one node in the Kubernetes system, specifically comprising:
in at least one node in the Kubernetes system, a container group copy is created that matches the container group label.
In this specific example, the task configuration information may further include a container group tag, i.e. a Lable field of the container group, where the Lable field may be used as an identifier of the container group, and when creating a container group copy in the Kubernetes system, a container group copy matching the container group tag may be created according to a preset Lable field value.
Step 230, when a task end response fed back by each container group copy is obtained, starting to time each container group copy, and destroying the corresponding container group copy when the time reaches the time of destroying.
According to the technical scheme of the embodiment, the Broadcast Job obtains at least one item of task configuration information matched with a task to be executed, wherein the task configuration information comprises: and the time for destroying is timed, in at least one node in the Kubernetes system, a container group copy is created according to template information of the container group copy, tasks to be executed are distributed to each container group copy for execution, when a task end response fed back by each container group copy is acquired, timing is started for each container group copy, when the timing is timed to the time for destroying, the corresponding container group copy is destroyed, the container group copy can be destroyed within a set time after the task is completed, and the container group copy does not occupy the Kubernetes resource for a long time.
Example III
Fig. 3 is a flowchart of a task distribution management method in a third embodiment of the present invention, where the steps after distributing tasks to be executed to the container group copies are provided in further detail on the basis of the above embodiments. The following describes a task distribution management method according to a third embodiment of the present invention with reference to fig. 3, including the following steps:
step 310, obtaining at least one piece of task configuration information matched with a task to be executed, where the task configuration information includes: and (5) timing the destruction time.
Optionally, the task configuration information further includes: the type of response when the task fails.
In this optional embodiment, another item of information included in the task configuration information is provided, which is a response type when the task fails, and the information is used to specify a specific response operation after the task failure is performed by the container group copy. For example, when the task of the container group fails, the copy of the container group may be destroyed directly, or a retry operation may be performed for a set number of times, and after the retry fails, the copy of the container group is destroyed.
Step 320, creating a container group copy in at least one node in the Kubernetes system, and distributing tasks to be executed to each container group copy for execution.
And 330, executing a failure response mechanism matched with the response type when the task fails when the task execution failure of the second target container group copy is detected.
In this embodiment, when the Broadcast Job resource object detects that the task execution of the second target container group copy fails, a failure response mechanism matching the response type when the task defined by the user fails is executed. For example, after detecting that the task of the second target container group copy fails to execute, the container group copy is not marked as failed to execute the task, but the container group copy is instructed to retry, and when the number of retries exceeds 3 and still fails to execute the task, the container group copy may be marked as failed to execute the task, or the container group copy may be destroyed directly.
In another specific example of the embodiment of the present disclosure, the task configuration information further includes: the pause control information includes pause enable or pause disable;
after distributing the task to be executed to each container group copy for execution, the method further comprises:
inquiring pause enabling information according to the detected pause instruction matched with the first target container group copy;
if the pause enabling information is pause enabling, instructing the first target container group copy to pause the distributed task;
if the pause enable information is pause disable, the pause instruction is ignored.
In this specific example, the task configuration information may further include pause control information, which is of boolean type and includes two states of pause enable and pause disable. When detecting a pause instruction matched with the first target container group, firstly inquiring the state of pause enabling information, if the pause enabling information is in the pause enabling state, indicating that the copy of the first target container group pauses the currently executed task, and if the pause enabling information is in the pause disabling state, directly ignoring the pause instruction and not carrying out pause processing on the currently executed task.
And 340, starting timing for each container group copy when the task end response fed back by each container group copy is obtained, and destroying the corresponding container group copy when timing to the time of destroying.
According to the technical scheme of the embodiment, the Broadcast Job obtains at least one item of task configuration information matched with a task to be executed, wherein the task configuration information comprises: and (3) timing destroying time, creating a container group copy in at least one node in the Kubernetes system, distributing a task to be executed to each container group copy for execution, executing a failure response mechanism matched with a response type when a task failure is detected, finally starting timing for each container group copy when a task end response fed back by each container group copy is acquired, and destroying the corresponding container group copy when timing to timing destroying time, wherein the container group copy can be destroyed within a set time after the task is completed, and the container group copy does not run for a long time, so that the Kubernetes resource is saved.
Example IV
Fig. 4 is a schematic structural diagram of a task distribution management device according to a fourth embodiment of the present invention, where the task distribution management device includes: the task configuration information acquisition module 410, the task to be executed distribution module 420 and the container group copy destruction module 430.
A task configuration information obtaining module 410, configured to obtain at least one task configuration information matched with a task to be performed, where the task configuration information includes: time of destroying is timed;
the task to be executed distributing module 420 is configured to create a container group copy in at least one node in the Kubernetes system, and distribute the task to be executed to each container group copy for execution;
the container group copy destroying module 430 is configured to start timing for each container group copy when a task end response fed back by each container group copy is obtained, and destroy the corresponding container group copy when the timing reaches the timing destroying time.
According to the technical scheme, the Broadcast Job acquires at least one item of task configuration information matched with a task to be executed, wherein the task configuration information comprises: and when the time is counted to the time of destroying, the corresponding container group copy is destroyed, the container group copy can be destroyed after the task is completed, the container group copy does not run on a server for a long time, and the Kubernetes resource is saved.
Optionally, the Broadcast Job resource object is obtained by initializing and creating a custom resource type CRD, and defining a resource set in the initialized Broadcast Job resource object;
the resource set comprises the following steps: at least one resource field name and data items to be filled corresponding to the resource field names respectively;
and the data item to be filled is associated with task configuration information in the Broadcast Job resource object.
Optionally, the task configuration information further includes: template information for the container group copy;
accordingly, the task distribution module to be executed 420 includes:
and the first container group copy creation unit is used for creating the container group copy according to the template information of the container group copy in at least one node in the Kubernetes system.
Optionally, the task configuration information further includes: the maximum number of concurrently running container group copies;
accordingly, the task distribution module to be executed 420 further includes:
the quantity calculating unit is used for calculating the ideal quantity of the container group copies according to the task quantity of the task to be executed;
a second container group copy creation unit configured to create one container group copy each in an ideal number of nodes in the Kubernetes system if the ideal number does not exceed the maximum number;
and a third container group copy creation unit for creating one container group copy each in the maximum number of nodes in the Kubernetes system if the ideal number exceeds the maximum number.
Optionally, the task configuration information further includes: pause control information including pause enable or pause disable;
correspondingly, the task distribution management device further comprises:
the enabling information inquiry module is used for inquiring pause enabling information according to the detected pause instruction matched with the first target container group copy after the task to be executed is distributed to each container group copy for execution;
a pause indication module, configured to instruct the first target container group copy to pause the distributed task if the pause enable information is pause enable;
and the instruction ignoring module is used for ignoring the pause instruction if the pause enabling information is pause prohibition.
Optionally, the task configuration information further includes: the response type when the task fails;
correspondingly, the task distribution management device further comprises:
and the identification response mechanism executing module is used for executing a failure response mechanism matched with the response type when the task of the second target container group copy fails when detecting that the task of the second target container group copy fails to execute after distributing the task to be executed to each container group copy for execution.
Optionally, the task configuration information further includes: a container group label;
accordingly, the task distribution module to be executed 420 further includes:
and a fourth container group copy creation unit, configured to create, in at least one node in the Kubernetes system, a container group copy that matches the container group label.
The task distribution management device provided by the embodiment of the invention can execute the task distribution management method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
Example five
Fig. 5 is a schematic structural diagram of an electronic device according to a fifth embodiment of the present invention, and as shown in fig. 5, the electronic device includes a processor 50 and a memory 51; the number of processors 50 in the device may be one or more, one processor 50 being taken as an example in fig. 5; the processor 50 and the memory 51 in the device may be connected by a bus or other means, for example in fig. 5.
The memory 51 is a computer readable storage medium, and may be used to store a software program, a computer executable program, and modules, such as program instructions/modules corresponding to a task distribution management method in an embodiment of the present invention (for example, a task configuration information acquisition module 410, a task distribution module to be executed 420, and a container group copy destruction module 430 in a task distribution management device). The processor 50 executes various functional applications of the apparatus and data processing by executing software programs, instructions and modules stored in the memory 51, i.e., implements the task distribution management method described above.
The method comprises the following steps:
acquiring at least one item of task configuration information matched with a task to be executed, wherein the task configuration information comprises: time of destroying is timed;
creating a container group copy in at least one node in the Kubernetes system, and distributing the task to be executed to each container group copy for execution;
and when a task end response fed back by each container group copy is acquired, starting to count each container group copy, and destroying the corresponding container group copy when counting to the timed destroying time.
The memory 51 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, at least one application program required for functions; the storage data area may store data created according to the use of the terminal, etc. In addition, memory 51 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage device. In some examples, memory 51 may further include memory located remotely from processor 50, which may be connected to the device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
Example six
A sixth embodiment of the present invention also provides a computer-readable storage medium having stored thereon a computer program for executing a task distribution management method when executed by a computer processor, the method comprising:
acquiring at least one item of task configuration information matched with a task to be executed, wherein the task configuration information comprises: time of destroying is timed;
creating a container group copy in at least one node in the Kubernetes system, and distributing the task to be executed to each container group copy for execution;
and when a task end response fed back by each container group copy is acquired, starting to count each container group copy, and destroying the corresponding container group copy when counting to the timed destroying time.
Of course, the storage medium provided by the embodiments of the present invention and including the computer executable instructions is not limited to the method operations described above, and may also perform the related operations in the task distribution management method provided by any embodiment of the present invention.
From the above description of embodiments, it will be clear to a person skilled in the art that the present invention may be implemented by means of software and necessary general purpose hardware, but of course also by means of hardware, although in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a FLASH Memory (FLASH), a hard disk or an optical disk of a computer, etc., and include several instructions for causing a computer device (which may be a personal computer, an application server, or a network device, etc.) to execute the method according to the embodiments of the present invention.
It should be noted that, in the above embodiment of the task distribution management device, each unit and module included are only divided according to the functional logic, but not limited to the above division, so long as the corresponding functions can be implemented; in addition, the specific names of the functional units are also only for distinguishing from each other, and are not used to limit the protection scope of the present invention.
Note that the above is only a preferred embodiment of the present invention and the technical principle applied. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, while the invention has been described in connection with the above embodiments, the invention is not limited to the embodiments, but may be embodied in many other equivalent forms without departing from the spirit or scope of the invention, which is set forth in the following claims.
Claims (8)
1. A task distribution management method, performed by a Broadcast Job resource object, comprising:
acquiring at least one item of task configuration information matched with a task to be executed, wherein the task configuration information comprises: time of timed destruction and response type when task fails; the task configuration information is a parameter of core resource configuration predefined in a Broadcast Job resource object aiming at the task to be executed; the task to be executed is a task which needs at least one container group copy to be executed;
creating a container group copy in at least one node in the Kubernetes system, and distributing the task to be executed to each container group copy for execution;
executing a failure response mechanism matched with the response type when the task fails when detecting that the task of the copy of the second target container group fails to execute;
when a task end response fed back by each container group copy is obtained, starting to count each container group copy, and destroying the corresponding container group copy when counting to the timed destroying time;
the Broadcast Job resource object is obtained by initializing and creating a custom resource type CRD, and defining a resource set in the initialized Broadcast Job resource object;
the resource set comprises the following steps: at least one resource field name and data items to be filled corresponding to the resource field names respectively; and the data item to be filled is associated with task configuration information in the Broadcast Job resource object.
2. The method according to claim 1, wherein the task configuration information further includes: template information for the container group copy;
creating a container group copy in at least one node in the Kubernetes system, specifically comprising:
in at least one node in the Kubernetes system, the container group copy is created according to template information of the container group copy.
3. The method according to claim 1, wherein the task configuration information further includes: the maximum number of concurrently running container group copies;
creating a container group copy in at least one node in the Kubernetes system, specifically comprising:
calculating the ideal number of the container group copies according to the task quantity of the task to be executed;
creating one container group copy each in an ideal number of nodes in the Kubernetes system if the ideal number does not exceed the maximum number;
if the ideal number exceeds the maximum number, one container group copy is created in each of the maximum number of nodes in the Kubernetes system.
4. The method according to claim 1, wherein the task configuration information further includes: pause control information including pause enable or pause disable;
after distributing the task to be executed to each container group copy for execution, the method further comprises:
inquiring pause enabling information according to the detected pause instruction matched with the first target container group copy;
if the pause enabling information is pause enabling, instructing the first target container group copy to pause the distributed task;
if the pause enable information is pause disable, the pause instruction is ignored.
5. The method according to claim 1, wherein the task configuration information further includes: a container group label;
creating a container group copy in at least one node in the Kubernetes system, specifically comprising:
in at least one node in the Kubernetes system, a container group copy is created that matches the container group label.
6. A task distribution management device, wherein the device is configured in a Broadcast Job resource object, the device comprising:
the task configuration information acquisition module is used for acquiring at least one item of task configuration information matched with a task to be executed, and the task configuration information comprises: time of timed destruction and response type when task fails; the task configuration information is a parameter of core resource configuration predefined in a Broadcast Job resource object aiming at the task to be executed; the task to be executed is a task which needs at least one container group copy to be executed;
the task to be executed distributing module is used for creating a container group copy in at least one node in the Kubernetes system and distributing the task to be executed to each container group copy for execution;
the failure response mechanism executing module is used for executing a failure response mechanism matched with the response type when the task fails when detecting that the task of the second target container group copy fails to execute;
the container group copy destroying module is used for starting to time each container group copy when acquiring a task end response fed back by each container group copy, and destroying the corresponding container group copy when the time reaches the time of timed destroying;
the Broadcast Job resource object is obtained by initializing and creating a custom resource type CRD, and defining a resource set in the initialized Broadcast Job resource object;
the resource set comprises the following steps: at least one resource field name and data items to be filled corresponding to the resource field names respectively; and the data item to be filled is associated with task configuration information in the Broadcast Job resource object.
7. An electronic device, the device comprising:
one or more processors;
a memory for storing one or more programs;
when executed by the one or more processors, causes the one or more processors to implement the task distribution management method of any of claims 1-5.
8. A computer-readable storage medium, on which a computer program is stored, characterized in that the program, when executed by a processor, implements the task distribution management method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010522290.1A CN111694666B (en) | 2020-06-10 | 2020-06-10 | Task distribution management method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010522290.1A CN111694666B (en) | 2020-06-10 | 2020-06-10 | Task distribution management method, device, equipment and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111694666A CN111694666A (en) | 2020-09-22 |
| CN111694666B true CN111694666B (en) | 2023-11-07 |
Family
ID=72480112
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010522290.1A Active CN111694666B (en) | 2020-06-10 | 2020-06-10 | Task distribution management method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111694666B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112099957B (en) * | 2020-11-02 | 2021-04-13 | 腾讯科技（深圳）有限公司 | Resource allocation method, system, device and electronic equipment |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108965485A (en) * | 2018-09-30 | 2018-12-07 | 北京金山云网络技术有限公司 | Management method, device and the cloud platform of container resource |
| CN109542605A (en) * | 2018-11-27 | 2019-03-29 | 长沙智擎信息技术有限公司 | A kind of container group life cycle management method based on Kubernetes system architecture |
| CN110489239A (en) * | 2019-08-22 | 2019-11-22 | 中国工商银行股份有限公司 | A kind of Container Management method, device and equipment |
| CN110764896A (en) * | 2018-07-25 | 2020-02-07 | 北京京东金融科技控股有限公司 | Resource allocation method, system, computer system and computer-readable storage medium |
| CN110888714A (en) * | 2019-11-26 | 2020-03-17 | 北京京东尚科信息技术有限公司 | Container scheduling method, device and computer-readable storage medium |
| CN110888743A (en) * | 2019-11-27 | 2020-03-17 | 中科曙光国际信息产业有限公司 | GPU resource using method, device and storage medium |
| CN111245900A (en) * | 2019-12-31 | 2020-06-05 | 北京健康之家科技有限公司 | Distributed message sending processing system and processing method thereof |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109491776B (en) * | 2018-11-06 | 2022-05-31 | 北京百度网讯科技有限公司 | Task arranging method and system |
-
2020
- 2020-06-10 CN CN202010522290.1A patent/CN111694666B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110764896A (en) * | 2018-07-25 | 2020-02-07 | 北京京东金融科技控股有限公司 | Resource allocation method, system, computer system and computer-readable storage medium |
| CN108965485A (en) * | 2018-09-30 | 2018-12-07 | 北京金山云网络技术有限公司 | Management method, device and the cloud platform of container resource |
| CN109542605A (en) * | 2018-11-27 | 2019-03-29 | 长沙智擎信息技术有限公司 | A kind of container group life cycle management method based on Kubernetes system architecture |
| CN110489239A (en) * | 2019-08-22 | 2019-11-22 | 中国工商银行股份有限公司 | A kind of Container Management method, device and equipment |
| CN110888714A (en) * | 2019-11-26 | 2020-03-17 | 北京京东尚科信息技术有限公司 | Container scheduling method, device and computer-readable storage medium |
| CN110888743A (en) * | 2019-11-27 | 2020-03-17 | 中科曙光国际信息产业有限公司 | GPU resource using method, device and storage medium |
| CN111245900A (en) * | 2019-12-31 | 2020-06-05 | 北京健康之家科技有限公司 | Distributed message sending processing system and processing method thereof |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111694666A (en) | 2020-09-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110781007B (en) | Task processing method, device, server, client, system and storage medium | |
| US9467460B1 (en) | Modularized database architecture using vertical partitioning for a state machine | |
| CN110502546A (en) | A kind of data processing method and device | |
| CN106528289B (en) | Resource operation processing method and device | |
| CN111831424B (en) | Task processing method, system and device | |
| CN105446806A (en) | Processing method and apparatus for application not responding | |
| CN111818117A (en) | Data updating method and device, storage medium and electronic equipment | |
| CN108665235B (en) | Resource processing node operation method, terminal device and medium | |
| CN111694666B (en) | Task distribution management method, device, equipment and medium | |
| CN111190753A (en) | Distributed task processing method and device, storage medium and computer equipment | |
| CN111177121A (en) | Order data feedback method and device, computer equipment and storage medium | |
| CN110941447A (en) | Directional release method, device and medium of application program and electronic equipment | |
| CN107547608A (en) | Distributed task scheduling treating method and apparatus | |
| CN109241042B (en) | Data processing method and device and electronic equipment | |
| CN113886590A (en) | Data summarizing method and device, computer equipment and storage medium | |
| CN106528551A (en) | Memory application method and apparatus | |
| CN110866007B (en) | Information management method, system and computer equipment for big data application and table | |
| CN104461741A (en) | Graphics device interface based computing device optimizing method and device | |
| CN104407808A (en) | Method and device for writing in data | |
| CN109408035B (en) | Flow configuration method, storage medium and server of business system | |
| CN115866017A (en) | Message processing method, message processing device, communication equipment and storage medium | |
| US11055133B2 (en) | Node-local-unscheduler for scheduling remediation | |
| CN114297067A (en) | Script testing method and device | |
| CN114722927A (en) | Collapse clustering method and device, electronic equipment and storage medium | |
| CN116263717A (en) | Order service processing method and device based on event |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20220915 Address after: 25 Financial Street, Xicheng District, Beijing 100033 Applicant after: CHINA CONSTRUCTION BANK Corp. Address before: 25 Financial Street, Xicheng District, Beijing 100033 Applicant before: CHINA CONSTRUCTION BANK Corp. Applicant before: Jianxin Financial Science and Technology Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |