Background
Cloud computing technology has experienced advances from the first hardware virtualization, IaaS (Infrastructure as a Service), OpenStack, PaaS (Platform-as-a-Service), to container technology. The container realizes the split charging of the application, the application is convenient to migrate among different environments, the light weight characteristic enables the container to consume less resources and bring greater convenience, and more enterprise users select to migrate the deployment of the application to a containerization platform. Therefore, the traditional deployment mode begins to be changed into container-based deployment, the development of technologies such as CI/CD (Continuous Integration/Continuous deployment) is promoted, and the automatic deployment based on the CI/CD tool becomes the application deployment first choice for developers.
An existing automated deployment system architecture diagram is shown in fig. 1, and an application program implements code downloading and integration, compiling and packaging, code scanning, mirror image making, and automated deployment through a platform. There are two common deployment approaches:
1) full deployment: updating all codes running on the line, verifying by a tester after updating, deploying successfully if verification is passed, and deploying again by adopting full rollback and emergency repair when verification fails;
2) gray level release: and updating only part of the online example codes, testing and verifying the part of the examples, continuously updating other clusters if the verification is passed, and only returning the updated example codes when the verification fails.
In the prior art, deployment of applications depends on manual operation, flexibility is not enough, deployment and testing depend on manual operation, online deployment is manually completed by operation and maintenance personnel, after deployment is completed, the testing personnel intervene to perform service verification and testing, and when problems occur, developers are required to position and analyze the problems. The manual intervention links are more, rapid online deployment cannot be realized, problems cannot be timely positioned and solved, and the stability of the service can be influenced during online deployment.
Disclosure of Invention
In view of the foregoing, embodiments of the present invention provide a method, an apparatus, and an electronic device for automated deployment of a service, which overcome the foregoing problems or at least partially solve the foregoing problems.
According to an aspect of an embodiment of the present invention, there is provided an automated deployment method of a service, the method including: partially deploying the service; acquiring the performance and log data of the service; determining a health state value of the system according to the performance of the service and the log data; when the health state value of the system is lower than a strategy threshold value formulated by a rollback strategy, deploying rollback according to the rollback strategy; otherwise, completing the full deployment of the rest services according to the deployment strategy.
In an optional manner, the determining a health status value of the system according to the performance of the service and the log data further includes: analyzing the success rate of service request response according to the service performance and the log data; and when the success rate of the service request response is lower than a preset threshold value, comprehensively judging the health state of the system to obtain the health state value of the system.
In an optional manner, the comprehensively determining the health state of the system to obtain the health state value of the system further includes: acquiring performance data of the system; analyzing the performance data of the system to obtain a database health state value, a network health state value and a system load health state value of the system; and determining the health state value of the system according to the service request response success rate, the database health state value, the network health state value, the system load health state value and respective weights.
In an alternative, the performance data of the system includes one or more of CPU occupancy of the system, usage amount of memory, I/O performance of a disk, and network response time.
In an optional manner, the determining a health status value of the system according to the performance of the service and the log data further includes: determining an initial system health state value according to the service performance and the log data; when the initial health state value of the system is lower than a strategy threshold value formulated by a rollback strategy, obtaining the concurrency of the current system and the historical load condition of the current time period of the system; and if the deviation of the current system concurrency and the historical load condition of the current time period of the system does not exceed a preset value, determining the health state value of the system as a value not lower than a strategy threshold value formulated by the rollback strategy.
In an optional manner, the method further comprises: and when the health state value of the system is lower than a strategy threshold value formulated by a rollback strategy, generating an alarm event according to an alarm strategy.
In an optional manner, the deploying rollback according to the rollback policy further includes: querying a historical version of the service; and backing the service instance with the updated code to the latest historical version according to the deployment strategy.
According to another aspect of the embodiments of the present invention, there is provided an automated deployment apparatus for services, the apparatus including: the deployment module is used for carrying out partial deployment on the service; the acquisition module is used for acquiring the performance and the log data of the service; the determining module is used for determining the health state value of the system according to the performance of the service and the log data; the rollback module is used for deploying rollback according to the rollback strategy when the health state value of the system is lower than a strategy threshold value formulated by the rollback strategy; otherwise, completing the full deployment of the rest services according to the deployment strategy.
According to another aspect of the embodiments of the present invention, there is provided an electronic device including: the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete mutual communication through the communication bus; the memory is configured to store at least one executable instruction that causes the processor to perform the operations of the automated deployment method of services as described above.
According to another aspect of the embodiments of the present invention, there is provided a computer storage medium, wherein at least one executable instruction is stored in the storage medium, and the executable instruction causes a processor to execute the operations of the automated deployment method of services as described above.
According to the embodiment of the invention, only partial deployment is carried out on the service, then the running condition of the service is monitored through automatic test, so that the health state value of the system is determined, whether the deployment is continued or returned is determined according to the health state of the system, the automation of the whole deployment process is realized, the efficiency of online deployment of the system is improved, the manual intervention link is reduced, and the rapid iteration of the system is realized. And when the deployment is in a problem, the system can timely return, and the normal operation of the service is ensured.
The foregoing description is only an overview of the technical solutions of the embodiments of the present invention, and the embodiments of the present invention can be implemented according to the content of the description in order to make the technical means of the embodiments of the present invention more clearly understood, and the detailed description of the present invention is provided below in order to make the foregoing and other objects, features, and advantages of the embodiments of the present invention more clearly understandable.
Detailed Description
Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.
The embodiment of the invention provides a method for realizing automatic service deployment based on a container technology, which is combined with continuous integration, continuous delivery and containerization technologies to realize automatic service deployment in a system. In the online deployment process, automatic compiling, packaging and testing of the service codes are realized through continuous integration, and automatic deployment is realized by butting a containerization platform. When the deployment is carried out, the whole deployment is not directly carried out, but the partial deployment is carried out firstly, after the partial deployment, the verification of the service is realized through an automatic test (the monitoring of the current running state of the system is realized by analyzing the log of the service in real time), and the corresponding processing strategy is automatically executed according to the success rate of the service, and the deployment is continuously executed or returned, so that the automation of the whole deployment process is realized. By the scheme of the embodiment of the invention, the efficiency of online deployment of the system is improved, the manual intervention links are reduced, and the rapid iteration and online deployment capabilities of the system are realized. And when the deployment is in a problem, the system can timely back off, and the normal operation of the service is guaranteed.
Fig. 2 shows an architecture diagram of a system applied in the embodiment of the present invention, and fig. 3 shows a business process diagram in the embodiment of the present invention. As shown in fig. 2 and 3, the downloading, compiling, deploying, and automated testing of code implementing services through persistent integration. And when the service is deployed, the service is deployed in each container instance according to the version, and the service is released to the production environment through the containerization platform. The interactive process between the service deployment module and the containerization platform is as follows: downloading the service code; compiling and packaging; manufacturing a mirror image; pushing the mirror image to a mirror image warehouse; informing the containerization platform of the services and the mirror images needing to be deployed in an interface mode; deploying the service by the containerization platform; and querying the deployment state of the containerization platform.
The service monitoring module comprises a service monitoring unit, a strategy maintenance unit and a service success rate analysis unit, wherein the service monitoring unit collects and analyzes related performance and log data of service examples and collects the data, the service success rate analysis unit analyzes the service success rate according to the collected data to realize the monitoring of the whole service success rate, the strategy maintenance unit carries out system monitoring alarm strategy, service success rate alarm strategy, deployment strategy (partial deployment or full deployment can be selected in the deployment process) and the formulation and maintenance of rollback strategy, when the service success rate is lower than a threshold value, an alarm event is generated, whether rollback operation is carried out on the deployment of the service is judged according to the strategy, if rollback is needed, a historical version is inquired, rollback operation is carried out on the service examples with updated codes according to the deployment strategy, and the service examples are rolled back to the latest historical version, and if the deployment is full, updating the service by the latest code, so that the partial deployment is performed each time, and testing and verifying after the deployment is performed until the deployment is completely completed. Follow-up, whether to continue deployment or to rollback, requires automated testing and validation to be redone.
Specifically, the service monitoring unit monitors the service running state, and acquires a real-time log of the service, and data such as a host CPU, an internal memory, an I/O and the like; the service success rate analysis unit analyzes the condition of the service for responding to the request data according to the service log acquired by the service monitoring unit and judges whether the request is successful or failed. The strategy maintenance unit makes and maintains a system monitoring alarm strategy, a service success rate alarm strategy, a deployment strategy and a rollback strategy. These units are all software module units.
The present invention will be described in further detail below with reference to specific examples. Fig. 4 is a flowchart illustrating an automated deployment method of a service provided by an embodiment of the present invention. As shown in fig. 4, the method includes:
step 401: the service is partially deployed.
In this step, partial deployment, rather than full deployment, is performed on the service. And when the part of the deployed services pass the test verification, continuing to deploy part of the rest services. Compared with a one-time full deployment mode, the problem can be quickly found if the problem occurs, and the problem can be timely returned, so that all deployments are not wasted. The efficiency and the success rate of deployment are improved. Generally, when service deployment is performed, hundreds of hosts may need to be deployed, and if such many hosts are deployed at the same time, problems are difficult to find. Therefore, the embodiment of the invention only deploys part of the hosts, for example, only deploys 10% of the hosts, and then deploys the next 10% of the hosts after test and verification are carried out. This is until all hosts have been deployed.
Step 402: and acquiring the performance and the log data of the service.
In this step, the performance of the service and the log data may be acquired by a service monitoring unit of the service monitoring module.
Step 403: and determining the health state value of the system according to the performance of the service and the log data.
Step 404: when the health state value of the system is lower than a strategy threshold value formulated by a rollback strategy, deploying rollback according to the rollback strategy; otherwise, completing the full deployment of the rest services according to the deployment strategy.
The policy maintenance unit establishes a policy threshold of the health status value of the system, wherein the policy threshold is a threshold for whether the fallback needs to be deployed. The strategy threshold value can be obtained through multiple experiments according to the actual running condition, and can also be manually appointed in the strategy maintenance unit according to the service requirement. The threshold may be set to, for example, 90%. If the health state value of the system is lower than the strategy threshold value after test verification, rollback needs to be deployed; if the health state value of the system is not lower than the strategy threshold, the deployment is successful, and the deployment of other services can be continuously completed without returning. And when the health state value of the system is lower than a strategy threshold value formulated by a rollback strategy, generating an alarm event according to an alarm strategy. Further, before rollback, a historical version of the service needs to be queried, and the service instance with the updated code is rolled back to the latest historical version according to the deployment policy.
According to the embodiment of the invention, only partial deployment is carried out on the service, then the running condition of the service is monitored through automatic test, so that the health state value of the system is determined, whether the deployment is continued or returned is determined according to the health state of the system, the automation of the whole deployment process is realized, the efficiency of online deployment of the system is improved, the manual intervention link is reduced, and the rapid iteration of the system is realized. And when the deployment is in a problem, the system can timely return, and the normal operation of the service is ensured.
The health status value of the system may be determined according to different settings and different parameters, for example, the health status value of the system may be determined only according to the health status of the service, may also be determined based on the health status of the service in combination with the conditions of the database, the network, the system load, and the like of the system, and may further be determined in combination with the current system concurrency and the historical load condition of the system in the current time period on the basis of the conditions of the health status of the service, the database, the network, the system load, and the like of the system. These are further described below separately.
Fig. 5 is a flow chart illustrating a method for automated deployment of services provided by another embodiment of the present invention. As shown in fig. 5, the health status value of the system in the method may be determined only according to the health status of the service, and the method includes:
step 501: the service is partially deployed.
Step 502: and acquiring the performance and the log data of the service.
Step 503: and analyzing the success rate of the service request response according to the service performance and the log data.
Step 504: judging whether the service request response success rate is lower than a preset threshold value or not; if yes, go to step 505; otherwise, step 501 is executed until deployment is completed.
Step 505: and deploying rollback according to the rollback strategy.
FIG. 6 shows a flow chart of a method for automated deployment of services provided by yet another embodiment of the present invention. As shown in fig. 6, the health status value of the system in the method is determined based on the health status of the service, in combination with the database, the network, the system load, and the like of the system, and the method includes:
step 601: the service is partially deployed.
Step 602: and acquiring the performance and log data of the service and the performance data of the system.
The performance data of the system comprises one or more of CPU occupancy rate, memory usage amount, I/O performance of a disk and network response time of the system.
Step 603: analyzing the success rate of service request response according to the service performance and the log data; and analyzing the performance data of the system to obtain a database health state value, a network health state value and a system load health state value of the system.
If the success rate of the service request response is analyzed only according to the service performance and the log data, the most accurate system health condition may not be reflected. Because whether the whole system is operating normally is also influenced by other factors, such as a database, a network, system load, and the like, the health status of the system after operating the service needs to be judged from multiple dimensions.
Step 604: and determining the health state value of the system according to the service request response success rate, the database health state value, the network health state value, the system load health state value and respective weights.
And according to the overall requirements and importance degree of each index on the system, distributing corresponding weight to each index. Before the threshold value is judged, the health degree of the current system is determined according to each index data and each influence weight ratio. Specifically, the index values may be multiplied by the weights thereof and summed to obtain the health state value of the system.
Step 605: judging whether the health state value of the system is lower than a strategy threshold value formulated by a rollback strategy or not; if yes, go to step 606; otherwise, step 601 is executed until deployment is completed.
Step 606: and deploying rollback according to the rollback strategy.
FIG. 7 illustrates a flow chart of a method for automated deployment of services provided by yet another embodiment of the present invention. As shown in fig. 7, the health status value of the system in the method is determined based on the health status of the service, the database of the system, the network, the system load, and other conditions, further combining the current system concurrency and the historical load condition of the system in the current time period, and the method includes:
step 701: the service is partially deployed.
Step 702: and acquiring the performance and log data of the service and the performance data of the system.
Step 703: analyzing the success rate of service request response according to the service performance and the log data; and analyzing the performance data of the system to obtain a database health state value, a network health state value and a system load health state value of the system.
Step 704: and determining an initial system health state value according to the service request response success rate, the database health state value, the network health state value, the system load health state value and respective weights.
Step 705: judging whether the initial health state value of the system is lower than a strategy threshold value formulated by a rollback strategy or not; if yes, go to step 706; otherwise, step 701 is executed until deployment is completed.
Step 706: and acquiring the concurrency of the current system and the historical load condition of the current time period of the system.
Since the service operation sometimes has some specific scenes, such as promotion of a shopping website, the system load may be large or the success rate of service request response may be reduced, but this may not indicate that the system health is not good because of belonging to a specific scene. Therefore, at this time, whether the system is healthy or not can be further judged according to the system concurrency and the historical load condition of the system in the current time period, and if the system concurrency is large (for example, exceeds a preset threshold), the historical load condition and the current load condition are almost equal and both exceed a preset range, the deployment can be continued without performing rollback processing.
Step 707: judging whether the deviation of the current system concurrency and the historical load condition of the current time period of the system exceeds a preset value; if yes, go to step 708; otherwise, step 701 is executed until deployment is completed.
Step 708: and deploying rollback according to the rollback strategy.
In some embodiments, in step 707, when the deviation between the current system concurrency and the historical load condition of the system in the current time period does not exceed a preset value, the health state value of the system may be further determined to be a value not lower than a policy threshold value formulated by the fallback policy, and further, whether the health state value of the system is lower than the policy threshold value formulated by the fallback policy is determined, if yes, the fallback is deployed according to the fallback policy, otherwise, the full deployment of the remaining services is completed according to the deployment policy.
The specific application example of the invention combines the continuous integration and containerization technology, greatly reduces the deployment difficulty of the service system, shortens the online deployment time, realizes the automatic operation of the online deployment process, can monitor in real time and carry out quick response processing according to strategies aiming at the abnormal conditions of the service in the deployment process, improves the deployment efficiency of the system and ensures the stable operation of the system. The method and the system implement multidimensional monitoring according to the running condition of the system, consider a plurality of reference factors, and make a plurality of strategies to realize the response to the abnormal condition, thereby ensuring the normal running of the on-line system and reducing the influence on the service processing in the on-line process.
Fig. 8 is a schematic structural diagram of an automated deployment apparatus for services provided by an embodiment of the present invention. As shown in fig. 8, the apparatus 800 includes: a deployment module 801, an acquisition module 802, a determination module 803, and a rollback module 804.
The deployment module 801 is used for performing partial deployment on the service; the obtaining module 802 is configured to obtain performance and log data of the service; the determining module 803 is configured to determine a health status value of the system according to the performance of the service and the log data; the rollback module 804 is configured to deploy rollback according to a rollback policy when the health state value of the system is lower than a policy threshold formulated by the rollback policy; otherwise, completing the full deployment of the rest services according to the deployment strategy.
In an optional manner, the determining module 803 is further configured to analyze a service request response success rate according to the performance of the service and log data; and when the success rate of the service request response is lower than a preset threshold value, comprehensively judging the health state of the system to obtain the health state value of the system.
In an optional manner, the determining module 803 is further configured to obtain performance data of the system; analyzing the performance data of the system to obtain a database health state value, a network health state value and a system load health state value of the system; and determining the health state value of the system according to the service request response success rate, the database health state value, the network health state value, the system load health state value and respective weights.
In an alternative, the performance data of the system includes one or more of CPU occupancy of the system, usage amount of memory, I/O performance of a disk, and network response time.
In an optional manner, the determining module 803 is further configured to determine an initial health state value of the system according to the performance of the service and the log data; when the initial health state value of the system is lower than a strategy threshold value formulated by a rollback strategy, obtaining the concurrency of the current system and the historical load condition of the current time period of the system; and if the deviation of the current system concurrency and the historical load condition of the current time period of the system does not exceed a preset value, determining the health state value of the system as a value not lower than a strategy threshold value formulated by the rollback strategy.
In an optional manner, the apparatus further includes an alarm module 805 configured to generate an alarm event according to an alarm policy when the health status value of the system is lower than a policy threshold value set by the fallback policy.
In an optional manner, the rollback module 804 is further configured to query a historical version of a service; and backing the service instance with the updated code to the latest historical version according to the deployment strategy.
It should be noted that the automatic service deployment apparatus provided in the embodiment of the present invention is an apparatus capable of executing the automatic service deployment method, and all embodiments of the automatic service deployment method are applicable to the apparatus and can achieve the same or similar beneficial effects.
According to the embodiment of the invention, only partial deployment is carried out on the service, then the running condition of the service is monitored through automatic test, so that the health state value of the system is determined, whether the deployment is continued or returned is determined according to the health state of the system, the automation of the whole deployment process is realized, the efficiency of online deployment of the system is improved, the manual intervention link is reduced, and the rapid iteration of the system is realized. And when the deployment is in a problem, the system can timely return, and the normal operation of the service is ensured.
An embodiment of the present invention provides a computer storage medium, where at least one executable instruction is stored in the storage medium, and the executable instruction causes a processor to execute the steps of the automated deployment method for services in any of the above method embodiments.
According to the embodiment of the invention, only partial deployment is carried out on the service, then the running condition of the service is monitored through automatic test, so that the health state value of the system is determined, whether the deployment is continued or returned is determined according to the health state of the system, the automation of the whole deployment process is realized, the efficiency of online deployment of the system is improved, the manual intervention link is reduced, and the rapid iteration of the system is realized. And when the deployment is in a problem, the system can timely return, and the normal operation of the service is ensured.
Embodiments of the present invention provide a computer program product comprising a computer program stored on a computer storage medium, the computer program comprising program instructions which, when executed by a computer, cause the computer to perform the steps of the method for automated deployment of services in any of the above-described method embodiments.
According to the embodiment of the invention, only partial deployment is carried out on the service, then the running condition of the service is monitored through automatic test, so that the health state value of the system is determined, whether the deployment is continued or returned is determined according to the health state of the system, the automation of the whole deployment process is realized, the efficiency of online deployment of the system is improved, the manual intervention link is reduced, and the rapid iteration of the system is realized. And when the deployment is in a problem, the system can timely return, and the normal operation of the service is ensured.
Fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present invention, and the specific embodiment of the present invention does not limit the specific implementation of the electronic device.
As shown in fig. 9, the electronic device may include: a processor (processor)902, a communication Interface 904, a memory 906, and a communication bus 908.
Wherein: the processor 902, communication interface 904, and memory 906 communicate with one another via a communication bus 908. A communication interface 904 for communicating with network elements of other devices, such as clients or other servers. The processor 902 is configured to execute the program 910, and may specifically execute the automated deployment method of the service in any of the method embodiments described above.
In particular, the program 910 may include program code that includes computer operating instructions.
The processor 902 may be a central processing unit CPU, or an Application Specific Integrated Circuit (ASIC), or one or more Integrated circuits configured to implement an embodiment of the invention. The electronic device comprises one or more processors, which can be the same type of processor, such as one or more CPUs; or may be different types of processors such as one or more CPUs and one or more ASICs.
A memory 906 for storing the program 410. The memory 906 may comprise high-speed RAM memory, and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.
According to the embodiment of the invention, only partial deployment is carried out on the service, then the running condition of the service is monitored through automatic test, so that the health state value of the system is determined, whether the deployment is continued or returned is determined according to the health state of the system, the automation of the whole deployment process is realized, the efficiency of online deployment of the system is improved, the manual intervention link is reduced, and the rapid iteration of the system is realized. And when the deployment is in a problem, the system can timely return, and the normal operation of the service is ensured.
The algorithms or displays presented herein are not inherently related to any particular computer, virtual system, or other apparatus. Various general purpose systems may also be used with the teachings herein. The required structure for constructing such a system will be apparent from the description above. In addition, embodiments of the present invention are not directed to any particular programming language. It is appreciated that a variety of programming languages may be used to implement the teachings of the present invention as described herein, and any descriptions of specific languages are provided above to disclose the best mode of the invention.
In the description provided herein, numerous specific details are set forth. It is understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the embodiments of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the invention and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: that the invention as claimed requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.
Those skilled in the art will appreciate that the modules in the device in an embodiment may be adaptively changed and disposed in one or more devices different from the embodiment. The modules or units or components of the embodiments may be combined into one module or unit or component, and furthermore they may be divided into a plurality of sub-modules or sub-units or sub-components. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where at least some of such features and/or processes or elements are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
Furthermore, those skilled in the art will appreciate that while some embodiments herein include some features included in other embodiments, rather than other features, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments may be used in any combination.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names. The steps in the above embodiments should not be construed as limiting the order of execution unless specified otherwise.