CN116737387A - Cloud computing resource prediction system and method based on power grid digital service - Google Patents
Cloud computing resource prediction system and method based on power grid digital service Download PDFInfo
- Publication number
- CN116737387A CN116737387A CN202310794397.5A CN202310794397A CN116737387A CN 116737387 A CN116737387 A CN 116737387A CN 202310794397 A CN202310794397 A CN 202310794397A CN 116737387 A CN116737387 A CN 116737387A
- Authority
- CN
- China
- Prior art keywords
- resource
- prediction
- service
- data
- cloud
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 58
- 230000006872 improvement Effects 0.000 claims abstract description 55
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 36
- 238000003066 decision tree Methods 0.000 claims abstract description 15
- 238000005457 optimization Methods 0.000 claims abstract description 15
- 238000013439 planning Methods 0.000 claims abstract description 11
- 238000013138 pruning Methods 0.000 claims abstract description 10
- 238000013468 resource allocation Methods 0.000 claims abstract description 7
- 238000012549 training Methods 0.000 claims description 34
- 230000008569 process Effects 0.000 claims description 19
- 238000012360 testing method Methods 0.000 claims description 11
- 238000011156 evaluation Methods 0.000 claims description 9
- 238000012544 monitoring process Methods 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 6
- 230000006870 function Effects 0.000 claims description 6
- 238000007781 pre-processing Methods 0.000 claims description 5
- 238000011161 development Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000007635 classification algorithm Methods 0.000 description 2
- 238000002790 cross-validation Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 239000002699 waste material Substances 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000005265 energy consumption Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008092 positive effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5072—Grid computing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/06—Energy or water supply
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S10/00—Systems supporting electrical power generation, transmission or distribution
- Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- Business, Economics & Management (AREA)
- Economics (AREA)
- General Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Public Health (AREA)
- Mathematical Physics (AREA)
- Water Supply & Treatment (AREA)
- General Health & Medical Sciences (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- Strategic Management (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention relates to a cloud computing resource prediction system and a method based on a power grid digital service, wherein the system comprises a data acquisition module, a learning system module, a primary distribution prediction module and an efficiency improvement prediction module, wherein the data acquisition module is responsible for collecting network requests of all resource examples, calculating real-time tpmC values and constructing a data set; the learning system module takes the collected data set as input, establishes a cloud computing resource prediction model based on decision tree generation and pruning algorithm, and learns the data set; the primary allocation prediction module predicts the resource specification of the cloud stage of the service system planning based on the prediction model; the efficiency improvement prediction module adjusts the resource specification of the cloud operation stage of the service of the operated system based on the prediction model, and generates an efficiency improvement suggestion optimization list, so that subjective guesses of the service system in initial allocation and later efficiency improvement are eliminated, the scientificity and objectivity of resource prediction are improved, the resource allocation is more reasonable, and the work efficiency improvement is more efficient.
Description
Technical Field
The invention belongs to the technical field of power information, relates to a cloud computing resource prediction technology, and particularly relates to a cloud computing resource prediction system and method based on a power grid digital service.
Background
Along with the continuous expansion of the cloud computing scale, the resource management process of the cloud computing is more complex, and meanwhile, the resource waste condition is generated. It is counted that there is 30% cost waste for organizations using cloud computing, one of the important reasons being that resource utilization is too low. The improvement of the resource utilization rate can effectively save the operation cost of enterprises and reduce the energy consumption expenditure caused by idle resources. How to reasonably manage and distribute cloud resources and effectively improve the utilization rate of the resources has become one of the main challenges that current research needs to face and solve.
When cloud is planned in the prior art, the resource requirement of the system is mainly given by service personnel according to the actual condition of the system. Normally, the resource demand situation should be based on system testing. However, in the actual working process, many cloud systems do not perform complete system tests for various reasons, so the basis of the application resources is not objective enough, and the proposed resource demands are also in a situation of deficiency and high, which is an important reason for low resource utilization rate.
In addition, after the system is cloud-loaded, along with the upgrading of the service and the change of the scale of the user, the resources originally allocated to the system are likely to no longer meet the actual requirements, and the resource configuration is very necessary to be adjusted according to the actual performance data of the service system, which is the main content of the efficiency improvement work. The efficiency improvement work demand firstly gives out resource allocation optimization suggestions, and the current optimization suggestion algorithm mainly carries out halving or proportional adjustment based on the average utilization rate of resources in a certain period of time, so that the adjustment effect is not ideal, and the adjustment cannot be carried out in place at one time.
Therefore, a scientific, objective and reasonable cloud computing resource prediction system and method based on the power grid digital service are needed to be found.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provide a cloud computing resource prediction system and a cloud computing resource prediction method based on a power grid digital service, wherein the system and the method are based on performance data generated by a cloud service system, and learn the mapping relation between service performance requirements and resource specification configuration, so that the prior subjective judgment can be abandoned, and a resource measurement result meeting the current state of a cloud platform can be obtained; on the other hand, the system and the method can play a guiding role in efficiency improvement, and when efficiency improvement is carried out, reasonable virtual machine specifications can be obtained according to performance data generated by projects and a measuring and calculating model, and adjustment of the specifications is relatively objective. So long as the performance data of the service does not change significantly, the repeated efficiency-enhancing operation can be omitted.
The invention solves the technical problems by adopting the following technical scheme:
the cloud computing resource prediction system based on the power grid digital service comprises a data acquisition module, a learning system module, a primary allocation prediction module and an efficiency improvement prediction module, wherein the data acquisition module is responsible for collecting network requests of all resource examples, calculating a real-time tpmC value, constructing a data set required by the learning system module and preparing for training; the learning system module takes the collected data set as input, establishes a cloud computing resource prediction model based on decision tree generation and pruning algorithm, and realizes the learning of the data set; the primary allocation prediction module predicts the resource specification of the cloud stage on the business system planning based on a cloud computing resource prediction model, and mainly comprises the input of the cloud resource performance requirement of a new business system and the display of a prediction result; the efficiency improvement prediction module adjusts the resource specification of the cloud operation stage of the service of the operated system based on the cloud computing resource prediction model, generates an efficiency improvement suggestion optimization list, and lays a foundation for the subsequent efficiency improvement work.
A cloud computing resource prediction method based on power grid digital service by using the system comprises the following steps:
(1) Collecting network requests of each resource instance by utilizing a data acquisition module, calculating a real-time tpmC value, and constructing a model training data set;
(2) Training the data set obtained in the step (1) by using a cloud computing resource prediction model in a learning system module, wherein the cloud computing resource prediction model is based on a decision tree generation algorithm and a pruning algorithm, and the data set can be expressed as D = { (x) 1 ，y 1 )，(x 2 ，y 2 )，...，(x N ，y N )}，
Wherein the method comprises the steps ofFor the input instance, n is the number of features, +.>In order to make a class mark,CPU core number representing cloud resource instance, +.>Memory capacity GB representing cloud resource instances;
(3) After the learning system module learns the data acquired by the data acquisition module, the data enters a subsequent prediction module, and the prediction module predicts cloud computing resources by using the initial allocation prediction module and the efficiency improvement prediction module respectively according to a cloud-up stage of service system planning and a cloud-up operation stage of the service system, and is mainly applied to initial resource allocation prediction of the service system and efficiency improvement resource adjustment prediction.
And each resource instance of the service system in the data acquisition module needs to be deployed with a special performance monitoring agent for collecting performance data of the service system in the operation period, summarizing the agent data of all the operated resource instances, preprocessing the agent data, and calculating the actual tpmC value of the resource instance.
Moreover, the specific method for calculating the actual tpmC value of the resource instance comprises the following steps: according to the method, the performance data of each resource instance are divided according to the resource utilization rate, data segments meeting the utilization rate evaluation standard and having a time length of more than ten minutes are selected, then the average tpmC value of each segment of data is calculated, finally one piece of input of training data is formed together with other characteristic values of a learning algorithm, in order to improve the prediction efficiency, the real-time performance data of each resource instance in the past period are required to be processed, whether the average resource utilization rate meets the evaluation standard is calculated, if yes, resource specification adjustment is not required, if not, the average tpmC value is calculated, the input data is formed together with other characteristic data, and the input data is substituted into a prediction system to be calculated, so that the changed resource specification is obtained.
Moreover, the characteristic selection process of the cloud computing resource prediction model adopts an information gain algorithm, wherein the information gain is defined as g (D, A) =H (D) -H (D|A), D is a training data set, H (D) is the empirical entropy of D, and H (D|A) is the empirical conditional entropy after the characteristic A is given.
In addition, the decision tree generation algorithm adopts an ID3 algorithm or a C4.5 algorithm, and the pruning algorithm mainly considers the integral loss function C of the decision tree α To minimize the lossThe function can be defined as
The initial allocation prediction module predicts the resource specification of the new service system based on a prediction method, mainly comprises the input of the cloud resource performance requirement of the new service system, and the display of a prediction result, wherein in the cloud stage of service system planning, a service side needs to provide a specific characteristic value according to the characteristic obtained by the algorithm of the learning system module, and the calculation of the tpmC value is derived from a TPC-C benchmark test published by a transaction performance committee.
And the efficiency improvement prediction module adjusts the resource specification of the operated system based on a prediction method, generates efficiency improvement suggestion optimization, lays a foundation for subsequent efficiency improvement work, and in a cloud operation stage on a service, the prediction system adjusts the resource specification of the operated system according to performance data during operation of the service system, generates an efficiency improvement suggestion optimization list, the optimization list should be grouped by the service system, displays performance data such as actual utilization rate of each subordinate resource instance, and constructs input data according to the data acquisition process for the resource instances which do not meet the resource utilization rate, and then re-inputs the input data for prediction to obtain the changed resource instance specification.
The cloud computing resource prediction system based on the power grid digital service is applied to realizing a possible software deployment architecture, and can be deployed by adopting a container cloud, wherein a service component mainly comprises three services and one job, and specifically comprises the following steps: data acquisition service, initial allocation prediction service, efficiency improvement prediction service and training operation.
The data acquisition service is used for acquiring monitoring agent data of each resource instance deployed to the cloud platform, and constructing training data and input data for efficiency improvement prediction according to the preprocessing process; after training data is obtained, training operation can be started, and a prediction model is obtained after training is finished, wherein the process is a disposable process, so that the operation mode is adopted for deployment; the initial allocation prediction service and the efficiency improvement prediction service are respectively used for providing resource pre-allocation of a new service system and efficiency improvement work of an existing service system.
The invention has the advantages and positive effects that:
when the existing resource service planning is cloud-going, the method is too subjective, even if a server performance evaluation system including LinPack is adopted to plan resources, the obtained resource configuration is obtained based on experience values, and expected performance can not be obtained necessarily in actual operation. In the efficiency improvement work, the adopted optimization algorithm is to be improved, and the specification adjustment is not fully performed based on the historical performance data of the service system. According to the invention, a set of resource prediction model based on a machine learning algorithm is established, the model takes actual performance data of the cloud service system as a learning data set, and a mapping relation between the performance data and resource specification configuration is established, so that subjective guesses of the service system in initial allocation and later efficiency improvement are eliminated, the scientificity and objectivity of resource prediction are improved, the resource allocation is more reasonable, and the work efficiency improvement is more efficient.
Drawings
FIG. 1 is a diagram of a system module of the present invention;
FIG. 2 is a schematic diagram of a method for measuring and calculating resources in a learning system module according to the invention;
FIG. 3 is a diagram of a deployment framework for an application implementation of the present invention.
Detailed Description
The invention is further illustrated by the following examples, which are intended to be illustrative only and not limiting in any way.
A cloud computing resource prediction system based on a power grid digital service is shown in fig. 1, and comprises a data acquisition module, a learning system module, a primary distribution prediction module and an efficiency improvement prediction module. The data acquisition module is responsible for collecting network requests of all resource examples, calculating real-time tpmC values, constructing a data set required by the learning system module, and preparing for training; the learning system module takes the collected data set as input to complete the development of a learning system, including the generation of decision trees, the realization of pruning algorithms, the evaluation of learning effects and the like; the primary allocation prediction module predicts the resource specification of the cloud stage of the service system planning based on a prediction method, and mainly comprises the input of the cloud resource performance requirement of a new service system and the display of a prediction result; the efficiency improvement prediction module adjusts the resource specification of the cloud operation stage of the service of the operated system based on the prediction method, generates an efficiency improvement suggestion optimization list, and lays a foundation for the subsequent efficiency improvement work.
Specifically, each resource instance of the service system in the data acquisition module needs to be deployed with a special performance monitoring agent for collecting performance data such as the number of network requests and the like of the service system during operation. Collecting this data serves two purposes: firstly, training data is constructed, and secondly, prediction efficiency is improved.
In order to construct model training data, the proxy data of all the running resource instances need to be summarized and preprocessed, wherein the most important operation is the calculation of the actual tpmC value of the resource instance. Firstly, dividing performance data of each resource instance according to resource utilization rate in order of time, selecting data segments which meet utilization rate evaluation standards and have a duration of more than ten minutes, calculating an average tpmC value of each segment of data, and finally forming one input of training data together with other characteristic values of a learning algorithm.
To implement the improvement of prediction efficiency, performance data of each resource instance in real time during a past period of time needs to be processed. Firstly, calculating whether the average resource utilization rate meets an evaluation standard, if yes, not needing to adjust the resource specification, and if not, calculating the average tpmC value of the resource specification, and substituting the average tpmC value and other characteristic data into a prediction system to calculate so as to obtain the changed resource specification.
The learning system module is a resource prediction model based on a decision tree classification algorithm, the principle of a resource measurement and calculation method is shown in figure 2, a collected data set is taken as input, and development of a learning system is completed, wherein the development comprises decision tree generation, pruning algorithm realization, learning effect evaluation and the like;
the data set can be expressed as D = { (x) by the resource prediction model of the decision tree classification algorithm 1 ，y 1 )，(x 2 ，y 2 )，…，(x N ，y N ) }, whereinFor the input instance, n is the number of features, +.>For class mark->The number of CPU cores representing an instance of a cloud resource,
representing the memory capacity (GB) of the cloud resource instance.
Generally, one cloud resource instance corresponds to one or more virtual machine instances, the number of cores of a CPU is at least 1, and the memory capacity is at least 1GB. However, for some cloud resource instances, such as the Redis instance, under the action of the slicing mechanism, it is possible that multiple instances thereof correspond to one virtual machine instance, the memory capacity of the virtual machine instance may be less than 1GB, and because the Redis is a pure memory type resource, the CPU core number of the Redis is 0.
The feature selection process of the model adopts an information gain algorithm, wherein the information gain is defined as g (D, A) =H (D) -H (D|A), D is a training data set, H (D) is the empirical entropy of D, and H (D|A) is the empirical conditional entropy after the feature A is given.
The decision tree generation algorithm can adopt an ID3 algorithm or a C4.5 algorithm, and the pruning algorithm mainly considers the integral loss function of the decision tree
C α For minimization, the loss function may be defined as
The data set of the model inputs the characteristics of the instance, and the information comprises a host group, a CPU (Central processing Unit) excess ratio, a resource type, a tpmC value and a service type.
In this embodiment, it is assumed that the CPU and the memory of the physical host under the same host group adopt the same model, otherwise, the feature set of the input instance should also include information such as CPU architecture, CPU model, memory model, and the like; the resource types comprise resource component types which can be provided by a cloud computing platform, such as an elastic cloud server ECS, a container computing engine CCE, a relational database RDS and the like; traffic types can be classified as computationally intensive, data intensive, and general service type.
The general service type generally provides business services for the outside, and most of the business services of the existing cloud platform belong to the type. The computation-intensive algorithm is generally used for the computation-complex algorithm, such as the model training of artificial intelligence, and the part of the cloud platform has less business. The IO intensive type is mainly used for storing data and has higher requirement on a hard disk. Data-intensive typically provides data transfer and analysis for services, such as databases or message queues.
Meanwhile, the collected training data set may be divided into a training set and a test set according to a fixed ratio using a simple cross-validation process. And after training is finished, testing by using the testing set. And when the accuracy rate of the set test result is larger than the specified threshold, the test is considered to pass, otherwise, a more complex cross-validation process can be adopted, and parameters of the learning algorithm are adjusted until the test result meets the test standard.
After the learning system module learns the data acquired by the data acquisition module, the data enters a subsequent prediction module, and the prediction module predicts cloud computing resources by using the initial allocation prediction module and the efficiency improvement prediction module respectively according to a cloud-up stage of service system planning and a cloud-up operation stage of the service system, and is mainly applied to initial resource allocation prediction of the service system and efficiency improvement resource adjustment prediction.
The primary allocation prediction module predicts the resource specification of the new service system based on a prediction method, and mainly comprises the input of cloud resource performance requirements of the new service system and the display of a prediction result.
In the cloud stage of service system planning, the service side needs to provide specific characteristic values according to the characteristics obtained by the algorithm of the learning system module. Wherein the calculation of the tpmC value is derived from TPC-C benchmark published by the transaction Performance Committee.
The system should provide a specific interface or interfaces, which is convenient for the service side to provide the resource allocation requirement data about the service system. Typically, each business service is allocated a separate resource, and the data that needs to be filled in at the business side mainly includes the service name, the resource type, and the performance data calculated with respect to the expected tpmC value, including the number of users U during the peak period, the duration T (minutes) during the peak period, the number of requests K sent by each user during the peak period, and the number of transactions S included in each request. Tpmc=u×k×s×f/(t×c). Wherein F and C are empirical values. F represents traffic development reservation, f=125% if 25% of the processing capacity is reserved. C represents the CPU utilization, where CPU performance is optimal when CPU utilization is at 75%, and system bottlenecks occur when utilization is higher than 80%.
For container cloud resource prediction, when a workload has multiple instances, the tpmC value of the workload is the sum R of the tpmC values of the instances, and the tpmC value of the container cloud cluster is the sum R of the tpmC values of the workloads. Then the specification of each node of the cluster isN＝max(n，R/r max ，rp max ) Wherein r is i For each predicted specification of workload, n is the number of workload, r max For maximum workload specification, rp max For the maximum number of workload instances, N is the number of cluster nodes, which is taken from N,/-or->And rp max Is a maximum value of (a).
And the efficiency improvement prediction module is used for adjusting the resource specification of the operated system based on the prediction method, generating an efficiency improvement suggestion optimization list and laying a foundation for the subsequent efficiency improvement work.
In the cloud operation stage of the service, the prediction system can be according to performance data during operation of the service system. To this end, we need to deploy specialized monitoring agents on each resource instance for collection. The performance data mainly relates to the calculation of the tpmC value for each resource instance. The tpmC value of the run phase is specified in this embodiment as the number of requests that the service receives per minute during its run during a certain history period. In particular, for a database, the number of transactions completed per minute is represented. For workloads in a container cloud environment, the number of requests received per minute for each workload is represented. When there are multiple instances of the workload, the tpmC value is the sum of the tpmC values of the instances.
When the prediction system is applied to the container cloud resource, the specification of each node of the container cloud cluster is as followsN＝max(n，R/r max ，rp max ) Wherein r is i For each predicted specification of workload, n is the number of workload, r max For maximum workload specification, rp max For the maximum number of workload instances, N is the number of cluster nodes, which is taken from N,/-or->And rp max Is a maximum value of (a).
The optimization list should be grouped by service system, and the performance data such as actual utilization rate of each subordinate resource instance is displayed. And for the resource examples which do not meet the resource utilization rate, constructing input data according to the data acquisition process, and then inputting the input data into a prediction system for prediction to obtain the changed resource example specification.
Fig. 3 is a schematic diagram of a possible software deployment architecture for an application implementation of an embodiment of the present invention. As can be seen from the figure, the software implementation of the prediction system and method can be deployed by using a container cloud, and the business components mainly comprise three services and a job, specifically comprising: data acquisition service, initial allocation prediction service, efficiency improvement prediction service and training operation.
The data acquisition service is used for acquiring monitoring agent data of each resource instance deployed to the cloud platform, and constructing training data and input data for efficiency improvement prediction according to the preprocessing process; after training data is obtained, training operation can be started, and a prediction model is obtained after training is finished, wherein the process is a disposable process, so that the operation mode is adopted for deployment; the initial allocation prediction service and the efficiency improvement prediction service are respectively used for providing resource pre-allocation of a new service system and efficiency improvement work of an existing service system.
Although embodiments of the present invention have been disclosed for illustrative purposes, those skilled in the art will appreciate that: various substitutions, changes and modifications are possible without departing from the spirit and scope of the invention and the appended claims, and therefore the scope of the invention is not limited to the disclosure of the embodiments.
Claims (10)
1. A cloud computing resource prediction system based on a power grid digital service is characterized in that: the system comprises a data acquisition module, a learning system module, a primary allocation prediction module and an efficiency improvement prediction module, wherein the data acquisition module is responsible for collecting network requests of all resource examples, calculating a real-time tpmC value, constructing a data set required by the learning system module and preparing for training; the learning system module takes the collected data set as input, establishes a cloud computing resource prediction model based on decision tree generation and pruning algorithm, and realizes the learning of the data set; the primary allocation prediction module predicts the resource specification of the cloud stage on the business system planning based on a cloud computing resource prediction model, and mainly comprises the input of the cloud resource performance requirement of a new business system and the display of a prediction result; the efficiency improvement prediction module adjusts the resource specification of the cloud operation stage of the service of the operated system based on the cloud computing resource prediction model, generates an efficiency improvement suggestion optimization list, and lays a foundation for the subsequent efficiency improvement work.
2. A method for predicting a cloud computing resource based on a grid digital service as set forth in claim 1, wherein: the method comprises the following steps:
(1) Collecting network requests of each resource instance by utilizing a data acquisition module, calculating a real-time tpmC value, and constructing a model training data set;
(2) Training the data set obtained in the step (1) by using a cloud computing resource prediction model in a learning system module, wherein the cloud computing resource prediction model is based on a decision tree generation algorithm and a pruning algorithm, and the data set can be expressed as D = { (x) 1 ，y 1 )，(x 2 ，y 2 )，…，(x N ，y N )}，
Wherein the method comprises the steps ofFor the input instance, n is the number of features, +.>In order to make a class mark,CPU core number representing cloud resource instance, +.>Memory capacity GB representing cloud resource instances;
(3) After the learning system module learns the data acquired by the data acquisition module, the data enters a subsequent prediction module, and the prediction module predicts cloud computing resources by using the initial allocation prediction module and the efficiency improvement prediction module respectively according to a cloud-up stage of service system planning and a cloud-up operation stage of the service system, and is mainly applied to initial resource allocation prediction of the service system and efficiency improvement resource adjustment prediction.
3. The cloud computing resource prediction method based on the power grid digital service according to claim 2, wherein the method is characterized by comprising the following steps of: each resource instance of the service system in the data acquisition module needs to be deployed with a special performance monitoring agent for collecting performance data of the service system in the operation period, summarizing agent data of all the operated resource instances, preprocessing the agent data, and calculating an actual tpmC value of the resource instance.
4. A cloud computing resource prediction method based on a power grid digital service according to claim 3, wherein: the concrete method for calculating the actual tpmC value of the resource instance comprises the following steps: according to the method, the performance data of each resource instance are divided according to the resource utilization rate, data segments meeting the utilization rate evaluation standard and having a time length of more than ten minutes are selected, then the average tpmC value of each segment of data is calculated, finally one piece of input of training data is formed together with other characteristic values of a learning algorithm, in order to improve the prediction efficiency, the real-time performance data of each resource instance in the past period are required to be processed, whether the average resource utilization rate meets the evaluation standard is calculated, if yes, resource specification adjustment is not required, if not, the average tpmC value is calculated, the input data is formed together with other characteristic data, and the input data is substituted into a prediction system to be calculated, so that the changed resource specification is obtained.
5. The cloud computing resource prediction method based on the power grid digital service according to claim 2, wherein the method is characterized by comprising the following steps of: the characteristic selection process of the cloud computing resource prediction model adopts an information gain algorithm, wherein the information gain is defined as g (D, A) =H (D) -H (D|A), D is a training data set, H (D) is the empirical entropy of D, and H (D|A) is the empirical conditional entropy after the characteristic A is given.
6. The cloud computing resource prediction method based on the power grid digital service according to claim 2, wherein the method is characterized by comprising the following steps of: the decision tree generation algorithm adopts an ID3 algorithm or a C4.5 algorithm, and the pruning algorithm mainly considers the integral loss function C of the decision tree α For minimization, the loss function may be defined as
7. The cloud computing resource prediction method based on the power grid digital service according to claim 2, wherein the method is characterized by comprising the following steps of: the initial allocation prediction module predicts the resource specification of the new service system based on a prediction method, mainly comprises the input of cloud resource performance requirements of the new service system, and the display of a prediction result, wherein in the cloud stage of service system planning, a service side is required to provide a specific characteristic value according to the characteristics obtained by the algorithm of the learning system module, and the calculation of a tpmC value is derived from a TPC-C benchmark test published by a transaction performance committee.
8. The cloud computing resource prediction method based on the power grid digital service according to claim 2, wherein the method is characterized by comprising the following steps of: the efficiency improvement prediction module adjusts the resource specification of the operated system based on a prediction method, generates efficiency improvement suggestion optimization, lays a foundation for subsequent efficiency improvement work, in a business cloud operation stage, the prediction system adjusts the resource specification of the operated system according to performance data during operation of the business system, generates an efficiency improvement suggestion optimization list, the optimization list should be grouped by the business system, displays performance data such as actual utilization rate of each subordinate resource instance, and constructs input data according to the data acquisition process for the resource instances which do not meet the resource utilization rate, and then re-inputs the input data for prediction to obtain the changed resource instance specification.
9. The cloud computing resource prediction system based on the power grid digital service as set forth in claim 1 is applied to realizing a possible software deployment architecture, wherein the realization can adopt container cloud deployment, and the service components mainly comprise three services and one job, and specifically comprise: data acquisition service, initial allocation prediction service, efficiency improvement prediction service and training operation.
10. The cloud computing resource prediction system based on the power grid digital service according to claim 8 is applied to realizing a possible software deployment architecture, and is characterized in that: the data acquisition service is used for acquiring monitoring agent data of each resource instance deployed to the cloud platform, and constructing training data and input data for efficiency improvement prediction according to the preprocessing process; after training data is obtained, training operation can be started, and a prediction model is obtained after training is finished, wherein the process is a disposable process, so that the operation mode is adopted for deployment; the initial allocation prediction service and the efficiency improvement prediction service are respectively used for providing resource pre-allocation of a new service system and efficiency improvement work of an existing service system.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310794397.5A CN116737387A (en) | 2023-06-30 | 2023-06-30 | Cloud computing resource prediction system and method based on power grid digital service |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310794397.5A CN116737387A (en) | 2023-06-30 | 2023-06-30 | Cloud computing resource prediction system and method based on power grid digital service |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116737387A true CN116737387A (en) | 2023-09-12 |
Family
ID=87911458
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310794397.5A Pending CN116737387A (en) | 2023-06-30 | 2023-06-30 | Cloud computing resource prediction system and method based on power grid digital service |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116737387A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117667425A (en) * | 2023-12-25 | 2024-03-08 | 江苏博云科技股份有限公司 | Configuration optimization method and system for heterogeneous cloud resources |
-
2023
- 2023-06-30 CN CN202310794397.5A patent/CN116737387A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117667425A (en) * | 2023-12-25 | 2024-03-08 | 江苏博云科技股份有限公司 | Configuration optimization method and system for heterogeneous cloud resources |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110990159B (en) | Historical data analysis-based container cloud platform resource quota prediction method | |
| CN110389820B (en) | Private cloud task scheduling method for resource prediction based on v-TGRU model | |
| CN110096349B (en) | Job scheduling method based on cluster node load state prediction | |
| CN104317658B (en) | A kind of loaded self-adaptive method for scheduling task based on MapReduce | |
| US7110913B2 (en) | Apparatus and method for managing the performance of an electronic device | |
| US9219663B1 (en) | Managing the performance of an electronic device | |
| CN103401938B (en) | Based on resource allocation system and the method thereof of traffic performance under distributed cloud framework | |
| CN104283946A (en) | Resource self-adaptive adjusting system and method of multiple virtual machines under single physical machine | |
| CN110047014A (en) | A kind of user's electricity data restorative procedure based on load curve and history electricity | |
| CN104038392A (en) | Method for evaluating service quality of cloud computing resources | |
| CN105718364A (en) | Dynamic assessment method for ability of computation resource in cloud computing platform | |
| CN108416465A (en) | A kind of Workflow optimization method under mobile cloud environment | |
| CN116737387A (en) | Cloud computing resource prediction system and method based on power grid digital service | |
| CN104407688A (en) | Virtualized cloud platform energy consumption measurement method and system based on tree regression | |
| CN105373432A (en) | Cloud computing resource scheduling method based on virtual resource state prediction | |
| CN118093326B (en) | Cloud rendering resource monitoring method and system based on computing power scheduling | |
| CN112332404A (en) | Intelligent management system and method for heating service | |
| CN108132840A (en) | Resource regulating method and device in a kind of distributed system | |
| WO2024212614A1 (en) | Hybrid elastic scaling method based on multi-dimensional resource prediction | |
| Hao et al. | Energy allocation and task scheduling in edge devices based on forecast solar energy with meteorological information | |
| CN117493020A (en) | Method for realizing computing resource scheduling of data grid | |
| CN117410957A (en) | Power resource load scheduling system for virtual power plant | |
| Ismaeel et al. | Real-time energy-conserving vm-provisioning framework for cloud-data centers | |
| CN116820767A (en) | Cloud resource management method and device, electronic equipment and storage medium | |
| CN114936460B (en) | User electricity utilization time sequence optimization method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |