CN104978604B - A kind of analog simulation method and device based on professional ability model - Google Patents
A kind of analog simulation method and device based on professional ability model Download PDFInfo
- Publication number
- CN104978604B CN104978604B CN201410136981.2A CN201410136981A CN104978604B CN 104978604 B CN104978604 B CN 104978604B CN 201410136981 A CN201410136981 A CN 201410136981A CN 104978604 B CN104978604 B CN 104978604B
- Authority
- CN
- China
- Prior art keywords
- data
- service
- model
- modeling
- business
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 61
- 238000004088 simulation Methods 0.000 title claims abstract description 24
- 238000012545 processing Methods 0.000 claims description 114
- 230000001419 dependent effect Effects 0.000 claims description 60
- 238000007781 pre-processing Methods 0.000 claims description 48
- 230000015654 memory Effects 0.000 claims description 35
- 238000012795 verification Methods 0.000 claims description 21
- 230000000737 periodic effect Effects 0.000 claims description 15
- 230000005540 biological transmission Effects 0.000 claims description 13
- 238000012216 screening Methods 0.000 claims description 11
- 238000000611 regression analysis Methods 0.000 claims description 6
- 238000012314 multivariate regression analysis Methods 0.000 claims description 3
- 230000008569 process Effects 0.000 abstract description 5
- 238000009825 accumulation Methods 0.000 abstract description 2
- 239000011159 matrix material Substances 0.000 description 65
- 238000013507 mapping Methods 0.000 description 37
- 238000012935 Averaging Methods 0.000 description 21
- 238000010586 diagram Methods 0.000 description 17
- 238000004364 calculation method Methods 0.000 description 10
- 238000000354 decomposition reaction Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 238000012544 monitoring process Methods 0.000 description 6
- 238000004458 analytical method Methods 0.000 description 4
- 230000001186 cumulative effect Effects 0.000 description 4
- 238000007689 inspection Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 238000010219 correlation analysis Methods 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000005065 mining Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000002203 pretreatment Methods 0.000 description 2
- 238000003672 processing method Methods 0.000 description 2
- 238000012163 sequencing technique Methods 0.000 description 2
- 230000009469 supplementation Effects 0.000 description 2
- 238000009960 carding Methods 0.000 description 1
- 238000005094 computer simulation Methods 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000013178 mathematical model Methods 0.000 description 1
- 150000007524 organic acids Chemical class 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a kind of analog simulation methods based on professional ability model, comprising: pre-processes to professional ability data;Professional ability model is established according to pretreated professional ability data;According to the professional ability model, prediction of result is carried out, and shows prediction result.The invention also discloses a kind of simulation devices based on professional ability model can accurately assess professional ability using the present invention, give warning in advance to the case where service request amount Rapid Accumulation.
Description
Technical Field
The invention relates to the technical field of computer simulation, in particular to a simulation method and a simulation device based on a service capability model.
Background
At present, China mobile has entered full-service operation, and the scale and the number of devices of a service support network system are continuously expanded. The enterprise IT department faces important challenges, needs to make business planning, respond to rapid business change and guarantee the continuous development of business while reasonably controlling cost, thereby further improving the business service level.
The existing service support system has a plurality of service processing capacity bottlenecks, once the service volume exceeds the system processing capacity under a specific condition, a similar avalanche effect often occurs, the system operation efficiency is rapidly reduced, the service request volume is rapidly accumulated and is greatly overtime or failed, and large-area customer complaints are caused. Because there is not accurate ability assessment mechanism at present, can't accomplish effectual early warning in advance, in case above-mentioned circumstances appears, is difficult to accomplish the ability promotion fast in the short time, so, just causes the reduction of customer service quality in the longer time. Therefore, the research and the establishment of the service capability model evaluation mechanism of the service support network have very important significance.
Disclosure of Invention
In view of this, embodiments of the present invention are expected to provide a simulation method and apparatus based on a service capability model, which can accurately evaluate service capability and perform early warning on the condition of rapid accumulation of service request volume.
In order to achieve the purpose, the technical scheme of the invention is realized as follows:
the embodiment of the invention provides a simulation method based on a service capability model, which comprises the following steps:
preprocessing the service capability data;
establishing a service capability model according to the preprocessed service capability data;
and predicting the result according to the service capability model, and displaying the predicted result.
In the foregoing solution, before the predicting the result according to the service capability model, the method further includes: and carrying out precision verification on the service capability model.
In the above scheme, the preprocessing the service capability data includes, but is not limited to:
sorting capacity model data, and/or screening holidays and/or repeating and missing processing, and/or dividing idle and busy periods of data, and/or analyzing correlation, and/or selecting optimal periods of data; and
and processing CPU data, memory data, input/output I/O data and IOPS data of the times of reading and writing I/O operation per second of the capacity model.
In the foregoing solution, the establishing a service capability model according to the preprocessed service capability data includes: and sequentially carrying out business modeling, service modeling and component modeling according to the preprocessed business capability data.
In the above scheme, the business modeling includes: establishing a service model by taking time as an independent variable and short message voice singles, voice singles and GPRS (general packet radio service) singles as dependent variables;
the service modeling comprises: establishing a capability model by taking the dependent variable in the business modeling as an independent variable and taking the single quantity of the processing link as the dependent variable; wherein, the processing link ticket includes but is not limited to preprocessing, sorting, weight checking, price grading warehousing, opening checking, account closing, BI file transmission and periodic fee sorting;
the component modeling comprises: and establishing a component model by taking the dependent variable in the service modeling as an independent variable and taking the CPU utilization rate, the memory utilization rate, the total I/O amount and the IOPS times as the dependent variable.
In the foregoing solution, the predicting the result according to the service capability model includes: and predicting the resource consumption trend of each host, the total resource consumption trend of each application link and the resource consumption distribution of single-stroke ticket processing.
The embodiment of the invention also provides a simulation device based on the service capability model, which comprises: the device comprises a data processing unit, a model establishing unit and a result predicting unit; wherein,
the data processing unit is used for preprocessing the service capability data;
the model establishing unit is used for establishing a service capability model according to the preprocessed service capability data;
and the result prediction unit is used for predicting the result according to the service capability model and displaying the prediction result.
In the foregoing solution, the apparatus further includes an accuracy verification unit, configured to perform accuracy verification on the service capability model before the result prediction unit performs result prediction according to the service capability model.
In the above scheme, the preprocessing the service capability data by the data processing unit includes but is not limited to:
sorting capacity model data, and/or screening holidays and/or repeating and missing processing, and/or dividing idle and busy periods of data, and/or analyzing correlation, and/or selecting optimal periods of data;
and processing CPU data, memory data, input/output I/O data and IOPS data of the times of reading and writing I/O operation per second of the capacity model.
In the above scheme, the step of establishing the service capability model according to the preprocessed service capability data by the model establishing unit includes: and sequentially carrying out business modeling, service modeling and component modeling according to the preprocessed business capability data.
In the above scheme, the modeling the service by the model building unit includes: establishing a service model by taking time as an independent variable and short message voice singles, voice singles and GPRS (general packet radio service) singles as dependent variables;
the service modeling performed by the model establishing unit comprises: establishing a capability model by taking the dependent variable in the business modeling as an independent variable and taking the single quantity of the processing link as the dependent variable; wherein, the processing link ticket includes but is not limited to preprocessing, sorting, weight checking, price grading warehousing, opening checking, account closing, BI file transmission and periodic fee sorting;
the modeling of the component by the model establishing unit comprises: and establishing a component model by taking the dependent variable in the service modeling as an independent variable and taking the CPU utilization rate, the memory utilization rate, the total I/O amount and the IOPS times as the dependent variable.
In the above scheme, the predicting the result by the result predicting unit according to the service capability model includes: and predicting the resource consumption trend of each host, the total resource consumption trend of each application link and the resource consumption distribution of single-stroke ticket processing.
The embodiment of the invention provides a simulation method and a simulation device based on a service capability model, which are used for preprocessing service capability data; establishing a service capability model according to the preprocessed service capability data; and predicting the result according to the service capability model and displaying the predicted result. The embodiment of the invention predicts the operation condition of the future service supporting network system by establishing a scientific service capability model and combining historical data, thereby not only better controlling the operation quality of the service supporting network system and reducing the operation and maintenance risks; on the basis of fully mastering the handling capacity of each service, the resource utilization rate of the system is reasonably balanced through reasonable resource distribution, the system investment cost is reduced to a great extent, and the operation and maintenance management level of the service support network is effectively improved.
Furthermore, before the result prediction is carried out according to the business capability model, the precision verification can be carried out on the established business capability model, so that the deviation condition of the result predicted or fitted by the business capability model and the real result can be further known.
Drawings
FIG. 1 is a schematic flow chart of a simulation method based on a business capability model according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart of a business modeling method according to an embodiment of the present invention;
FIG. 3 is a schematic flow chart of a service modeling method according to an embodiment of the invention;
FIG. 4 is a flowchart illustrating a component modeling method according to an embodiment of the present invention;
FIG. 5 is a schematic flow chart of a capability model accuracy verification method according to an embodiment of the present invention;
FIGS. 6-a to 6-d are diagrams illustrating the estimation result of the resource consumption trend of each host according to the embodiment of the present invention;
FIGS. 7-a to 7-d are diagrams showing the predicted result of the total resource consumption trend of each application link according to the embodiment of the present invention;
FIGS. 8-a to 8-d are views showing the resource consumption distribution of single call ticket processing according to the embodiment of the present invention;
fig. 9 is a schematic structural diagram of a simulation device based on a service capability model according to an embodiment of the present invention.
Detailed Description
In the embodiment of the invention, the service capability data is preprocessed; establishing a service capability model according to the preprocessed service capability data; and predicting the result according to the service capability model, and displaying the predicted result.
Furthermore, before the result prediction is carried out according to the service capability model, the accuracy verification can be carried out on the established service capability model.
The embodiment of the invention discloses a simulation method based on a service capability model, which is characterized in that a mathematical model with a regression analysis method as a core is established, and the operation condition of a service support network system on a future corresponding date is predicted according to collected historical data; and under the support of service data, analyzing and early warning the service support network system.
The embodiment of the invention firstly needs to analyze and comb the service capability index, and the service capability modeling system architecture mainly comprises a service layer, a service layer and a component layer; the service layer comprises service indexes, service planning, service transaction types and single service workflow, and the core service transaction types are screened out according to the relation between the operation condition of the service support network system and a plurality of service types; the service layer comprises system response time, application processing capacity and the number of concurrent users, can reflect the performance pressure of a service support network system component and the transaction state of the service layer, can collect the workflow of multiple service transaction types into the index of the service layer, avoids the offline or newly increased influence of the service, and converts the pressure of the service layer into the pressure of the service layer; the component layer comprises performance data of each component of the service support network system, and can directly reflect the performance pressure load of the service support network system component. In the aspect of selecting the business capability data of the modeling, the business capability data is selected according to the analysis and carding conditions of the business layer, the service layer and the component layer, so that the reliability and the accuracy of the model can be effectively improved.
The simulation method based on the service capability model of the present invention is further described in detail with reference to the accompanying drawings and the specific embodiments.
The simulation method based on the service capability model in the embodiment of the invention, as shown in fig. 1, comprises the following steps:
step 101: preprocessing the service capability data;
here, the service capability data includes: short message phone bill quantity, voice phone bill quantity, General Packet Radio Service (GPRS) phone bill quantity and other phone bill quantities in a past period of time; processing link call bill quantity, including pretreatment, sorting, weight checking, price grading warehousing, opening inspection, account closing, BI file transmission and periodic fee sorting; CPU utilization rate, memory utilization rate, Input/Output (I/O) data, and number of read/write I/O operations Per Second (IOPS, Input/Output Operati/Ons Per Second) data; the past period of time may be within the last three years, or may be within other time ranges, and the embodiment of the present invention takes the service capability data within the last three years as an example, but is not limited to this range.
Correspondingly, the preprocessing of the service capability data includes but is not limited to: sequencing the service capacity data, and/or screening holidays and/or repeating and missing processing, and/or dividing idle and busy periods of data, and/or analyzing the correlation, and/or selecting the optimal period of data;
the preprocessing of the service capability data comprises the following processes:
a. and the service capability data are sequenced according to the time sequence, so that the phenomenon of time disorder is avoided, and a foundation is provided for subsequent data processing.
b. And respectively screening out the service capacity data of the working day and the holiday according to the time field.
c. Judging whether the service capability data is missing or overlapped, if so, namely, the monitoring values in the same time period repeatedly appear, keeping one record, and deleting redundant repeated records;
for the case of missing service capability data, in the case of missing only one value, interpolation supplementation is performed by using a mean value interpolation method, for example: missing value of 4 months occurrence = (time control in 3 months + historical monitoring value of 5 months)/2; if the data are missing for more than one day, performing single processing on a fake period, and if the data are missing for more than 5 hours, deleting the data in the whole day and processing the data according to a processing method for missing for more than one day; if the time is less than 5 hours, interpolation can be performed according to different interpolation methods as described below.
Specifically, the interpolation function is determined using the least squares method:
the first step is as follows: the independent variable and the dependent variable mean values are respectively calculated,
where X is time and Y is the desired interpolation;
the second step is that: computingWherein n is the number of independent variables, XiIs the ith independent variable;
the third step: computingYiIs the ith dependent variable;
the fourth step: calculating a slope factor a, wherein the formula is as follows:
the fifth step: calculating an intercept factor b, and calculating the formula:
or, determining an interpolation function based on a Lagrange interpolation method, and adopting a formula: y = a ((logx + log (x +1))/2) + b.
d. Dividing idle busy time periods of data;
the purpose of dividing the idle and busy periods of the data is to predict the idle and busy data respectively, so as to reduce the inaccuracy caused by predicting after averaging all the data;
mining busy and idle rules of a service support network system according to data of a past time interval; the data of a certain past period can be data of 1-3 months in the past, but is not limited to the range, and a required period can be set according to actual requirements;
specifically, the original data is averaged according to hours; for example, monitoring data is collected in units of 5 minutes or 1 minute, and averaged by hour; in the embodiment of the invention, taking 5 minutes as an example, overlapping 1-3 months of data collected by a system according to the data of different days and the same hour for calculating an average value, and counting the system operation trend in the same day;
determining a base line for judging the idle busy time period according to the historical data; here, the baseline criteria may be determined according to actual demand, for example, a period of time in which the collected data is greater than 70% of the historical average may be taken as a busy period, and the rest of the time may be an idle period.
e. Performing correlation analysis on the service capability data, and screening out the service capability data with the incidence relation;
here, the process of the business capability data correlation analysis includes: starting with the first set of business capability data in the business capability data, the business capability data of selection 1/3 establishes a correlation coefficient of the IT performance data and the business data, respectively. Wherein the IT performance data includes: CPU utilization rate, memory utilization rate, I/O data and IOPS data, wherein the service data comprises: short message phone bill quantity, voice phone bill quantity, GPRS phone bill quantity and other phone bill quantity. In calculating the correlation coefficient, the data of the previous 1/3 are respectively selected, and the correlation coefficient value of each data set is respectively calculated according to the step size of 1 until the last data set.
The correlation coefficient criteria are shown in table 1:
correlation | Negative values | Positive values |
Is not related | -0.09~-0.0 | 0.0~0.09 |
Low correlation | -0.3~-0.1 | 0.1~0.3 |
Moderate correlation | -0.5~-0.3 | 0.3~0.5 |
Significant correlation | -1.0~-0.5 | 0.5~1.0 |
TABLE 1
In practical application, a correlation coefficient is selected as a correlation factor according to actual requirements; for example: the factor having a correlation coefficient absolute value of 0.3 or more is selected as the correlation factor.
f. Selecting an optimal data time period of the capacity model;
here, since the change of the partial service support network system does not have detailed document record and query, and thus the reasonability and scientificity of model data input are affected, the accuracy of the model is improved by the optimal data selection of the historical data.
The specific calculation method comprises the following steps:
f1, determining the value range of m: starting from the last group of M groups of original data, the original data are selected forwardThe data of (1), e.g. the original data has data No. 1-30, the data No. 30 is used as the last group of data, and the data is selectedGet the originalIf the number of the obtained data sets is a decimal number, rounding up, namely the ceil generationRounding up the table. For example: when the M =31, the signal strength of the signal is high,then
Wherein, the value range of m is as follows: m is1,m1+1,m1M, e.g., the data of the first set of data M1 is {20, 21, … …, 30}, the data of the second set of data M1+1 is {19,20, … …, 30}, and so on, and the goodness of fit is calculated
f2, calculating the goodness of fit:
f21. calculating the mean value of the X axis and the mean value of the Y axis of the sample data, and recording as
f22. Calculating the residual square sum of sample data X and Y respectively:
f23. calculating the sum of variance:
f24. calculating a correlation coefficient RmThe concrete calculation formula is as follows:
f25. goodness of fit equal to the square of the standard covariance
f26. Selecting the optimal data set, calculated according to step f25And screening out the maximum value, and obtaining the data group number m at the moment, namely the most appropriate data group number m, as an optimal data set.
The step of preprocessing the service capability data is not limited to a specific sequence, that is, the steps a to f may be executed in any sequence or in parallel according to actual requirements.
The preprocessing the service capability data further comprises: processing CPU data, memory data, I/O data and IOPS data of the capability model; the method specifically comprises the following steps: processing CPU data, memory data, I/O data and IOPS data of the capacity model, wherein the peak data of busy hour in one day and the valley data of idle hour in one day;
specifically, according to the busy/idle time period determined in the preprocessing, the hour peak value of the service is averaged according to the busy/idle time period; mapping the time of the maximum/small value of the service transaction to the CPU performance utilization rate of the component, the MEM performance utilization rate of the component, the I/O of the component and the IOPS of the component; and mapping the service with the average value of busy/idle time peaks of the CPU, MEM and I/O, IOPS according to time.
The processing of the CPU data of the capacity model comprises processing of peak data of busy hour in one day and processing of valley data of idle hour of the CPU. Specifically, based on the business transaction, the maximum/minimum value of the business per hour is selected; according to the busy and idle time periods determined in the preprocessing, averaging the hour peak values of the services according to the busy/idle time periods; mapping the time of the maximum/small value of the business transaction to the CPU performance utilization rate of the component; converting the CPU utilization rate corresponding to the peak/valley time of the service into the CPU utilization time; and (4) mapping the peak/valley value average values of busy/idle time of the service and the CPU according to time.
For example, processing the peak data of the busy hour of the CPU in one day includes: based on the service transaction, selecting the maximum value of the service per hour, namely the service transaction of 5 minutes, selecting the maximum value MAX according to the hour, and recording the time of the maximum service value; and according to the busy and idle periods determined in the preprocessing, averaging the hour peak values of the services according to the busy periods. For example: the busy time period is 8: 00-23: 00, and the average value statistics is carried out on the hourly peak value of the business transaction according to 8: 00-23: 00; mapping the time of the maximum value of the business transaction to the CPU performance utilization rate of the component, if the system is a pricing and account closing system, mapping the time of the business transaction to the CPU utilization rates of the two physical machines, and averaging the CPU utilization rates, namely, the two physical machines are regarded as one device for processing; converting the CPU utilization rate corresponding to the service peak time into the CPU utilization time, namely the CPU utilization rate 300 s; the average value of the busy hour peak values of the service and the CPU is mapped according to time, so that a service capability model can be conveniently established later.
The processing of the free valley data of the CPU in one day comprises the following steps: based on the business transaction, selecting the minimum business value of each hour, namely the business transaction of 5 minutes, selecting MIN according to the hour, and recording the time of the minimum business value; and according to the busy and idle time periods determined in the preprocessing, averaging the hour valley values of the services according to the idle time periods. For example: the idle time period is 8: 00-23: 00, and the average value statistics is carried out on the business transaction hour valley value according to 8: 00-23: 00; mapping the time of the minimum value of the business transaction to the CPU performance utilization rate of the component, if the system is a pricing and account closing system, mapping the time of the business transaction to the CPU utilization rates of the two physical machines, and averaging the CPU utilization rates, namely, the two physical machines are regarded as one device for processing; converting the CPU utilization rate corresponding to the service peak time into the CPU utilization time, namely the CPU utilization rate 300 s; and mapping the business and the daily idle valley value average value of the CPU according to time, so as to be convenient for establishing a business capability model later.
The processing of the memory data of the capability model comprises: and processing peak data of busy hour in one day of the memory and processing valley data of idle hour. Specifically, based on the business transaction, the maximum/minimum value of the business per hour is selected; according to the busy and idle time periods determined in the preprocessing, averaging the hour peak values of the services according to the busy/idle time periods; mapping the time of the maximum/small value of the business transaction to the MEM performance utilization rate of the component; converting the MEM usage rate corresponding to the peak/bottom time of the service into the usage capacity of the MEM; and (4) mapping the peak value/valley value average values of the busy/idle times of the business and the MEM according to time.
For example, processing the peak data of busy hours in one day of the memory includes: based on the service transaction, selecting the maximum value of the service per hour, namely the service transaction of 5 minutes, selecting the maximum value MAX according to the hour, and recording the time of the maximum service value; and according to the busy and idle periods determined in the preprocessing, averaging the hour peak values of the services according to the busy periods. For example: the busy time period is 8: 00-23: 00, and the average value statistics is carried out on the hourly peak value of the business transaction according to 8: 00-23: 00; mapping the time of the maximum value of the business transaction to the MEM performance utilization rate of the component, if the system is a pricing and reconciliation system, mapping the time of the business transaction to the MEM utilization rates of the two physical machines, and averaging the MEM utilization rates, namely, the two physical machines are regarded as one device for processing; converting the MEM utilization rate corresponding to the service peak time into the MEM utilization capacity, namely MEM utilization rate configured capacity, wherein for a system consisting of two physical machines for pricing and accounting, the configured capacity is the sum of the configurations of the two physical machines, for example, the MEM configured capacity of a single physical machine is 6G, and the two physical machines are 12G; the average value of the busy hour peak value of the business and the MEM is mapped according to time, so that a business capability model can be conveniently established later.
The processing of the idle valley data in the memory within one day comprises the following steps: based on the business transaction, selecting the minimum business value of each hour, namely the business transaction of 5 minutes, selecting the minimum value MIN of 12 numerical values according to the hours, and recording the time of the minimum business value; and according to the busy and idle time periods determined in the preprocessing, averaging the hour valley values of the services according to the idle time periods. For example: the idle time period is 8: 00-23: 00, and the average value statistics is carried out on the business transaction hour valley value according to 8: 00-23: 00; mapping the time of the minimum value of the business transaction to the MEM performance utilization rate of the component, if the system is a pricing and account closing system, mapping the time of the business transaction to the MEM utilization rates of the two physical machines, and averaging the MEM utilization rates, namely, the two physical machines are regarded as one device for processing; and converting the MEM utilization rate corresponding to the service valley time into the MEM utilization capacity, namely, the MEM utilization rate is configured with capacity, and mapping is performed on the service and the daily idle valley average of the MEM according to time, so that a service capability model is conveniently established later.
The processing the I/O data of the capability model comprises: and processing peak data of busy hour in one day of the memory and processing valley data of idle hour. Specifically, based on the business transaction, the maximum/minimum value of the business per hour is selected; according to the busy and idle time periods determined in the preprocessing, averaging the hour peak values of the services according to the busy/idle time periods; mapping the time of the maximum/small value of the business transaction to the I/O of the component; and (4) mapping the peak/valley value average values of busy/idle times of the business and the I/O according to time.
The processing of the peak data of the busy hour in one day of the I/O comprises the following steps: based on the service transaction, selecting the maximum value of the service per hour, namely the service transaction of 5 minutes, selecting the maximum value MAX of 12 values according to the hour, and recording the time of the maximum service value; and according to the busy and idle periods determined in the preprocessing, averaging the hour peak values of the services according to the busy periods. For example: the busy time period is 8: 00-23: 00, and the average value statistics is carried out on the hourly peak value of the business transaction according to 8: 00-23: 00; mapping the time of the maximum value of the business transaction to the I/O of the component, if the system is a pricing and reconciliation system, mapping the time of the business transaction to the I/O of the two physical machines, and summing the I/O, namely, the two physical machines are regarded as one device for processing; the peak value average values of busy days and time of the service and the I/O are mapped according to time, so that a service capability model can be conveniently established later.
The processing of the I/O one-day idle valley data comprises the following steps: based on the business transaction, selecting the minimum business value of each hour, namely the business transaction of 5 minutes, selecting the minimum value MIN of 12 numerical values according to the hours, and recording the time of the minimum business value; and according to the busy and idle time periods determined in the preprocessing, averaging the hour peak values of the services according to the idle time periods. For example: the idle time period is 8: 00-23: 00, and the average value statistics is carried out on the business transaction hour valley value according to 8: 00-23: 00; mapping the time of the minimum value of the business transaction to the I/O of the component, if the system is a pricing and reconciliation system, mapping the time of the business transaction to the I/O of the two physical machines, and summing the I/O, namely, the two physical machines are regarded as one device for processing; and mapping the daily idle valley average values of the service and the I/O according to time, so as to be convenient for establishing a service capability model later.
The processing of the IOPS data of the capability model comprises: and processing peak data of busy hour in one day of the memory and processing valley data of idle hour. Specifically, based on the business transaction, the maximum/minimum value of the business per hour is selected; according to the busy and idle time periods determined in the preprocessing, averaging the hour peak values of the services according to the busy/idle time periods; mapping the time of the maximum/small value of the service transaction to the IOPS of the component; and (4) mapping the peak/valley value average values of busy/idle times of the service and the IOPS according to time.
The processing of peak data of busy hour in one day of IOPS comprises the following steps: based on the service transaction, selecting the maximum value of the service per hour, namely the service transaction of 5 minutes, selecting the maximum value MAX of 12 values according to the hour, and recording the time of the maximum service value; and according to the busy and idle periods determined in the preprocessing, averaging the hour peak values of the services according to the busy periods. For example: the busy time period is 8: 00-23: 00, and the average value statistics is carried out on the hourly peak value of the business transaction according to 8: 00-23: 00; mapping the time of the maximum value of the service transaction to the IOPS of the component, if the system is a pricing and reconciliation system, mapping the time of the service transaction to the IOPS of the two physical machines, and taking the sum of the IOPS, namely, the two physical machines are regarded as one device for processing; the peak value average values of busy days of the service and the IOPS are mapped according to time, so that a service capability model can be conveniently established later.
The processing of the data of the valley value of the idle time in one day of the IOPS comprises the following steps: based on the business transaction, selecting the minimum business value of each hour, namely the business transaction of 5 minutes, selecting the minimum value MIN of 12 numerical values according to the hours, and recording the time of the minimum business value; (note: if there is more than one service index, such as more than one service index of a reconciliation system, the numerical value of the service index of the system is accumulated according to a uniform time point, then the minimum value is selected, the average value of the small valley value of the service is taken according to the idle time period determined in the preprocessing, for example, the idle time period is 8: 00-23: 00, the average value statistics is carried out according to the small peak value of the service transaction 8: 00-23: 00, the time of the minimum value of the service transaction is mapped to the IOPS of the component, if the system is a pricing and reconciliation system, the time of the service transaction is mapped to the IOPS of two physical machines, the IOPS is taken as the sum value, namely the two physical machines are regarded as one device for processing, the average value of the free valley value of the service and the IOPS is mapped according to the time, and the service capability model is conveniently established later.
Step 102: establishing a service capability model according to the preprocessed service capability data;
specifically, business modeling, service modeling and component modeling are sequentially performed according to the preprocessed business capability data.
The business modeling comprises the following steps: establishing a service model by taking time as an independent variable and short message voice singles, voice singles and GPRS voice singles as dependent variables;
here, the service modeling mainly depends on a least square method, the independent variable X takes a time period of nearly three years, and the dependent variable Y takes a short message phone bill quantity, a voice phone bill quantity, a GPRS phone bill quantity and other phone bill quantities of nearly three years, fig. 2 is a schematic flow chart of the service modeling method according to the embodiment of the present invention, and as shown in fig. 2, the service modeling according to the embodiment of the present invention includes the following steps:
step 102 Aa: the independent variable and the dependent variable mean values are respectively calculated,
step 102 Ab: according to the obtained independent variable and dependent variable mean values, adoptingFormula (II)Calculating a slope factor a; wherein n is the number of independent variables, XiIs the ith independent variable, and Yi is the ith dependent variable;
step 102 Ac: adopting a formula according to the obtained slope factor aCalculating an intercept factor b;
step 102 Ad: obtaining a service model formula according to the slope factor a and the obtained intercept factor b: y = aX + b.
In all embodiments of the present invention, the three-year time period is taken as an example, and the time range is not limited, and in the actual data acquisition, the length of the selected time period may be determined according to the actual requirement.
The service modeling comprises: establishing a capability model by taking the dependent variable in the business modeling as an independent variable and taking the single quantity of the processing link as the dependent variable; wherein, the processing link ticket includes but is not limited to preprocessing, sorting, weight checking, price grading warehousing, opening checking, account closing, BI file transmission and periodic fee sorting;
the service modeling adopts a multiple regression analysis method to carry out modeling, wherein an independent variable x is a dependent variable y in the service modeling, namely a short message phone bill quantity, a voice phone bill quantity, a GPRS phone bill quantity and other phone bill quantities. y is a processing link phone bill quantity of nearly three years in the existing system, including preprocessing, sorting, duplicate checking, pricing warehousing, opening inspection, reconciliation, BI document transmission, and periodic fee sorting, and fig. 3 is a schematic flow chart of the service modeling method according to the embodiment of the present invention, and as shown in fig. 3, the service modeling according to the embodiment of the present invention includes the following steps:
step 102 Ba: determining a model correlation coefficient r;
specifically, all services causing server resource consumption in the same time period are determined according to the service relationship of the charging system, a correlation coefficient r between the services and the resource consumption is calculated, and the services with the absolute value of the correlation coefficient greater than a threshold value r are used as independent variables, wherein:which represents the mean of the independent variables,representing the mean of the dependent variables, xiIs the i-th independent variable, yiIs the ith dependent variable.
Here, all the services causing the consumption of the server resources in the same time period include: preprocessing, sorting, checking, warehousing price, opening and checking, closing accounts, transmitting BI files and sorting the periodic fee.
Step 102 Bb: determining a model multiple regression coefficient;
the first step is as follows: obtaining an input data matrix X, Y from the processed data:
the 1 st column of X is m sets of raw data of the 1 st argument, the 2 nd column is m sets of raw data of the 2 nd argument, and the rest columns of data are analogized in sequence, and the input data matrix X is an m × n matrix, n is the number of arguments, that is, the input argument data has m rows and n columns. The values of m, n will be used in the following calculation steps. The input data matrix X will be used in the following second and sixth steps.
The Y matrix is a matrix formed by m rows of data of the dependent variable data in the original data.
The second step is that: carrying out QR decomposition on the matrix data X in the first step;
and obtaining a Q matrix and an R matrix of the matrix X in the first step by using a QR decomposition algorithm, wherein the Q matrix is an m multiplied by m matrix, and the R matrix is an m multiplied by n matrix. The form is as follows:
the third step: selecting partial data of the matrix in the second step to obtain a new matrix R';
the new matrix R' is a matrix consisting of the first n rows and n columns of the R matrix in the second step, i.e.:
the fourth step: calculating a matrix D;
and the Q matrix is a matrix Q obtained by QR decomposition in the second step, and the Y matrix is an original data matrix Y in the first step. Here, QTUsing the rule of transposing the matrix, the calculation matrix D being a multiplication of the matrix by the matrixAnd (5) operation rules.
Taking the first n rows of data of the matrix D to obtain a new matrix D':
the fifth step: calculating and calculating output equation coefficients: b1,b2,...,bn
And substituting the matrix D' in the fourth step and the matrix R in the third step into the following equation coefficients.
In the calculation of bkIn the course of (1), if R is presentk,kIs equal to zero, then bk=0;
And a sixth step: calculating and fitting to obtain dependent variable result
The matrix operation form is:
wherein the X matrix is an independent variable input matrix X in the first step,for the equation of arrival calculated in the fifth stepIs a systemAnd (4) counting. x is the number of1,i,x1,i,...,xn,iRespectively represent independent variable x1,x2,..,xnRepresents the ith fit of the dependent variableAnd (4) data.
The seventh step: calculating goodness of fit R2;
Wherein,the i-th fitting data, y, representing the dependent variable obtained in the sixth stepiRepresents the ith data in the original factor number.
Eighth step: calculating the standard error Sy:
Wherein,the i-th fitting data, y, representing the dependent variable obtained in the sixth stepiRepresents the ith data in the original factor number. m sets of input data, n being the number of arguments, m having been guaranteed in the input data of the first step>n, so that the denominator is 0 does not occur here.
The ninth step: for the input new n independent variable values respectively being x10,x20,...,xn0Predicting the value y of the dependent variable by calculating the output0:y0=b1×x10+b2,i×x20+…+bn×xn0;
Wherein, b1,b2,...,bnFor the result of the fifth calculation, x10,x20,...,xn0Requiring re-entry of the argument data for the user.
Step 102 Bc: carrying out data prediction of the model;
here, the multiple regression coefficient is forced to 0.
Firstly, determining the number m of data of the prediction dependent variable data; an integer of m = n (1/3) is taken, where n is the number of data items of the dependent variable in calculating the multiple regression coefficient.
Next, data prediction is performed by substituting the independent variable data X into equation y =0+ b1x1+b2x2+…+bpxpObtaining a prediction result; wherein, b1,b2…bpThe obtained multiple regression coefficients are obtained.
The component modeling comprises: and establishing a component model by taking the dependent variable in the service modeling as an independent variable and taking the corresponding CPU utilization rate, memory utilization rate, total I/O amount and IOPS times as the dependent variable.
The component modeling and the service modeling are the same, and a multivariate regression analysis method is adopted for modeling, wherein an independent variable x is a dependent variable y of the service modeling, and the y is the CPU utilization rate, the memory utilization rate, the total I/O amount and the IOPS times of nearly three years; fig. 4 is a schematic flowchart of a component modeling method according to an embodiment of the present invention, and as shown in fig. 4, the component modeling method according to the embodiment of the present invention includes the following steps:
step 102 Ca: determining a model correlation coefficient r;
here, the step of determining the model correlation coefficient r is the same as step 102 Ba.
Step 102 Cb: determining a model multiple regression coefficient;
the first step is as follows: obtaining an input data matrix X, Y from the processed data:
wherein, the 1 st column of X is all 1, the 2 nd column is m sets of original data of the 1 st argument, the 3 rd column is m sets of original data of the 2 nd argument, and the other columns of data are analogized in turn, and the input data matrix X is a matrix of m × (n +1), that is, the input argument data has m rows and n columns. The values of m, n will be used in the following calculation steps. The input data matrix X will be used in the following second and sixth steps.
Wherein the Y matrix is a matrix formed by m rows of data of dependent variable data in the original data, and the matrix Y
The second step is that: carrying out QR decomposition on the matrix data X in the first step;
obtaining a Q matrix and an R matrix of the matrix X in the first step by using a QR decomposition algorithm, wherein the Q matrix is an m multiplied by m matrix, and the R matrix is an m multiplied by (n +1) matrix; the form is as follows:
the third step: selecting partial data of the matrix in the second step to obtain a new matrix R';
the new matrix R' is a matrix consisting of the first n +1 rows and n +1 columns of the R matrix in the second step, i.e.:
the fourth step: calculating a matrix D;
the Q matrix is obtained by QR decomposition in the second step, and the Y matrix is the original data matrix Y in the first step; taking the first n +1 rows of data of the matrix D to obtain a new matrix D':
the fifth step: calculating the output equation coefficients: b0,b1,b2,...,bn;
And substituting the matrix D' in the fourth step and the matrix R in the third step into the following equation coefficients.
In the calculation of bkIn the course of (1), if R is presentk,kIs equal to zero, then bk=0;
And a sixth step: calculating and fitting to obtain dependent variable result
The matrix operation form is:
wherein; the X matrix is the argument input matrix X in the first step,for the system of equations calculated in the fifth stepAnd (4) counting. Here, x1,i,x1,i,...,xn,iRespectively represent independent variable x1,x2,..,xnRepresents the ith group of the dependent variableAnd (4) fitting the data.
The seventh step: calculating goodness of fit R2;
Firstly, calculating the average value of the dependent variable y in m groups of data in the original data:
recalculate goodness of fit R2:
Wherein,the i-th fitting data, y, representing the dependent variable obtained in the sixth stepiRepresents the ith data in the original factor number.
Eighth step: calculating the standard error Sy;
Wherein,the i-th fitting data, y, representing the dependent variable obtained in the sixth stepiRepresents the ith data in the original factor number. m sets of input data, n being the number of arguments, m having been guaranteed in the input data of the first step>n +1, so that the denominator is 0 does not occur here.
The ninth step: for the input new n independent variable values respectively being x10,x20,...,xn0Predicting the value y of the dependent variable by calculating the output0:y0=b0+b1×x10+b2,i×x20+…+bn×xn0;
Wherein, b0,b1,b2,...,bnFor the result of the fifth calculation, x10,x20,...,xn0New input argument data is needed for the user.
Step 102 Cc: carrying out data prediction of the model;
here, the multiple regression coefficient is forced to 0.
Firstly, determining the number m of data of the prediction dependent variable data; an integer of m = n (1/3) is taken, where n is the number of data items of the dependent variable in calculating the multiple regression coefficient.
Then data prediction is carried out, and the independent variable data X is substituted into the equation y =0+ b1x1+b2x2+…+bpxpObtaining a prediction result; wherein, b1,b2…bpThe obtained multiple regression coefficients are obtained.
Step 103: performing precision verification on the service capability model;
in practical applications, this step may be used as an optional step, and the purpose of this step is to perform precision verification is to further understand the deviation situation of the result predicted by the business capability model or the fitted result from the real result.
And if the relatively large error accumulated distribution exists in the relatively small error, the prediction result or the fitting result is more ideal, namely the prediction result or the fitting result is more toward the true value.
In practical application, the relative error is plotted as a horizontal axis and the relative error cumulative distribution is plotted as a vertical axis, and the error condition of the predicted result or the fitted result and the actual result can be more visually observed through a chart. FIG. 5 is a schematic flow chart of a capability model accuracy verification method according to an embodiment of the present invention, including the following steps:
step 103A: calculating a temporary relative error:
wherein n is the number of real results.
Step 103B: determining a maximum value max (RE) and a minimum value min (RE) of RE;
step 103C: calculating the relative error aj:
M +1 data are generated, where m =5n, and n is the number of real results.
Step 103D: calculating a relative error cumulative distribution;
a) calculating the temporary relative error RE smaller than or equal to ajThe number k of data is that the RE is taken as a whole, find the RE is smaller than or equal to ajThe number k of data. Wherein j =1,2,. ereisma, m + 1;
b)wherein n is the number of real results.
Step 103E: with a relative error of ajAs a horizontal axis, relative error cumulative distribution CDFjPlotted as the vertical axis.
Step 104: and predicting the result according to the service capability model, and displaying the predicted result.
The predicting the result according to the service capability model comprises the following steps: and predicting the resource consumption trend of each host, the total resource consumption trend of each application link and the resource consumption distribution of single-stroke ticket processing.
After model accuracy verification is carried out on a modeling formula, analysis and modeling are carried out on data of an existing charging system, the model is applied to trend prediction of resource consumption, resource consumption prediction of a single ticket and overall resource consumption prediction of each application aiming at a single host, the data of 24 days in 2 months in 2012 is predicted by taking the data of 24 days in 2 months in 2011 as an example, and CPU, MEM and I/O, IOPS consumed by processing the single ticket are analyzed.
Wherein the estimating of the resource consumption trend of each host comprises the following steps: according to the modeling step of the capability model, a component model formula y =0+ b is applied1x1+b2x2+…+bpxpCalculating and estimating resource consumption of the host as-jf1, taking an independent variable x as a duplicate checking, sorting and preprocessing phone bill quantity, and respectively calculating the CPU utilization rate, the memory utilization rate, the total I/O quantity and the total IOPS times; FIGS. 6-a to 6-d areThe embodiment of the invention discloses a display diagram of the estimated result of the resource consumption trend of each host; FIG. 6-a is a diagram illustrating a CPU (core count) utilization rate prediction result according to an embodiment of the present invention; FIG. 6-b is a graph illustrating MEM (KB) usage prediction results in accordance with an embodiment of the present invention; FIG. 6-c is a schematic diagram of the predicted I/O (KB) result of the present invention; FIG. 6-d is a schematic diagram of IOPS (times) prediction results according to an embodiment of the present invention; in figures 6-a to 6-d,in order to be a basis for the resource overhead,in order to check the weight of the data,in order to carry out the sorting,in order to carry out the pre-treatment,in order to configure the capacity of the system,as a usable threshold valueThe amount of the organic acid is the total amount,
the forecasting of the overall resource consumption trend of each application link comprises the following steps: the proportion of host resources consumed by the total telephone bill in unit time and a trend prediction chart of each application link are obtained by applying a component modeling formula y =0+ b after model precision verification1x1+b2x2+…+bpxpPerforming prediction, wherein the independent variable x is preprocessing, sorting, duplicate checking, warehousing of batch price, opening check, reconciliation, BI file transmission, periodic fee sorting, and calculating the number of CPU cores consumed in unit time, the number of memories consumed in unit time, the total I/O amount in unit time andthe number of IOPS readings per unit time; FIGS. 7-a to 7-d are diagrams showing the predicted result of the total resource consumption trend of each application link according to the embodiment of the present invention; FIG. 7-a is a diagram illustrating a prediction result of overall resource consumption of a CPU (core count) in each application link according to an embodiment of the present invention; FIG. 7-b is a diagram illustrating the prediction result of MEM (KB) total resource consumption in each application link according to the embodiment of the present invention; FIG. 7-c is a diagram illustrating the predicted result of total resource consumption of each application link I/O (KB); FIG. 7-d is a diagram illustrating the IOPS (times) total resource consumption prediction result of each application link according to the embodiment of the present invention; in figures 7-a to 7-d,in order to sort the periodic fee,in order to transmit the BI file(s),in order to make the account of the account,to open the check;in order to put in storage,in order to be a wholesale price,in order to check the weight of the data,in order to carry out the sorting,in order to carry out the pre-treatment,
the single ticket processing resource consumption distribution comprises the following steps: processing sheetThe analysis conditions of CPU, MEM and I/O, IOPS consumed by the pen ticket are shown in figures 8-a to 8-d, and a component modeling formula y =0+ b after model precision verification is applied1x1+b2x2+…+bpxpForecasting, wherein the independent variable x is preprocessing, sorting, duplicate checking, warehousing of batch price, opening inspection, reconciliation, BI file transmission and periodic charge sorting, and the number of CPU cores consumed in unit time, the number of memories consumed in unit time, the total I/O amount in unit time and the IOPS reading times in unit time are calculated; FIG. 8 is a diagram illustrating a distribution result of single-stroke ticket processing resource consumption according to an embodiment of the present invention; wherein, fig. 8-a is a schematic diagram of resource consumption distribution of a single-stroke ticket processing CPU (core number); FIG. 8-b is a schematic diagram illustrating resource consumption distribution of single-stroke ticket processing MEM (KB) according to the embodiment of the present invention; FIG. 8-c is a schematic diagram of resource consumption distribution of a single-stroke ticket processing I/O (KB); FIG. 8-d is a schematic diagram of IOPS resource consumption distribution for single-stroke call ticket processing according to the present invention;
the embodiment of the invention also provides a simulation device based on the service capability model, which comprises: a data processing unit 91, a model building unit 92, a result prediction unit 93, wherein,
the data processing unit 91 is configured to perform preprocessing on the service capability data;
the data processing unit 91 preprocesses the service capability data, including but not limited to: sorting capacity model data, and/or screening holidays and/or repeating and missing processing, and/or dividing idle and busy periods of data, and/or analyzing correlation, and/or selecting optimal periods of data;
specifically, the preprocessing of the service capability data by the data processing unit 91 includes the following processes:
a. sequencing the service capability data according to the time sequence, avoiding the phenomenon of time disorder and providing a basis for subsequent data processing;
b. respectively screening out service capacity data of working days and holidays according to the time fields;
c. judging whether the service data is missing or overlapped, if so, namely, the monitoring values in the same time period repeatedly appear, keeping one record, and deleting redundant repeated records;
for the case of missing service capability data, in the case of missing only one value, interpolation supplementation is performed by using a mean value interpolation method, for example: missing value of 4 months occurrence = (time control in 3 months + historical monitoring value of 5 months)/2; if the data are missing for more than one day, performing single processing on a fake period, and if the data are missing for more than 5 hours, deleting the data in the whole day and processing the data according to a processing method for missing for more than one day; if the time is less than 5 hours, interpolation can be performed according to different interpolation methods as described below. In particular, the method comprises the following steps of,
the interpolation function is determined using the least squares method:
or, determining an interpolation function based on a Lagrange interpolation method, wherein the formula is as follows: y = a ((logx + log (x +1))/2) + b.
d. Dividing idle busy time periods of data; here, the purpose of dividing the idle and busy periods of the data is to predict the idle and busy data respectively, so as to reduce the inaccuracy caused by averaging all the data and then predicting. Mining busy and idle rules of a service support network system according to data of a past time interval; the data of the past certain period may be data of past 1-3 months, but is not limited to this range, and the required period may be set according to actual requirements.
Specifically, the original data is averaged according to hours; for example, monitoring data is collected in units of 5 minutes or 1 minute, and averaged by hour; in the embodiment of the invention, taking 5 minutes as an example, overlapping 1-3 months of data collected by a system according to the data of different days and the same hour for calculating an average value, and counting the system operation trend in the same day;
determining a base line for judging idle busy time periods according to historical data; here, the baseline standard may be determined according to actual requirements, for example, a time period in which the acquired data is greater than 70% of the historical average value may be taken as a busy time period, and the rest of the time period may be an idle time period;
e. performing correlation analysis on the service capability data, and screening out the service capability data with the incidence relation;
the process of the data processing unit 91 for analyzing the service capability data correlation includes: starting with the first set of business capability data in the business capability data set, the business capability data of selection 1/3 establishes the correlation coefficient of the IT performance data and the business data respectively. Wherein the IT performance data includes: CPU utilization rate, memory utilization rate, I/O data and IOPS data, wherein the service data comprises: short message phone number, voice phone number, General Packet Radio Service (GPRS) phone number, and other phone numbers. In the calculation of the correlation coefficient, the previous 1/3 data are respectively selected, and the correlation coefficient value of each group of data is respectively calculated according to the step size of 1 until the last group of data.
And selecting the relevant factors according to actual requirements, for example, selecting the factors with the absolute value of the relevant coefficient of more than 0.3 as the relevant factors.
f. Selecting the optimal data time period of the capacity model; because the change of part of systems has no detailed literature record and query, the reasonability and scientificity of model data input are influenced, and therefore, the accuracy of the model is improved through the optimal data selection of historical data.
The step of performing the preprocessing on the service capability data by the data processing unit 91 is not limited to a specific sequence, and the steps a to f may be performed in any sequence or in parallel according to actual requirements.
The data processing unit 91 preprocesses the capability model data further includes: processing CPU data, memory data, input/output I/O data and IOPS data of the times of reading and writing I/O operation per second of the capacity model; the data processing unit 91 is specifically used for processing the CPU data, the memory data, the I/O data and the IOPS data of the capacity model, wherein the peak data in busy hour in one day and the valley data in idle hour in one day;
specifically, the data processing unit 91 averages the hour peak values of the services according to the busy/idle periods determined in the preprocessing; mapping the time of the maximum/small value of the service transaction to the CPU performance utilization rate of the component, the MEM performance utilization rate of the component, the I/O of the component and the IOPS of the component; and mapping the service with the average value of busy/idle time peaks of the CPU, MEM and I/O, IOPS according to time.
The processing of the CPU data of the capacity model comprises processing of peak data of busy hour in one day and processing of valley data of idle hour of the CPU. Specifically, based on the business transaction, the maximum/minimum value of the business per hour is selected; according to the busy and idle time periods determined in the preprocessing, averaging the hour peak values of the services according to the busy/idle time periods; mapping the time of the maximum/small value of the business transaction to the CPU performance utilization rate of the component; converting the CPU utilization rate corresponding to the peak/valley time of the service into the CPU utilization time; and (4) mapping the peak/valley value average values of busy/idle time of the service and the CPU according to time.
The processing of the memory data of the capacity model comprises processing of peak data of busy hour in one day of memory and processing of valley data of idle hour. Specifically, based on the business transaction, the maximum/minimum value of the business per hour is selected; according to the busy and idle time periods determined in the preprocessing, averaging the hour peak values of the services according to the busy/idle time periods; mapping the time of the maximum/small value of the business transaction to the MEM performance utilization rate of the component; converting the MEM usage rate corresponding to the peak/bottom time of the service into the usage capacity of the MEM; and (4) mapping the peak value/valley value average values of the busy/idle times of the business and the MEM according to time.
The processing of the I/O data of the capacity model comprises processing of peak data of busy hour in one internal storage day and processing of valley data of idle hour. Specifically, based on the business transaction, the maximum/minimum value of the business per hour is selected; according to the busy and idle time periods determined in the preprocessing, averaging the hour peak values of the services according to the busy/idle time periods; mapping the time of the maximum/small value of the business transaction to the I/O of the component; and (4) mapping the peak/valley value average values of busy/idle times of the business and the I/O according to time.
The processing of the IOPS data of the capacity model comprises processing of peak data of busy hour within one day of memory and processing of valley data of idle hour. Specifically, based on the business transaction, the maximum/minimum value of the business per hour is selected; according to the busy and idle time periods determined in the preprocessing, averaging the hour peak values of the services according to the busy/idle time periods; mapping the time of the maximum/small value of the service transaction to the IOPS of the component; and (4) mapping the peak/valley value average values of busy/idle times of the service and the IOPS according to time.
The model establishing unit 92 is configured to establish a service capability model according to the preprocessed service capability data;
the step of establishing the service capability model by the model establishing unit 92 according to the preprocessed service capability data includes: and sequentially carrying out business modeling, service modeling and component modeling according to the preprocessed business capability data.
The modeling of the service by the model building unit 92 includes: establishing a service model by taking time as an independent variable and short message voice singles, voice singles and GPRS (general packet radio service) singles as dependent variables;
here, the business modeling mainly depends on a least square method, and the independent variable X takes a time period of nearly three years, and the dependent variable Y takes a short message phone bill quantity, a voice phone bill quantity, a GPRS phone bill quantity and other phone bill quantities of nearly three years.
The modeling of the service by the model establishing unit 92 includes: establishing a capability model by taking the dependent variable in the business modeling as an independent variable and taking the single quantity of the processing link as the dependent variable; wherein, the processing link ticket includes but is not limited to preprocessing, sorting, weight checking, price grading warehousing, opening checking, account closing, BI file transmission and periodic fee sorting;
the service modeling adopts a multiple regression analysis method to carry out modeling, wherein an independent variable x is a dependent variable y in the service modeling, namely a short message phone bill quantity, a voice phone bill quantity, a GPRS phone bill quantity and other phone bill quantities. And y, taking the processing link phone bill quantity of nearly three years in the existing system, including pretreatment, sorting, weight checking, price grading warehousing, opening inspection, reconciliation, BI file transmission and periodic fee sorting.
The modeling of the components by the model building unit 92 includes: and establishing a component model by taking the dependent variable in the service modeling as an independent variable and taking the CPU utilization rate, the memory utilization rate, the total I/O amount and the IOPS times as the dependent variable.
And modeling the component and the service, and performing modeling by adopting a multiple regression analysis method, wherein an independent variable x is a dependent variable y of the service modeling, and the y is the CPU utilization rate, the memory utilization rate, the total I/O amount and the IOPS times of nearly three years.
And the result predicting unit 93 is configured to perform result prediction according to the service capability model, and display a prediction result.
The result predicting unit 93 performs result prediction according to the service capability model, including: and predicting the resource consumption trend of each host, the total resource consumption trend of each application link and the resource consumption distribution of single-stroke ticket processing.
After model accuracy verification, the modeling formula analyzes and models data of the existing charging system, and the model is applied to trend prediction of resource consumption for a single host, prediction of resource consumption of a single ticket and total resource consumption prediction of each application.
Wherein the estimating of the resource consumption trend of each host comprises the following steps: according to the modeling step of the capability model, a component model formula y =0+ b is applied1x1+b2x2+…+bpxpTo aim atThe host as-jf1 calculates and estimates the resource consumption, and respectively calculates the CPU utilization rate, the memory utilization rate, the total I/O amount and the total IOPS times by taking the independent variable x as the duplicate checking, sorting and preprocessing phone bill amount.
The forecasting of the overall resource consumption trend of each application link comprises the following steps:
the proportion of host resources consumed by the total telephone bill in unit time and the trend prediction chart of each application link are as follows, and a component modeling formula y =0+ b after model precision verification is applied1x1+b2x2+…+bpxpAnd predicting, wherein the independent variable x is preprocessing, sorting, duplicate checking, warehousing of batch price, opening and checking, reconciliation, BI file transmission and periodic charge sorting, and the number of CPU cores consumed in unit time, the number of memories consumed in unit time, the total I/O amount in unit time and the IOPS reading times in unit time are calculated.
The prediction of the single ticket processing resource consumption distribution comprises the following steps: the analysis conditions of CPU, MEM and I/O, IOPS consumed by processing a single ticket are as follows, and a component modeling formula y =0+ b after model precision verification is applied1x1+b2x2+…+bpxpAnd predicting, wherein the independent variable x is preprocessing, sorting, duplicate checking, warehousing of batch price, opening and checking, reconciliation, BI file transmission and periodic charge sorting, and the number of CPU cores consumed in unit time, the number of memories consumed in unit time, the total I/O amount in unit time and the IOPS reading times in unit time are calculated.
Further, the apparatus further includes an accuracy verification unit 94, configured to perform accuracy verification on the service capability model;
here, the accuracy verification of the service capability model is performed before the result prediction unit 93 performs result prediction according to the service capability model, so as to further understand the deviation between the predicted or fitted result of the service capability model and the true result.
And if the relatively large error accumulated distribution exists in the relatively small error, the prediction result or the fitting result is more ideal, namely the prediction result or the fitting result is more toward the true value. In practical application, the relative error is plotted as a horizontal axis and the relative error cumulative distribution is plotted as a vertical axis, and the error condition of the predicted result or the fitted result and the actual result can be more visually observed through a chart.
The implementation functions of the processing modules in the simulation device based on the business capability model shown in fig. 9 can be understood by referring to the related description of the simulation method based on the business capability model. Those skilled in the art will understand that the functions of each processing unit in the simulation device based on the service capability model shown in fig. 9 can be realized by a program running on a processor, and can also be realized by specific logic circuits, such as: may be implemented by a Central Processing Unit (CPU), Microprocessor (MPU), Digital Signal Processor (DSP), or Field Programmable Gate Array (FPGA).
In the embodiments provided in the present invention, it should be understood that the disclosed method and apparatus can be implemented in other ways. The above-described device embodiments are merely illustrative, for example, the division of the unit is only a logical functional division, and there may be other division ways in actual implementation, such as: multiple units or components may be combined, or may be integrated into another system, or some features may be omitted, or not implemented. In addition, the communication connections between the components shown or discussed may be through interfaces, indirect couplings or communication connections of devices or units, and may be electrical, mechanical or other.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed on a plurality of network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, all the functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately regarded as one unit, or two or more units may be integrated into one unit; the integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
Those of ordinary skill in the art will understand that: all or part of the steps for realizing the method embodiments can be completed by hardware related to program instructions, the program can be stored in a computer readable storage medium, and the program executes the steps comprising the method embodiments when executed; and the aforementioned storage medium includes: various media that can store program codes, such as a removable Memory device, a Read-Only Memory (ROM), a magnetic disk, or an optical disk.
Alternatively, the integrated unit according to the embodiment of the present invention may be stored in a computer-readable storage medium if it is implemented in the form of a software functional unit and sold or used as a separate product. Based on such understanding, the technical solutions of the embodiments of the present invention may be essentially implemented or a part contributing to the prior art may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the methods described in the embodiments of the present invention. And the aforementioned storage medium includes: a removable storage device, a ROM, a magnetic or optical disk, or other various media that can store program code.
The simulation method and apparatus based on the service capability model described in the embodiment of the present invention are only for example, but not limited thereto, and the present invention is within the protection scope of the present invention as long as the simulation method and apparatus based on the service capability model are involved.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention.
Claims (8)
1. A simulation method based on a business capability model is characterized by comprising the following steps:
preprocessing the service capability data;
establishing a service capability model according to the preprocessed service capability data;
predicting the result according to the service capability model, and displaying the predicted result;
wherein, the establishing of the service capability model according to the preprocessed service capability data comprises: sequentially performing business modeling, service modeling and component modeling according to the preprocessed business capability data;
the business modeling comprises the following steps: establishing a service model by using a least square method by taking time as an independent variable and short message voice singles, voice singles and GPRS (general packet radio service) singles as dependent variables;
the service modeling comprises: taking the dependent variable in the business modeling as an independent variable, taking the single quantity of the processing link as the dependent variable, and establishing a capability model by adopting a multiple regression analysis method; wherein, the processing link ticket includes but is not limited to preprocessing, sorting, weight checking, price grading warehousing, opening checking, account closing, BI file transmission and periodic fee sorting;
the component modeling comprises: and establishing a component model by using the dependent variable in service modeling as an independent variable and using the CPU utilization rate, the memory utilization rate, the total I/O amount and the IOPS times as the dependent variable by adopting a multivariate regression analysis method, wherein the component model represents the business capability model.
2. The method of claim 1, wherein before the predicting the result according to the business capability model, the method further comprises: and carrying out precision verification on the service capability model.
3. The method according to claim 1 or 2, wherein the preprocessing of the service capability data includes but is not limited to:
sorting capacity model data, and/or screening holidays and/or repeating and missing processing, and/or dividing idle and busy periods of data, and/or analyzing correlation, and/or selecting optimal periods of data; and
and processing CPU data, memory data, input/output I/O data and IOPS data of the times of reading and writing I/O operation per second of the capacity model.
4. The method according to claim 1 or 2, wherein the predicting the result according to the service capability model comprises: and predicting the resource consumption trend of each host, the total resource consumption trend of each application link and the resource consumption distribution of single-stroke ticket processing.
5. An apparatus for simulation based on a business capability model, the apparatus comprising: the device comprises a data processing unit, a model establishing unit and a result predicting unit; wherein,
the data processing unit is used for preprocessing the service capability data;
the model establishing unit is used for establishing a service capability model according to the preprocessed service capability data;
the result prediction unit is used for predicting the result according to the service capability model and displaying the prediction result;
the model establishing unit establishes the service capability model according to the preprocessed service capability data, and comprises the following steps: sequentially performing business modeling, service modeling and component modeling according to the preprocessed business capability data;
the modeling unit for performing business modeling comprises: establishing a service model by using a least square method by taking time as an independent variable and short message voice singles, voice singles and GPRS (general packet radio service) singles as dependent variables;
the service modeling performed by the model establishing unit comprises: taking the dependent variable in the business modeling as an independent variable, taking the single quantity of the processing link as the dependent variable, and establishing a capability model by adopting a multiple regression analysis method; wherein, the processing link ticket includes but is not limited to preprocessing, sorting, weight checking, price grading warehousing, opening checking, account closing, BI file transmission and periodic fee sorting;
the modeling of the component by the model establishing unit comprises: and establishing a component model by using the dependent variable in service modeling as an independent variable and using the CPU utilization rate, the memory utilization rate, the total I/O amount and the IOPS times as the dependent variable by adopting a multivariate regression analysis method, wherein the component model represents the business capability model.
6. The apparatus according to claim 5, further comprising a precision verification unit configured to perform precision verification on the service capability model before the result prediction unit performs result prediction according to the service capability model.
7. The apparatus of claim 5 or 6, wherein the data processing unit preprocesses the service capability data including but not limited to:
sorting capacity model data, and/or screening holidays and/or repeating and missing processing, and/or dividing idle and busy periods of data, and/or analyzing correlation, and/or selecting optimal periods of data;
and processing CPU data, memory data, input/output I/O data and IOPS data of the times of reading and writing I/O operation per second of the capacity model.
8. The apparatus according to claim 5 or 6, wherein the result prediction unit performs result prediction according to the service capability model, and comprises: and predicting the resource consumption trend of each host, the total resource consumption trend of each application link and the resource consumption distribution of single-stroke ticket processing.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201410136981.2A CN104978604B (en) | 2014-04-04 | 2014-04-04 | A kind of analog simulation method and device based on professional ability model |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201410136981.2A CN104978604B (en) | 2014-04-04 | 2014-04-04 | A kind of analog simulation method and device based on professional ability model |
Publications (2)
Publication Number | Publication Date |
---|---|
CN104978604A CN104978604A (en) | 2015-10-14 |
CN104978604B true CN104978604B (en) | 2019-09-17 |
Family
ID=54275086
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201410136981.2A Active CN104978604B (en) | 2014-04-04 | 2014-04-04 | A kind of analog simulation method and device based on professional ability model |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN104978604B (en) |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106897293B (en) * | 2015-12-17 | 2020-09-11 | 中国移动通信集团公司 | Data processing method and device |
CN109002925A (en) * | 2018-07-26 | 2018-12-14 | 北京京东金融科技控股有限公司 | Traffic prediction method and apparatus |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101364229A (en) * | 2008-10-06 | 2009-02-11 | 中国移动通信集团设计院有限公司 | Database host resource prediction method based on time capability analysis |
CN101854652A (en) * | 2010-06-23 | 2010-10-06 | 天元莱博(北京)科技有限公司 | Telecommunications network service performance monitoring system |
CN102111284A (en) * | 2009-12-28 | 2011-06-29 | 北京亿阳信通软件研究院有限公司 | Method and device for predicting telecom traffic |
CN102437873A (en) * | 2010-09-29 | 2012-05-02 | 大连大学 | Service volume modeling and flow control method based on satellite network |
CN103024762A (en) * | 2012-12-26 | 2013-04-03 | 北京邮电大学 | Service feature based communication service forecasting method |
US9001658B2 (en) * | 2010-03-31 | 2015-04-07 | Alcatel Lucent | Method for reducing energy consumption in packet processing linecards |
-
2014
- 2014-04-04 CN CN201410136981.2A patent/CN104978604B/en active Active
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101364229A (en) * | 2008-10-06 | 2009-02-11 | 中国移动通信集团设计院有限公司 | Database host resource prediction method based on time capability analysis |
CN102111284A (en) * | 2009-12-28 | 2011-06-29 | 北京亿阳信通软件研究院有限公司 | Method and device for predicting telecom traffic |
US9001658B2 (en) * | 2010-03-31 | 2015-04-07 | Alcatel Lucent | Method for reducing energy consumption in packet processing linecards |
CN101854652A (en) * | 2010-06-23 | 2010-10-06 | 天元莱博(北京)科技有限公司 | Telecommunications network service performance monitoring system |
CN102437873A (en) * | 2010-09-29 | 2012-05-02 | 大连大学 | Service volume modeling and flow control method based on satellite network |
CN103024762A (en) * | 2012-12-26 | 2013-04-03 | 北京邮电大学 | Service feature based communication service forecasting method |
Also Published As
Publication number | Publication date |
---|---|
CN104978604A (en) | 2015-10-14 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Huang et al. | Resource prediction based on double exponential smoothing in cloud computing | |
CN105678398A (en) | Power load forecasting method based on big data technology, and research and application system based on method | |
CN108388974A (en) | Top-tier customer Optimum Identification Method and device based on random forest and decision tree | |
CN112651563B (en) | Load prediction method and device, computer readable storage medium and electronic equipment | |
CN109118012B (en) | Industrial dynamic multi-dimensional energy consumption cost prediction method, system, storage medium and terminal | |
CN102982489A (en) | Power customer online grouping method based on mass measurement data | |
CN111680841A (en) | Short-term load prediction method and system based on principal component analysis and terminal equipment | |
CN111162925A (en) | Network flow prediction method and device, electronic equipment and storage medium | |
CN108154311A (en) | Top-tier customer recognition methods and device based on random forest and decision tree | |
CN110618867A (en) | Method and device for predicting resource usage amount | |
US20140289007A1 (en) | Scenario based customer lifetime value determination | |
CN112365070A (en) | Power load prediction method, device, equipment and readable storage medium | |
CN111062564A (en) | Method for calculating power customer appeal sensitive value | |
CN114169802B (en) | Power grid user demand response potential analysis method, system and storage medium | |
CN111506876A (en) | Data prediction analysis method, system, equipment and readable storage medium | |
CN104978604B (en) | A kind of analog simulation method and device based on professional ability model | |
CN109583773A (en) | A kind of method, system and relevant apparatus that taxpaying credit integral is determining | |
CN113450004A (en) | Power credit report generation method and device, electronic equipment and readable storage medium | |
CN108830663B (en) | Electric power customer value evaluation method and system and terminal equipment | |
WO2023229474A1 (en) | Methods, systems and computer program products for determining models for predicting reoccurring transactions | |
CN109948926A (en) | A kind of highway concrete-bridge maintenance technology Selection Method based on project period | |
CN104680400A (en) | Method for short-term or long-term prediction of electricity sales amount changes of enterprises based on grey correlation | |
CN114254857A (en) | Power equipment inventory condition evaluation method and server | |
CN113627821A (en) | Method and system for identifying abnormal electricity utilization based on electricity utilization behavior characteristics | |
CN106651055A (en) | Prediction method and system for short-term power sale quantity |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |