CN110597703A - Regression testing method and device - Google Patents
Regression testing method and device Download PDFInfo
- Publication number
- CN110597703A CN110597703A CN201810606840.0A CN201810606840A CN110597703A CN 110597703 A CN110597703 A CN 110597703A CN 201810606840 A CN201810606840 A CN 201810606840A CN 110597703 A CN110597703 A CN 110597703A
- Authority
- CN
- China
- Prior art keywords
- data
- tested
- threshold
- historical test
- result
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012360 testing method Methods 0.000 title claims abstract description 231
- 238000005315 distribution function Methods 0.000 claims abstract description 47
- 230000002159 abnormal effect Effects 0.000 claims abstract description 39
- 238000000034 method Methods 0.000 claims abstract description 23
- 238000002790 cross-validation Methods 0.000 claims description 77
- 238000012549 training Methods 0.000 claims description 31
- 238000012795 verification Methods 0.000 claims description 27
- 238000010200 validation analysis Methods 0.000 claims description 20
- 238000004891 communication Methods 0.000 claims description 7
- 230000002547 anomalous effect Effects 0.000 claims description 4
- 230000004044 response Effects 0.000 description 9
- 238000010586 diagram Methods 0.000 description 8
- 238000004590 computer program Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000005856 abnormality Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000010998 test method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Prevention of errors by analysis, debugging or testing of software
- G06F11/3668—Testing of software
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Prevention of errors by analysis, debugging or testing of software
- G06F11/3668—Testing of software
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The embodiment of the invention discloses a regression testing method and a regression testing device. Wherein the method comprises the following steps: acquiring a to-be-tested data set of a to-be-tested item; obtaining the probability corresponding to each data according to each data in the data set to be tested and the probability distribution function corresponding to the item to be tested; wherein the probability distribution function is pre-established; if the probability corresponding to the data is judged to be larger than the threshold value, the test result of the data is normal, otherwise, the test result of the data is abnormal. The device is used for executing the method. The regression testing method and the device provided by the embodiment of the invention improve the efficiency of regression testing.
Description
Technical Field
The embodiment of the invention relates to the technical field of computers, in particular to a regression testing method and device.
Background
The regression test is used as a component of the software life cycle, and has a great proportion in the whole software test process, and multiple regression tests can be carried out at each stage of software development.
Taking the telecommunication industry as an example, regression testing of core services is required after a new version is online, wherein the quantity of the free resources left for inquiring users is a common test case, but because packages and refueling bags handled by each user are different, the consumption speed is different, and the quantity of the free resources left for returning to the system cannot be accurately verified. In the prior art, the following solutions are mainly adopted: (1) semi-automatic testing: for the part which cannot be accurately verified by the automatic test, the manual test is assisted, and professional testers further check the part to improve the test accuracy; (2) setting a possible range interval: setting a possible range interval through experience summary of testers, and if the test result is not in the range, determining that the test result is abnormal; (3) and calculating according to the related parameters: and obtaining other related parameter information, calculating an expected value of the result to be verified according to the association relation, and if the actual result is different from the expected value, determining that the test is not passed. However, the above technical solutions have the following disadvantages: (1) for a semi-automatic test mode, test result data is extracted and is checked by professional testers, the test efficiency is obviously reduced, the requirement of the test method on the testers is high, and the testers unfamiliar with the service can not accurately judge whether the test result is correct or not; (2) the method for verifying the result by setting the range interval mainly depends on the experience of testers, and the range interval needs to be adjusted along with the evolution of the service; therefore, no matter the early setting or the later maintenance, the method needs to consume large manpower, and in addition, special conditions cannot be identified in a fixed range interval, so that false alarm is easily caused; (3) for the method of calculating the expected value according to other parameters, the calculation process under a complex service scene is very complicated, even the calculation cannot be performed at all, and the accuracy of the judgment of the test result is indirectly influenced by the abnormality or the error of other parameters.
Therefore, how to provide a regression testing method, which can test data without a fixed range to improve the efficiency of regression testing, is an important issue to be solved in the industry.
Disclosure of Invention
In view of the defects in the prior art, embodiments of the present invention provide a regression testing method and apparatus.
In one aspect, an embodiment of the present invention provides a regression testing method, including:
acquiring a to-be-tested data set of a to-be-tested item;
obtaining the probability corresponding to each data according to each data in the data set to be tested and the probability distribution function corresponding to the item to be tested; wherein the probability distribution function is pre-established;
if the probability corresponding to the data is judged to be larger than the threshold value, the test result of the data is normal, otherwise, the test result of the data is abnormal.
In another aspect, an embodiment of the present invention provides a regression testing apparatus, including:
the first acquisition unit is used for acquiring a to-be-tested data set of a to-be-tested item;
the first obtaining unit is used for obtaining the probability corresponding to each data according to each data in the data set to be tested and the probability distribution function corresponding to the item to be tested; wherein the probability distribution function is pre-established;
and the test unit is used for judging whether the probability corresponding to the data is greater than a threshold value or not, if so, judging that the test result of the data is normal, and otherwise, judging that the test result of the data is abnormal.
In another aspect, an embodiment of the present invention provides an electronic device, including: a processor, a memory, and a communication bus, wherein:
the processor and the memory are communicated with each other through the communication bus;
the memory stores program instructions executable by the processor, and the processor calls the program instructions to perform the regression testing method provided in the embodiments described above.
In yet another aspect, embodiments of the present invention provide a non-transitory computer-readable storage medium storing computer instructions that cause the computer to perform a regression testing method as provided in the embodiments above.
According to the regression testing method and device provided by the embodiment of the invention, the to-be-tested data set of the to-be-tested item can be obtained, the probability corresponding to each data is obtained according to each data in the to-be-tested data set and the probability distribution function corresponding to the to-be-tested item, and then if the probability corresponding to the obtained data is judged to be larger than the threshold value, the testing result of the data is normal, otherwise, the testing result of the data is abnormal, so that the efficiency of the regression testing is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a schematic flow chart of a regression testing method according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart of a regression testing method according to another embodiment of the present invention;
FIG. 3 is a schematic flow chart of a regression testing method according to another embodiment of the present invention;
FIG. 4 is a schematic structural diagram of a regression testing apparatus according to an embodiment of the present invention;
FIG. 5 is a schematic structural diagram of a regression testing apparatus according to another embodiment of the present invention;
FIG. 6 is a schematic structural diagram of a regression testing apparatus according to another embodiment of the present invention;
fig. 7 is a schematic physical structure diagram of an electronic device according to an embodiment of the invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present invention without any creative effort belong to the protection scope of the embodiments of the present invention.
Fig. 1 is a schematic flow chart of a regression testing method according to an embodiment of the present invention, and as shown in fig. 1, the regression testing method according to the embodiment of the present invention includes:
s101, acquiring a to-be-tested data set of a to-be-tested project;
specifically, after the software system is upgraded, the regression testing device may send a query message carrying a to-be-tested item to the software system, and after receiving the message, the software system returns a response message to the regression testing device, where the response message includes related data of the to-be-tested item. The regression testing device can obtain each data of the item to be tested by performing keyword retrieval on the response messages line by line, and each data of the item to be tested forms a data set to be tested. The keywords are preset and correspond to the items to be detected; the items to be tested include but are not limited to free resource residual amounts such as telephone charge balance, voice and flow of a user, and data in the data set to be tested of the items to be tested have no fixed range but obey probability distribution.
S102, obtaining the probability corresponding to each data according to each data in the data set to be tested and the probability distribution function corresponding to the item to be tested; wherein the probability distribution function is pre-established;
specifically, after obtaining the data set to be tested, the regression testing device inputs each data in the data set to be tested into a probability distribution function, and obtains a probability corresponding to each data in the data set to be tested. The probability distribution function is pre-established and corresponds to the item to be measured.
S103, if the probability corresponding to the data is judged to be larger than a threshold value, the test result of the data is normal, otherwise, the test result of the data is abnormal.
Specifically, after obtaining the probability corresponding to each data in the data set to be tested, the regression testing device compares the probability corresponding to each data in the data set to be tested with a threshold, if the probability corresponding to the data in the data set to be tested is greater than the threshold, the test result of the data is normal, which indicates that the data is correct, and if the probability corresponding to the data in the data set to be tested is less than or equal to the threshold, the test result of the data is abnormal, which indicates that the data is wrong. After obtaining the test result of each data in the data set to be tested, the regression testing apparatus may send the test result to a tester in the form of an email.
For example, a software system of a mobile communication company a is version-upgraded, after the version-upgrading, a telephone charge balance of a user included in the software system is one of the items to be tested, in order to perform a regression test on the telephone charge balance of the user, the regression test device sends an inquiry message carrying the telephone charge balance of the user to the software system after the version-upgrading, and the software system returns a response message including data of the telephone charge balances of all the users in response to the inquiry message. The regression testing device is used for determining the regression condition according to the keywords: and telephone charge balance, wherein telephone charge balance retrieval is carried out line by line in the response message comprising the telephone charge balance data of all the users, so that each data of the telephone charge balance of the users can be obtained, and each data of the telephone charge balance of the users forms a to-be-tested data set of the telephone charge balance of the users. The regression testing device inputs each data distribution in the data set to be tested into a probability distribution function corresponding to the telephone charge balance of the userWherein, mu and sigma2The data in the data set to be tested are input, and the probability P corresponding to each data in the data set to be tested can be calculated. The regression testing means compares the probability P with a threshold epsilon if P>If P is less than or equal to epsilon, the test result of the data corresponding to the probability P is abnormal. The regression testing device compares the probability corresponding to each data in the data set to be tested with a threshold epsilon respectively, and can obtain the testing result of each data in the data set to be tested.
According to the regression testing method provided by the embodiment of the invention, the to-be-tested data set of the to-be-tested item can be obtained, the probability corresponding to each data is obtained according to each data in the to-be-tested data set and the probability distribution function corresponding to the to-be-tested item, and then if the probability corresponding to the obtained data is judged to be greater than the threshold value, the testing result of the data is normal, otherwise, the testing result of the data is abnormal, so that the efficiency of the regression testing is improved.
Fig. 2 is a schematic flow chart of a regression testing method according to another embodiment of the present invention, and as shown in fig. 2, the establishing of the probability distribution function includes:
s201, acquiring a historical test data set of the item to be tested and a historical test result of each data in the historical test data set;
specifically, in the past, after the software system is upgraded, a conventional regression testing method, such as a semi-automatic testing method, is used to perform a regression test on the item to be tested, and each historical test data of the item to be tested and a historical test result of each historical test data are recorded. The regression testing device can obtain each historical testing data of the item to be tested and each corresponding historical testing result, and each historical testing data of the item to be tested forms a historical testing data set of the item to be tested.
S202, acquiring a first preset amount of data from the historical test data set to form a training set, wherein the historical test result of each data in the training set is normal;
specifically, after obtaining the historical test data set of the item to be tested and the historical test result of each data in the historical test data set, the regression test device selects data with a first preset number of historical test results as normal from the historical test data set, and the data with the first preset number of historical test results as normal form a training set. The first preset number is set according to practical experience, and the embodiment of the invention is not limited.
S203, obtaining the probability distribution function according to the training set.
Specifically, the regression testing device may utilize detection software of a probability distribution function, such as Minitab, SPSS, and SAS, to detect the training set, obtain a probability distribution function obeyed by the training set, and calculate relevant parameters of the probability distribution function, so as to obtain a probability distribution function corresponding to the item to be tested.
For example, the item to be tested is the free flow residual quantity, the regression testing device obtains a training set corresponding to the free flow residual quantity, and determines that the training set obeys gaussian distribution by using SAS software, so that the probability distribution function corresponding to the free flow residual quantity is as follows:
where μ is the mathematical expectation and σ2Is the variance. The regression testing device calculates and obtains mu and sigma in the formula by using data in a training set corresponding to the free flow residual quantity2Wherein:
wherein x isiAnd the ith value in the training set corresponding to the free flow residual amount is represented, m represents the number of data in the training set corresponding to the free flow residual amount, i is a positive integer and is less than or equal to m.
Fig. 3 is a schematic flowchart of a regression testing method according to another embodiment of the present invention, and as shown in fig. 3, the threshold is determined according to the following steps:
s301, removing the training set from the historical test data set to obtain a cross validation set;
specifically, the regression testing device may remove the training set from the historical test data set after obtaining the historical test data set and the training set, thereby obtaining a cross validation set.
S302, obtaining the probability corresponding to each data in the cross validation set according to the cross validation set and the probability distribution function;
specifically, after obtaining the cross validation set, the regression testing apparatus inputs each data in the cross validation set into the probability function, respectively, to obtain a probability corresponding to each data in the cross validation set.
S303, obtaining a verification result of each data in the cross validation set according to the probability corresponding to each data in the cross validation set and a threshold to be determined; wherein the threshold to be determined is preset;
specifically, after obtaining the probability corresponding to each data in the cross validation set, the regression testing apparatus compares the probability corresponding to each data in the cross validation set with a threshold to be determined, if the probability corresponding to the data in the cross validation set is greater than the threshold to be determined, the validation result of the data in the cross validation set is normal, and if the probability corresponding to the data in the cross validation set is less than or equal to the threshold to be determined, the validation result of the data in the cross validation set is abnormal. Wherein the threshold to be determined is preset.
S304, obtaining an F-measure value corresponding to the threshold to be determined according to the verification result and the historical test result of each data in the cross verification set;
specifically, the regression testing apparatus compares the verification result of each data in the cross-validation set with the historical test result, and there may be a case where the verification result and the historical test result of the data are both normal, the verification result and the historical test result of the data are both abnormal, the verification result of the data is normal, and the historical test result of the data is abnormal. Representing the quantity of data with normal history test results and normal verification results in the cross-validation set as t, and representing the quantity of data with abnormal history test results and abnormal verification results in the cross-validation set as fnThe number of data for which the historical test result is abnormal and the verification result is normal is represented as fpThen test the overall rateAccuracy of testThe regression testing means may be according to the formulaAnd calculating to obtain an F-measure value F corresponding to the threshold to be determined.
S305, calculating F-measure values corresponding to a second preset number of the to-be-determined thresholds respectively, and taking the to-be-determined threshold corresponding to the largest F-measure value in the second preset number of the F-measure values as the threshold.
Specifically, the regression testing device repeats steps S301, S302, S303, and S304, respectively calculates F-measure values corresponding to a second preset number of the to-be-determined thresholds, obtains the F-measure values of the second preset number, then obtains a maximum F-measure value from the F-measure values of the second preset number, and takes the to-be-determined threshold corresponding to the maximum F-measure value as the threshold. The second preset number is set according to an actual situation, and the embodiment of the present invention is not limited.
On the basis of the foregoing embodiments, further, the obtaining, according to the probability corresponding to each data in the cross validation set and the threshold to be determined, the validation result of each data in the cross validation set includes:
if the probability corresponding to the data in the cross validation set is judged to be larger than the threshold value to be determined, the validation result of the data in the cross validation set is normal, otherwise, the validation result of the data in the cross validation set is abnormal.
Specifically, the regression testing device compares the probability corresponding to each data in the cross validation set with the threshold to be determined, if the probability corresponding to the data in the cross validation set is greater than the threshold to be determined, the validation result of the data in the cross validation set is normal, and if the probability corresponding to the data in the cross validation set is less than or equal to the threshold to be determined, the validation result of the data in the cross validation set is abnormal.
On the basis of the foregoing embodiments, further, the obtaining, according to the verification result and the historical test result of each data in the cross-validation set, the F-measure value corresponding to the threshold to be determined includes:
according to the formulaCalculating to obtain an F-measure value F corresponding to the threshold to be determined, wherein,r represents a test completeness, t represents the number of data in the cross validation set with the normal historical test result and the normal validation result, fnRepresenting the amount of data in the cross-validation set for which the historical test results are anomalous and for which the validation results are anomalous;p represents the test accuracy, fpRepresenting the amount of data in the cross-validation set that the historical test result is abnormal and the validation result is normal; and the cross validation set comprises data with normal historical test results and data with abnormal historical test results.
Specifically, the regression testing apparatus may obtain the number t of data in the cross-validation set in which the historical test result is normal and the validation result is normal, and obtain the number f of data in the cross-validation set in which the historical test result is abnormal and the validation result is abnormalnThe number f of data in the cross-validation set with the historical test result being abnormal and the validation result being normalpThen according toCalculating to obtain the test comprehensive rate r according to a formulaCalculating to obtain the test accuracy p, and then obtaining the test accuracy p according to a formulaAnd calculating to obtain an F-measure value F corresponding to the threshold to be determined. And the cross validation set comprises data with normal historical test results and data with abnormal historical test results.
Fig. 4 is a schematic structural diagram of a regression testing apparatus according to an embodiment of the present invention, and as shown in fig. 4, the regression testing apparatus according to the embodiment of the present invention includes a first obtaining unit 401, a first obtaining unit 402, and a testing unit 403, where:
the first obtaining unit 401 is configured to obtain a to-be-tested data set of a to-be-tested item; the first obtaining unit 402 is configured to obtain, according to each data in the to-be-tested data set and a probability distribution function corresponding to the to-be-tested item, a probability corresponding to each data; wherein the probability distribution function is pre-established; the test unit 403 is configured to determine that the test result of the data is normal if it is determined that the probability corresponding to the data is greater than the threshold, and otherwise, determine that the test result of the data is abnormal.
Specifically, after the software system is upgraded, the first obtaining unit 401 may send an inquiry packet carrying the item to be tested to the software system, and after receiving the packet, the software system returns a response packet to the regression testing apparatus, where the response packet includes the relevant data of the item to be tested. The first obtaining unit 401 may obtain each data of the item to be tested by performing keyword retrieval on the response packet line by line, where each data of the item to be tested constitutes a data set to be tested. The keywords are preset and correspond to the items to be detected; the items to be tested include but are not limited to free resource residual amounts such as telephone charge balance, voice and flow of a user, and data in the data set to be tested of the items to be tested have no fixed range but obey probability distribution.
After obtaining the data set to be tested, the first obtaining unit 402 respectively inputs each data in the data set to be tested into a probability distribution function, and obtains a probability corresponding to each data in the data set to be tested. The probability distribution function is pre-established and corresponds to the item to be measured.
After obtaining the probability corresponding to each data in the data set to be tested, the testing unit 403 compares the probability corresponding to each data in the data set to be tested with a threshold, if the probability corresponding to the data in the data set to be tested is greater than the threshold, the test result of the data is normal, which indicates that the data is correct, and if the probability corresponding to the data in the data set to be tested is less than or equal to the threshold, the test result of the data is abnormal, which indicates that the data is wrong. After obtaining the test result of each data in the data set to be tested, the testing unit 403 may send the test result to the tester in the form of an email.
According to the regression testing device provided by the embodiment of the invention, the to-be-tested data set of the to-be-tested item can be obtained, the probability corresponding to each data is obtained according to each data in the to-be-tested data set and the probability distribution function corresponding to the to-be-tested item, and then if the probability corresponding to the obtained data is judged to be greater than the threshold value, the testing result of the data is normal, otherwise, the testing result of the data is abnormal, so that the efficiency of regression testing is improved.
Fig. 5 is a schematic structural diagram of a regression testing apparatus according to another embodiment of the present invention, and as shown in fig. 5, on the basis of the foregoing embodiments, the regression testing apparatus according to the embodiment of the present invention further includes a second obtaining unit 404, a third obtaining unit 405, and a second obtaining unit 406, where:
the second obtaining unit 404 is configured to obtain a historical test data set of the item to be tested and a historical test result of each data in the historical test data set; the third obtaining unit 405 is configured to obtain a first preset number of data from the historical test data set to form a training set, where a historical test result of each data in the training set is normal; the second obtaining unit 406 is configured to obtain the probability distribution function according to the training set.
Specifically, in the past, after the software system is upgraded, a conventional regression testing method, such as a semi-automatic testing method, is used to perform a regression test on the item to be tested, and each historical test data of the item to be tested and a historical test result of each historical test data are recorded. The second obtaining unit 404 may obtain each historical test data of the item to be tested and a corresponding historical test result, where each historical test data of the item to be tested constitutes a historical test data set of the item to be tested.
After obtaining the historical test data set of the item to be tested and the historical test result of each data in the historical test data set, the third obtaining unit 405 selects a first preset number of data with normal historical test results from the historical test data set, and the training set is formed by the data with normal historical test results of the first preset number. The first preset number is set according to practical experience, and the embodiment of the invention is not limited.
The second obtaining unit 406 may use detection software of a probability distribution function, such as Minitab, sps, and SAS, to detect the training set, obtain a probability distribution function obeyed by the training set, and calculate relevant parameters of the probability distribution function, so as to obtain a probability distribution function corresponding to the item to be tested.
Fig. 6 is a schematic structural diagram of a regression testing apparatus according to still another embodiment of the present invention, as shown in fig. 6, on the basis of the foregoing embodiments, further, the regression testing apparatus according to the embodiment of the present invention further includes a third obtaining unit 407, a fourth obtaining unit 408, a fifth obtaining unit 409, a sixth obtaining unit 410, and a calculating unit 411, where:
a third obtaining unit 407 is configured to remove the training set from the historical test data set to obtain a cross validation set; the fourth obtaining unit 408 is configured to obtain a probability corresponding to each data in the cross validation set according to the cross validation set and the probability distribution function; the fifth obtaining unit 409 is configured to obtain a verification result of each data in the cross validation set according to the probability corresponding to each data in the cross validation set and a threshold to be determined; wherein the threshold to be determined is preset; the sixth obtaining unit 410 is configured to obtain an F-measure value corresponding to the threshold to be determined according to a verification result and a historical test result of each data in the cross-validation set; the calculating unit 411 is configured to calculate F-measure values corresponding to respective second preset numbers of the to-be-determined thresholds, and use a to-be-determined threshold corresponding to a largest F-measure value in the second preset numbers of F-measure values as the threshold.
The third obtaining unit 407 may remove the training set from the historical test data set after obtaining the historical test data set and the training set, thereby obtaining a cross validation set.
After obtaining the cross validation set, the fourth obtaining unit 408 inputs each data in the cross validation set into the probability function, and obtains a probability corresponding to each data in the cross validation set.
After obtaining the probability corresponding to each data in the cross validation set, the fifth obtaining unit 409 compares the probability corresponding to each data in the cross validation set with a threshold to be determined, if the probability corresponding to the data in the cross validation set is greater than the threshold to be determined, the validation result of the data in the cross validation set is normal, and if the probability corresponding to the data in the cross validation set is less than or equal to the threshold to be determined, the validation result of the data in the cross validation set is abnormal. Wherein the threshold to be determined is preset.
The sixth obtaining unit 410 compares the verification result of each data in the cross-validation set with the historical test result, and there may be a case where the verification result and the historical test result of the data are both normal, the verification result and the historical test result of the data are both abnormal, the verification result of the data is normal, and the historical test result of the data is abnormal. Representing the quantity of data with normal history test results and normal verification results in the cross-validation set as t, and representing the quantity of data with abnormal history test results and abnormal verification results in the cross-validation set as fnThe number of data for which the historical test result is abnormal and the verification result is normal is represented as fpThen test the overall rateAccuracy of testThe regression testing means may be according to the formulaAnd calculating to obtain an F-measure value F corresponding to the threshold to be determined.
The calculating unit 411 repeats steps S301, S302, S303, and S304, calculates F-measure values corresponding to a second preset number of the to-be-determined thresholds, respectively, obtains the F-measure values of the second preset number, then obtains a maximum F-measure value from the F-measure values of the second preset number, and takes the to-be-determined threshold corresponding to the maximum F-measure value as the threshold. The second preset number is set according to an actual situation, and the embodiment of the present invention is not limited.
The embodiment of the regression testing apparatus provided in the embodiment of the present invention may be specifically configured to execute the processing flows of the above method embodiments, and the functions of the embodiment are not described herein again, and refer to the detailed description of the above method embodiments.
Fig. 7 is a schematic physical structure diagram of an electronic device according to an embodiment of the present invention, as shown in fig. 7, the electronic device includes a processor 701, a memory 702 and a communication bus 703;
the processor 701 and the memory 702 complete mutual communication through a communication bus 703;
processor 701 is configured to call program instructions in memory 702 to perform the methods provided by the above-described method embodiments, including, for example: acquiring a to-be-tested data set of a to-be-tested item; obtaining the probability corresponding to each data according to each data in the data set to be tested and the probability distribution function corresponding to the item to be tested; wherein the probability distribution function is pre-established; if the probability corresponding to the data is judged to be larger than the threshold value, the test result of the data is normal, otherwise, the test result of the data is abnormal.
The present embodiment discloses a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, enable the computer to perform the method provided by the above-mentioned method embodiments, for example, comprising: acquiring a to-be-tested data set of a to-be-tested item; obtaining the probability corresponding to each data according to each data in the data set to be tested and the probability distribution function corresponding to the item to be tested; wherein the probability distribution function is pre-established; if the probability corresponding to the data is judged to be larger than the threshold value, the test result of the data is normal, otherwise, the test result of the data is abnormal.
The present embodiments provide a non-transitory computer-readable storage medium storing computer instructions that cause the computer to perform the methods provided by the above method embodiments, for example, including: acquiring a to-be-tested data set of a to-be-tested item; obtaining the probability corresponding to each data according to each data in the data set to be tested and the probability distribution function corresponding to the item to be tested; wherein the probability distribution function is pre-established; if the probability corresponding to the data is judged to be larger than the threshold value, the test result of the data is normal, otherwise, the test result of the data is abnormal.
In addition, the logic instructions in the memory may be implemented in the form of software functional units and may be stored in a computer readable storage medium when sold or used as a stand-alone product. Based on such understanding, the technical solutions of the embodiments of the present invention may be essentially implemented or make a contribution to the prior art, or may be implemented in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, an apparatus, or a network device) to execute all or part of the steps of the methods described in the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention, and not to limit the same; although embodiments of the present invention have been described in detail with reference to the foregoing embodiments, those skilled in the art will understand that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A regression testing method, comprising:
acquiring a to-be-tested data set of a to-be-tested item;
obtaining the probability corresponding to each data according to each data in the data set to be tested and the probability distribution function corresponding to the item to be tested; wherein the probability distribution function is pre-established;
if the probability corresponding to the data is judged to be larger than the threshold value, the test result of the data is normal, otherwise, the test result of the data is abnormal.
2. The method of claim 1, wherein establishing the probability distribution function comprises:
acquiring a historical test data set of the item to be tested and a historical test result of each data in the historical test data set;
acquiring a first preset amount of data from the historical test data set to form a training set, wherein the historical test result of each data in the training set is normal;
and obtaining the probability distribution function according to the training set.
3. The method of claim 2, wherein the threshold is determined according to the following steps:
removing the training set from the historical test data set to obtain a cross validation set;
obtaining the probability corresponding to each data in the cross validation set according to the cross validation set and the probability distribution function;
obtaining a verification result of each data in the cross validation set according to the probability corresponding to each data in the cross validation set and a threshold to be determined; wherein the threshold to be determined is preset;
obtaining an F-measure value corresponding to the threshold to be determined according to the verification result and the historical test result of each data in the cross verification set;
and calculating F-measure values corresponding to the second preset number of the thresholds to be determined respectively, and taking the threshold to be determined corresponding to the largest F-measure value in the second preset number of the F-measure values as the threshold.
4. The method according to claim 3, wherein the obtaining the verification result of each data in the cross-validation set according to the probability corresponding to each data in the cross-validation set and the threshold to be determined comprises:
if the probability corresponding to the data in the cross validation set is judged to be larger than the threshold value to be determined, the validation result of the data in the cross validation set is normal, otherwise, the validation result of the data in the cross validation set is abnormal.
5. The method according to claim 3, wherein the obtaining the F-measure value corresponding to the threshold to be determined according to the verification result and the historical test result of each data in the cross-validation set comprises:
according to the formulaCalculating to obtain an F-measure value F corresponding to the threshold to be determined, wherein,r represents a test completeness, t represents the number of data in the cross validation set with the normal historical test result and the normal validation result, fnRepresenting the amount of data in the cross-validation set for which the historical test results are anomalous and for which the validation results are anomalous;p represents the test accuracy, fpRepresenting the amount of data in the cross-validation set that the historical test result is abnormal and the validation result is normal; and the cross validation set comprises data with normal historical test results and data with abnormal historical test results.
6. A regression testing apparatus, comprising:
the first acquisition unit is used for acquiring a to-be-tested data set of a to-be-tested item;
the first obtaining unit is used for obtaining the probability corresponding to each data according to each data in the data set to be tested and the probability distribution function corresponding to the item to be tested; wherein the probability distribution function is pre-established;
and the test unit is used for judging whether the probability corresponding to the data is greater than a threshold value or not, if so, judging that the test result of the data is normal, and otherwise, judging that the test result of the data is abnormal.
7. The apparatus of claim 6, further comprising:
the second acquisition unit is used for acquiring a historical test data set of the item to be tested and a historical test result of each data in the historical test data set;
a third obtaining unit, configured to obtain a first preset number of data from the historical test data set to form a training set, where a historical test result of each data in the training set is normal;
a second obtaining unit, configured to obtain the probability distribution function according to the training set.
8. The apparatus of claim 7, further comprising:
a third obtaining unit, configured to remove the training set from the historical test data set to obtain a cross validation set;
a fourth obtaining unit, configured to obtain, according to the cross validation set and the probability distribution function, a probability corresponding to each data in the cross validation set;
a fifth obtaining unit, configured to obtain a verification result of each data in the cross validation set according to a probability corresponding to each data in the cross validation set and a threshold to be determined; wherein the threshold to be determined is preset;
a sixth obtaining unit, configured to obtain, according to a verification result and a historical test result of each data in the cross-validation set, an F-measure value corresponding to the threshold to be determined;
and the calculating unit is used for calculating F-measure values corresponding to the second preset number of the thresholds to be determined respectively, and taking the threshold to be determined corresponding to the largest F-measure value in the second preset number of the F-measure values as the threshold.
9. An electronic device, comprising: a processor, a memory, and a communication bus, wherein:
the processor and the memory are communicated with each other through the communication bus;
the memory stores program instructions executable by the processor, the processor invoking the program instructions to perform the method of any of claims 1 to 5.
10. A non-transitory computer-readable storage medium storing computer instructions that cause a computer to perform the method of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810606840.0A CN110597703A (en) | 2018-06-13 | 2018-06-13 | Regression testing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810606840.0A CN110597703A (en) | 2018-06-13 | 2018-06-13 | Regression testing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110597703A true CN110597703A (en) | 2019-12-20 |
Family
ID=68849267
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810606840.0A Pending CN110597703A (en) | 2018-06-13 | 2018-06-13 | Regression testing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110597703A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111637926A (en) * | 2020-06-03 | 2020-09-08 | 广东电网有限责任公司清远供电局 | Switch cabinet monitoring system and method |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070179398A1 (en) * | 2005-08-24 | 2007-08-02 | Margolis Robert H | Method for assessing the accuracy of test results |
| CN101201762A (en) * | 2006-12-13 | 2008-06-18 | 株式会社东芝 | Device and method for detecting abnormal symptom |
| CN101651568A (en) * | 2009-07-01 | 2010-02-17 | 青岛农业大学 | Method for predicting network flow and detecting abnormality |
| CN102339288A (en) * | 2010-07-21 | 2012-02-01 | 中国移动通信集团辽宁有限公司 | Method and device for detecting abnormal data of data warehouse |
| US20150309918A1 (en) * | 2014-04-25 | 2015-10-29 | Wipro Limited | Method of optimizing execution of test cases and a system thereof |
| CN106021090A (en) * | 2016-05-04 | 2016-10-12 | 上海瀚银信息技术有限公司 | Software interface test system and method |
| CN106933745A (en) * | 2017-03-30 | 2017-07-07 | 携程计算机技术（上海）有限公司 | WEB automatic interface testing methods and system |
| CN107197473A (en) * | 2017-06-15 | 2017-09-22 | 三星电子（中国）研发中心 | A kind of terminal abnormal state determines method and apparatus |
| CN107239387A (en) * | 2017-05-25 | 2017-10-10 | 深圳市金立通信设备有限公司 | A kind of data exception detection method and terminal |
| CN107291911A (en) * | 2017-06-26 | 2017-10-24 | 北京奇艺世纪科技有限公司 | A kind of method for detecting abnormality and device |
| JP2017194341A (en) * | 2016-04-20 | 2017-10-26 | 株式会社Ihi | Abnormality diagnosis method, abnormality diagnosis device, and abnormality diagnosis program |
| CN107426019A (en) * | 2017-07-06 | 2017-12-01 | 国家电网公司 | Network failure determines method, computer equipment and computer-readable recording medium |
| CN107943966A (en) * | 2017-11-28 | 2018-04-20 | 合肥工业大学 | Abnormal individual character decision method and device based on microblogging text |
-
2018
- 2018-06-13 CN CN201810606840.0A patent/CN110597703A/en active Pending
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070179398A1 (en) * | 2005-08-24 | 2007-08-02 | Margolis Robert H | Method for assessing the accuracy of test results |
| CN101201762A (en) * | 2006-12-13 | 2008-06-18 | 株式会社东芝 | Device and method for detecting abnormal symptom |
| CN101651568A (en) * | 2009-07-01 | 2010-02-17 | 青岛农业大学 | Method for predicting network flow and detecting abnormality |
| CN102339288A (en) * | 2010-07-21 | 2012-02-01 | 中国移动通信集团辽宁有限公司 | Method and device for detecting abnormal data of data warehouse |
| US20150309918A1 (en) * | 2014-04-25 | 2015-10-29 | Wipro Limited | Method of optimizing execution of test cases and a system thereof |
| JP2017194341A (en) * | 2016-04-20 | 2017-10-26 | 株式会社Ihi | Abnormality diagnosis method, abnormality diagnosis device, and abnormality diagnosis program |
| CN106021090A (en) * | 2016-05-04 | 2016-10-12 | 上海瀚银信息技术有限公司 | Software interface test system and method |
| CN106933745A (en) * | 2017-03-30 | 2017-07-07 | 携程计算机技术（上海）有限公司 | WEB automatic interface testing methods and system |
| CN107239387A (en) * | 2017-05-25 | 2017-10-10 | 深圳市金立通信设备有限公司 | A kind of data exception detection method and terminal |
| CN107197473A (en) * | 2017-06-15 | 2017-09-22 | 三星电子（中国）研发中心 | A kind of terminal abnormal state determines method and apparatus |
| CN107291911A (en) * | 2017-06-26 | 2017-10-24 | 北京奇艺世纪科技有限公司 | A kind of method for detecting abnormality and device |
| CN107426019A (en) * | 2017-07-06 | 2017-12-01 | 国家电网公司 | Network failure determines method, computer equipment and computer-readable recording medium |
| CN107943966A (en) * | 2017-11-28 | 2018-04-20 | 合肥工业大学 | Abnormal individual character decision method and device based on microblogging text |
Non-Patent Citations (3)
| Title |
|---|
| JUNFEI_WANG: "基于高斯分布的异常检测(Anomaly Detection)算法", 《HTTPS://WWW.CNBLOGS.COM/RHYSWANG/P/7076414.HTML》 * |
| WYL1813240346: "斯坦福大学机器学习笔记——异常检测算法（高斯分布、多元高斯分布、异常检测算法）", 《HTTPS://BLOG.CSDN.NET/WYL1813240346/ARTICLE/DETAILS/79059647》 * |
| 于冰洁等: "基于高斯过程模型的异常检测算法", 《计算机工程与设计》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111637926A (en) * | 2020-06-03 | 2020-09-08 | 广东电网有限责任公司清远供电局 | Switch cabinet monitoring system and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112162878B (en) | Database fault discovery method and device, electronic equipment and storage medium | |
| CN106294120B (en) | Method, apparatus and computer program product for testing code | |
| CN110471821B (en) | Abnormality change detection method, server, and computer-readable storage medium | |
| CN110457175B (en) | Service data processing method and device, electronic equipment and medium | |
| CN113762525B (en) | Federal learning model training method with differential privacy protection | |
| CN113536081A (en) | Data center data management method and system based on artificial intelligence | |
| CN113507455A (en) | Network security detection method and system based on big data | |
| CN116483697A (en) | Method and system for evaluating artificial intelligence model | |
| CN116340934A (en) | Terminal abnormal behavior detection method, device, equipment and storage medium | |
| CN115858311A (en) | Operation and maintenance monitoring method and device, electronic equipment and readable storage medium | |
| CN110597703A (en) | Regression testing method and device | |
| CN110348215A (en) | Exception object recognition methods, device, electronic equipment and medium | |
| CN113112126A (en) | Water supply pipe leakage early warning method and system based on artificial intelligence | |
| CN110765005B (en) | Software reliability evaluation method and device | |
| CN118445412A (en) | Method and device for detecting risk text, storage medium and electronic equipment | |
| CN117009221A (en) | Processing method, device, equipment, storage medium and program product for product test | |
| CN115923592A (en) | Battery state detection method and device for electric vehicle | |
| CN111124918B (en) | Test data prediction method and device and processing equipment | |
| CN113961565A (en) | Data detection method, system, computer system and readable storage medium | |
| CN114938339A (en) | Data processing method and related device | |
| CN112597046A (en) | Test method, test device, computer equipment and storage medium | |
| CN111258916A (en) | Automatic testing method and device, storage medium and equipment | |
| WO2023151622A1 (en) | Page testing method, apparatus and device, and storage medium | |
| CN112463577B (en) | Sample data processing method and device and electronic equipment | |
| CN118822819A (en) | User information data accuracy detection method and device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191220 |