WO2023123943A1

WO2023123943A1 - Interface automation testing method and apparatus, and medium, device and program

Info

Publication number: WO2023123943A1
Application number: PCT/CN2022/102161
Authority: WO
Inventors: 杨璟斐; 曾凌子; 江旻; 杨杨
Original assignee: 深圳前海微众银行股份有限公司
Priority date: 2021-12-27
Filing date: 2022-06-29
Publication date: 2023-07-06
Also published as: CN114168486A

Abstract

Provided in the present application are an interface automation testing method and apparatus, and a medium, a device and a program. The interface automation testing method provided in the embodiments of the present application comprises: executing, in a system link, each test scenario use case in a test use case set, and acquiring a use case running result according to a preset secure program running mechanism; determining a test baseline corresponding to each interface of the system link according to the running result of each test scenario use case and a use case scenario rule in a production version code environment; executing, in the system link, the running of a program of a version to be tested, and acquiring a test result of each interface; and finally, generating an assertion according to the test baseline and test result which correspond to each interface, so that a full-link assertion of the system link can be automatically and accurately generated without the need of manual maintenance, thereby meeting the requirements of automatically covering an interface test corresponding to a full link.

Description

Interface automated testing method, device, medium, equipment and program

This application claims the priority of the Chinese patent application with the application number 202111609529.X and the application name "Interface automation test method, device, medium, equipment and program" submitted to the China Patent Office on December 27, 2021, the entire content of which Incorporated in this application by reference.

technical field

The present application relates to the technical field of interface testing, in particular to an interface automatic testing method, device, medium, equipment and program.

Background technique

With the development of computer technology, more and more technologies are applied in the financial field. The traditional financial industry is gradually transforming into financial technology (Finteh), and the interface testing technology is no exception. However, due to the security and real-time requirements of the financial industry, It also places higher demands on technology.

At present, in the interface automation testing of financial technology, it is usually based on static code analysis to generate a database (Data Base, DB) table structure template, and then correlate with the DB full dynamic table for verification, so as to manually configure the interface to correspond to the business The process involves the system, and the tools are used to back up and compare the dynamic table data in all databases under the system, so as to complete the test of each system.

However, for the static code analysis of the interface to generate the DB table structure template, the generated checkpoints are not perfect, and cannot automatically cover the requirements of the interface test corresponding to the entire link.

technical solution

The embodiment of the present application provides an interface automatic testing method, device, medium, equipment and program to solve the technical problem that the existing technology cannot automatically cover the corresponding interface testing requirements of the whole link.

In the first aspect, the embodiment of the present application provides an interface automated testing method, including:

Obtain a test case set, the test case set includes multiple types of test case subsets, each type of test case subset includes a plurality of test scenario use cases, and each test scenario use case is configured with a corresponding use case business serial number;

Execute each test scenario use case in the test case set in the system link, and obtain the use case operation result according to the mechanism of the preset safe operation program, the use case operation result includes the database operation result and the message result returned by each interface;

Determine the test baseline corresponding to each interface of the system link according to the running results of each test scenario use case in the production version code environment and the use case scenario rules;

Execute and run the program of the version to be tested in the system link, and obtain the test results of each interface;

An assertion is generated according to the test baseline and the test result corresponding to each interface.

In a possible design, the determination of the test baseline corresponding to each interface of the system link according to the running results of each test scenario use case in the production version code environment and the use case scenario rules includes:

Directly remove the first type of noise result in the use case operation result according to the preset denoising rule, wherein the first type of noise result is an operation result that has nothing to do with the test logic of the system link;

Carrying out horizontal analysis on the operation results of the use cases to determine the corresponding operation result attributes of each interface, the horizontal analysis includes cluster analysis on the operation results generated by running different test scenario use cases in the same test case subset;

The test baseline corresponding to each interface of the system link is determined according to the use case running result and the corresponding behavior result attribute.

In a possible design, the horizontal analysis is performed on the operation result of the use case to determine the attributes of the operation result corresponding to each interface, including:

After running the preset first number of test scenario cases in the same test case subset, obtain the test case running results of each interface;

Determine the corresponding result equivalent rate according to the result distribution of the use case operation results of each interface;

Determine the corresponding operation result attribute according to the result equivalent rate of each interface and the preset equivalent rate condition.

In a possible design, the determination of the corresponding operation result attributes according to the result equivalent rate of each interface and the preset equivalent rate condition includes:

If the result equivalent rate is 100%, it is determined that the corresponding operation result attribute is a fixed value class, and the test baseline corresponding to the fixed value class is a fixed value;

If the result equivalence rate is greater than or equal to the preset equivalence rate, it is determined that the corresponding operation result attribute is an enumeration class, and the test baseline corresponding to the enumeration class is an enumeration list;

If the result equivalent rate is less than the preset equivalent rate, it is determined that the corresponding use case operation result is noise.

In a possible design, after the horizontal analysis of the use case operation results to determine the operation result attributes corresponding to each interface, it further includes:

Carrying out longitudinal analysis on the running results of the use cases belonging to the enumerated class, the longitudinal analysis including cluster analysis on the running results generated by running the same test scenario use case;

If the running results of all the use cases belonging to the enumeration class are the same, backing up the running results of the use cases to the enumeration list;

If the running results of the use cases belonging to the enumeration class are not the same, it is judged whether the running results of each use case are obtained when the preset backup conditions are met, and the preset backup conditions are used to determine that the running of the use cases is in an end state;

If the judgment result is yes, back up the running results of each use case to the enumeration list;

If the judgment result is no, then back up the use case running results obtained by satisfying the preset backup conditions to the enumeration list.

In a possible design, before executing and running the program of the version to be tested in the system link, and obtaining the test results of each interface, it also includes:

Generate basic assertions according to the affiliation relationship between the returned message results and the preset result enumeration set in the use case running results corresponding to each test scenario use case;

It is determined that the test results corresponding to the test scenario use cases in which the basic assertion conforms to the preset assertion result are compared with the test baseline corresponding to each interface to generate the assertion.

In a possible design, the basic assertion is generated according to the affiliation relationship between the returned message result and the preset result enumeration set in the use case running result corresponding to each test scenario use case, including:

If the test scenario use case is a pass business use case, then if the returned message result in the corresponding use case operation result does not belong to the preset result enumeration set, then the test scenario use case is a failure use case; If the returned message result belongs to the preset result enumeration set, then the test scenario use case is a successful use case; the preset result enumeration set is a business success set;

If the test scenario use case is a failure business use case, then if the returned message result in the corresponding use case operation result does not belong to the preset result enumeration set, then the test scenario use case is a successful use case; If the returned message result belongs to the preset result enumeration set, the test scenario use case is a failure use case;

If the test scenario use case is a use case that does not conform to the protocol specification, if the returned message result in the corresponding use case running result does not belong to the preset result enumeration set, the test scenario use case is a failure use case; if the corresponding use case running result If the returned message result in the test belongs to the preset result enumeration set, then the test scenario use case is a successful use case; the preset result enumeration set is the message does not meet the protocol specification exception set;

If the test scenario use case is a use case conforming to the protocol specification, if the returned message result in the corresponding use case operation result does not belong to the preset result enumeration set, the test scenario use case is a successful use case; if the corresponding use case operation result in If the returned message result belongs to the preset result enumeration set, the test scenario use case is a failed use case.

In a possible design, after the assertion is generated according to the test baseline corresponding to each interface and the test result, it further includes:

It is determined that the interface part where the difference between the test baseline indicated by the assertion and the test result is a system change point or a system vulnerability.

In a possible design, before directly removing the first type of noise results in the use case running results according to the preset denoising rules, it further includes:

If the result of the use case operation includes a large structured field, split the large structured field for comparison with the preset denoising rule, so as to remove the first category in each field after splitting noise result.

In the second aspect, the embodiment of the present application also provides an interface automated testing device, including:

The obtaining module is used to obtain a test case set, the test case set includes multiple types of test case subsets, each type of test case subset includes a plurality of test scenario use cases, and each test scenario use case is configured with a corresponding use case business serial number;

The processing module is used to execute each test scenario use case in the test case set in the system link, and obtain the use case operation result according to the mechanism of the preset safe operation program, and the use case operation result includes the database operation result and each interface Return message result;

The processing module is also used to determine the test baseline corresponding to each interface of the system link according to the running results of each test scenario use case in the production version code environment and the use case scenario rules;

The obtaining module is also used to execute and run the program of the version to be tested in the system link, and obtain the test results of each interface;

The processing module is further configured to generate an assertion according to the test baseline and the test result corresponding to each interface.

In a possible design, the processing module is specifically used for:

The test baseline corresponding to each interface of the system link is determined according to the use case running result and the corresponding result attribute.

In a possible design, the processing module is specifically used for:

In a possible design, the processing module is also used for:

In a possible design, if the test scenario use case is a business use case, if the returned message result in the corresponding use case operation result does not belong to the preset result enumeration set, then the test scenario use case is a failure use case; If the returned message result in the corresponding use case operation result belongs to the preset result enumeration set, then the test scenario use case is a successful use case; the preset result enumeration set is a business success set;

In a possible design, the processing module is further configured to determine that the interface portion of the difference between the test baseline indicated by the assertion and the test result is a system change point or a system vulnerability.

In a possible design, if the use case running result includes a structured large field, the structured large field is split for comparison with the preset denoising rule, so as to remove the split Type 1 noise results in the following fields.

In a third aspect, the embodiment of the present application further provides an electronic device, including:

Processor; and,

a memory for storing executable instructions of the processor;

Wherein, the processor is configured to execute any interface automation testing method in the first aspect by executing the executable instructions.

In a fourth aspect, the embodiment of the present application further provides a storage medium on which a computer program is stored, and when the program is executed by a processor, any one of the interface automation testing methods in the first aspect is implemented.

In the fifth aspect, the embodiment of the present application further provides a computer program product, including a computer program, and when the computer program is executed by a processor, any interface automation testing method in the first aspect is implemented.

An interface automation testing method, device, medium, equipment, and program provided by the embodiment of the present application obtain the use cases by executing each test scenario use case in the test case set in the system link, and obtain the use cases according to the mechanism of the preset safe running program Run the results, and then determine the test baseline corresponding to each interface of the system link according to the use case running results and the use case scenario rules, then execute the version program to be tested in the system link, and obtain the test results of each interface, and finally, according to each interface Corresponding test baselines and test results generate assertions, so as to realize automatic and accurate generation of full-link assertions of system links, without manual maintenance, and thus meet the requirements of automatically covering the corresponding interface tests of the entire link.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description These are some embodiments of the present application. Those skilled in the art can also obtain other drawings based on these drawings without any creative effort.

FIG. 1 is a schematic diagram of a test flow showing an interface automation test method according to an example embodiment of the present application;

Fig. 2 is a schematic flowchart of an interface automated testing method according to an example embodiment of the present application;

Fig. 3 is a schematic flow diagram of an assertion self-learning module according to an example embodiment of the present application;

Fig. 4 is another schematic flowchart of an interface automated testing method according to an example embodiment of the present application;

Fig. 5 is a schematic structural diagram of an interface automation testing device according to an example embodiment of the present application;

Fig. 6 is a schematic structural diagram of an electronic device according to an example embodiment of the present application.

Embodiments of the present invention

In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments It is only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without creative work, including but not limited to combinations of multiple embodiments, all fall within the protection scope of this application.

The terms "first", "second", "third", "fourth", etc. (if any) in the description and claims of this application and the above drawings are used to distinguish similar objects, and not necessarily Used to describe a specific sequence or sequence. It is to be understood that the data so used are interchangeable under appropriate circumstances such that the embodiments of the application described herein, for example, can be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having", as well as any variations thereof, are intended to cover a non-exclusive inclusion, for example, a process, method, system, product or device comprising a sequence of steps or elements is not necessarily limited to the expressly listed instead, may include other steps or elements not explicitly listed or inherent to the process, method, product or apparatus.

With the development of computer technology, more and more technologies are applied in the financial field. The traditional financial industry is gradually transforming into financial technology (Finteh), and interface testing technology is no exception. However, due to the security and real-time requirements of the financial industry, It also places higher demands on technology. At present, in the automated testing of financial technology interfaces, it is usually based on static code analysis to generate databases (Data Base, referred to as DB) table structure template, and then correlate with the DB full dynamic table for verification, so as to correspond to the way the business process involves the system through the manual configuration interface, and then use the tool to backup and compare the dynamic table data in all databases under the system. In order to complete the testing of each system, for example, the above operations are performed through the open-source automated testing framework SOFAACTS of Ant Financial.

However, for the static code analysis of the interface to generate the DB table structure template, the generated checkpoints are not perfect, and cannot automatically cover the requirements of the interface test corresponding to the entire link. In addition, the release of the above-mentioned prior art also has the problem of high maintenance costs, requiring manual maintenance of the expected value of the check point. When the protocol version changes, SOFAACTS needs to manually trigger the generation of a new template. In addition, to verify the full dynamic table of the associated DB, it is necessary to back up all the dynamic tables under the configuration system, which is likely to waste storage space.

In view of the above-mentioned technical problems, the interface automation testing method, device, medium, equipment and program provided in the embodiment of the present application, compared with the above-mentioned testing process in the prior art, does not require manual input of assertions. The solutions in the existing technology manually enter the assertion, the labor cost is high, and the integrity of the assertion cannot be guaranteed, and the checkpoint is often missed. full link. In the embodiment provided by this application, each test scenario use case in the test case set is executed in the system link, and the operation result of the use case is obtained according to the mechanism of the preset safe operation program, and then, according to the operation result of the use case and the use case scenario rules Determine the test baseline corresponding to each interface of the system link, and then execute the version program to be tested in the system link, and obtain the test results of each interface, and finally, generate an assertion according to the test baseline and test results corresponding to each interface, so as to realize the system The full link assertion of the link is automatically and accurately generated without manual maintenance, thus meeting the requirement of automatically covering the corresponding interface test of the whole link.

In order to understand this application more accurately, the following terms are defined and explained as follows:

(1) Assertion (Assert): Expressed as some Boolean expressions, used to indicate that the value of the expression can be believed to be true at a certain point in the program.

(2) Interface Testing (Interface Testing): It is a test type for testing the interface between system components. Interface testing is mainly used to detect the interaction points between external systems and internal subsystems. The focus of testing is to check data exchange, transfer and control management processes, and inter-system logical dependencies.

(3) Interface Auto Testing (Interface Auto Testing): Based on interfaces and protocols, run systems and applications under preset conditions to evaluate the results of a test, where the preconditions should include normal conditions and abnormal conditions.

(4) Sandbox (SANDBOX): Sandbox refers to a technology, which is a Java Virtual Machine (Java Virtual Machine, JVM) platform that is open sourced by Alibaba. The solution is essentially a form of AOP implementation.

Fig. 1 is a schematic diagram of a testing process illustrating an interface automation testing method according to an exemplary embodiment of the present application. As shown in Figure 1, in the interface automation testing method provided in this embodiment, the test scenario use cases can be generated in batches through the interface automation platform, that is, the test cases in each scenario can be automatically batch generated through the interface automation platform, and each Test scenario use cases are configured with corresponding use case business serial numbers. It is worth noting that it is precisely because the embodiment of this application is based on the interface automation platform, through which a large number of test scenario use cases will be automatically generated. Therefore, the generated assertions are difficult to maintain manually. Therefore, it is urgently needed The assertion is generated and maintained through the interface automated testing method provided by the embodiment of the present application.

Specifically, in the embodiment of the present application, assertion self-learning is adopted to realize the automatic generation of assertions of interface return messages and various database operation results. Among them, the main method is to generate assertion expected values for interface return messages and various database operation results through assertion self-learning, and use the expected values as the baseline, and then generate assertions by comparing and verifying the execution results of use cases through Diff.

Fig. 2 is a schematic flowchart of an interface automation testing method according to an example embodiment of the present application. As shown in Figure 2, the interface automated testing method provided in this embodiment includes:

Step 101, acquiring a test case set.

In this step, a set of test cases is obtained. The test case set includes multiple types of test case subsets. Each type of test case subset includes multiple test scenario cases. Each test scenario use case is configured with a corresponding use case business flow Number.

Step 102: Execute each test scenario use case in the test case set in the system link, and obtain the test case running result according to the preset safe running program mechanism.

Specifically, each test scenario use case in the test case set can be executed in the system link, and the test case running result can be obtained according to the preset safe running program mechanism (for example: sandbox), wherein the test case running result includes the database operation result And each interface returns the message result.

Wherein, FIG. 3 is a schematic flowchart of the assertion self-learning module according to an example embodiment of the present application. As shown in Figure 3, the database operation results of the full link and the message results returned by each interface in the full link can be automatically obtained through the assertion self-learning module, and used as the basic data for the analysis of the assertion results. Specifically, the test plan can be pulled up periodically to execute the test scenario use case, and the business serial number bizNo of the test scenario use case can be passed as an input parameter to the assertion self-learning module, so as to automatically analyze the test case full-link interface return message results and the database result of the operation. In addition, the assertion self-learning module takes the use case business serial number as an input parameter, and obtains the link system list, the SQL list of each interface operation, and the return message of each interface through the sandbox. Among them, the system list is combined with the database to obtain the link system library, table, and field collection. The SQL list obtained by the sandbox is associated with the library table field set to remove information such as aliases in the SQL, so as to obtain the library, table, and field information of each interface operation under the link of the use case scenario. Then, the database operation results of the link and the message results returned by each interface are stored as the use case running results. Specifically, the information contained in the test case running result backup may include: test case CASE_ID, business serial number BIZ_NO, and return message RSP_MSG.

Step 103: Determine the test baseline corresponding to each interface of the system link according to the use case running result and the use case scenario rules.

Step 104, execute and run the program of the version to be tested in the system link, and obtain the test results of each interface.

Step 105, generating assertions according to the test baselines and test results corresponding to each interface.

In step 103-step 105, the test baseline corresponding to each interface of the system link can be determined first according to the use case running results and the use case scenario rules. For the use case scenario rules, the interface can return messages and various Database operation results are processed, for example, automatic denoising processing can be performed first to remove fields that do not involve logic, and then, through horizontal analysis and vertical analysis, determine the operation result attributes corresponding to each interface, and determine the assertion expectations for each interface value, so that the expected value of the assertion is determined as the test baseline corresponding to each interface of the system link, and entered into the baseline. Then, by executing and running the version program to be tested in the system link, and obtaining the test results of each interface, an assertion is generated according to the test baseline and test results corresponding to each interface.

In this embodiment, each test scenario use case in the test case set is executed in the system link, and the operation result of the use case is obtained according to the mechanism of the preset safe operation program, and then the system is determined according to the operation result of the use case and the use case scenario rules. The test baseline corresponding to each interface of the link, and then execute the version program to be tested in the system link, and obtain the test results of each interface, and finally, generate an assertion according to the test baseline and test results corresponding to each interface, so as to realize the system link The full-link assertion is automatically and accurately generated without manual maintenance, thus meeting the requirement of automatically covering the corresponding interface test of the whole link.

Fig. 4 is another schematic flowchart of an interface automation testing method according to an example embodiment of the present application. As shown in Figure 4, the interface automated testing method provided in this embodiment includes:

Step 201, acquiring a test case set.

Step 202: Execute each test scenario use case in the test case set in the system link, and obtain the test case running result according to the preset safe running program mechanism.

The database operation results of the full link and the message results returned by each interface in the full link can be automatically obtained through the assertion self-learning module, and used as the basic data for the analysis of the assertion results. Specifically, the test plan can be pulled up periodically to execute the test scenario use case, and the business serial number bizNo of the test scenario use case can be passed as an input parameter to the assertion self-learning module, so as to automatically analyze the test case full-link interface return message results and the database result of the operation. In addition, the assertion self-learning module takes the use case business serial number as an input parameter, and obtains the link system list, the SQL list of each interface operation, and the return message of each interface through the sandbox. Among them, the system list is combined with the database to obtain the link system library, table, and field collection. The SQL list obtained by the sandbox is associated with the library table field set to remove information such as aliases in the SQL, so as to obtain the library, table, and field information of each interface operation under the link of the use case scenario. Then, the database operation results of the link and the message results returned by each interface are stored as the use case running results. Specifically, the information contained in the test case running result backup may include: test case CASE_ID, business serial number BIZ_NO, and return message RSP_MSG.

Step 203, directly remove the first type of noise results in the use case running results according to the preset noise removal rules.

In this step, the first type of noise results in the use case running results may be directly removed according to the preset denoising rules, wherein the first type of noise results are running results that have nothing to do with the test logic of the system link.

Specifically, it can automatically identify the serial number, time stamp, IOU number, logic card number and other fields in the use case running results, and directly perform noise removal processing after identification according to the rules. For example:

1. If the length is greater than 20 characters and consists of letters or numbers, it can be identified as a serial number;

2. For numbers with a length of 8, 10, 13, 14, and 17 digits, and the string time conversion function can handle it normally; if the length is 16 or 19 digits, spaces, "-" and ":", it can be regarded as time;

3. For the length equal to 20, the first two digits are DCN_NO, which can be regarded as the IOU number;

4. For a length of 16 digits, the first 6 are equal to the product number, which can be identified as a logical card number.

Step 204 , horizontally analyzing the running result of the use case to determine the attributes of the running result corresponding to each interface.

Specifically, all use cases under the interface can be launched in batches n times according to the execution plan, where n can be individually configured according to each interface, and the initial value is 2 if not configured. For the convenience of explanation, the default value n=2 can be used as an example. The system automatically adjusts according to the number of interface use cases, n>=2, so as to ensure that the total number of operation times of interface use cases is greater than N, and N can also be configured. If the default initial value is not configured for 100. It is worth understanding that n is the number of times of launching use cases corresponding to each interface, and N is the minimum threshold of the number of times of launching use cases of all interfaces in the whole link. In addition, the number of run cases can be dynamically adjusted to ensure the amount of sample data and the correctness of the training results.

After the execution plan is finished running and the test case running results are obtained, the test case running results can be analyzed horizontally to determine the properties of the running results corresponding to each interface. The horizontal analysis includes running different test scenario cases in the same test case subset. Cluster analysis of the running results.

Optionally, after running the preset first number of test scenario use cases in the same test case subset, obtain the use case running results of each interface, and then determine the corresponding result equivalent rate according to the result distribution of the use case running results of each interface, And determine the corresponding operation result attribute according to the result equivalent rate of each interface and the preset equivalent rate condition.

Specifically, if the result equivalence rate is 100%, it is determined that the corresponding operation result attribute is a fixed value class, and the test baseline corresponding to the fixed value class is a fixed value; if the result equivalence rate is greater than or equal to the preset equivalence rate, then Determine that the corresponding operation result attribute is an enumeration class, and the test baseline corresponding to the enumeration class is an enumeration list; if the result equivalent rate is less than the preset equivalent rate, determine that the corresponding use case operation result is noise, which can be removed .

In a possible design, horizontal clustering analysis can be performed on all the obtained running results of the use cases above. If the message returns or the equivalent rate of the DB field is 100%, it is a fixed value class and directly backed up to the baseline. If the message is returned or the equivalent rate of the DB field is greater than 20%, it is an enumeration class, and the enumeration list is maintained. If the message is returned or the equivalent rate of the DB field is less than 20%, it is an irregular type, which is directly identified as noise and can be removed.

Step 205 , longitudinally analyzing the running results of the use cases belonging to the enumerated class.

In the step, after performing a horizontal analysis on the running results of the use cases to determine the corresponding running result attributes of each interface, it is necessary to conduct a longitudinal analysis on the running results of the use cases belonging to the enumeration class, wherein the longitudinal analysis includes running the same test scenario Cluster analysis performed on the run results generated by the use case. If the running results of all use cases belonging to the enumeration class are the same, back up the running results of the use cases to the enumeration list; if the running results of the use cases belonging to the enumeration class are different, then judge whether the running results of each use case meet the preset backup conditions The preset backup conditions are used to determine that the use case operation is in the end state; if the judgment result is yes, then the running results of each use case will be backed up to the enumeration list; if the judgment result is no, the preset backup conditions will be met. The obtained use case running results are backed up to the enumeration list.

Specifically, the same test scenario use case can be run multiple times, and the use case running result after each run can be obtained. If the running results are consistent every time, you can directly back up the fixed use case running results to the baseline. If the running results are inconsistent, it may be an intermediate state of running or a random hit enumeration. At this point, the user can be notified and intervened to perform batch denoising. And if it is running in the intermediate state, the user can set the preset backup conditions according to the enumeration results, and the system will perform backup after judging that the preset conditions are met, thereby eliminating the intermediate state. If the user determines that the enumeration result is randomly hit, the enumeration list will be backed up as the expected value of the assertion as the baseline, and the subsequent test case running results will be considered successful as long as one of the enumeration lists is hit.

Step 206: Generate a basic assertion according to the affiliation relationship between the returned message result and the preset result enumeration set in the use case running results corresponding to each test scenario use case.

In this step, the basic assertion may be generated according to the affiliation relationship between the returned message result in the use case operation result corresponding to each test scenario use case and the preset result enumeration set, and determine that the basic assertion conforms to the test scenario use case of the preset assertion result. The corresponding test results are continuously compared with the test baselines corresponding to each interface to generate assertions.

Specifically, after the test cases are executed in batches, the message return values of all the test scenario cases, that is, the fields containing the code string, are clustered. The code enumeration is obtained through denoising analysis and horizontal analysis, combined with self-learning of the use case alias: business success, business failure, message does not conform to the protocol specification exception, and system failure returns code enumeration in four categories. Among them, taking "the message does not conform to the protocol specification exception" as an example, the use cases can be classified according to the identification keywords such as "extra long", "outside the enumeration range", and "does not conform to the data type", among which , the return code of each type of use case is theoretically unique. After the codes obtained through denoising analysis and horizontal analysis are grouped according to the use cases, the codes in each group are theoretically equal. In practice, you can analyze the differences of the codes under each set, and the code enumeration value with a consistency greater than 90% means that the corresponding type of use case can reasonably return the enumeration set. Subsequent use case assertions are automatically judged. According to such use cases, the use cases whose return code is equal to this code are successful use cases, and other use cases whose return codes are not equal are failure use cases.

In a possible design, if the test scenario use case is a pass business use case, if the returned message result in the corresponding use case operation result does not belong to the preset result enumeration set, the test scenario use case is a failure use case; if the corresponding If the returned message result in the running result of the use case belongs to the preset result enumeration set, the test scenario use case is a successful use case; the preset result enumeration set is the business success set.

In a possible design, if the test scenario use case is a failed business use case, if the returned message result in the corresponding use case operation result does not belong to the preset result enumeration set, the test scenario use case is a successful use case; if the corresponding If the returned message result in the test case running result belongs to the preset result enumeration set, the test scenario use case is a failed use case.

In one possible design, if the test scenario use case is a use case that does not conform to the protocol specification, if the returned message result in the corresponding use case running result does not belong to the preset result enumeration set, the test scenario use case is a failure use case; If the returned message result in the running result of the corresponding use case belongs to the preset result enumeration set, the test scenario use case is a successful use case; the preset result enumeration set is an exception set of messages that do not conform to the protocol specification.

In one possible design, if the test scenario use case is a use case conforming to the protocol specification, if the returned message result in the corresponding use case operation result does not belong to the preset result enumeration set, the test scenario use case is a successful use case; if If the returned message result in the corresponding use case running result belongs to the preset result enumeration set, the test scenario use case is a failed use case.

In a possible specific implementation, for business and mock use cases, if the use case type is a pass business use case, the use case result message code is not in the business success set, and the use case fails; the use case result message code is in the business success set, If the success set enumeration is unique, the result is successful, and if the enumeration is not unique, the result is suggestion success. If the use case type is a failure business use case, the use case result message code is not in the business failure set, and the use case fails; the use case result message code is in the business failure set, if the enumeration is unique, the result is successful, and if the enumeration is not unique, the result is suggestion success .

For field use cases, if the use case type is a field does not conform to the protocol specification use case, the use case result message code is not in the message does not meet the protocol specification exception set, and the result is failure; the use case result message code is in the message does not meet the protocol specification Exception collection, if the enumeration is unique, the result is successful, if the enumeration is not unique, the result is suggestion success. If the use case type is a use case whose field conforms to the protocol specification, the use case result message code is in the message does not conform to the protocol specification exception set, and the result is a suggestion failure; the use case result message code is not in the message does not meet the protocol specification exception set, and the result is a suggestion success.

In addition, it is worth noting that the above basic assertion generation method can also be applied to single test, smoke or SIT test in addition to regression test. Among them, in the regression test, it mainly plays an accelerating role. After all, the subsequent Diff judgment takes a long time. Basic assertions can be used to quickly determine whether it is necessary to do accurate Diff, thereby reducing the consumption of invalid Diff resources.

Step 207, execute and run the program of the version to be tested in the system link, and obtain the test results of each interface.

Step 208, generating assertions according to the test baselines and test results corresponding to each interface.

In steps 207-208, the test baseline corresponding to each interface of the system link can be determined first according to the use case running results and the use case scenario rules. For the use case scenario rules, the interface can return messages and various Database operation results are processed, for example, automatic denoising processing can be performed first to remove fields that do not involve logic, and then, through horizontal analysis and vertical analysis, determine the operation result attributes corresponding to each interface, and determine the assertion expectations for each interface value, so that the expected value of the assertion is determined as the test baseline corresponding to each interface of the system link, and entered into the baseline. Then, by executing and running the version program to be tested in the system link, and obtaining the test results of each interface, an assertion is generated according to the test baseline and test results corresponding to each interface.

Specifically, after the assertion is generated according to the test baseline and test result corresponding to each interface, it can be determined that the interface part whose test baseline indicated by the assertion is different from the test result is a system change point or a system vulnerability. When performing regression, after executing the use case, the new assertion of the whole link is obtained, and the assertion expectation is automatically compared with the baseline. The difference is the system change point or system bug.

In a possible scenario, the field value (value) of the use case running result is inconsistent: the database operation result of the use case baseline is the field "a=1" of the A table. After the version to be tested is running, the newly backed up use case running result is "a=2" in the A table field, and the system judges the use case running result as failure. Then you can listen to the way of reporting and guide users to locate the reason. Users can judge that it is a system bug, report the defect, and re-run the comparison after the defect is repaired. After the comparison is consistent, it is considered to have passed successfully. It is also possible that the user judges that the requirement change causes the field "a=2" in table A to be a normal result, then the user directly sets the result as success, and the system updates the running result of the use case into the baseline, so that the field a in table A in the baseline Change to "a=2".

In another scenario, the field (key) of the use case running result is inconsistent: the B interface on the use case link adds a message return field "b" in the version to be tested. After the use case is run, "b=1" is added to the B interface return message in the new backup result. In the baseline, the A interface does not have the "b" field, that is, the comparison is inconsistent, and the user is prompted to locate. If the user judges that this is the new content of this version, the result will be directly updated to the baseline, that is, the baseline will be used as the assertion result in the future.

Optionally, if the use case running results include structured large fields, then split the structured large fields for comparison with preset denoising rules, so as to remove the first category in each field after splitting noise result. Specifically, for structured large fields, you can identify that the field may be a large field based on "{" or the escape character "\", and then convert the String into a dictionary. If the conversion is successful, you can also obtain it in the same way as in this step. Denoising assertions.

In this embodiment, each test scenario use case in the test case set is executed in the system link, and the operation result of the use case is obtained according to the mechanism of the preset safe operation program, and then the system is determined according to the operation result of the use case and the use case scenario rules. The test baseline corresponding to each interface of the link, and then execute the version program to be tested in the system link, and obtain the test results of each interface, and finally, generate an assertion according to the test baseline and test results corresponding to each interface, so as to realize the system link The full-link assertion is automatically and accurately generated without manual maintenance, thus meeting the requirement of automatically covering the corresponding interface test of the whole link. Moreover, in this embodiment, the baseline of the full-link interface assertion can be automatically generated, and then the comparison method is to compare the full-link database operation results and interface return message results. In addition, the automatic analysis and removal of noise is carried out on the running results of the use cases, and preliminary judgments are made through the self-learning method of basic assertions, thereby reducing invalid Diff comparisons. In addition, if the use case running results include structured large fields, the structured large fields can also be split and compared to support large field denoising Diff comparison.

Fig. 5 is a schematic structural diagram of a block chain state data processing device provided by an embodiment of the present application. The block chain state data processing device 500 can be realized by software, hardware or a combination of both.

Fig. 5 is a schematic structural diagram of an interface automation testing device according to an example embodiment of the present application. As shown in Figure 5, the interface automation testing device provided in this embodiment includes:

The obtaining module 301 is used to obtain a test case set, the test case set includes multiple types of test case subsets, each type of test case subset includes a plurality of test scenario use cases, and each test scenario use case is configured with a corresponding Use case business serial number;

The processing module 302 is configured to execute each test scenario use case in the test case set in the system link, and obtain the use case operation result according to the preset safe operation program mechanism, and the use case operation result includes the database operation result and each The interface returns the message result;

The processing module 302 is further configured to determine a test baseline corresponding to each interface of the system link according to the use case running result and the use case scenario rules;

The obtaining module 301 is also used to execute and run the version program to be tested in the system link, and obtain the test results of each interface;

The processing module 302 is further configured to generate an assertion according to the test baseline and the test result corresponding to each interface.

In a possible design, the processing module 302 is specifically configured to:

In a possible design, the processing module 302 is further configured to:

In a possible design, the processing module 302 is further configured to determine that the interface portion of the difference between the test baseline indicated by the assertion and the test result is a system change point or a system vulnerability.

This embodiment provides an interface automation testing device, which can be used to execute the steps in the foregoing method embodiments. For details not disclosed in the device embodiments of the present application, please refer to the above-mentioned method embodiments of the present application.

Fig. 6 is a schematic structural diagram of an electronic device according to an example embodiment of the present application. As shown in FIG. 6, an electronic device 400 provided in this embodiment includes:

processor 401; and,

The memory 402 is used to store executable instructions of the processor, and the memory may also be flash (flash memory);

Wherein, the processor 401 is configured to execute each step in the above method by executing the executable instructions.

Optionally, the memory 402 can be independent or integrated with the processor 401 .

When the memory 402 is a device independent of the processor 401, the electronic device 400 may further include:

The bus 403 is used to connect the processor 401 and the memory 402 .

This embodiment also provides a readable storage medium, in which a computer program is stored, and when at least one processor of the electronic device executes the computer program, the electronic device executes each step in the above method.

This embodiment also provides a program product, where the program product includes a computer program, and the computer program is stored in a readable storage medium. At least one processor of the electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program so that the electronic device implements each step in the above method.

This embodiment also provides a computer program, including program code. When the computer runs the computer program, the program code executes each step in the above method.

Those of ordinary skill in the art can understand that all or part of the steps for implementing the above method embodiments can be completed by program instructions and related hardware. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps including the above-mentioned method embodiments; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other various media that can store program codes.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, rather than limiting them; although the application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: It is still possible to modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the various embodiments of the present application. scope.

Claims

An interface automated testing method, characterized in that it comprises:

Obtain a test case set, the test case set includes multiple types of test case subsets, each type of test case subset includes a plurality of test scenario use cases, and each test scenario use case is configured with a corresponding use case business serial number;

Execute each test scenario use case in the test case set in the system link, and obtain the use case operation result according to the mechanism of the preset safe operation program, the use case operation result includes the database operation result and the message result returned by each interface;

Determine the test baseline corresponding to each interface of the system link according to the running results of each test scenario use case in the production version code environment and the use case scenario rules;

Execute and run the program of the version to be tested in the system link, and obtain the test results of each interface;

An assertion is generated according to the test baseline and the test result corresponding to each interface.
The interface automated testing method according to claim 1, wherein said determination of the test baselines corresponding to each interface of said system link according to the operation results of each test scenario use case in the production version code environment and the use case scenario rules includes:

Directly remove the first type of noise result in the use case operation result according to the preset denoising rule, wherein the first type of noise result is an operation result that has nothing to do with the test logic of the system link;

Carrying out horizontal analysis on the operation results of the use cases to determine the corresponding operation result attributes of each interface, the horizontal analysis includes cluster analysis on the operation results generated by running different test scenario use cases in the same test case subset;

The test baseline corresponding to each interface of the system link is determined according to the use case operation result and the corresponding operation result attribute.
The interface automated testing method according to claim 2, wherein the horizontal analysis of the operation results of the use cases to determine the attributes of the operation results corresponding to each interface includes:

After running the preset first number of test scenario cases in the same test case subset, obtain the test case running results of each interface;

Determine the corresponding result equivalent rate according to the result distribution of the use case operation results of each interface;

Determine the corresponding operation result attribute according to the result equivalent rate of each interface and the preset equivalent rate condition.
The interface automated testing method according to claim 3, wherein the determination of the corresponding operation result attributes according to the result equivalent rate of each interface and the preset equivalent rate condition includes:

If the result equivalent rate is 100%, it is determined that the corresponding operation result attribute is a fixed value class, and the test baseline corresponding to the fixed value class is a fixed value;

If the result equivalence rate is greater than or equal to the preset equivalence rate, it is determined that the corresponding operation result attribute is an enumeration class, and the test baseline corresponding to the enumeration class is an enumeration list;

If the result equivalent rate is less than the preset equivalent rate, it is determined that the corresponding use case operation result is noise.
The interface automated testing method according to claim 4, characterized in that, after the horizontal analysis of the operation results of the use cases to determine the attributes of the operation results corresponding to each interface, it further includes:

Carrying out longitudinal analysis on the running results of the use cases belonging to the enumerated class, the longitudinal analysis including cluster analysis on the running results generated by running the same test scenario use case;

If the running results of all the use cases belonging to the enumeration class are the same, backing up the running results of the use cases to the enumeration list;

If the running results of the use cases belonging to the enumeration class are not the same, it is judged whether the running results of each use case are obtained when the preset backup conditions are met, and the preset backup conditions are used to determine that the running of the use cases is in an end state;

If the judgment result is yes, back up the running results of each use case to the enumeration list;

If the judgment result is no, then back up the use case running results obtained by satisfying the preset backup conditions to the enumeration list.
According to the interface automated testing method described in any one of claims 2-5, it is characterized in that, before executing and running the version program to be tested in the system link, and obtaining the test results of each interface, it also includes:

Generate basic assertions according to the affiliation relationship between the returned message results and the preset result enumeration set in the use case running results corresponding to each test scenario use case;

It is determined that the test results corresponding to the test scenario use cases in which the basic assertion conforms to the preset assertion result are compared with the test baseline corresponding to each interface to generate the assertion.
The interface automation testing method according to claim 6, wherein the basic assertion is generated according to the affiliation relationship between the return message result and the preset result enumeration set in the use case operation results corresponding to each test scenario use case, including:

If the test scenario use case is a pass business use case, then if the returned message result in the corresponding use case operation result does not belong to the preset result enumeration set, then the test scenario use case is a failure use case; If the returned message result belongs to the preset result enumeration set, then the test scenario use case is a successful use case; the preset result enumeration set is a business success set;

If the test scenario use case is a failure business use case, then if the returned message result in the corresponding use case operation result does not belong to the preset result enumeration set, then the test scenario use case is a successful use case; If the returned message result belongs to the preset result enumeration set, the test scenario use case is a failure use case;

If the test scenario use case is a use case that does not conform to the protocol specification, if the returned message result in the corresponding use case running result does not belong to the preset result enumeration set, the test scenario use case is a failure use case; if the corresponding use case running result If the returned message result in the test belongs to the preset result enumeration set, then the test scenario use case is a successful use case; the preset result enumeration set is the message does not meet the protocol specification exception set;

If the test scenario use case is a use case conforming to the protocol specification, if the returned message result in the corresponding use case operation result does not belong to the preset result enumeration set, the test scenario use case is a successful use case; if the corresponding use case operation result in If the returned message result belongs to the preset result enumeration set, the test scenario use case is a failed use case.
The interface automated testing method according to claim 7, wherein, after the assertion is generated according to the test baseline corresponding to each interface and the test result, further comprising:

It is determined that the interface part where the difference between the test baseline indicated by the assertion and the test result is a system change point or a system vulnerability.
The interface automation testing method according to any one of claims 2-5, characterized in that, before directly removing the first type of noise results in the use case running results according to the preset denoising rules, it also includes:

If the result of the use case operation includes a large structured field, split the large structured field for comparison with the preset denoising rule, so as to remove the first category in each field after splitting noise result.
An interface automated testing device is characterized in that it comprises:

The obtaining module is used to obtain a test case set, the test case set includes multiple types of test case subsets, each type of test case subset includes a plurality of test scenario use cases, and each test scenario use case is configured with a corresponding use case business serial number;

The processing module is used to execute each test scenario use case in the test case set in the system link, and obtain the use case operation result according to the mechanism of the preset safe operation program, and the use case operation result includes the database operation result and each interface Return message result;

The processing module is also used to determine the test baseline corresponding to each interface of the system link according to the running results of each test scenario use case in the production version code environment and the use case scenario rules;

The obtaining module is also used to execute and run the program of the version to be tested in the system link, and obtain the test results of each interface;

The processing module is further configured to generate an assertion according to the test baseline and the test result corresponding to each interface.
An electronic device, characterized in that it comprises:

processor; and

a memory for storing the computer program of the processor;

Wherein, the processor is configured to implement the interface automation testing method according to any one of claims 1 to 9 by executing the computer program.
A computer-readable storage medium, on which a computer program is stored, characterized in that, when the computer program is executed by a processor, the interface automation testing method described in any one of claims 1 to 9 is implemented.
A computer program product, comprising a computer program, characterized in that, when the computer program is executed by a processor, the interface automation testing method described in any one of claims 1 to 9 is implemented.
A computer program, characterized in that it includes program code, and when the computer runs the computer program, the program code executes the interface automation testing method according to any one of claims 1 to 9.