JP6695437B2 - Management computer and test environment determination method - Google Patents

Description

  The present invention relates to a test environment management technique for performing a test defined by one or more test cases.

  Unlike waterfall type development, system development such as DevOps requires rapid and continuous application release. Therefore, a huge amount of regression tests are executed using a plurality of test environments.

  With the spread of virtualization technology and cloud computing, not only on-premise systems but also test environments composed of resources such as virtual machines (VM) on cloud services are being used.

  The technique described in Patent Document 1 is known to meet the demand for efficient system development. In Patent Document 1, "In a retest method for determining the test coverage of a program, a retest range at the time of module modification is extracted based on test data and related information of a test target module, and the extracted test The data is re-executed, and the executed test result for this test data and the test result of this time are compared and verified. "

JP-A-6-35754

  In system development, tests are performed using multiple test environments, so there is a user demand to reduce the execution cost of system development. Here, the system development execution cost includes an operation cost such as a usage fee, a management cost such as a user operation, and a processing cost such as a load on a computer when a test case is executed. In order to solve the above-mentioned user request, it is necessary to determine the test environment in consideration of the contents of the test defined in the test case, the state of the test environment, the execution cost of system development, and the like.

  For example, when focusing on operating costs, there are the following problems. When executing a test case using a test environment that uses commercial software, it is necessary to pay the same charges as in the production environment, which increases operating costs. As a method of solving the above-mentioned problem, a method of using a test environment using OSS (Open Source Software) can be considered. However, depending on the content of the test defined in the test case, the OSS may not be available. In addition, there is a possibility that the test environment cannot be used because the resources of the test environment using OSS are insufficient at the time of executing the test case.

  The number of combinations of test contents and test environment states is enormous, and a user's burden is heavy for determining a test environment that matches a user request. Therefore, there is a problem that the management cost also increases. In addition, when determining a test environment in which a test case is executed by the round robin method, test cases with a high load are concentrated in one test environment, resulting in an increase in processing cost.

  The present invention realizes a system and method for automatically determining a test environment that matches a user request in consideration of the content of the test and the state of the test environment.

  A typical example of the invention disclosed in the present application is as follows. That is, a management computer connected to a computer system in which a plurality of test environments that execute a test case group including one or more test cases is connected, wherein the computer system includes a plurality of computers, and the management computer is A processor, a memory connected to the processor, and a network interface connected to the processor, the candidates being determined based on the identification information of the test case group and the characteristics of the test cases included in the test case group. The processor holds group information including an entry composed of identification information of a test environment and condition information including a dynamic condition which is a condition relating to the state of the test environment, and the processor executes the first test case group. When the trigger is detected, the first group of information is referred to by referring to the group information. A candidate test environment of a Stokes group, obtains state information indicating a state of the candidate test environment, and based on the condition information and the state information, the first test case group from the candidate test environment. It is characterized by determining the execution test environment for executing.

  According to the present invention, it is possible to automatically determine the test environment that matches the user request. As a result, the execution cost of system development can be reduced. Problems, configurations and effects other than those described above will be clarified by the following description of the embodiments.

2 is a block diagram showing an example of a computer system of Example 1. FIG. FIG. 7 is a diagram showing an example of test case information according to the first embodiment. FIG. 7 is a diagram showing an example of condition information according to the first embodiment. FIG. 6 is a diagram showing an example of test environment information according to the first embodiment. FIG. 6 is a diagram showing an example of license information according to the first embodiment. 6 is a diagram showing an example of group information of Example 1. FIG. FIG. 7 is a diagram showing an example of test execution queue information of the first embodiment. FIG. 6 is a sequence diagram illustrating a flow of processing executed by the management server of the first embodiment. FIG. 8 is a sequence diagram illustrating a flow of processing executed by the management server according to the first embodiment. 5 is a flowchart illustrating an example of static classification processing executed by the static classification module according to the first embodiment. 6 is a flowchart illustrating an example of a group generation process executed by the group generation module according to the first embodiment. 6 is a flowchart illustrating an example of a group evaluation process executed by the group evaluation module according to the first embodiment. 6 is a flowchart illustrating an example of dynamic classification processing executed by the dynamic classification module according to the first embodiment. 6 is a flowchart illustrating an example of dynamic classification processing executed by the dynamic classification module according to the first embodiment. 6 is a flowchart illustrating an example of processing executed by an information acquisition module according to the first embodiment. 6 is a flowchart illustrating an example of processing executed by the cost calculation module according to the first embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. The identification information used in the table used in the embodiment may be omitted if the entry of the table can be specified by using one or more columns in the table. Further, the identification information may be automatically assigned an identification number in ascending order. In each drawing, the same reference numerals are attached to the common configurations.

  In the first embodiment, the configuration and processing when the user request is to reduce the operating cost will be described. When using the test environment on the cloud type system, the operation cost can be reduced by using the test environment with a low usage fee or by shortening the usage time of the test environment. Here, a method and system for reducing the operating cost by using a test environment with a low usage fee will be described. The same functions and effects can be realized with the same configuration for other user requests.

  FIG. 1 is a block diagram illustrating an example of a computer system according to the first embodiment.

  The computer system according to the first embodiment includes a management server 100, an on-premises type system 101, and a cloud type system 102, which are connected to each other via a management network 103. The management network 103 may be divided into a network connected to the on-premises system 101 and a network connected to the cloud system 102.

  In this embodiment, the test is performed by the test environment executing one or more test cases. Here, the test case is information defining the content of the test including pre-execution conditions, input values, expected results, post-execution conditions, and the like. The test environment is an environment for performing a test.

  The on-premises type system 101 and the cloud type system 102 are systems that realize a test environment. In this embodiment, two types of test environments, a staging environment and an alternative environment, are built on each system. The staging environment is an environment that uses commercial software and is the same environment as the production environment. The alternative environment is an environment that uses OSS. The production environment is the environment in which the system actually operates.

  The on-premises system 101 is a system operated by the company to which the system developer belongs. The cloud type system 102 is a system operated by a cloud service provider. The on-premises system 101 and the cloud system 102 each include a plurality of computers, a plurality of storage devices, a plurality of network devices, and the like. The test environment may be a test environment that uses a physical device itself as a resource or a test environment that uses a virtual device as a resource. In the first embodiment, a process for a test environment built on the cloud type system 102 will be described. The processing for the test environment constructed on the two systems will be described in the second embodiment.

  The management server 100 determines the test environment in which the test case is executed. Here, the hardware configuration and software configuration of the management server 100 will be described.

  The management server 100 includes a CPU (Central Processing Unit) 111, a memory 112, a network interface 113, and an IO interface 114. The above hardware is connected to each other via an internal bus.

  The CPU 111 executes the program stored in the memory 112. The CPU 111 operates as a module that realizes a predetermined function by operating according to a program stored in the memory 112. For example, the CPU 111 functions as the information acquisition module 126 by operating according to the information acquisition program. The same applies to other programs.

  The memory 112 stores programs executed by the CPU 111 and information. The memory 112 also includes a work area that is temporarily used by the program. The programs and information included in the memory 112 will be described later.

  The network interface 113 is an interface for communicating with an external device via an IP network or the like. The IO interface 114 is an interface connected to the input device 104 and the display device 105. The IO interface 114 may be connected to an external storage medium such as a USB memory and an external storage device.

  Here, the input device 104 is a device for inputting data to the management server 100. A keyboard, a mouse, a touch panel, etc. can be used as the input device 104. The display device 105 is a device for displaying the processing result of the management server 100 and the like. A display, a touch panel, or the like can be used as the display device 105.

  Although the number of each hardware is one in FIG. 1, it may be two or more. For example, when the management server 100 connects to a plurality of networks, the management server 100 may have a plurality of network interfaces 113 connected to each network.

  Here, the programs and information stored in the memory 112 will be described. The memory 112 stores a program that implements the control module 120 and a management information group 130.

  The control module 120 is a module that controls the entire management server 100 and includes a plurality of modules. Specifically, the control module 120 includes a test execution module 121, a static classification module 122, a dynamic classification module 123, a group generation module 124, a group evaluation module 125, an information acquisition module 126, and a cost calculation module 127.

  The test execution module 121 causes the determined test environment to execute a test case in cooperation with each module. The static classification module 122 selects a test environment candidate for executing each test case based on the content of the test defined by the test case and the condition information 132. The dynamic classification module 123 determines the test environment in which the test case is executed based on the test environment state and the condition information 132 such as the operating state of the test environment. In the following description, the contents of the test defined by the test case are described as test case characteristics.

  The group generation module 124 generates a test case group including one or more test cases. In this embodiment, the test is carried out in test case group units. The group evaluation module 125 selects a test environment candidate for executing the test case group based on the processing result of the static classification module 122. In the following description, candidate test environments that execute test case groups are referred to as candidate test environments.

  The information acquisition module 126 acquires status information indicating the status of the test environment. The cost calculation module 127 calculates a system development execution cost. In this embodiment, the cost calculation module 127 calculates the usage charge of the test environment.

  The management information group 130 includes test case information 131, condition information 132, test environment information 133, license information 134, group information 135, and test execution queue information 136. Details of each information will be described with reference to FIGS. 2 to 7.

  The programs and information stored in the memory 112 are computer-readable non-transitory storage media such as HDD (Hard Disk Drive), SSD (Solid State Drive), non-volatile memory, IC card, SD card, and DVD. Alternatively, it may be stored in an external storage device. In this case, the CPU 111 reads the program and information from the external storage device or the like and loads the program and information into the memory 112.

  FIG. 2 is a diagram illustrating an example of the test case information 131 according to the first embodiment. Note that, in FIG. 2, the test case information 131 is divided into two tables in consideration of the space of the drawing, but it is actually managed as one table.

  The test case information 131 is information for managing a test case, and includes a test case ID 201, automation 202, performance 203, unique function 204, importance 205, prerequisite 206, alternative environment 207, execution environment 208, error 209, and estimated time. 210, system ID 211, use case / function 212, and grouping 213. The entry may include columns other than those described above.

  The test case ID 201 stores identification information for identifying the test case. The test case ID 201 is also used as identification information for identifying the entry of the test case information 131.

  The automation 202 stores information indicating whether or not the test case can be automated. The automation 202 stores either “possible” or “impossible”.

  The performance 203 stores information indicating whether the test defined in the test case is a performance test. The performance 203 stores either “YES” or “NO”. The performance test is conducted for the purpose of measuring the performance of the system in the production environment. Therefore, the performance test needs to be performed in a staging environment equivalent to the production environment. As a test that needs to use the staging environment, a test or the like can be considered for the purpose of confirming security.

  The unique function 204 stores information indicating whether the test defined in the test case is a test for confirming the unique function of the commercial software. The unique function 204 stores either “YES” or “NO”.

  The importance 205 stores the importance indicating the execution order of test cases (test execution order) and the like. The degree of importance represents the high occurrence rate of bugs or the degree of influence of the entire system. The importance 205 stores an integer of “1” or more. In this embodiment, the closer the value is to 1, the higher the importance. A value set by the user may be set as the importance 205, or a value calculated based on the dependency between test cases may be set as the importance 205. For example, a method of setting a value based on the number registered in the precondition 206 can be considered.

  The precondition 206 stores the identification information of the test case that needs to be executed before executing the test case corresponding to the entry. If there is no test case that needs to be executed in advance, the precondition 206 is blank. For example, if there is a first test case that defines a test that adds a record in the database and a second test case that defines a test that deletes a record added in the first test case, the second test case The case is based on the first test case. Therefore, the identification information of the first test case is stored in the precondition 206 of the entry corresponding to the second test case.

  The alternative environment 207 stores information indicating whether the test case can be executed in the alternative environment 207. The alternative environment 207 stores either “possible” or “impossible”.

  The execution environment 208 stores the identification information of the test environment in which the test case was executed last time. The execution environment 208 stores the identification information set in the test environment ID 401 (see FIG. 4) described later.

  The error 209 stores information indicating whether the test result is the expected result in the test case. When the test result is not the expected result in the test case, “error” is stored in the error 209, and when the test result is the expected result in the test case, the error 209 is blank.

  The predicted time 210 stores a predicted value of the execution time of the test defined by the test case. The execution time of the test executed last time may be stored in the estimated time 210, or the statistical value of the execution time of the test executed so far may be stored in the estimated time 210. Here, the statistical value indicates an average value, a maximum value, a median value, or the like.

  The system ID 211 stores identification information of a system or software that is a test target defined in the test case.

  The use case / function 212 stores identification information such as a specific function that is a test target defined in the test case.

  The grouping 213 stores information indicating whether or not the entry has been processed by the group generation module 124. The grouping 213 stores either “executed” or “not executed”.

  In this embodiment, automation 202, performance 203, unique function 204, importance 205, and precondition 206 are used as test case characteristics.

  FIG. 3 is a diagram illustrating an example of the condition information 132 according to the first embodiment.

  The condition information 132 is information for managing conditions for determining the test environment in which the test case group is executed, and includes an entry including a condition ID 301, a condition type 302, an item 303, and a condition 304. The entry may include columns other than those described above.

  The condition ID 301 stores identification information for uniquely identifying the condition.

  The condition type 302 stores information indicating the type of condition. Either “static” or “dynamic” is stored in the condition type 302. “Static” is a condition used by the static classification module 122, and “dynamic” is a condition used by the dynamic classification module 123. In the following description, a condition corresponding to an entry in which “static” is set in the condition type 302 is described as a static condition, and a condition corresponding to an entry in which “dynamic” is set in the condition type 302 is described. Described as a static condition. Static conditions are conditions such as parameters that do not depend on time. Conditions relating to test case characteristics are mainly set as static conditions. The dynamic conditions are conditions such as time-dependent parameters. Conditions relating to the state of the test environment are mainly set as dynamic conditions.

  The item 303 stores information indicating the name and type of condition. The item 303 is displayed on a GUI or the like for the user to input a user request.

  The condition 304 stores specific contents of the condition. In this embodiment, the condition is described as a character string for explanation, but data other than the character string such as a conditional expression may be stored. The user may determine the numerical value set in the condition 304.

  FIG. 4 is a diagram illustrating an example of the test environment information 133 according to the first embodiment.

  The test environment information 133 is information for managing the configuration of the test environment, the state of the test environment, etc., and includes the test environment ID 401, the test environment type 402, the configuration 403, the replacement target software 404, the CPU usage rate 405, and the memory usage rate 406. Contains an entry consisting of The entry may include columns other than those described above.

  The test environment ID 401 stores identification information that uniquely identifies the test environment.

  The test environment type 402 stores information indicating the type of test environment. In the test environment type 402 of this embodiment, either “staging” or “substitution” is stored.

  The configuration 403 stores the configuration information of the test environment. In this embodiment, the type of OS (Operating System) operating in the test environment is stored in the configuration 403. The configuration 403 may store the hardware configuration and software configuration of the test environment.

  The replacement target software 404 stores the identification information of the commercial software or OSS that replaces the OSS. When the test environment type 402 of the entry is “staging”, the replacement target software 404 stores the identification information of the commercial software to replace the OSS. When the test environment type 402 of the entry is “substitute”, the substitution target software 404 stores the OSS identification information.

  The CPU usage rate 405 stores the usage rate of the CPU included in the server or the like that constitutes the test environment. For example, when the server is a physical server, the usage rate of the physical CPU is stored in the CPU usage rate 405, and when the server is a VM, the usage rate of the virtual CPU is stored in the CPU usage rate 405.

  The memory usage rate 406 stores the usage rate of the memory included in the server or the like that constitutes the test environment. For example, when the server is a physical server, the usage rate of physical memory is stored in the memory usage rate 406, and when the server is VM, the usage rate of virtual memory is stored in the memory usage rate 406.

  FIG. 5 is a diagram illustrating an example of the license information 134 according to the first embodiment.

  The license information 134 is information for managing the license fee of software used in the test environment, and includes an entry including a license ID 501, software 502, a usage state 503, a test environment ID 504, and a fee 505. The entry may include columns other than those described above.

  The license ID 501 stores identification information for uniquely identifying the license.

  The software 502 stores identification information of software included in the licensed test environment. When software is provided as an optional service of the cloud system 102, service identification information is stored in the software 502.

  The usage state 503 stores information indicating whether or not the license is used. The use state 503 stores either “in use” or “unused”.

  The test environment ID 504 stores identification information of the test environment for which the license is a target. The test environment ID 504 stores the identification information set in the test environment ID 401.

  The fee 505 stores the usage fee of the licensed test environment.

  FIG. 6 is a diagram illustrating an example of the group information 135 according to the first embodiment.

  The group information 135 is information for managing the test case group generated by the group generation module 124, and includes an entry including a group ID 601, a group importance 602, an alternative environment 603, and a test case 604. The entry may include columns other than those described above.

  The group ID 601 stores identification information for uniquely identifying the test case group.

  The group importance 602 stores the importance of the test case group. The group importance 602 is used to determine the execution order between test case groups.

  The alternative environment 603 stores information indicating whether the alternative environment can be used as a test case group. The alternative environment 603 stores either “possible” or “impossible”. The value set in the alternative environment 603 is determined based on the alternative environment 207 of the test case included in the test case group. In this embodiment, the type of the test environment is used as the information for identifying the candidate test environment, so the entry includes the alternative environment 603.

  The test case 604 stores the identification information of the test cases that form the test case group. The test case 604 has as many columns as the test cases included in the test case group. The identification information set in the test case ID 201 is stored in each column of the test case 604. In this embodiment, the test cases are executed sequentially from the left of the test case 604.

  FIG. 7 is a diagram illustrating an example of the test execution queue information 136 according to the first embodiment.

  The test execution queue information 136 is information for managing a test execution queue for registering test cases, and includes an entry including a queue ID 701, a test case ID 702, a group ID 703, and an execution environment ID 704. The entry may include columns other than those described above.

  The queue ID 701 stores identification information for uniquely identifying the queue.

  The test case ID 702 stores the identification information of the test case put in the queue. The test case ID 702 stores the identification information set in the test case ID 201.

  The group ID 703 stores the identification information of the test case group to which the test case submitted to the queue belongs. The identification information set in the group ID 601 is stored in the group ID 703.

  The execution environment ID 704 stores the identification information of the test environment in which the test corresponding to the test case is executed. The execution environment ID 704 stores the identification information set in the test environment ID 401.

  8A and 8B are sequence diagrams illustrating the flow of processing executed by the management server 100 according to the first embodiment.

  When the test execution module 121 detects the start trigger of the test (test case group) (step S801), the test execution module 121 transmits a start command to the static classification module 122 (step S802). Further, the test execution module 121 displays a screen for the user to input the budget and accepts the budget input.

  The test execution module 121 may detect a test start instruction input by the user as a test start timing. Further, the test execution module 121 may determine that the test has started when the time set in the execution schedule has passed or when the system to be tested commits the source to the version management software. ..

  If the management server 100 holds information for managing the budget, the screen for entering the budget may not be displayed.

  Upon receiving the start command from the test execution module 121, the static classification module 122 executes static classification processing (step S803). In the static classification process, the static classification module 122 selects an executable test environment for each test case based on the test case characteristics. Details of the static classification process will be described with reference to FIG.

  The static classification module 122 transmits a completion notification to the test execution module 121 after the static classification processing is completed (step S804).

  When the test execution module 121 receives the completion notification from the static classification module 122, the test execution module 121 sends a start command to the group generation module 124 (step S805).

  When the group generation module 124 receives the start command from the test execution module 121, the group generation module 124 executes a group generation process (step S806). In the group generation process, the group generation module 124 generates a test case group. Details of the group generation process will be described with reference to FIG.

  The group generation module 124 transmits a completion notice to the test execution module 121 after the group generation process is completed (step S807).

  When the test execution module 121 receives the completion notification from the group generation module 124, the test execution module 121 transmits a start command to the group evaluation module 125 (step S808).

  When receiving the start command from the test execution module 121, the group evaluation module 125 executes the group evaluation process (step S809). In the group evaluation process, the group evaluation module 125 selects a candidate test environment. Details of the group evaluation process will be described with reference to FIG.

  The group evaluation module 125 sends a completion notification to the test execution module 121 after the group evaluation processing is completed (step S810).

  When the test execution module 121 receives the completion notification from the group evaluation module 125, the test execution module 121 transmits a start command to the dynamic classification module 123 (step S811).

  When receiving the start command from the test execution module 121, the dynamic classification module 123 executes the dynamic classification process (step S812). In the dynamic classification process, the dynamic classification module 123 determines the test environment in which the test case group is executed based on the state of the candidate test environment. The dynamic classification module 123 also registers the test cases included in the test case group in the test execution queue. Details of the dynamic classification process will be described with reference to FIGS. 12A and 12B.

  The dynamic classification module 123 transmits a completion notification to the test execution module 121 after the dynamic classification process is completed (step S813).

  The test execution module 121 refers to the test execution queue information 136 and executes the test case put in the test execution queue (step S814).

  With the above processing, the test case group can be executed in an appropriate test environment.

  FIG. 9 is a flowchart illustrating an example of static classification processing executed by the static classification module 122 according to the first embodiment.

  The static classification module 122 starts the loop processing of the test case (step S901).

  Specifically, the static classification module 122 refers to the test case information 131 and selects one target test case. Further, the static classification module 122 acquires from the condition information 132 an entry in which “static” is set in the condition type 302.

  In the loop processing, the target test case may be selected from all the test cases registered in the test case information 131, or the target test case may be selected from the test cases related to the changed software. Good. The processing load can be reduced by narrowing down only the test cases related to the changed software.

  In the processing from step S903 to step S909, the static classification module 122 selects a test environment capable of executing the target test case based on the static condition.

  First, the static classification module 122 refers to the test case information 131 and determines whether the automation 202 of the entry corresponding to the target test case is “possible” (step S902).

  If the automation 202 of the entry corresponding to the target test case is determined to be “impossible”, the static classification module 122 proceeds to step S910. In this case, "impossible" is set as the alternative environment 207 of the entry.

  When it is determined that the automation 202 of the entry corresponding to the target test case is “enabled”, the static classification module 122 determines whether the target test case can be executed in the alternative environment (step S903). Specifically, the following processing is executed.

  The static classification module 122 determines whether the performance 203 of the entry corresponding to the target test case is “YES”.

  If the performance 203 of the entry is determined to be “YES”, the static classification module 122 determines that the target test case cannot be executed in the alternative environment. When the performance 203 of the entry is determined to be “NO”, the static classification module 122 determines whether the unique function 204 of the entry is “YES”.

  If the entry's unique function 204 is determined to be "YES," the static classification module 122 determines that the target test case cannot be executed in the alternative environment. If the entry's unique function 204 is determined to be "NO," the static classification module 122 determines that the target test case can be executed in the alternative environment. The above is the description of the process of step S903.

  When it is determined that the target test case cannot be executed in the alternative environment, the static classification module 122 sets “impossible” in the alternative environment 207 of the entry corresponding to the target test case (step S909). After that, the static classification module 122 proceeds to step S910.

  When it is determined that the target test case can be executed in the alternative environment, the static classification module 122 executes the test case in each of the staging environment and the alternative environment (step S904). The static classification module 122 shifts to a waiting state until the test result is output from the staging environment and the alternative environment.

  When the test result is output from the staging environment and the alternative environment, the static classification module 122 determines whether the test case needs to be reviewed based on the test result (step S905).

  Specifically, the static classification module 122 determines whether the expected result in the test case has been output.

  If it is determined that the test case needs to be reviewed, the static classification module 122 proceeds to step S910. In this case, no particular value may be set in the alternative environment 207 of the entry.

  When it is determined that the test case does not need to be reviewed, the static classification module 122 determines whether the test results of the staging environment and the alternative environment match (step S906). Note that the test results do not have to match exactly, but may have similar results.

  If it is determined that the test results of the staging environment and the alternative environment do not match, the static classification module 122 sets “impossible” to the alternative environment 207 of the entry (step S909). After that, the static classification module 122 proceeds to step S910.

  When it is determined that the test results of the staging environment and the alternative environment match, the static classification module 122 sets "possible" to the alternative environment 207 of the entry (step S907). After that, the static classification module 122 proceeds to step S910.

  In step S910, the static classification module 122 determines whether processing has been completed for all selectable test cases (step S910).

  When it is determined that the processing has not been completed for all selectable test cases, the static classification module 122 returns to step S901, selects a new target test case, and executes the same processing.

  When it is determined that the processing has been completed for all selectable test cases, the static classification module 122 ends the test case loop processing, and sends a completion notification to the test execution module 121 (step S911). After that, the static classification module 122 ends the static classification process.

  As described above, the static classification module 122 selects the test environment in which the test case can be executed based on the test case characteristics. In this embodiment, the static classification module 122 controls such that the alternative environment is preferentially selected. This can reduce the operating cost of the test case.

  Note that, in FIG. 9, the static classification module 122 selects either the staging environment or the alternative environment, but the present invention is not limited to this. By changing the condition set in the condition information 132, the static classification module 122 can select the test environment based on various viewpoints.

  FIG. 10 is a flowchart illustrating an example of a group generation process executed by the group generation module 124 according to the first embodiment.

  The group generation module 124 sets "unexecuted" in the grouping 213 of all the entries of the test case information 131 (step S1001).

  Next, the group generation module 124 identifies a test case not registered in the precondition 206 (step S1002).

  Specifically, the group generation module 124 refers to the precondition 206 of each entry, and specifies a test case that is not registered in the precondition 206 of any entry. The test case not registered in the precondition 206 corresponds to the test case executed last in the test case group.

  Next, the group generation module 124 starts loop processing of the specified test case (step S1003).

  Specifically, the group generation module 124 selects one test case from the identified test cases and executes the processing from step S1004 to step S1011.

  The group generation module 124 sets the selected test case as a target test case (step S1004), and also generates a test case group including the target test case (step S1005).

  Specifically, the group generation module 124 adds an entry to the group information 135 and sets identification information in the group ID 601 of the added entry. The group generation module 124 adds a column to the test case 604 and sets the identification information of the target test case. Further, the group generation module 124 refers to the test case information 131 and sets “executed” to the grouping 213 of the entries of the target test case.

  Next, the group generation module 124 determines whether or not there is a test case having a dependency relationship with the target test case (step S1006).

  Specifically, the group generation module 124 searches for an entry in which the identification information of the target test case is stored in the precondition 206. If the entry does not exist, the group generation module 124 determines that there is no test case having a dependency relationship with the target test case.

  When it is determined that there is a test case having a dependency relationship with the target test case, the group generation module 124 adds the retrieved test case to the test case group (step S1007).

  Specifically, the group generation module 124 acquires the identification information of the test case searched in step S1006 from the test case information 131. The group generation module 124 adds the identification information of the acquired test case to the test case 604 of the entry added to the group information 135 in step S1005. The test case identification information is set in order from the right. That is, the identification information of the newly set test case is set to the rightmost.

  Next, the group generation module 124 refers to the test case information 131 and sets "executed" to the grouping 213 of the entries of the target test case (step S1008). If the grouping 213 is already “executed”, this step can be omitted.

  Next, the group generation module 124 sets the test case retrieved in step S1007 as a new target test case (step S1009). After that, the group generation module 124 returns to step S1006 and executes the same processing.

  If it is determined in step S1006 that there is no test case that has a dependency relationship with the target test case, the group generation module 124 refers to the test case information 131 and sets “completed” in the grouping 213 of the target test case entries. Is set (step S1010). If the grouping 213 is already “executed”, this step can be omitted.

  Next, the group generation module 124 calculates the importance of the test case group generated in step S1005 (step S1011). Specifically, the following processing is executed.

  The group generation module 124 acquires the identification information of the test case included in the test case group from the test case 604 of the entry added to the group information 135.

  The group generation module 124 refers to the test case information 131 to search for an entry whose test case ID 201 matches the identification information of the acquired test case, and acquires a value from the importance 205 of the searched entry. The group generation module 124 uses the acquired values to calculate the importance of the test case group. For example, the total value, the average value, the maximum value, the minimum value, etc. of the importance of the test cases are calculated as the importance of the test case group.

  The group generation module 124 sets the calculated importance to the group importance 602 of the entry added to the group information 135. The above is the description of the process of step S1011.

  Next, the group generation module 124 determines whether the processing has been completed for all the test cases specified in step S1002 (step S1012).

  When it is determined that the processing has not been completed for all the test cases specified in step S1002, the group generation module 124 returns to step S1003 and selects a new test case from the specified test cases, Perform similar processing.

  When it is determined that the processing has been completed for all the test cases specified in step S1002, the group generation module 124 ends the test case loop processing and sends a completion notification to the test execution module 121 (step S1013). .. After that, the group generation module 124 ends the group generation process.

  In the case of the test case information 131 shown in FIG. 2, “TC2”, “TC3”, and “TC5” are specified in step S1002. The "TC2" test case group includes "TC2" and "TC1", the "TC3" test case group includes only "TC3", and the "TC5" test case group includes "TC1", "TC4", and Includes "TC5". “TC1” is included in two test case groups.

  The group generation module 124 may set the grouping 213 to “not executed” only for the test case entry related to the changed software.

  As described above, the group generation module 124 can generate a test case group including one or more test cases based on the dependency relationship between the test cases.

  FIG. 11 is a flowchart illustrating an example of the group evaluation process executed by the group evaluation module 125 according to the first embodiment.

  The group evaluation module 125 starts the loop processing of the test case group (step S1101).

  Specifically, the group evaluation module 125 refers to the group information 135 and selects one target test case group.

  Next, the group evaluation module 125 determines whether a test case that can be executed only in the staging environment is included in the target test case group (step S1102). Specifically, the following processing is executed.

  The group evaluation module 125 refers to the group information 135 and acquires the identification information of the test case from the test case 604 of the entry of the target test case group. The group evaluation module 125 refers to the test case information 131, and searches for an entry whose test case ID 201 matches the identification information of the acquired test case.

  The group evaluation module 125 refers to the alternative environment 207 of the searched entry and determines whether or not there is an entry in which the alternative environment 207 is “impossible”. If there is an entry in which the alternative environment 207 is “impossible”, the group evaluation module 125 determines that the target test case group includes test cases that can be executed only in the staging environment. The above is the description of the process of step S1102.

  When it is determined that the target test case group does not include the test case that can be executed only in the staging environment, the group evaluation module 125 refers to the group information 135, and the target test case group entry “Allowed environment 603” is displayed. Is set (step S1103). That is, the alternative environment is selected as the candidate test environment for the target test case group. After that, the group evaluation module 125 proceeds to step S1105.

  When it is determined that the target test case group includes a test case that can be executed only in the staging environment, the group evaluation module 125 refers to the group information 135, and the target test case group entry's alternative environment 603 indicates “impossible”. Is set (step S1104). That is, the staging environment is selected as the candidate test environment for the target test case group. After that, the group evaluation module 125 proceeds to step S1105.

  In step S1105, the group evaluation module 125 determines whether the processing has been completed for all test case groups (step S1105).

  If it is determined that the processing has not been completed for all the test case groups, the group evaluation module 125 returns to step S1101, selects a new target test case group, and executes the same processing.

  When it is determined that the processing has been completed for all the test case groups, the group evaluation module 125 ends the loop processing of the test case groups and sends a completion notification to the test execution module 121 (step S1106). After that, the group evaluation module 125 ends the group evaluation process.

  12A and 12B are flowcharts illustrating an example of the dynamic classification process executed by the dynamic classification module 123 according to the first embodiment.

  The dynamic classification module 123 rearranges the entries of the group information 135 in descending order of group importance (step S1201). In this embodiment, the smaller the value, the higher the importance. Therefore, the entries are rearranged in ascending order of the group importance 602. The process of step S1201 is a process for preferentially executing a test case having high importance.

  Next, the dynamic classification module 123 starts the loop processing of the test case group (step S1202).

  Specifically, the dynamic classification module 123 refers to the group information 135 and selects one target test case. The dynamic classification module 123 also acquires from the condition information 132 an entry in which “dynamic” is set in the condition type 302.

  In steps S1203 to S1212, the dynamic classification module 123 determines a test environment in which the selected test case group is executed based on the dynamic condition.

  Next, the dynamic classification module 123 refers to the group information 135 and determines whether the alternative environment 603 of the entry corresponding to the target test case group is “possible” (step S1203). That is, it is determined whether the alternative environment is a candidate test environment. The process is a process for specifying a candidate test environment.

  When it is determined that the alternative environment 603 of the entry corresponding to the target test case group is “impossible”, the dynamic classification module 123 refers to the test environment information 133 and determines the staging environment for executing the target test case group. It is determined (step S1205). After that, the dynamic classification module 123 proceeds to step S1211. Specifically, the following processing is executed.

  The dynamic classification module 123 refers to the test environment information 133 and searches for an entry whose test environment type 402 is “staging”. The dynamic classification module 123 acquires the CPU usage rate 405 and the memory usage rate 406 of the retrieved entry. The dynamic classification module 123 identifies a staging environment that satisfies the dynamic condition that the item 303 is “operating state” based on the acquired CPU usage rate and memory usage rate.

  The dynamic classification module 123 selects the staging environment with the smallest CPU usage rate from the identified staging environments. The dynamic classification module 123 determines the selected staging environment to be the staging environment that executes the selected test case group. Note that the above-described processing is an example, and the present invention is not limited to this. The above is the description of the process of step S1205.

  When it is determined that the alternative environment 603 of the entry corresponding to the target test case group is “possible”, the dynamic classification module 123 determines whether the alternative environment can be used (step S1204).

  Specifically, the dynamic classification module 123 refers to the condition information 132 and substitutes based on the schedule set in the condition 304 of the entry in which the condition type 302 is “dynamic” and the item 303 is “schedule”. Determine if the environment can be used. For example, the dynamic classification module 123 can determine whether an alternative environment can be used by comparing the current time or the scheduled start time of the test case group with the schedule.

  If it is determined that the alternative environment cannot be used, the dynamic classification module 123 proceeds to step S1205.

  When it is determined that the alternative environment can be used, the dynamic classification module 123 determines the alternative environment in which the target test case group is executed (step S1206).

  Specifically, the dynamic classification module 123 refers to the test environment information 133 and searches for an entry whose test environment type 402 is “substitute”. The dynamic classification module 123 selects an alternative environment in which the value of the CPU usage rate 405 or the memory usage rate 406 of the retrieved entry is the smallest. The dynamic classification module 123 determines the selected alternative environment as the alternative environment for executing the target test case group. The above is the description of the process of step S1206.

  Next, the dynamic classification module 123 determines whether or not the resources required for the test exist in the determined alternative environment (step S1207).

  Specifically, the dynamic classification module 123 refers to the test environment information 133 and acquires the values of the CPU usage rate 405 and the memory usage rate 406 of the entry corresponding to the determined alternative environment. The dynamic classification module 123 determines, based on the acquired CPU usage rate and memory usage rate, whether or not the item 303 satisfies the dynamic condition of “operating state”. When the item 303 does not satisfy the dynamic condition of being “operating state”, the dynamic classification module 123 determines that the determined alternative environment does not have the resources necessary for the test.

  When it is determined that the resources necessary for the test exist in the determined alternative environment, the dynamic classification module 123 proceeds to step S1211.

  When it is determined that the resources necessary for the test do not exist in the determined alternative environment, the dynamic classification module 123 calls the cost calculation module 127 (step S1208). At this time, the dynamic classification module 123 inputs the identification information of the target test case group to the cost calculation module 127. The dynamic classification module 123 shifts to a waiting state until it receives a response including a usage fee from the cost calculation module 127. The processing executed by the cost calculation module 127 will be described with reference to FIG.

  When the dynamic classification module 123 receives the response including the usage fee from the cost calculation module 127, the dynamic classification module 123 determines whether the configuration of the alternative environment can be added within the budget (step S1209).

  Specifically, the dynamic classification module 123 determines whether the usage fee is less than or equal to the budget. If the usage fee is less than or equal to the budget, the dynamic classification module 123 determines that the alternative environment can be added within the budget.

  When it is determined that the alternative environment cannot be added within the budget, the dynamic classification module 123 proceeds to step S1211.

  When it is determined that the alternative environment can be added within the budget, the dynamic classification module 123 adds the test environment (step S1210), and the added test environment becomes the test environment that executes the selected test case group. decide. After that, the dynamic classification module 123 proceeds to step S1211. A well-known technique may be used for the method of adding the test environment, and thus the description thereof will be omitted.

  In the present embodiment, the usage fee of the alternative environment is lower than the usage fee of the staging environment, so that the alternative environment is normally generated in step S1210.

  As shown in steps S1203 to S1210, the dynamic classification module 123 determines a test environment in which to execute the test case group based on the dynamic condition.

  In step S1211, the dynamic classification module 123 starts loop processing of the test cases included in the target test case group (step S1211).

  Specifically, the dynamic classification module 123 refers to the group information 135 and selects one piece of test case identification information (target test case) set in the test case 604 of the entry of the target test case group. At this time, the dynamic classification module 123 selects the identification information of the test cases in the order of execution of the test cases. That is, the test case identification information is selected in order from the left. This is because the test cases are set in the test case 604 in the reverse order of the execution order of the test cases.

  Next, the dynamic classification module 123 registers the target test case in the test execution queue information 136 (step S1212).

  Specifically, the dynamic classification module 123 adds an entry to the test execution queue information 136 and sets identification information in the queue ID 701 of the added entry. Further, the dynamic classification module 123 sets the identification information of the target test case in the test case ID 702 and the identification information of the target test case group in the group ID 703. Further, the dynamic classification module 123 sets the identification information of the determined test environment in the execution environment ID 704 of the added entry.

  Next, the dynamic classification module 123 determines whether the processing has been completed for all test cases included in the target test case group (step S1213).

  When it is determined that the processing has not been completed for all test cases included in the target test case group, the dynamic classification module 123 returns to step S1211, selects a new target test case, and executes the same processing. To do.

  When it is determined that the processing has been completed for all the test cases included in the target test case group, the dynamic classification module 123 determines whether the processing has been completed for all the test case groups (step S1214).

  When it is determined that the processing has not been completed for all the test case groups, the dynamic classification module 123 returns to step S1202, newly selects the target test case group, and executes the same processing.

  When it is determined that the processing has been completed for all the test case groups, the dynamic classification module 123 sends a completion notification to the test execution module 121 (step S1216). After that, the dynamic classification module 123 ends the dynamic classification process.

  As described above, the dynamic classification module 123 can determine the test environment in which the test case group is actually executed based on the dynamic condition.

  FIG. 13 is a flowchart illustrating an example of processing executed by the information acquisition module 126 according to the first embodiment.

  The information acquisition module 126 starts the process described below periodically or when receiving a start command from the test execution module 121.

  The information acquisition module 126 starts the loop processing of the test environment (step S1301).

  Specifically, the information acquisition module 126 refers to the test environment information 133 and selects one target test environment.

  The information acquisition module 126 acquires status information indicating the operating status of the test environment from the target test environment (step S1302).

  The state of the test environment indicates CPU usage rate, memory usage rate, network band, IO performance, and the like. Since the method of acquiring the state information is publicly known, detailed description will be omitted.

  Next, the information acquisition module 126 registers the acquired status information in the test environment information 133 (step S1303). In this embodiment, values are set to the CPU usage rate 405 and the memory usage rate 406 of the entry of the target test environment.

  Next, the information acquisition module 126 determines whether processing has been completed for all test environments (step S1304).

  When it is determined that the processing has not been completed for all the test environments, the information acquisition module 126 returns to step S1301, selects a new target test environment, and executes the same processing.

  When it is determined that the processing has been completed for all the test environments, the information acquisition module 126 ends the processing.

  As described above, the information acquisition module 126 can acquire the status information of the test environment.

  FIG. 14 is a flowchart illustrating an example of processing executed by the cost calculation module 127 according to the first embodiment.

  The cost calculation module 127 calculates the usage charge (operation cost) of the alternative environment for executing the target test case group selected in step S1202 (step S1401). Specifically, the following processing is executed.

  The cost calculation module 127 refers to the group information 135 and acquires the test case identification information from the test case 604 of the entry of the test case group selected in step S1202.

  The cost calculation module 127 refers to the test case information 131, and acquires the value of the predicted time 210 of the entry whose test case ID 201 matches the identification information of the acquired test case. The cost calculation module 127 calculates the predicted time of the test case group by summing the acquired values.

  The cost calculation module 127 refers to the license information 134 and acquires the value of the charge 505 of each entry. The cost calculation module 127 calculates the usage charge of each test environment by multiplying the charge per unit time by the estimated time. The above is the description of the process of step S1401.

  The cost calculation module 127 selects the test environment with the lowest usage charge, and outputs the identification information and the usage charge of the selected test environment to the dynamic classification module 123 (step S1402). Then, the cost calculation module 127 ends the process.

  The dynamic classification module 123 can add a test environment having the same configuration as the determined test environment based on the identification information of the test environment.

  The first embodiment has described the method of determining the test environment when the user requests the reduction of the operating cost. The management server 100 can change the test environment determined according to a user request. The information and the content of processing in the case of generalization will be described.

  The alternative environment 207 of the test case information 131 is replaced with the selected test environment ID. The selected test environment ID stores the identification information of the test environment in which the test case can be executed. The alternative environment 603 of the group information 135 is replaced with the candidate test environment ID. The candidate test environment ID stores specific identification information of the test environment or the type of the test environment.

  The group evaluation module 125 selects a candidate test environment from the test environments set in the selected test environment ID instead of steps S1102 to S1104. Further, the group evaluation module 125 sets the identification information of the selected candidate test environment in the candidate test environment ID of the group information 135. For example, in the case of a user request for the purpose of reducing the processing cost, a test environment that can execute all test cases and has the smallest CPU usage rate is selected.

  The dynamic classification module 123 identifies the candidate test environment by referring to the group information 135 in step S1203. In step S1206, the dynamic classification module 123 searches the test environments other than the candidate test environment for the test environment based on the dynamic condition, and determines the execution test environment from the searched test environments.

  Here, a processing example of a user request other than that described in the first embodiment will be described. For example, for the purpose of reducing the test time (reducing the processing cost), the static condition is not changed, and the dynamic condition is set to "CPU usage rate <30% and memory usage rate 30%", "test environment = 2". It is possible to set it as ". In the case of the above-mentioned conditions, the management server 100 sets an alternative environment as a candidate test environment, and selects two alternative environments that use a small amount of resources from the candidate test environments.

  Further, the name of software or the type of function may be set as a static condition. Thereby, the test environment having the designated software or function can be set as the candidate test environment.

  According to the first embodiment, the management server 100 selects a candidate test environment in which to execute a test case group based on the content of the test, and further executes a test in which the test case group is executed based on the status of the test environment and the user request. The environment can be determined.

  Since the candidate test environment is determined in advance, the management server 100 can quickly determine the test environment. Further, the management server 100 can determine the test environment that matches the user request according to the state of the test environment.

  For example, when the user requests the reduction of the operating cost, the management server 100 selects the alternative environment. When the user requests reduction of the test time, the management server 100 selects a test environment with a small amount of used resources or a test environment with a low load.

  The second embodiment is different in that a test environment is generated in each of the on-premises system 101 and the cloud system 102.

  The configuration of the computer system of the second embodiment and the configuration of the management server 100 are the same as those of the first embodiment. The test case information 131, license information 134, group information 135, and test execution queue information 136 are the same as in the first embodiment. In the second embodiment, the condition information 132 and the test environment information 133 are different.

  In the second embodiment, a system type column is newly added to the test environment information 133. In the column, for example, either "on-premises" or "cloud" is stored. This makes it possible to identify on which system the test environment was created.

  In the second embodiment, in order to reduce the usage fee required for the test, “on-premises” is set as the static condition in the condition information 132. This is because using the on-premise type system 101 can reduce the charge.

  The flow of processing executed by each module of the second embodiment is the same as that of the first embodiment. However, the processes executed by the static classification module 122 and the dynamic classification module 123 are partially different.

  Specifically, the static classification module 122 selects a test environment on the on-premises system 101 as a candidate test environment. Further, the dynamic classification module 123 determines the test environment in which the test case group is executed from the test environments on the on-premises system 101. When the test environment on the on-premises system 101 is insufficient, the dynamic classification module 123 determines the test environment to execute the test case group from the test environments on the cloud system 102.

  In order to realize the above-described control, for example, in step S1208, the dynamic classification module 123 can generate a test environment in the on-premises system 101 that satisfies the dynamic condition in which the item 303 is “operating state”. To judge.

  If the determination result is YES, the dynamic classification module 123 proceeds to step S1211. If the determination result is NO, the dynamic classification module 123 proceeds to step S1209. At this time, the dynamic classification module 123 inputs the system type “cloud” to the cost calculation module 127.

  The second embodiment also has the same effect as the first embodiment.

  It should be noted that the present invention is not limited to the above-described embodiments, but includes various modifications. In addition, for example, the above-described embodiment is a detailed description of the configuration in order to explain the present invention in an easy-to-understand manner, and is not necessarily limited to one having all the configurations described. Further, a part of the configuration of the embodiment can be added, deleted, or replaced with another configuration.

  Further, the above-described respective configurations, functions, processing units, processing means, etc. may be realized by hardware by designing a part or all of them, for example, with an integrated circuit. The present invention can also be realized by a program code of software that realizes the functions of the embodiments. In this case, the storage medium recording the program code is provided to the computer, and the CPU included in the computer reads the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the function of the above-described embodiment, and the program code itself and the storage medium storing the program code constitute the present invention. As a storage medium for supplying such a program code, for example, a flexible disk, a CD-ROM, a DVD-ROM, a hard disk, an SSD (Solid State Drive), an optical disk, a magneto-optical disk, a CD-R, a magnetic tape, A non-volatile memory card, ROM or the like is used.

  Further, the program code for realizing the functions described in the present embodiment can be implemented by a wide range of programs or script languages such as assembler, C / C ++, perl, Shell, PHP, and Java.

  Furthermore, by distributing the program code of the software that realizes the functions of the embodiments via a network, the program code is stored in a storage means such as a hard disk or a memory of a computer or a storage medium such as a CD-RW or a CD-R. Alternatively, the CPU included in the computer may read and execute the program code stored in the storage unit or the storage medium.

  In the above-described embodiments, the control lines and information lines are shown to be necessary for explanation, and not all the control lines and information lines on the product are necessarily shown. All configurations may be connected to each other.

Claims (14)

A management computer connected to a computer system in which a plurality of test environments for executing a test case group including one or more test cases is constructed,
The computer system includes a plurality of computers,
The management computer is
A processor, a memory connected to the processor, and a network interface connected to the processor,
Group information including an entry composed of the identification information of the test case group and the identification information of the candidate test environment determined based on the characteristics of the test cases included in the test case group, and the condition related to the state of the test environment. Holds condition information, including dynamic conditions that are
The processor is
When the execution trigger of the first test case group is detected, the candidate test environment of the first test case group is identified by referring to the group information,
Obtaining status information indicating the status of the candidate test environment,
A management computer that determines an execution test environment for executing the first test case group from the candidate test environments based on the condition information and the state information. The management computer according to claim 1,
The management computer holds test environment information for managing the configuration and status of the test environment,
The processor is
With reference to the test environment information, determine whether or not the candidate test environment that satisfies the dynamic condition,
A management computer, wherein when it is determined that the candidate test environment that satisfies the dynamic condition exists, the execution test environment is determined from among the candidate test environments that satisfy the dynamic condition. The management computer according to claim 2, wherein
The processor is
When it is determined that the candidate test environment that satisfies the dynamic condition does not exist, a test environment that satisfies the dynamic condition is generated based on the configuration of the candidate test environment and the dynamic condition,
A management computer, characterized in that the generated test environment is determined as the execution test environment. The management computer according to claim 3,
The first test case group includes a plurality of test cases,
The condition information includes a static condition that is a condition related to the characteristics of the test case,
The processor is
Analyzing the characteristics of each test case included in the first test case group,
Selecting the candidate test environment based on the results of the analysis and the static conditions,
A management computer, wherein an entry composed of identification information of the first test case group and identification information of the selected candidate test environment is registered in the group information. The management computer according to claim 4,
The management computer holds test case information including an entry composed of identification information of the test case, dependency between the test cases, and identification information of a test environment capable of executing the test case,
The processor is
For each of a plurality of test cases managed by the test case information, specify a test environment that satisfies the static conditions,
Generating the first test case group based on the dependencies between the test cases,
Selecting the candidate test environment from test environments satisfying the static condition of each test case included in the first test case group,
A management computer, wherein an entry composed of identification information of the test case group and identification information of the candidate test environment is registered in the group information. The management computer according to claim 3,
The management computer holds cost information for calculating the usage fee of the test environment,
The dynamic condition includes a condition specifying a resource usage rate of the test environment,
The plurality of test environments include a first test environment and a second test environment that costs less to execute the test case than the first test environment,
The processor is
When the candidate test environment of the first test case group is the second test environment, the status information indicating the status of the second test environment is acquired,
Determining whether or not the first test case group can be executed using the second test environment based on the state information and the dynamic condition,
When it is determined that the first test case group cannot be executed using the second test environment, a cost for adding the same test environment as the candidate test environment is calculated based on the cost information,
A management computer, wherein when the cost is determined to be less than or equal to a threshold, a second test environment is newly added and the second test environment is determined as the execution test environment. The management computer according to claim 6,
The dynamic condition includes a condition indicating a time when the second test environment is available,
The processor is
When the candidate test environment of the first test case group is the second test environment, it is determined whether or not it is a time zone in which the second test environment can be used,
When it is determined that the second test environment can be used, it is determined that the second test environment is the execution test environment,
A management computer, wherein when it is determined that the second test environment is not in a usable time zone, the first test environment is determined to be the execution test environment. A method for determining a test environment in a management computer connected to a computer system in which a plurality of test environments for executing a test case group including one or more test cases is constructed,
The computer system includes a plurality of computers,
The management computer is
A processor, a memory connected to the processor, and a network interface connected to the processor,
Group information including an entry composed of the identification information of the test case group and the identification information of the candidate test environment determined based on the characteristics of the test cases included in the test case group, and the condition regarding the state of the test environment. Holds condition information, including dynamic conditions that are
The test environment determination method is
A first step of identifying a candidate test environment of the first test case group by referring to the group information when the processor detects an execution trigger of the first test case group;
A second step in which the processor obtains state information indicating a state of the candidate test environment;
A third step in which the processor determines an execution test environment for executing the first test case group from the candidate test environments based on the condition information and the state information. How to determine the test environment. The test environment determination method according to claim 8,
The management computer holds test environment information for managing the configuration and status of the test environment,
The third step is
The processor refers to the test environment information to determine whether or not the candidate test environment that satisfies the dynamic condition exists;
When it is determined that the candidate test environment satisfying the dynamic condition exists, the processor determines the execution test environment from the candidate test environments satisfying the dynamic condition. How to determine the test environment. The test environment determination method according to claim 9,
The third step is
When it is determined that the candidate test environment that satisfies the dynamic condition does not exist, the processor generates a test environment that satisfies the dynamic condition based on the configuration of the candidate test environment and the dynamic condition. When,
The processor determines the generated test environment as the execution test environment, the test environment determining method. The test environment determination method according to claim 10, wherein
The first test case group includes a plurality of test cases,
The condition information includes a static condition that is a condition related to the characteristics of the test case,
The test environment determination method is
The processor analyzing a characteristic of each test case included in the first test case group;
The processor selecting the candidate test environment based on the results of the analysis and the static conditions;
The processor registering, in the group information, an entry consisting of identification information of the first test case group and identification information of the selected candidate test environment, the test environment. How to decide. The test environment determination method according to claim 11,
The management computer holds test case information including an entry composed of identification information of the test case, dependency between the test cases, and identification information of a test environment capable of executing the test case,
The test environment determination method is
The processor specifying a test environment satisfying the static condition for each of a plurality of test cases managed by the test case information;
The processor generating the first test case group based on dependencies between the test cases;
The processor selecting the candidate test environment from test environments satisfying the static condition of each test case included in the first test case group;
And a step of the processor registering, in the group information, an entry composed of identification information of the test case group and identification information of the candidate test environment. The test environment determination method according to claim 10, wherein
The management computer holds cost information for calculating the usage fee of the test environment,
The dynamic condition includes a condition specifying a resource usage rate of the test environment,
The plurality of test environments include a first test environment and a second test environment that costs less to execute the test case than the first test environment,
In the second step, the processor obtains state information indicating a state of the second test environment when the candidate test environment of the first test case group is the second test environment. Including,
The third step is
The processor determining whether the first test case group can be executed using the second test environment based on the state information and the dynamic condition;
A cost for the processor to add the same test environment as the candidate test environment based on the cost information when it is determined that the first test case group cannot be executed using the second test environment. And a step of calculating
When the cost is determined to be less than or equal to a threshold value, the processor newly adds a second test environment and determines the second test environment as the execution test environment. And how to determine the test environment. The test environment determination method according to claim 13,
The dynamic condition includes a condition indicating a time when the second test environment is available,
The third step is
The processor, if the candidate test environment of the first test case group is the second test environment, determining whether or not it is a time zone in which the second test environment can be used;
The processor determines that the second test environment is the execution test environment when it is determined that the second test environment can be used.
A method for determining a test environment, comprising: determining that the first test environment is the execution test environment when the processor determines that the second test environment cannot be used. ..

Images