Материал: + Optimized Regression Testusing Test Case Prioritization

Available online at www.sciencedirect.com

ScienceDirect

Procedia Computer Science 79 (2016) 152 – 160

7th International Conference on Communication, Computing and Virtualization 2016

Optimized Regression Test using Test Case Prioritization

Ahlam Ansaria,Anam Khanb,Alisha Khanc,Konain Mukadamd

a,b,c,dM.H. Saboo Siddik College of Engineering,8,Saboo Siddik Polytechnic Road,Byculla,Mumbai-400008,India. aahlam.shakeel@mhssce.ac.in ,banam.khan@mhssce.ac.in, calisha.khan@mhssce.ac.in, dkonain.mukadam@mhssce.ac.in

Abstract

Regression testing ensures that changes made in the fixes or any enhancement changes do not impact the previously working functionality. Whenever software is modified, a set of test cases are run to assure that changes don’t affect the other parts of the software. Hence all existing test cases need to be tested as well as new test cases need to be created. It is nonviable to re-execute every test case for a given software, because if there are more number of test cases to be tested, the more effort and time is required. This problem can be solved by prioritizing test cases. Test case prioritization techniques reorder the priority of a test case in an attempt to ensure that maximum faults are uncovered by the high prioritized test cases. In this paper we propose an optimized test case prioritization technique using Ant Colony Optimization (ACO) to reduce the cost, effort and time taken to perform regression testing and also uncover maximum faults.

© 2016 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license

© 2016 The Authors. Published by Elsevier B.V.

(http://creativecommons.org/licenses/by-nc-nd/4.0/).

Peer-review- underunderresponsibilityof theof theOrganizingOrganizingCommitteeCommitteeof ICCCVof ICCCV2016 2016.

Keywords:Regression testing;Test case prioritization;Ant colony optimization

1. Introduction

When the software is designed then the software development life cycle is used to develop the complete software. For the software development, different phases are followed. After the designing phase testing phase is used to test the whole design code. There are different types of testing that are used to test the code. The regression testing is one of the types of the testing that is used to test the software code1.

Regression testing is a type of testing in which previously executed test cases are re-run so as to verify that existing functionalities work fine. This testing is done to make sure that the process of fixing existing bugs doesn’t result in new bugs, affecting, previously flawless, functions. Regression Testing is required when there is

* Corresponding author. Tel.: +0-000-000-0000 ; fax: +0-000-000-0000 .

E-mail address: ahlam.shakeel@mhssce.ac.in

1877-0509 © 2016 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

Peer-review under responsibility of the Organizing Committee of ICCCV 2016 doi:10.1016/j.procs.2016.03.020

ξChange in requirements and code is modified according to requirement

ξNew features are added to the software

ξDefect fixing Performance issue fix2.

There are various techniques for regression testing. The different regression testing techniques are listed below3:

ξRetest All

ξRegression Test Selection

ξTest Case Prioritization

ξHybrid Approach.

Fig 1 shows the various Regression Testing Techniques

Fig 1. Regression Testing Techniques

ξRetest All: One of the methods of performing regression testing is to retest all the test cases i.e. all the tests in the existing test suite should be re-executed. Running all test cases in an existing test suite, however, can consume a lot of time.

ξRegression Test Selection: Instead of re-executing the entire test suite, it is better to select part of test suite to be run. Test cases selected can be categorized as I) Reusable Test Cases II) Obsolete Test Cases. Reusable Test cases can be used in succeeding regression cycles. Obsolete Test Cases can't be used in succeeding cycles.

ξPrioritization of Test Cases: This way deals with prioritizing the test cases such that test cases with higher priority, according to some criterion are executed earlier than those with lower priority to meet some performance goal. So test case prioritization technique does not discard test cases, they can avoid the drawback of test case minimization technique.

ξHybrid Approach: The fourth regression testing is the Hybrid Approach of both Regression Test Selection and Test Case Prioritization.

Various prioritization criteria may be applied to the regression test suite with the objective of meeting a certain criterion. Test cases can be prioritized in terms of random, optimal, statement coverage total or additional, branch coverage total or additional, failure rates, or total fault exposing potential of the test cases4. Many techniques have been used for prioritization according to one or more of the chosen criteria. In this paper we make use of ACO to prioritize the given test cases. ACO is a process based on real life of ants, precisely on their food searching process.

2. Regression Testing

The maintenance phase of a software product needs to go through the inevitable regression testing process. It is required to retest the existing test suite whenever any addition, deletion or modifications are made to the software. Re-running the test cases from the existing test suite to build confidence in the correctness of the modified software

is referred to as regression testing5.

Regression testing is a type of testing that guarantees that any enhancement or defect fixes in the software or its environment do not impact the previously working functionalities of the software. While testing the working of a specific function it might introduce new defects in other functions.

Fig. 2. Various activities that are carried out during regression testing.

Regression testing focuses on the criteria that the changes done in a particular function should not introduce any defects in the previous flawlessly working functions. Various regression testing techniques can be seen in Fig. 2.

3. Test Case Prioritization

Test case prioritization schedules test case according to some criterion. The objective of this technique is to enhance the possibility that if the test cases are prioritized in some order, they will meet the specified goals within stipulated time and cost than they would if they were not prioritized.

Test case prioritization can address a wide variety of objectives, as given below:

ξSoftware developers/testers intend to increase the rate of fault detection.

ξDetecting the high-risk faults earlier in testing life cycle.

ξTo increase the possibility of regression errors related to specific code changes very early in testing process

ξTo enhance the coverage of coverable code at a faster rate.

ξTo make a system more reliable6.

3.1. Test case prioritization techniques

Test case prioritization technique involves the selection of test cases that uncover maximum faults in the components of the software and assigning high priorities to test cases that have minimum execution time and are used maximum times. Execution of these prioritized test cases proves to be sufficient for testing the functionalities of components of a software in a lesser time. Test case prioritization thus helps in reducing the time and efforts thereby reducing the cost of testing.

The nine different test case prioritization techniques are no prioritization, random prioritization, optimal prioritization, total branch coverage prioritization, total fault-exposing-potential (FEP) prioritization, additional fault-exposing-potential (FEP) prioritization, total statement coverage prioritization, and additional statement coverage prioritization7.

3.2. Algorithms for test case prioritization

There are many algorithms for test case prioritization which are being developed and unfolded by various researchers in the field. They are Greedy Algorithm, Additional Greedy Algorithm, Optimal Algorithm, Hill Climbing Algorithm, Genetic Algorithm, PORT version1.1, Ant Colony optimization3.

4. Ant Colony Optimization

Ant Colony Optimization is an optimal path searching technique that is based on the natural behaviour of ants on a food hunt. During food hunt all the ants follow a certain path and each ant leaves a chemical substance behind it which is known as “pheromone”. The ants following the same path are able to trace the path by smelling the odour of pheromone that was left behind8.

The shortest path will be discovered through teamwork and pheromone evaporation process. As seen in Fig. 3 a. Random path followed by ants 8there are two possible paths from the ant nest to the food source. Assume in the beginning of food search, ant choose their path randomly depositing pheromone on each path. But in the return trip, the ants will choose the path which has more residual pheromone. Since the pheromone evaporates according to the length of paths, thus each path will have different amount of residual pheromone.

Fig 3. (a) Random path followed by ants (b) Shorter path followed by ants

The longer the path, will thus have more pheromone evaporation and less residual pheromones. The other follower ants will choose the shorter path according to the amount of residual pheromones as seen in Fig. 3 b shorter path followed by ants8.

5. Optimized Test Case Evaluation Metric

It is a must to assess effectiveness of the sequence/ordering of the test case using some metric. The proposed technique is evaluated by metric of Average Percentage of Fault Detected (APFD).

5.1 Need for optimization of test cases

Re-running all existing test cases is often costly and sometimes even infeasible due to time and resource constraints. Thus arises the need to optimize test case to maximize coverage and minimize redundancy with minimum time and resources utilised. An optimized test case will not only help provide a baseline assessment of the current version, but some of those tests will also be reused on the next release or version of the software. Such optimized test case will enable testers to rapidly re-verify the changes and quickly respond to new function requirements.

5.2 Test case Evaluation attributes

Following are the attributes of test case evaluation:

ξSize: Size of the set of test cases (test suites) corresponds to the number of test cases present in it.

ξEffort: Effort is the ratio of size of test suite and number of test cases to be executed in a man day. It is calculated as follows:

ξTime: Time required to test a set of test cases depends on the size of test suites and time taken per person to run and test the test cases. It is calculated as:

ξCost: Cost for executing a set of test cases depends upon the effort and cost of per man day. Cost of execution is calculated as:

5.3 APFD matrix

To quantify the goal of increasing a subset of the test suite's rate of fault detection, we use a metric called APFD developed by Elbaum et al.9 that measures the rate of fault detection per percentage of test suite execution. The APFD is calculated by taking the weighted average of the percentage of faults detected during the execution of the test suite. Higher APFD values imply faster (better) fault detection rates. APFD can be calculated as follows:

Where,

T Æ The test suite under evaluation

m Æ the number of faults contained in the program under test P n ÆThe total number of test cases

Tfi Æ The position of the first test in T that exposes fault i.

So as the formula for APFD shows that calculating APFD is only possible when prior knowledge of faults is available. The APFD metric relies on two assumptions: (1) all faults have equal costs. (2) All test cases have equal costs. These assumptions are manifested in the fact that the metric plots the percentage of faults detected against the percentage of the test suite run.

6. Proposed Technique

The paper mainly focuses at optimizing the efficiency of regression testing by sorting the list of test cases, required for a successful regression test of the software. This is done by taking input of test case matrices and sorting it in accordance to certain criterion. Thus the efficiency of the test will be optimized.

Initially a test case is chosen that covers maximum faults. Since the aim is to uncover all faults, thus it is checked if all faults are covered by it or not. If not, then choose the next test case that covers the remaining faults and repeat this until all faults have been covered. Once all faults are covered, calculate the total number of faults covered by each test case which is stored in total fault test case matrix. This procedure will lead to many combinations of test case called paths that cover all faults.

Many such paths are explored while iterating and the best path from all paths explored is selected. The pheromone value is updated on the best path selected. The selection of best path is based on minimum execution time, maximum probability of usage and highest pheromone value.

This process of prioritizing using Ant Colony Optimization is further explained with help of an algorithm.

6.1 Proposed Algorithm

Inputs: Fault Test case matrix, Execution time test case matrix, Usage matrix, Pheromone matrix. Output: Prioritized list of test case.

Step 1: Input to the system the following:

a)Fault Test case matrix

b)Execution time test case matrix

c)Usage matrix