Selecting Test cases for Regression Testing

Post Your Queries | QA and Testing - Table of Contents

SPONSORED LINKS

.. Continuing the Test Management Series

It was found from recent survey (from small and big Software testing companies) that good bunch of the defects reported by clients were due to last minute bug fixes creating side effects. And hence, selecting the test cases for regression testing is not easy. It is an art and one QA Lead/QA Manager should be perfect in this.

To select test cases for Regression Testing, QA Lead/QA Manager should know the following:

Knowledge on the bug fixes and how it affects the application.
In which area/functionality of application, the defects are frequently occurring.
Which area/functionality has undergone many/recent code changes?
Which features of the product/application are mandatory requirements of the client?
Which features of the product/application are Important to the customer?
In which functionality, the bugs are fixed in rush/hurry?

Selecting test cases for regression testing depends more on the criticality of bug fixes than the criticality of the defect itself. Fix of a minor bug can result in major side effect and a bug fix for an Extreme defect or a high severity defect can have no or a just a minor side effect. So the Test Lead/engineer or test Manager needs to balance these aspects for selecting the test cases and test scenarios for regression testing.

Solution: Make good relations with Development team lead/Development Technical manager. They can easily help QA team in identifying the above. A proper impact analysis should be done.

While selecting test cases and test scenarios for Regression Testing, we should not select only those test cases which fail while regular test cycles because those test cases and scenarios can have no or less relevance to the bug fixes. We need to select more positive test cases than negative test cases for final regression test cycle. It is also recommended and a best software testing practise that the regular test cycles (which are conducted before regression testing) should have right mix of both positive and negative test scenarios. Negative test scenarios are those test cases which are introduced with intention to break the application.

From a recent survey, it is found that several companies have "constant test cases set" for regression testing and they are executed irrespective of the number and type of bug fixes. Sometimes this approach may not find all bugs (side effects) due to the bug fixes in the application. Also in some cases it is observed that the effort spend on executing test cases for regression testing can be minimized if analysis or a study is done to find out what test cases are relevant for regression testing and what are not. This can be done by Impact Analysis.

It is a good approach to Planning Regression testing is from the beginning of project before the test cycles instead of hat you plan after the completion of regular test cycles. Best practise is –

Classify the test cases and test scenarios into various Priorities based on importance and customer usage. Here it is suggested the test cases be classified into three classes:

Priority 0 – In this category, those test cases falls which checks basic functionality and are executed for pre-system acceptance and when product goes thru major change. These test cases basically checks whether the application is stable enough for further testing or not. These are Sanity Test cases which delivers high project value to client and the entire software development/testing and quality assurance team.

Priority-1 – In this category, we can add test cases of those functionalities (which are very important to customer) in which we are getting major/critical bugs, the bugs of critical functionalities which are fixed in rush.

Priority-2 – These test cases deliver moderate project value. Executed part of Software Testing cycle and selected for regression testing on need basis.

There are various right approaches to regression testing which needs to be decided on "case to case" basis and we can prioritize the test cases:

Case 1: If the criticality and impact of the bug fixes are LOW, then it is enough a Software Tester selects a few test cases from Test Case Database (TCDB) and executes them. These test cases can fall under any Priority (Priority 0, Priority 1 or Priority 2).

Case 2: If the criticality and the impact of the bug fixes are Medium, then we need to execute all Priority 0 and Priority 1 test cases. If bug fixes need some additional test cases from Priority-2, then those test cases can also be selected and executed for regression testing. Selecting Priority-2 test cases in this case is desirable or optional but not must.

Case 3: If the criticality and impact of the bug fixes are High, then testing team need to execute all the Priority 0, Priority 1 and carefully selected Priority 2 test cases. Priority 2 test cases cannot be skipped in this case. So be careful while choosing the test cases.

Case 4: QA Lead or QA Manager can also go through the complete log of changes happened (can be obtained from Configuration Management Team) because of bug fixes and select the test cases to conduct regression testing. This is a detailed and sometimes complex process but can give very good results. Also don’t forget the One Hour Regression Test Strategy.

This is illustrated in the picture below;