How do Testing Metrics help?
Testing metrics can be used to track project quality, assess organization’s adherence to quality standards and ultimately contribute to reduction in the number of defects discovered post project implementation. At the commencement of a program, organizations need to have consensus on the metrics that need to be generated – the frequency, formats and stakeholder specific metrics. Identifying key metrics that need to be captured and analyzed can help developers and testers to effectively manage enterprise quality standards. Analyzing and reporting the right metrics also helps key stakeholders understand the value of testing.
Software Testing focus on Defect Prevention instead of detection?
Metrics pertaining to testing should ideally be generated using a Test Management Tool that is the centralized warehouse of all testing related documentation. This would ensure that metrics are generated from a common source and are made available across the enterprise in a standard format and frequency that is easily assimilated.
The focus of software testing is now shifting towards defect prevention rather than defect detection:
- Reduce the cost of testing (and the associated timelines)
- Lowering the cost of maintenance (arising from incorrect implementation of requirements or inadequate testing that results in too many Change Requests) post implementation.
Getting the most from a Root Case Analysis
The quality of requirements implementation can be measured by conducting a Root Case Analysis (RCA) on the defects reported:
- Identify the phase of the software development life cycle (requirements, design, development and testing) where the defects originated.
- Defect Removal Efficiency (DRE) tracked through each stage of testing (unit testing, system integration testing and user acceptance testing) is a direct indicator of the effectiveness of testing.
The Benefits of focusing on Defect Removal Efficiency
Defect Removal Efficiency provides an indication of the defects that leak through to the subsequent stage of testing. Organizations that focus on DRE as key metric and set internal benchmarks have started adopting a “Shift Left” approach, i.e., focus on discovering defects early in the testing life cycle and eliminate discovery of defects post implementation.
This would result in reduced maintenance costs and improved stakeholder satisfaction post implementation. However, the percentage of organizations adopting this approach is still quite small. To ensure that end users of applications spend less time testing the same, organizations engage external service providers to perform the early stages of testing.
This strategy works well by ensuring that most of the critical and high severity defects are unearthed early in the testing life cycle and get fixed before testing is done by business users. Defect Removal Efficiency is a key metric that can be used to assess the effectiveness of external service providers.
Popular testing related metrics like defects severity, aging, SLA breaches for providing defect fixes and testing productivity are focused more on the testing process. These metrics need to be tracked more for process improvements and benchmarking.
Improving the final results of the SDLC is what software development teams have been working. Finding the right strategy to ensure high quality development is the goal of most developers. Tracking the quality of software development and testing efforts is absolutely essential to gauge the current state of an organization’s Quality Assurance (QA) processes. At every level of development, accurate data is required to ascertain the quality of current processes and identify areas that require improvement.