Effects of Negative Testing on TDD: An Industrial Experiment
Adnan Causevic, Rakesh Shukla, Sasikumar Punnekkat, and Daniel Sundmark
Research Paper
Summary
In our recent academic experiments, an existence of positive test bias, that is lack of negative test cases, was identified when a test driven development approach was used. At the same time, when defect detecting ability of individual test cases was calculated, it was noted that the probability of a negative test case to detect a defect was substantially higher than that of a positive test case.
The goal of this study is to investigate the existence of positive test bias in test driven development within an industrial context, and measure defect detecting ability of both positive and negative test cases. An industrial experiment was conducted at Infosys Ltd. India, whose employees voluntarily singed up to participate in the study and were randomly assigned to groups utilizing test driven development, test driven development with negative testing, and test last development. Source code and test cases created by each participant during the study were collected and analysed.
The collected data indicate a statistically significant difference between the number of positive and negative test cases created by industrial participants, confirming the existence of positive test bias. The difference in defect detecting ability of positive and negative test cases is also statistically significant. As a result, similarly to our previous academic study, 29% of all test cases were negative, contributing by revealing as much as 71% of all the defects found by all test cases. With this industrial experiment, we confirmed the existence of a positive test bias in an industrial context, as well as significantly higher defect detecting ability of negative test cases.
Presenter
Adnan Causevic is a PhD student at Mälardalen University, Sweden, member of the Software Engineering Division at Mälardalen Research and Technology Centre (MRTC). He graduated from the Sarajevo University, Faculty of Electrical Engineering, Computer Science and Informatics Department in 2006. His research interests include software verification and validation, software testing, agile testing and empirical studies in software engineering.
Mälardalen University, Sweden, adnan.causevic@mdh.se
Other Authors
Rakesh Shukla, Infosys Ltd. India, rakesh_shukla@infosys.com, https://www.infosys.com
Sasikumar Punnekkat, Mälardalen University, Sweden, sasikumar.punnekkat@mdh.se
Daniel Sundmark, Mälardalen University, Sweden, daniel.sundmark@mdh.se
