Write a Blog >>
ESEC/FSE 2021
Thu 19 - Sat 28 August 2021 Clowdr Platform
Fri 27 Aug 2021 11:00 - 11:10 - Human Aspects—Software Practices 2 Chair(s): Ingrid Nunes, Ivica Crnkovic
Fri 27 Aug 2021 23:00 - 23:10 - Human Aspects—Software Practices 2 Chair(s): Kostadin Damevski

Software testing is an essential software quality assurance practice. Testing helps expose faults earlier, allowing developers to repair the code and reduce future maintenance costs. However, repairing (i.e., making failing tests pass) may not always be done immediately. Bugs may require multiple rounds of repairs and even remain unfixed due to the difficulty of bug-fixing tasks. To help test maintenance, along with code comments, the majority of testing frameworks (e.g., JUnit and TestNG) have also introduced annotations such as @Ignore to disable failing tests temporarily. Although disabling tests may help alleviate maintenance difficulties, they may also introduce technical debt. With the faster release of applications in modern software development, disabling tests may become the salvation for many developers to meet project deliverables. In the end, disabled tests may become outdated and a source of technical debt, harming long-term maintenance. Despite its harmful implications, there is little empirical research evidence on the prevalence,
evolution, and maintenance of disabling tests in practice. To fill this gap, we perform the first empirical study on test disabling practice. We develop a tool to mine 122K commits and detect 3,111 changes that disable tests from 15 open-source Java systems. Our main findings are: (1) Test disabling changes are 19% more common than regular test refactorings, such as renames and type changes. (2) Our life-cycle analysis shows that 41% of disabled tests are never brought back to evaluate software quality, and most disabled tests stay disabled for several years. (3)We unveil the motivations behind test disabling practice and the associated technical debt by manually studying evolutions of 349 unique disabled tests, achieving a 95% confidence level and a 5% confidence interval. Finally, we present some actionable implications for researchers and developers.

Fri 27 Aug

Displayed time zone: Athens change

11:00 - 12:00
Human Aspects—Software Practices 2Research Papers / Journal First +12h
Chair(s): Ingrid Nunes Universidade Federal do Rio Grande do Sul (UFRGS), Brazil, Ivica Crnkovic Chalmers University of Technology
11:00
10m
Paper
How Disabled Tests Manifest in Test Maintainability Challenges?
Research Papers
Dong Jae Kim Concordia University, Bo Yang Concordia University, Jinqiu Yang Concordia University, Tse-Hsun (Peter) Chen Concordia University
DOI Pre-print
11:10
10m
Paper
Formal methods in dependable systems engineering: a survey of professionals from Europe and North America
Journal First
Mario Gleirscher University of Bremen, Diego Marmsoler University of Exeter
11:20
10m
Paper
Promises and Challenges of Microservices: An Exploratory Study
Journal First
YINGYING WANG University of British Columbia, Harshavardhan Kadiyala The University of British Columbia, Julia Rubin University of British Columbia
11:30
30m
Live Q&A
Q&A (Human Aspects—Software Practices 2)
Research Papers

23:00 - 00:00
Human Aspects—Software Practices 2Journal First / Research Papers
Chair(s): Kostadin Damevski Virginia Commonwealth University
23:00
10m
Paper
How Disabled Tests Manifest in Test Maintainability Challenges?
Research Papers
Dong Jae Kim Concordia University, Bo Yang Concordia University, Jinqiu Yang Concordia University, Tse-Hsun (Peter) Chen Concordia University
DOI Pre-print
23:10
10m
Paper
Formal methods in dependable systems engineering: a survey of professionals from Europe and North America
Journal First
Mario Gleirscher University of Bremen, Diego Marmsoler University of Exeter
23:20
10m
Paper
Promises and Challenges of Microservices: An Exploratory Study
Journal First
YINGYING WANG University of British Columbia, Harshavardhan Kadiyala The University of British Columbia, Julia Rubin University of British Columbia
23:30
30m
Live Q&A
Q&A (Human Aspects—Software Practices 2)
Research Papers