Output Sampling for Output Diversity in Automatic Unit Test Generation
Fri 27 Aug 2021 04:20 - 04:30 - Testing—Test Generation 1 Chair(s): Shiva Nejati
Diverse test sets are able to expose bugs that test sets generated with structural coverage techniques cannot discover. Input-diverse test set generators have been shown to be effective for this, but also have limitations: e.g., they need to be complemented with semantic information derived from the Software Under Test. We demonstrate how to drive the test set generation process with semantic information in the form of output diversity. We present the first totally automatic output sampling for output diversity unit test set generation tool, called OutGen. OutGen transforms a program into an SMT formula in bit-vector arithmetic. It then applies universal hashing in order to generate an output-based diverse set of inputs. The result offers significant diversity improvements when measured as a high output uniqueness count. It achieves this by ensuring that the test set’s output probability distribution is uniform, i.e. highly diverse. The use of output sampling, as opposed to any of input sampling, CBMC, CAVM, behaviour diversity or random testing improves mutation score and bug detection by up to 4150% and 963% respectively on programs drawn from three different corpora: the R-project, SIR and CodeFlaws. OutGen test sets achieve an average mutation score of up to 92%, and 70% of the test sets detect the defect. Moreover, OutGen is the only automatic unit test generation tool that is able to detect bugs on the real number C functions from the R-project.
Thu 26 AugDisplayed time zone: Athens change
16:00 - 17:00 | Testing—Test Generation 1Journal First / Research Papers +12h Chair(s): Rachel Tzoref-Brill IBM Research, Myra Cohen Iowa State University | ||
16:00 10mPaper | Graph-Based Seed Object Synthesis for Search-Based Unit Testing Research Papers Yun Lin National University of Singapore, You Sheng Ong National University of Singapore, Jun Sun Singapore Management University, Gordon Fraser University of Passau, Jin Song Dong National University of Singapore DOI Pre-print | ||
16:10 10mPaper | An Extended Abstract of ''Theoretical and Empirical Analyses of the Effectiveness of Metamorphic Relation Composition'' Journal First Kun Qiu Hefei University of Technology, Zheng Zheng Beihang University, Tsong Yueh Chen Swinburne University of Technology, Pak-Lok Poon School of Engineering & Technology, Central Queensland University, Australia Link to publication DOI | ||
16:20 10mPaper | Output Sampling for Output Diversity in Automatic Unit Test Generation Journal First Hector Menendez Middlesex University London, Michele Boreale Università di Firenze, Daniele Gorla Department of Computer Science, Sapienza University of Rome, David Clark University College London | ||
16:30 30mLive Q&A | Q&A (Testing—Test Generation 1) Research Papers |
Fri 27 AugDisplayed time zone: Athens change
04:00 - 05:00 | Testing—Test Generation 1Research Papers / Journal First Chair(s): Shiva Nejati University of Ottawa | ||
04:00 10mPaper | Graph-Based Seed Object Synthesis for Search-Based Unit Testing Research Papers Yun Lin National University of Singapore, You Sheng Ong National University of Singapore, Jun Sun Singapore Management University, Gordon Fraser University of Passau, Jin Song Dong National University of Singapore DOI Pre-print | ||
04:10 10mPaper | An Extended Abstract of ''Theoretical and Empirical Analyses of the Effectiveness of Metamorphic Relation Composition'' Journal First Kun Qiu Hefei University of Technology, Zheng Zheng Beihang University, Tsong Yueh Chen Swinburne University of Technology, Pak-Lok Poon School of Engineering & Technology, Central Queensland University, Australia Link to publication DOI | ||
04:20 10mPaper | Output Sampling for Output Diversity in Automatic Unit Test Generation Journal First Hector Menendez Middlesex University London, Michele Boreale Università di Firenze, Daniele Gorla Department of Computer Science, Sapienza University of Rome, David Clark University College London | ||
04:30 30mLive Q&A | Q&A (Testing—Test Generation 1) Research Papers |