LS-Sampling: An Effective Local Search Based Sampling Approach for Achieving High t-wise Coverage
Fri 27 Aug 2021 05:00 - 05:10 - Testing—Test Generation 2 Chair(s): Shiva Nejati
There has been a rapidly increasing demand for developing highly configurable software systems, which urgently calls for effective testing methods. In practice, t-wise coverage has been widely recognized as a useful metric to evaluate the quality of a test suite for testing highly configurable software systems, and achieving high t-wise coverage is important for ensuring test adequacy. However, state-of-the-art methods usually cost a fairly long time to generate large test suites for high pairwise coverage (i.e., 2-wise coverage), which would lead to ineffective and inefficient testing of highly configurable software systems. In this paper, we propose a novel local search based sampling approach dubbed LS-Sampling for achieving high t-wise coverage. Extensive experiments on a large number of public benchmarks, which are collected from real-world, highly configurable software systems, show that LS-Sampling achieves higher 2-wise and 3-wise coverage than the current state of the art. LS-Sampling is effective, since on average it achieves the 2-wise coverage of 99.64% and the 3-wise coverage of 97.87% through generating a small test suite consisting of only 100 test cases (90% smaller than the test suites generated by its state-of-the-art competitors). Furthermore, LS-Sampling is efficient, since it only requires an average execution time of less than one minute to generate a test suite with high 2-wise and 3-wise coverage.
Thu 26 AugDisplayed time zone: Athens change
17:00 - 18:00 | Testing—Test Generation 2Journal First / Research Papers / Demonstrations +12h Chair(s): Gunel Jahangirova USI Lugano, Michael Pradel University of Stuttgart | ||
17:00 10mPaper | LS-Sampling: An Effective Local Search Based Sampling Approach for Achieving High t-wise Coverage Research Papers Chuan Luo Microsoft Research, Binqi Sun Microsoft Research, Bo Qiao Microsoft Research, Junjie Chen Tianjin University, Hongyu Zhang University of Newcastle, Jinkun Lin Institute of Software at Chinese Academy of Sciences, Qingwei Lin Microsoft Research, Dongmei Zhang Microsoft Research DOI | ||
17:10 10mPaper | GLIB: Towards Automated Test Oracle for Graphically-Rich Applications Research Papers Ke Chen Fuxi AI Lab of Netease, Yufei Li University of Texas at Dallas, Yingfeng Chen Fuxi AI Lab of Netease, Changjie Fan Netease, Zhipeng Hu Netease, Wei Yang University of Texas at Dallas DOI | ||
17:20 5mPaper | CrossASR++: A Modular Differential Testing Framework for Automatic Speech Recognition Demonstrations Muhammad Hilmi Asyrofi Singapore Management University, Zhou Yang Singapore Management University, David Lo Singapore Management University DOI Pre-print Media Attached | ||
17:25 5mPaper | Practical Constraint Solving for Generating System Test Data Journal First Ghanem Soltana SnT, University of Luxembourg, Mehrdad Sabetzadeh University of Ottawa, Lionel Briand University of Ottawa, Canada / University of Luxembourg, Luxembourg | ||
17:30 30mLive Q&A | Q&A (Testing—Test Generation 2) Research Papers |
Fri 27 AugDisplayed time zone: Athens change
05:00 - 06:00 | Testing—Test Generation 2Research Papers / Demonstrations / Journal First Chair(s): Shiva Nejati University of Ottawa | ||
05:00 10mPaper | LS-Sampling: An Effective Local Search Based Sampling Approach for Achieving High t-wise Coverage Research Papers Chuan Luo Microsoft Research, Binqi Sun Microsoft Research, Bo Qiao Microsoft Research, Junjie Chen Tianjin University, Hongyu Zhang University of Newcastle, Jinkun Lin Institute of Software at Chinese Academy of Sciences, Qingwei Lin Microsoft Research, Dongmei Zhang Microsoft Research DOI | ||
05:10 10mPaper | GLIB: Towards Automated Test Oracle for Graphically-Rich Applications Research Papers Ke Chen Fuxi AI Lab of Netease, Yufei Li University of Texas at Dallas, Yingfeng Chen Fuxi AI Lab of Netease, Changjie Fan Netease, Zhipeng Hu Netease, Wei Yang University of Texas at Dallas DOI | ||
05:20 5mPaper | CrossASR++: A Modular Differential Testing Framework for Automatic Speech Recognition Demonstrations Muhammad Hilmi Asyrofi Singapore Management University, Zhou Yang Singapore Management University, David Lo Singapore Management University DOI Pre-print Media Attached | ||
05:25 5mPaper | Practical Constraint Solving for Generating System Test Data Journal First Ghanem Soltana SnT, University of Luxembourg, Mehrdad Sabetzadeh University of Ottawa, Lionel Briand University of Ottawa, Canada / University of Luxembourg, Luxembourg | ||
05:30 30mLive Q&A | Q&A (Testing—Test Generation 2) Research Papers |