Write a Blog >>
ESEC/FSE 2021
Thu 19 - Sat 28 August 2021 Clowdr Platform
Thu 26 Aug 2021 17:20 - 17:25 - Testing—Test Generation 2 Chair(s): Gunel Jahangirova, Michael Pradel
Fri 27 Aug 2021 05:20 - 05:25 - Testing—Test Generation 2 Chair(s): Shiva Nejati

Developers need to perform adequate testing to ensure the quality of Automatic Speech Recognition (ASR) systems. However, manually collecting required test cases is tedious and time-consuming. Our recent work proposes CrossASR, a differential testing method for ASR systems. This method first utilizes Text-to-Speech (TTS) to generate audios from texts automatically and then feed these audios into different ASR systems for cross-referencing to uncover failed test cases. It also leverages a failure estimator to find failing test cases more efficiently. Such a method is inherently self-improvable: the performance can increase by leveraging more advanced TTS and ASR systems. So, in this accompanying tool demo paper, we further engineer CrossASR and propose CrossASR++, an easy-to-use ASR testing tool that can be conveniently extended to incorporate different TTS and ASR systems, and failure estimators. We also make CrossASR++ chunk texts from a given corpus dynamically and enable the estimator to work in a more effective and flexible way. We demonstrate that the new features can help CrossASR++ discover more failed test cases. Using the same TTS and ASR systems, CrossASR++ can uncover $26.2%$ more failed test cases for 4 ASRs than the original tool. Moreover, by simply adding one more ASR for cross-referencing, we can increase the number of failed test cases uncovered for each of the 4 ASR systems by $25.07%$, $39.63%$, $20.95%$ and $8.17%$ respectively. We also extend CrossASR++ with 5 additional failure estimators. Compared to worst estimator, the best one can discover $10.41%$ more failed test cases within the same amount of time. The demo video for CrossASR++ can be viewed at https://youtu.be/ddRk-f0QV-g and the source code can be found at https://github.com/soarsmu/CrossASRplus.

Thu 26 Aug

Displayed time zone: Athens change

17:00 - 18:00
Testing—Test Generation 2Journal First / Research Papers / Demonstrations +12h
Chair(s): Gunel Jahangirova USI Lugano, Michael Pradel University of Stuttgart
17:00
10m
Paper
LS-Sampling: An Effective Local Search Based Sampling Approach for Achieving High t-wise Coverage
Research Papers
Chuan Luo Microsoft Research, Binqi Sun Microsoft Research, Bo Qiao Microsoft Research, Junjie Chen Tianjin University, Hongyu Zhang University of Newcastle, Jinkun Lin Institute of Software at Chinese Academy of Sciences, Qingwei Lin Microsoft Research, Dongmei Zhang Microsoft Research
DOI
17:10
10m
Paper
GLIB: Towards Automated Test Oracle for Graphically-Rich ApplicationsArtifacts Available
Research Papers
Ke Chen Fuxi AI Lab of Netease, Yufei Li University of Texas at Dallas, Yingfeng Chen Fuxi AI Lab of Netease, Changjie Fan Netease, Zhipeng Hu Netease, Wei Yang University of Texas at Dallas
DOI
17:20
5m
Paper
CrossASR++: A Modular Differential Testing Framework for Automatic Speech Recognition
Demonstrations
Muhammad Hilmi Asyrofi Singapore Management University, Zhou Yang Singapore Management University, David Lo Singapore Management University
DOI Pre-print Media Attached
17:25
5m
Paper
Practical Constraint Solving for Generating System Test Data
Journal First
Ghanem Soltana SnT, University of Luxembourg, Mehrdad Sabetzadeh University of Ottawa, Lionel Briand University of Ottawa, Canada / University of Luxembourg, Luxembourg
17:30
30m
Live Q&A
Q&A (Testing—Test Generation 2)
Research Papers

Fri 27 Aug

Displayed time zone: Athens change

05:00 - 06:00
Testing—Test Generation 2Research Papers / Demonstrations / Journal First
Chair(s): Shiva Nejati University of Ottawa
05:00
10m
Paper
LS-Sampling: An Effective Local Search Based Sampling Approach for Achieving High t-wise Coverage
Research Papers
Chuan Luo Microsoft Research, Binqi Sun Microsoft Research, Bo Qiao Microsoft Research, Junjie Chen Tianjin University, Hongyu Zhang University of Newcastle, Jinkun Lin Institute of Software at Chinese Academy of Sciences, Qingwei Lin Microsoft Research, Dongmei Zhang Microsoft Research
DOI
05:10
10m
Paper
GLIB: Towards Automated Test Oracle for Graphically-Rich ApplicationsArtifacts Available
Research Papers
Ke Chen Fuxi AI Lab of Netease, Yufei Li University of Texas at Dallas, Yingfeng Chen Fuxi AI Lab of Netease, Changjie Fan Netease, Zhipeng Hu Netease, Wei Yang University of Texas at Dallas
DOI
05:20
5m
Paper
CrossASR++: A Modular Differential Testing Framework for Automatic Speech Recognition
Demonstrations
Muhammad Hilmi Asyrofi Singapore Management University, Zhou Yang Singapore Management University, David Lo Singapore Management University
DOI Pre-print Media Attached
05:25
5m
Paper
Practical Constraint Solving for Generating System Test Data
Journal First
Ghanem Soltana SnT, University of Luxembourg, Mehrdad Sabetzadeh University of Ottawa, Lionel Briand University of Ottawa, Canada / University of Luxembourg, Luxembourg
05:30
30m
Live Q&A
Q&A (Testing—Test Generation 2)
Research Papers