GLIB: Towards Automated Test Oracle for Graphically-Rich Applications
Fri 27 Aug 2021 05:10 - 05:20 - Testing—Test Generation 2 Chair(s): Shiva Nejati
Graphically-rich applications such as games are ubiquitous with attractive visual effects of Graphical User Interface (GUI) that offers a bridge between software applications and end-users. However, various types of graphical glitches may arise from such GUI complexity and have become one of the main component of software compatibility issues. Our study on bug reports from game development teams in NetEase Inc. indicates that graphical glitches frequently occur during the GUI rendering and severely degrade the quality of graphically-rich applications such as video games.
Existing automated testing techniques for such applications focus mainly on generating various GUI test sequences and check whether the test sequences can cause crashes.
These techniques require constant human attention to captures non-crashing bugs such as bugs causing graphical glitches.
In this paper, we present the first step in automating the test oracle for detecting non-crashing bugs in graphically-rich applications.
Specifically, we propose \texttt{GLIB} based on a code-based data augmentation technique to detect game GUI glitches.
We perform an evaluation of \texttt{GLIB} on 20 real-world game apps (with bug reports available) and the result shows that \texttt{GLIB} can achieve 100% precision and 99.5% recall in detecting non-crashing bugs such as game GUI glitches. Practical application of \texttt{GLIB} on another 14 real-world games (without bug reports) further demonstrates that \texttt{GLIB} can effectively uncover GUI glitches, with 48 of 53 bugs reported by \texttt{GLIB} having been confirmed and fixed so far.
Thu 26 AugDisplayed time zone: Athens change
17:00 - 18:00 | Testing—Test Generation 2Journal First / Research Papers / Demonstrations +12h Chair(s): Gunel Jahangirova USI Lugano, Michael Pradel University of Stuttgart | ||
17:00 10mPaper | LS-Sampling: An Effective Local Search Based Sampling Approach for Achieving High t-wise Coverage Research Papers Chuan Luo Microsoft Research, Binqi Sun Microsoft Research, Bo Qiao Microsoft Research, Junjie Chen Tianjin University, Hongyu Zhang University of Newcastle, Jinkun Lin Institute of Software at Chinese Academy of Sciences, Qingwei Lin Microsoft Research, Dongmei Zhang Microsoft Research DOI | ||
17:10 10mPaper | GLIB: Towards Automated Test Oracle for Graphically-Rich Applications Research Papers Ke Chen Fuxi AI Lab of Netease, Yufei Li University of Texas at Dallas, Yingfeng Chen Fuxi AI Lab of Netease, Changjie Fan Netease, Zhipeng Hu Netease, Wei Yang University of Texas at Dallas DOI | ||
17:20 5mPaper | CrossASR++: A Modular Differential Testing Framework for Automatic Speech Recognition Demonstrations Muhammad Hilmi Asyrofi Singapore Management University, Zhou Yang Singapore Management University, David Lo Singapore Management University DOI Pre-print Media Attached | ||
17:25 5mPaper | Practical Constraint Solving for Generating System Test Data Journal First Ghanem Soltana SnT, University of Luxembourg, Mehrdad Sabetzadeh University of Ottawa, Lionel Briand University of Ottawa, Canada / University of Luxembourg, Luxembourg | ||
17:30 30mLive Q&A | Q&A (Testing—Test Generation 2) Research Papers | ||
Fri 27 AugDisplayed time zone: Athens change
05:00 - 06:00 | Testing—Test Generation 2Research Papers / Demonstrations / Journal First Chair(s): Shiva Nejati University of Ottawa | ||
05:00 10mPaper | LS-Sampling: An Effective Local Search Based Sampling Approach for Achieving High t-wise Coverage Research Papers Chuan Luo Microsoft Research, Binqi Sun Microsoft Research, Bo Qiao Microsoft Research, Junjie Chen Tianjin University, Hongyu Zhang University of Newcastle, Jinkun Lin Institute of Software at Chinese Academy of Sciences, Qingwei Lin Microsoft Research, Dongmei Zhang Microsoft Research DOI | ||
05:10 10mPaper | GLIB: Towards Automated Test Oracle for Graphically-Rich Applications Research Papers Ke Chen Fuxi AI Lab of Netease, Yufei Li University of Texas at Dallas, Yingfeng Chen Fuxi AI Lab of Netease, Changjie Fan Netease, Zhipeng Hu Netease, Wei Yang University of Texas at Dallas DOI | ||
05:20 5mPaper | CrossASR++: A Modular Differential Testing Framework for Automatic Speech Recognition Demonstrations Muhammad Hilmi Asyrofi Singapore Management University, Zhou Yang Singapore Management University, David Lo Singapore Management University DOI Pre-print Media Attached | ||
05:25 5mPaper | Practical Constraint Solving for Generating System Test Data Journal First Ghanem Soltana SnT, University of Luxembourg, Mehrdad Sabetzadeh University of Ottawa, Lionel Briand University of Ottawa, Canada / University of Luxembourg, Luxembourg | ||
05:30 30mLive Q&A | Q&A (Testing—Test Generation 2) Research Papers | ||