Write a Blog >>
ESEC/FSE 2021
Mon 23 - Sat 28 August 2021 Athens, Greece

This program is tentative and subject to change.

Thu 26 Aug 2021 12:20 - 12:40 - Human—HCI and Mobile
Fri 27 Aug 2021 00:20 - 00:40 - Human—HCI and Mobile

For ensuring the reliability of Android apps, there has been tremendous, continuous progress
on improving automated GUI testing in the past decade.
Specifically, dozens of testing techniques and tools
have been developed and demonstrated to be effective
in detecting crash bugs and outperform their respective prior work
in the number of detected crashes.
However, an overarching question ``How effectively and thoroughly
can these tools find crash bugs in practice?'' has not been
well-explored, which requires a ground-truth benchmark with real-world bugs.
Since prior studies focus on tool comparisons w.r.t some selected apps, they cannot
provide direct, in-depth answers to this question.

To complement existing work and tackle the above question,
this paper offers the first ground-truth empirical evaluation of
automated GUI testing for Android.
To this end, we devote substantial manual effort to
set up the Themis benchmark set, including (1) a carefully constructed dataset
with 52 real, reproducible crash bugs (taking
two person-months for its collection and validation), and (2)
a unified, extensible infrastructure with six
recent state-of-the-art testing tools.
The whole evaluation has taken over 10,920 CPU hours.
We find a considerable gap in these tools
finding the collected real bugs — 18 bugs cannot be detected by any tool.
Our systematic analysis further identifies five major common
challenges that these tools face, and reveals additional findings
such as factors affecting these tools in bug finding and opportunities
for tool improvements.
Overall, this work offers new concrete insights, most of which
are previously unknown/unstated and difficult to obtain.
Our study presents a new, complementary perspective from prior
studies to understand
and analyze the effectiveness of existing testing tools,
as well as a benchmark for future research on this topic.
The Themis benchmark is publicly available at
https://github.com/the-themis-benchmarks/home.

This program is tentative and subject to change.

Conference Day
Thu 26 Aug

Displayed time zone: Athens change

12:00 - 13:00
Human—HCI and MobileResearch Papers / Industry Papers +12h
12:00
20m
Talk
An Empirical Study of GUI Widget Detection for Industrial Mobile Games
Industry Papers
Jiaming YeKyushu University, Ke ChenFuxi AI Lab of Netease, Xiaofei XieKyushu University, Lei MaUniversity of Alberta, Ruochen HuangUniversity of Alberta, Yingfeng ChenFuxi AI Lab of Netease, Yinxing XueUniversity of Science and Technology of China, Jianjun ZhaoKyushu University
DOI
12:20
20m
Talk
Benchmarking Automated GUI Testing for Android against Real-World Bugs
Research Papers
Ting SuEast China Normal University, Jue WangNanjing University, Zhendong SuETH Zurich
DOI Pre-print Media Attached
12:40
20m
Talk
Data-Driven Accessibility Repair Revisited: On the Effectiveness of Generating Labels for Icons in Android Apps
Research Papers
Forough MehralianUniversity of California at Irvine, Navid SalehnamadiUniversity of California at Irvine, Sam MalekUniversity of California at Irvine
DOI

Conference Day
Fri 27 Aug

Displayed time zone: Athens change

00:00 - 01:00
Human—HCI and MobileResearch Papers / Industry Papers
00:00
20m
Talk
An Empirical Study of GUI Widget Detection for Industrial Mobile Games
Industry Papers
Jiaming YeKyushu University, Ke ChenFuxi AI Lab of Netease, Xiaofei XieKyushu University, Lei MaUniversity of Alberta, Ruochen HuangUniversity of Alberta, Yingfeng ChenFuxi AI Lab of Netease, Yinxing XueUniversity of Science and Technology of China, Jianjun ZhaoKyushu University
DOI
00:20
20m
Talk
Benchmarking Automated GUI Testing for Android against Real-World Bugs
Research Papers
Ting SuEast China Normal University, Jue WangNanjing University, Zhendong SuETH Zurich
DOI Pre-print Media Attached
00:40
20m
Talk
Data-Driven Accessibility Repair Revisited: On the Effectiveness of Generating Labels for Icons in Android Apps
Research Papers
Forough MehralianUniversity of California at Irvine, Navid SalehnamadiUniversity of California at Irvine, Sam MalekUniversity of California at Irvine
DOI