Write a Blog >>
ESEC/FSE 2021
Mon 23 - Sat 28 August 2021 Athens, Greece

This program is tentative and subject to change.

Wed 25 Aug 2021 16:15 - 16:30 - AI—Software Engineering for Machine Learning 2
Thu 26 Aug 2021 04:15 - 04:30 - AI—Software Engineering for Machine Learning 2

The increasingly wide uptake of Machine Learning (ML) has raised the significance of the problem of tackling bias (i.e., unfairness), making it a primary software engineering concern. In this paper, we introduce Fairea, a model behaviour mutation approach to benchmarking ML bias mitigation methods. We also report on a large-scale empirical study to test the effectiveness of 12 widely-studied bias mitigation methods. Our results reveal that, surprisingly, bias mitigation methods have a poor effectiveness in 49% of the cases. In particular, 15% of the mitigation cases have worse fairness-accuracy trade-offs than the baseline established by Fairea; 34% of the cases have a decrease in accuracy and an increase in bias.

Fairea has been made publicly available for software engineers and researchers to evaluate their bias mitigation methods.

This program is tentative and subject to change.

Conference Day
Wed 25 Aug

Displayed time zone: Athens change

16:00 - 17:00
AI—Software Engineering for Machine Learning 2Journal First / Research Papers / Ideas, Visions and Reflections +12h
16:00
15m
Talk
Selecting Test Inputs for DNNs using Differential Testing with Subspecialized Model Instances
Ideas, Visions and Reflections
Yu-Seung MaElectronics and Telecommunications Research Institute, Shin YooKorea Advanced Institute of Science and Technology, Taeho KimElectronics and Telecommunications Research Institute (ETRI)
16:15
15m
Talk
Fairea: A Model Behaviour Mutation Approach to Benchmarking Bias Mitigation Methods
Research Papers
Max HortUniversity College London, Jie M. ZhangUniversity College London, UK, Federica SarroUniversity College London, Mark HarmanUniversity College London
16:30
15m
Talk
Fair Preprocessing: Towards Understanding Compositional Fairness of Data Transformers in Machine Learning Pipeline
Research Papers
Sumon BiswasIowa State University, USA, Hridesh RajanIowa State University, USA
DOI Pre-print Media Attached
16:45
15m
Talk
The Current State of Industrial Practice in Artificial Intelligence Ethics
Journal First
Ville VakkuriUniversity of Jyvaskyla, Kai-Kristian KemellUniversity of Jyvaskyla, Joni KultanenUniversity of Jyvaskyla, Pekka AbrahamssonUniversity of Jyväskylä

Conference Day
Thu 26 Aug

Displayed time zone: Athens change

04:00 - 05:00
AI—Software Engineering for Machine Learning 2Research Papers / Ideas, Visions and Reflections / Journal First
04:00
15m
Talk
Selecting Test Inputs for DNNs using Differential Testing with Subspecialized Model Instances
Ideas, Visions and Reflections
Yu-Seung MaElectronics and Telecommunications Research Institute, Shin YooKorea Advanced Institute of Science and Technology, Taeho KimElectronics and Telecommunications Research Institute (ETRI)
04:15
15m
Talk
Fairea: A Model Behaviour Mutation Approach to Benchmarking Bias Mitigation Methods
Research Papers
Max HortUniversity College London, Jie M. ZhangUniversity College London, UK, Federica SarroUniversity College London, Mark HarmanUniversity College London
04:30
15m
Talk
Fair Preprocessing: Towards Understanding Compositional Fairness of Data Transformers in Machine Learning Pipeline
Research Papers
Sumon BiswasIowa State University, USA, Hridesh RajanIowa State University, USA
DOI Pre-print Media Attached
04:45
15m
Talk
The Current State of Industrial Practice in Artificial Intelligence Ethics
Journal First
Ville VakkuriUniversity of Jyvaskyla, Kai-Kristian KemellUniversity of Jyvaskyla, Joni KultanenUniversity of Jyvaskyla, Pekka AbrahamssonUniversity of Jyväskylä