FairPath: A Human-in-the-Loop Framework for Adversarial Fairness Testing in Machine Learning Pipelines

This thesis investigates how fairness is addressed within machine learning pipelines and proposes a framework for adversarial fairness testing through a human-in-the-loop approach. Although several fairness metrics and mitigation techniques have been proposed, existing approaches often present technical barriers that limit collaboration between machine learning practitioners and non-technical stakeholders, hindering the identification and mitigation of fairness issues before deployment. To address this gap, this research introduces a framework integrating fairness assessment, bias mitigation, explanatory feedback, and decision support mechanisms to enable accessible and structured fairness evaluation. A tool named FairPath was developed and evaluated through quantitative benchmarking and qualitative heuristic evaluation against comparable approaches. This thesis addresses five research questions: RQ1: How is bias identified and measured in the software fairness literature? RQ2: What data improvement approaches are proposed to enhance fairness? RQ3: What effects do data improvement approaches have on bias mitigation and representativeness enhancement? RQ4: What ethical considerations are examined in research on data improvement approaches? RQ5: Can a fairness tool enable accessible bias mitigation while preserving model utility? By addressing these questions, this research aims to bridge fairness assessment and stakeholder collaboration, reducing barriers to effective bias identification and mitigation while supporting the development of transparent, accountable, and socially responsible machine learning systems.