On the limits of privacy in federated learning : personalization and adversarial analysis

With the rapid adoption of Artificial Intelligence across diverse applications, concerns around data privacy have become increasingly prominent. Traditional centralized machine learning approaches require aggregating large volumes of user data at a central server, posing risks related to privacy, scalability, and regulatory compliance. Federated Learning has emerged as a decentralized alternative, enabling clients to train models locally while only sharing model updates with a central server. It addresses some of the concerns by enabling multiple participants to collaboratively train a shared model while keeping their raw data locally stored. However, data locality alone does not guarantee privacy. Information about local training records may still be revealed through model parameters, gradients, updates, or predictions.This makes FL vulnerable to privacy attacks such as model poisoning, data reconstruction and membership inference attacks. This thesis investigates these limitations in FL through two complementary perspectives: adversarial evaluation of the framework and personalization of privacy protection. In this thesis, we first examine the validity of various data anonymization techniques for facial images through the re-identification attacks. A weight-optimized record linkage method is developed to determine whether anonymized image records can be linked to their corresponding original records. The evaluation considers different adversarial settings, including cases in which an attacker possesses only partial information. The findings demonstrate that visually modifying an image or separating identity-related information does not necessarily prevent re-identification. We also experiment with Shapley values to check whether they could be useful in detecting model poisoning attacks.Then, we develop two privacy-enhancing mechanisms for federated time-series forecasting. The first framework is centered around using differential privacy with transformers specifically for traffic flow data whereas the other framework is a microaggregation-based model agnostic k-anonymous federated forecasting framework. The results demonstrate that useful forecasting performance can be maintained while raw data remain decentralized and additional protection is applied at the client level. One of the major challenges that exist in FL is heterogeneity. There have been several studies in the literature on data and device heterogeneity in FL. However, the heterogeneity that exists in the privacy requirements of different clients is widely unexplored. Building on these findings, this thesis introduces a personalized privacy-preserving federated learning framework that allows individual clients to select their preferred privacy mechanism and protection level. The proposed FedPUP framework supports the simultaneous use of differential privacy and k-anonymity within a federated round. We also used Membership Inference attacks to empirically evaluate the information leakage associated with different privacy choices and to identify combinations of ε and k that provide comparable levels of protection. We also propose a privacy-aware server-side aggregation algorithm that incorporates client-level privacy characteristics during the aggregation process in each training round. Our empirical contributions are also complemented by an exploratory study investigating the practical requirements for deploying privacy-preserving federated learning systems in the industry. Our study highlights the importance of transparency, accountability, trust, costs, and privacy–utility trade-offs. Overall, this thesis demonstrates that federated learning should not be considered inherently private merely because data remain decentralized. By combining adversarial evaluation with personalized privacy-preserving mechanisms, it provides a more comprehensive understanding of privacy risks and safeguards in FL. The findings offer practical guidance for developing federated learning systems that are more privacy-aware, adaptable, and suitable for real-world deployment.