Enhancing Privacy and Efficiency in Federated Learning: Quantization and Low-Rank Adaptation Approaches
Abstract
Federated Learning (FL) enables collaborative AI model training on decentralized data but faces challenges in ensuring privacy and operational efficiency, especially for large models over wireless networks. This thesis addresses these critical issues by enhancing both aspects in FL systems. Firstly, we provide a rigorous analysis of the interplay between model update quantization and Differential Privacy (DP). Using Rényi Differential Privacy (RDP), we formally demonstrate that lower quantization bit levels can enhance privacy guarantees when applied to Gaussian-noised updates. This theoretical finding, contrary to some prior notions, is empirically validated using Membership Inference Attacks (MIA), confirming that quantization can be a privacy-enhancing mechanism in addition to a communication-saving one. Secondly, we propose a novel framework for differentially private federated fine-tuning (FedFT) of Foundation Models using Low-Rank Adaptation (LoRA). This framework uniquely leverages inherent wireless channel noise as a natural source for DP, reducing the need for artificial noise injection. We analyze the convergence of this split FedFT-LoRA architecture and address the critical problem of noise amplification by proposing a modified LoRA update strategy where only one low-rank matrix is trained while another initialized as scaled orthogonal. Simulations show this approach achieves superior model accuracy under stringent privacy budgets compared to baseline methods. Collectively, this work offers new theoretical insights and practical solutions for deploying private and resource-efficient FL, particularly for large-scale models in wireless environments.
Keywords
Not available
Specialisations
Similar theses
- Addressing data heterogeneity, scarcity, and training efficiency in privacy-preserving federated learning Chen, Haokun · Ludwig-Maximilians-Universität München · 2025
- A Comprehensive Framework for Targeted Transfer Learning: Privacy-Preserving, Secure, and Efficient Domain Adaptation Sayyed Farid Ahamed · Old Dominion University · 2026
- Federated Learning of Fuzzy Rule-based Systems: Advances in Privacy, Security and Interpretability Mattia Daole · University of Pisa · 2026
- Topology-dependent privacy risks in decentralized federated learning JOSÉ INÁCIO ANTUNES DE GOUVEIA · 2026
- Privacy-Preserving Federated Learning With Backdoor Resilience Yordanov Yordan · Karlsruhe Institute of Technology · 2026
- Designing for Privacy in Tabular Federated Learning Ivo Osterberg Nilsson ; Maximilian Birr Engvall · Jönköping University · 2026
