Not stated in source 2025 Needs review

Enhancing Privacy and Efficiency in Federated Learning: Quantization and Low-Rank Adaptation Approaches

Visit original source

Abstract

Federated Learning (FL) enables collaborative AI model training on decentralized data but faces challenges in ensuring privacy and operational efficiency, especially for large models over wireless networks. This thesis addresses these critical issues by enhancing both aspects in FL systems. Firstly, we provide a rigorous analysis of the interplay between model update quantization and Differential Privacy (DP). Using Rényi Differential Privacy (RDP), we formally demonstrate that lower quantization bit levels can enhance privacy guarantees when applied to Gaussian-noised updates. This theoretical finding, contrary to some prior notions, is empirically validated using Membership Inference Attacks (MIA), confirming that quantization can be a privacy-enhancing mechanism in addition to a communication-saving one. Secondly, we propose a novel framework for differentially private federated fine-tuning (FedFT) of Foundation Models using Low-Rank Adaptation (LoRA). This framework uniquely leverages inherent wireless channel noise as a natural source for DP, reducing the need for artificial noise injection. We analyze the convergence of this split FedFT-LoRA architecture and address the critical problem of noise amplification by proposing a modified LoRA update strategy where only one low-rank matrix is trained while another initialized as scaled orthogonal. Simulations show this approach achieves superior model accuracy under stringent privacy budgets compared to baseline methods. Collectively, this work offers new theoretical insights and practical solutions for deploying private and resource-efficient FL, particularly for large-scale models in wireless environments.

Keywords

Not available

Specialisations

Similar theses