Evaluating Federated Learning and Multimodal Data for Privacy-Aware Video-Based Fall Detection

Falls remain a growing major public health risk in midst of an aging global population,increasing the demand for reliable fall detection systems that can operate withinprivacy-sensitive healthcare environments. While federated learning has emerged as apromising approach for training healthcare machine learning applications without sharingsensitive data, existing research on multimodal federated learning for fall detection hasprimarily focused on wearable sensors or sensor-image combinations. In contrast,multimodal video-audio fall detection has largely been evaluated under centralisedtraining settings, leaving uncertainty regarding its effectiveness in distributed dataenvironments.This thesis investigates whether Federated Averaging can support privacy-awarevideo-based and video-audio multimodal fall detection while maintaining acceptabledetection performance. Unlike many vision-based approaches that rely on raw imagedata, the proposed framework uses human pose keypoints and bounding-box features asprivacy-aware visual representations, reducing exposure of identifiable information whilepreserving motion patterns relevant to fall detection. A multimodal extension combinesvisual and audio representations through feature-level fusion to evaluate whether audiocan improve robustness under degraded visual conditions.Experiments conducted on the Le2i dataset show that centralised video-based falldetection achieved an average accuracy of 93.6%, while multimodal detection achieved92.6%. Although audio provided limited benefits under normal visual conditions,multimodal fusion improved robustness when video data were affected by noise.Federated learning introduced performance trade-offs compared with centralised training;however, federated models achieved higher recall and maintained effective fall detectionperformance. Under federated training, the multimodal model achieved an averageaccuracy of 85.3%, outperforming both unimodal video and audio models.The findings demonstrate that privacy-aware fall detection based on skeletal andbounding-box representations can be effectively combined with federated learning, whilemultimodal fusion can improve robustness in challenging environments. This contributesnew evidence regarding the feasibility of deploying privacy-preserving video-based falldetection systems in healthcare settings where centralised collection of sensitive data isimpractical.