Alzheimer's Disease (AD) and related dementia are a growing global health challenge due to the aging population. In this paper, we present ADMarker, the first end-to-end system that integrates multi-modal sensors and new federated learning algorithms for detecting multidimensional AD digital biomarkers in natural living environments. ADMarker features a novel three-stage multi-modal federated learning architecture that can accurately detect digital biomarkers in a privacy-preserving manner. Our approach collectively addresses several major real-world challenges, such as limited data labels, data heterogeneity, and limited computing resources. We built a compact multi-modality hardware system and deployed it in a four-week clinical trial involving 91 elderly participants. The results indicate that ADMarker can accurately detect a comprehensive set of digital biomarkers with up to 93.8% accuracy and identify early AD with an average of 88.9% accuracy. ADMarker offers a new platform that can allow AD clinicians to characterize and track the complex correlation between multidimensional interpretable digital biomarkers, demographic factors of patients, and AD diagnosis in a longitudinal manner.

Federated learning in multi-modal systems faces challenges due to modality heterogeneity, where edge devices have different sensor setups. Labeling multi-modal data is labor-intensive and impractical, leading to the label scarcity issue on edge clients. This paper presents a novel semi-supervised federated learning framework to address these issues. It uses complementary data, like RGB images and depth sensors, with a pseudo-labeling algorithm to improve cross-modal learning. Applied to the human action recognition task, the framework outperforms baselines. It enables efficient federated learning, handling labeling difficulties and missing modalities, offering robust performance in real-world scenarios.