Postural tremor of hands, a typical symptom of Parkinson's disease (PD), is assessed using the Movement Disorder Society-sponsored revision of the Unified PD Rating Scale (MDS-UPDRS). However, the manual assessment process is both subjective and time-consuming, underscoring the necessity for automated rating models in clinical practice. To accommodate large-scale applications, such models must effectively identify and mitigate the impact of confounding factors in complex clinical settings, ensuring stable and robust assessment performance. Therefore, we propose a causality-representation graph convolutional network (GCN) scheme for automated video-based assessment of postural tremor of hands. This scheme systematically eliminates the effects of confounding factors by extracting discriminative structure and representation from hand skeleton graphs, which are clinically significant and causally related to tremor assessment, ultimately achieving stable multiclass tremor scoring. Specifically, a causality-informed graph structure mining (CI-GSM) module was first proposed to automatically extract nodes with discriminative tremor features from skeletal graphs, construct hand skeleton graphs causally related to tremor assessment, and thereby suppress interference from irrelevant nodes. Afterward, a causality-representation enhancement (CRE) module was designed to improve the construction of graph representations that correlate strongly with tremor assessment, thereby further reducing noise during evaluation. Our method achieves an accuracy of 64.01% and an acceptable accuracy of 98.62% on a large clinical dataset, demonstrating satisfactory performance on independent test sets collected from multiple centers. In conclusion, our proposed approach offers a convenient and stable solution for objective assessment of PD-associated tremors and holds significant potential for large-scale applications in remote PD assessment.

Leg agility is a key indicator of bradykinesia, which in turn is a cardinal manifestation of Parkinson’s disease (PD). In fact, automated video assessment of the leg-agility task is critically required for improving the efficiency and objectivity of PD diagnosis. Therefore, we propose a causality-informed graph convolutional network to extract discriminative clinically-meaningful motion features from human skeletons in videos, finally achieving stable leg-agility 5-point scoring. The proposed scheme systematically mines causal features of each skeleton graph from graph node, structure, and representation levels. Specifically, we firstly developed a causality-informed node selection mechanism to mine the graph nodes representing the discriminative features, and thus identify nodes causally correlated to the clinical assessment aspects and suppress the interference from other nodes. Afterwards, a causality-informed structure generation mechanism was designed to generate a graph structure encoding the connections between the discriminative nodes, hence maintaining the discriminability of features associated with these causality-informed nodes. Finally, we employed a clinically-driven self-supervised learning scheme to embed clinical prior knowledge into the proposed model and hence boost the clinical significance of the causality-informed graph nodes, structures, and representations. The proposed method achieved a 71.11% accuracy and a 98.93% acceptable accuracy on a large clinical video dataset. Its effectiveness was also confirmed on an independent test set, and the obtained results exhibited interpretability from modeling and clinical perspectives. In conclusion, our method provides a highly stable scheme for objective video quantification of bradykinesia. Our source code will be released athttps://github.com/SJTUBME-QianLab/PD-CIGCN.

In the clinically widely used rating scale (MDS-UPDRS), the pronation-supination movement task of hands is required for assessment of bradykinesia, which is a typical clinical symptom of Parkinson’s disease (PD). Due to inter-rater variability in the task rating process, objective automated rating models are critically needed. Still, the performance of such models would be limited if prior knowledge of the clinical rating principles is not adequately accounted for. Therefore, we propose a clinically guided method which fully exploits the MDS-UPDRS rating principles to achieve consistent and accurate automated rating. First, a multi-scale framework which employs two graph convolutional networks (GCNs) as two streams is developed to extract transient and persistent features related to these rating principles. In particular, abnormal transient features are detected through a specialized multiple-instance-learning GCN. Moreover, the multiple-instance-learning GCN is equipped with an accumulation-aware ordered multiple-instance pooling module, which estimates sample-level severity by accounting for both the intra-instance intrinsic severity order and the inter-instance accumulation effects. Besides, an instance context encoding module is designed to combine the phase information in the pronation-supination movement cycles with instances’ motion features. This facilitates the differentiation between PD-induced halts and natural periodic halts. Our method demonstrated excellent performance on both a large clinical dataset and an additional independent test dataset. Our proposed scheme only requires consumer-level cameras, and therefore exhibits high potential for large-scale applications in PD telemedicine.

Freezing of gait (FoG) is a common disabling symptom of Parkinson's disease (PD). It is clinically characterized by sudden and transient walking interruptions for specific human body parts, and it presents the localization in time and space. Due to the difficulty in extracting global fine-grained features from lengthy videos, developing an automated five-point FoG scoring system is quite challenging. Therefore, we propose a novel video-based automated five-classification FoG assessment method with a causality-enhanced multiple-instance-learning graph convolutional network (GCN). This method involves developing a temporal segmentation GCN to segment each video into three motion stages for stage-level feature modeling, followed by a multiple-instance-learning framework to divide each stage into short clips for instance-level feature extraction. Subsequently, an uncertainty-driven multiple-instance-learning GCN is developed to capture spatial and temporal fine-grained features through GCN scheme and uncertainty learning, respectively, for acquiring global representations. Finally, a causality-enhanced graph generation strategy is proposed to exploit causal inference for mining and enhancing human structures causally related to clinical assessment, thereby extracting spatial causal features. Extensive experimental results demonstrate the excellent performance of the proposed method on five-classification FoG assessment with an accuracy of 62.72% and an acceptable accuracy of 91.32%, which is confirmed by independent testing. Additionally, it enables temporal and spatial localization of FoG events to a certain extent, facilitating reasonable clinical interpretations. In conclusion, our method provides a valuable tool for automated FoG assessment in PD, and the proposed causality-related component exhibits promising potential for extension to other general and medical fine-grained action recognition tasks.

Segmenting peripancreatic vessels in CT, including the superior mesenteric artery (SMA), the coeliac artery (CA), and the partial portal venous system (PPVS), is crucial for preoperative resectability analysis in pancreatic cancer. However, the clinical applicability of vessel segmentation methods is impeded by the low generalizability on multi-center data, mainly attributed to the wide variations in image appearance, namely the spurious correlation factor. Therefore, we propose a causal-invariance-driven generalizable segmentation model for peripancreatic vessels. It incorporates interventions at both image and feature levels to guide the model to capture causal information by enforcing consistency across datasets, thus enhancing the generalization performance. Specifically, firstly, a contrast-driven image intervention strategy is proposed to construct image-level interventions by generating images with various contrast-related appearances and seeking invariant causal features. Secondly, the feature intervention strategy is designed, where various patterns of feature bias across different centers are simulated to pursue invariant prediction. The proposed model achieved high DSC scores (79.69%, 82.62%, and 83.10%) for the three vessels on a cross-validation set containing 134 cases. Its generalizability was further confirmed on three independent test sets of 233 cases. Overall, the proposed method provides an accurate and generalizable segmentation model for peripancreatic vessels and offers a promising paradigm for increasing the generalizability of segmentation models from a causality perspective. Our source codes will be released at https://github.com/ SJTUBME-QianLab/PC_VesselSeg.

Rigidity is one of the common motor disorders in Parkinson's disease (PD), which lead to life quality deterioration. The widely-used rating-scale-based approach for rigidity assessment still depends on the availability of experienced neurologists and is limited by rating subjectivity. Given the recent successful applications of quantitative susceptibility mapping (QSM) in auxiliary PD diagnosis, automated assessment of PD rigidity can be essentially achieved through QSM analysis. However, a major challenge is the performance instability due to the confounding factors (e.g., noise and distribution shift) which conceal the truly-causal features. Therefore, we propose a causality-aware graph convolutional network (GCN) framework, where causal feature selection is combined with causal invariance to ensure that causality-informed model decisions are reached. Firstly, a GCN model that integrates causal feature selection is systematically constructed at three graph levels: node, structure, and representation. In this model, a causal diagram is learned to extract a subgraph with truly-causal information. Secondly, a non-causal perturbation strategy is developed along with an invariance constraint to ensure the stability of the assessment results under different distributions, and thus avoid spurious correlations caused by distribution shifts. The superiority of the proposed method is shown by extensive experiments and the clinical value is revealed by the direct relevance of selected brain regions to rigidity in PD. Besides, its extensibility is verified on other two tasks: PD bradykinesia and mental state for Alzheimer's disease. Overall, we provide a clinically-potential tool for automated and stable assessment of PD rigidity. Our source code will be available at https://github.com/SJTUBME-QianLab/Causality-Aware-Rigidity.

Pancreatic cancer is the emperor of all cancer maladies, mainly because there are no characteristic symptoms in the early stages, resulting in the absence of effective screening and early diagnosis methods in clinical practice. Non-contrast computerized tomography (CT) is widely used in routine check-ups and clinical examinations. Therefore, based on the accessibility of non-contrast CT, an automated early diagnosismethod for pancreatic cancer is proposed. Among this, we develop a novel causalitydriven graph neural network to solve the challenges of stability and generalization of early diagnosis, that is, the proposed method achieves stable performance for datasets from different hospitals, which highlights its clinical significance. Specifically, a multiple-instance-learning framework is designed to extract fine-grained pancreatic tumor features. Afterwards, to ensure the integrity and stability of the tumor features, we construct an adaptivemetric graph neural network that effectively encodes prior relationships of spatial proximity and feature similarity for multiple instances, and hence adaptively fuses the tumor features. Besides, a causal contrastivemechanism is developed to decouple the causality-driven and non-causal components of the discriminative features, suppress the non-causal ones, and hence improve the model stability and generalization. Extensive experiments demonstrated that the proposed method achieved the promising early diagnosis performance, and its stability and generalizability were independently verified on amulti-center dataset. Thus, the proposed method provides a valuable clinical tool for the early diagnosis of pancreatic cancer. Our source codes will be released at https://github.com/SJTUBME-QianLab/ CGNN-PC-Early-Diagnosis.

Abnormal posture is a common movement disorder in the progress of Parkinson's disease (PD), and this abnormality can increase the risk of falls or even disabilities. The conventional assessment approach depends on the judgment of well-trained experts via canonical scales. However, this approach requires extensive clinical expertise and is highly subjective. Considering the potential of quantitative susceptibility mapping (QSM) in PD diagnosis, this study explored the QSM-based method for the automated classification between PD patients with and without postural abnormalities. Nevertheless, a major challenge is that unstable non-causal features typically lead to less reliable performance. Therefore, we propose a causality-driven graph-convolutional-network framework based on multi-instance learning, where performance stability is enhanced through the invariant prediction principle and causal interventions. Specifically, we adopt an intervention strategy that combines a non-causal intervenor with causal prediction. A stability constraint is proposed to ensure robust integrated prediction under different interventions. Moreover, an intra-class homogeneity constraint is enforced for each individually-learned causality scoring module to promote the extraction of group-level general features, and hence achieve a balance between subject-specific and group-level features. The proposed method demonstrated promising performance through extensive experiments on a real clinical dataset. Also, the features extracted by our method coincide with those reported in previous medical studies on PD posture abnormalities. In general, our work provides a clinically-valuable approach for automated, objective, and reliable diagnosis of postural abnormalities in Parkinsonians. Our source code is publicly available at https://github.com/SJTUBME-QianLab/CausalGCN-PDPA.

The onset and progression of Parkinson’s disease (PD) gradually affect the patient’s motor functions and quality of life. The PD motor symptoms are usually assessed using the Movement Disorder Society-sponsored revision of the Unified Parkinson’s Disease Rating Scale (MDS-UPDRS). Automated MDS-UPDRS assessment has been recently required as an invaluable tool for PD diagnosis and telemedicine, especially with the recent novel coronavirus pandemic outbreak. This paper proposes a novel vision-based method for automated assessment of the arising-from-chair task, which is one of the key MDS-UPDRS components. The proposed method is based on a self-supervised metric learning scheme with a graph convolutional network (SSM-GCN). Specifically, for human skeleton sequences extracted from videos, a self-supervised intra-video quadruplet learning strategy is proposed to construct a metric learning formulation with prior knowledge, for improving the spatial-temporal representations. Afterwards, a vertex-specific convolution operation is designed to achieve effective aggregation of all skeletal joint features, where each joint or feature is weighted differently based on its relative factor of importance. Finally, a graph representation supervised mechanism is developed to maximize the potential consistency between the joint and bone information streams. Experimental results on a clinical dataset demonstrate the superiority of the proposed method over the existing sensor-based methods, with an accuracy of 70.60% and an acceptable accuracy of 98.65%. The analysis of discriminative spatial connections makes our predictions more clinically interpretable. This method can achieve reliable automated PD assessment using only easily-obtainable videos, thus providing an effective tool for real-time PD diagnosis or remote continuous monitoring.

One of the common motor symptoms of Parkinson’s disease (PD) is bradykinesia. Automated bradykinesia assessment is critically needed for helping neurologists achieve objective clinical diagnosis and hence provide timely and appropriate medical services. This need has become especially urgent after the outbreak of the coronavirus pandemic in late 2019. Currently, the main factor limiting the accurate assessment is the difficulty of mining the fine-grained discriminative motion features. Therefore, we propose a novel contrastive graph convolutional network for automated and objective toe-tapping assessment, which is one of the most important tests of lower-extremity bradykinesia. Specifically, based on joint sequences extracted from videos, a supervised contrastive learning strategy was followed to cluster together the features of each class, thereby enhancing the specificity of the learnt class-specific features. Subsequently, a multi-stream joint sparse learning mechanism was designed to eliminate potentially similar redundant features of joint position and motion, hence strengthening the discriminability of features extracted from different streams. Finally, a spatial-temporal interaction graph convolutional module was developed to explicitly model remote dependencies across time and space, and hence boost the mining of fine-grained motion features. Comprehensive experimental results demonstrate that this method achieved remarkable classification performance on a clinical video dataset, with an accuracy of 70.04% and an acceptable accuracy of 98.70%. These results obviously outperformed other existing sensor- and video-based methods. The proposed video-based scheme provides a reliable and objective tool for automated quantitative toe-tapping assessment, and is expected to be a viable method for remote medical assessment and diagnosis.

Automated assessment of patients with Parkinson's disease (PD) is urgently required in clinical practice to improve the diagnostic efficiency and objectivity and to remotely monitor the motor disorder symptoms and general health of these patients, especially in view of the travel restrictions due to the recent coronavirus epidemic. Gait motor disorder is one of the critical manifestations of PD, and automated assessment of gait is vital to realize automated assessment of PD patients. To this end, we propose a novel two-stream spatial-temporal attention graph convolutional network (2s-ST-AGCN) for video assessment of PD gait motor disorder. Specifically, the skeleton sequence of human body is extracted from videos to construct spatial-temporal graphs of joints and bones, and a two-stream spatial-temporal graph convolutional network is then built to simultaneously model the static spatial information and dynamic temporal variations. The multi-scale spatial-temporal attention-aware mechanism is also designed to effectively extract the discriminative spatial-temporal features. The deep supervision strategy is then embedded to minimize classification errors, thereby guiding the weight update process of the hidden layer to promote significant discriminative features. Besides, two model-driven terms are integrated into this deep learning framework to strengthen multi-scale similarity in the deep supervision and realize sparsification of discriminative features. Extensive experiments on the clinical video dataset show that the proposed model exhibits good performance with an accuracy of 65.66% and an acceptable accuracy of 98.90%, which is much better than that of the existing sensor- and vision-based methods for Parkinsonian gait assessment. Thus, the proposed method is potentially useful for assessing PD gait motor disorder in clinical practice.