Abstract Parkinson’s Disease (PD) is associated with gait movement disorders, such as bradykinesia, stiffness, tremors and postural instability. Hence, a kinematic gait analysis for PD characterization is key to support diagnosis and to carry out an effective treatment planning. Nowadays, automatic classification and characterization strategies are based on deep learning representations, following supervised rules, and assuming large and stratified data. Nonetheless, such requirements are far from real clinical scenarios. Additionally, supervised rules may introduce bias into architectures from expert’s annotations. This work introduces a self-supervised generative representation to learn gait-motion-related patterns, under the pretext task of video reconstruction. Following an anomaly detection framework, the proposed architecture can avoid inter-class variance, learning hidden and complex kinematics locomotion relationships. In this study, the proposed model was trained and validated with an owner dataset (14 Parkinson and 23 control). Also, an external public dataset (16 Parkinson, 30 control, and 50 Knee-arthritis) was used only for testing, measuring the generalization capability of the method. During training, the method learns from control subjects, while Parkinson subjects are detected as anomaly samples. From owner dataset, the proposed approach achieves a ROC-AUC of 95% in classification task. Regarding the external dataset, the architecture evidence generalization capabilities, achieving a 75% of ROC-AUC (shapeness and homoscedasticity of 66.7%), without any additional training. The proposed model has remarkable performance in detecting gait parkinsonian patterns, recorded in markerless videos, even competitive results with classes non-observed during training.

Abstract The classification of clinically significant prostate cancer (csPCa) lesions remains one of the most important challenges in prostate cancer diagnosis. For this, multimodal convolutional neural networks (CNNs) have achieved outstanding results. Nevertheless, the data used in these studies may only partially represent the total burden of csPCa cases. Hence, it is necessary to design reliable models that perform well in limited data scenarios and involving information from different centers (multicentric). A deep Riemannian geometric learning architecture was introduced to capture the intermediate relationships between bi-parametric MRI (bp-MRI) deep representations coded from a 3D multimodal convolutional backbone and considering their geometry. For this, several multimodal bp-MRI fusion strategies were explored to assess their ability to classify csPCa lesions in scenarios where the percentage of available training data was progressively reduced and multicentric data were involved. The proposed method outperformed baseline CNN techniques with an AUC-ROC of 0.96. More remarkably, the method remained stable even only using 10% of the available training data. Additionally, considering multicentric information, this approach also demonstrates generalization ability by losing only 5.4% of the AUC testing data from different acquisition centers, compared to the 10.4% loss of the baseline method. A new deep learning-based method that improves generalization under scenarios with limited data translates to better support for clinicians in accurately classifying csPCa lesions on unseen data.

Parkinson’s disease (PD) is the second most common neurodegenerative disorder, mainly characterized by motor alterations. Despite multiple efforts, there is no definitive biomarker to diagnose, quantify, and characterize the disease early. Recently, abnormal fixational oculomotor patterns have emerged as a promising disease biomarker with high sensitivity, even at early stages. Nonetheless, the complex patterns and potential correlations with the disease remain largely unexplored, among others, because of the limitations of standard setups that only analyze coarse measures and poorly exploit the associated PD alterations. This work introduces a new strategy to represent, analyze and characterize fixational patterns from non-invasive video analysis, adjusting a geometric learning strategy. A deep Riemannian framework is proposed to discover potential oculomotor patterns aimed at withstanding data scarcity and geometrically interpreting the latent space. A convolutional representation is first built, then aggregated onto a symmetric positive definite matrix (SPD). The latter encodes second-order statistics of deep convolutional features and feeds a non-linear hierarchical architecture that processes SPD data by maintaining them into their Riemannian manifold. The complete representation discriminates between Parkinson and Healthy (Control) fixational observations, even at PD stages 2.5 and 3. Besides, the proposed geometrical representation exhibit capabilities to statistically differentiate observations among Parkinson’s stages. The developed tool demonstrates coherent results from explainability maps back-propagated from output probabilities.

Abstract More than 70 million people use at least one sign language (SL) as their main channel of communication. Nevertheless, the absence of effective mechanisms to translate massive information among sign, written and spoken languages is the main cause of a negligible inclusion of deaf people into society. Therefore, SL automatic recognition systems have widely proposed to support the characterisation of the sign structure. Today, natural and continuous SL recognition is an open research problem due to multiple spatio‐temporal shape variations, challenging visual sign characterisation, as well as the non‐linear correlation among signs to express a message. A compact sign is introduced to text architecture that explores motion as an alternative to support sign translation. Such characterisation results are robust to appearance variance with relative support to geometrical variations. The proposed representation focuses on the main spatio‐temporal regions to each corresponding word. The proposed architecture was evaluated in a built SL data set (LSCDv1) dedicated to motion study and also in the state‐of‐the‐art RWTH‐Phoenix. From the LSCDv1 data set, the best configuration reports a BLEU‐4 score of 63.04 in a testing set. Regarding RWTH‐Phoenix, the proposed strategy achieved a BLEU‐4 score in a test of 4.56, improving the results under similar reduced conditions.

This study presents a spatio‐temporal motion descriptor that is computed from a spatially‐constrained decomposition and applied to online classification and recognition of human activities. The method starts by computing a dense optical flow without explicit spatial regularisation. Potential human actions are detected at each frame as spatially consistent moving regions of interest (RoIs). Each of these RoIs is then sequentially partitioned to obtain a spatial representation of small overlapped subregions with different sizes. Each of these region parts is characterised by a set of flow orientation histograms. A particular RoI is then described along the time by a set of recursively calculated statistics that collect information from the temporal history of orientation histograms, to form the action descriptor. At any time, the whole descriptor can be extracted and labelled by a previously trained support vector machine. The method was evaluated using three different public datasets: (i) the ViSOR dataset was used for global classification obtaining an average accuracy of 95% and for recognition in long sequences, achieving an average per‐frame accuracy of 92.3%. (ii) The KTH dataset was used for global classification and (iii) the UT‐datasets were used for recognition task, obtaining an average accuracy of 80% (frame rate).