Non-intrusive load monitoring (NILM) enables appliance-level behaviour analysis by examining the aggregated electrical consumption signals. These techniques hold significant potential for applications ranging from electrical load management to remote human health monitoring. Despite its potential, NILM faces challenges in adapting to evolving appliance baselines, including the integration of new devices or the replacement of existing ones. These challenges aggravate when dealing with a large number of appliances, or even more if there are overlapping energy consumption profiles, thus reducing the effectiveness of load monitoring techniques. In real-world scenarios, the scarcity of labelled data further intensifies these issues, increasing the risk of overfitting. This limits the ability of NILM models to generalise and perform effectively on unseen data. To address these limitations, this work presents some methods for accurately classifying known appliances while identifying unknown ones by using features derived from electrical current signals. The framework includes a feature extraction stage that explores neural networks with both supervised and unsupervised learning techniques to derive latent representations. Additionally, the appliance distinction stage optimises data distribution for recognised known appliances and evaluates two distinct approaches (a supervised method and a semi-supervised one) for detecting unseen appliances. Experimental evaluations demonstrate promising results, achieving over 95% accuracy for the supervised feature extraction method and 83% for the unsupervised one in classifying known appliances, even under limited data conditions. Furthermore, both approaches performed well in detecting unseen appliances, with detection rates exceeding 90% for the supervised classification method and 70% for the semi-supervised method for certain categories. • A novel NILM approach combines classification with anomaly detection capabilities. • The analysis spans approaches from supervised to unsupervised learning techniques. • The proposals identify known and novel devices, enhancing the system’s adaptability. • Techniques are employed to minimise overfitting while maintaining robustness. • The study addresses challenges in limited-sample, multi-category NILM scenarios.

This work presents a novel proposal aimed at automating the assessment of human functional limitations. It represents an interdisciplinary effort to develop and clinically evaluate an automated system for objective functional assessment, and is being carried out within the EYEFUL research project. Functional evaluation is a complex task that requires a holistic approach, considering various components like cognition, motor skills, and executive functions. The EYEFUL project has created an intelligent space within a fully equipped apartment, which includes the deployment of advanced technology (cameras, microphones, and wearable devices), to capture data from subjects undergoing assessment. In this paper, we provide an extensive overview of the project's architecture, including the selection of Activities of Daily Living (ADLs), the recruitment of subjects, the validation methodology, and the experimental setup. We describe the specialized apartment and sensor hardware deployed for data collection, along with the initial machine learning systems for data analysis. Additionally, we analyze the existing state-of-the-art research in automatic functional performance evaluation and present our proposal. This encompasses ADL selection, database design, recording considerations, and an overview of technological modules to be implemented. We also discuss the current status of select modules, including quantitative and qualitative results and references to ongoing work. In conclusion, the EYEFUL project is pioneering automated functional assessment, integrating advanced technology and interdisciplinary collaboration. It aims to provide healthcare professionals with accurate, comprehensive, and clinically validated evaluations of human functionality during ADLs, addressing the complex nature of functional limitations.

Abstract This work proposes and compare two different approaches for real-time human action recognition (HAR) from raw depth video sequences. Both proposals are based on the convolutional long short-term memory unit, namely ConvLSTM, with differences in the architecture and the long-term learning. The former uses a video-length adaptive input data generator (stateless) whereas the latter explores thestatefulability of general recurrent neural networks but is applied in the particular case of HAR. This stateful property allows the model to accumulate discriminative patterns from previous frames without compromising computer memory. Furthermore, since the proposal uses only depth information, HAR is carried out preserving the privacy of people in the scene, since their identities can not be recognized. Both neural networks have been trained and tested using the large-scale NTU RGB+D dataset. Experimental results show that the proposed models achieve competitive recognition accuracies with lower computational cost compared with state-of-the-art methods and prove that, in the particular case of videos, the rarely-used stateful mode of recurrent neural networks significantly improves the accuracy obtained with the standard mode. The recognition accuracies obtained are 75.26% (CS) and 75.45% (CV) for the stateless model, with an average time consumption per video of 0.21 s, and 80.43% (CS) and 79.91%(CV) with 0.89 s for the stateful one.

Shape-from-Template (SfT) solves 3D vision from a single image and a deformable 3D object model, called a template. Concretely, SfT computes registration (the correspondence between the template and the image) and reconstruction (the depth in camera frame). It constrains the object deformation to quasi-isometry. Real-time and automatic SfT represents an open problem for complex objects and imaging conditions. We present four contributions to address core unmet challenges to realise SfT with a Deep Neural Network (DNN). First, we propose a novel DNN called DeepSfT, which encodes the template in its weights and hence copes with highly complex templates. Second, we propose a semi-supervised training procedure to exploit real data. This is a practical solution to overcome the render gap that occurs when training only with simulated data. Third, we propose a geometry adaptation module to deal with different cameras at training and inference. Fourth, we combine statistical learning with physics-based reasoning. DeepSfT runs automatically and in real-time and we show with numerous experiments and an ablation study that it consistently achieves a lower 3D error than previous work. It outperforms in generalisation and achieves great performance in terms of reconstruction and registration error with wide-baseline, occlusions, illumination changes, weak texture and blur.

Abstract This work describes an end-to-end approach for real-time human action recognition from raw depth image-sequences. The proposal is based on a 3D fully convolutional neural network, named 3DFCNN, which automatically encodes spatio-temporal patterns from raw depth sequences. The described 3D-CNN allows actions classification from the spatial and temporal encoded information of depth sequences. The use of depth data ensures that action recognition is carried out protecting people’s privacy, since their identities can not be recognized from these data. The proposed 3DFCNN has been optimized to reach a good performance in terms of accuracy while working in real-time. Then, it has been evaluated and compared with other state-of-the-art systems in three widely used public datasets with different characteristics, demonstrating that 3DFCNN outperforms all the non-DNN-based state-of-the-art methods with a maximum accuracy of 83.6% and obtains results that are comparable to the DNN-based approaches, while maintaining a much lower computational cost of 1.09 seconds, what significantly increases its applicability in real-world environments.

This work presents a new people re-identification method, using depth and intensity images, both of them captured with a single static camera, located in an overhead position. The proposed solution arises from the need that exists in many areas of application to carry out identification and re-identification processes to determine, for example, the time that people remain in a certain space, while fulfilling the requirement of preserving people’s privacy. This work is a novelty compared to other previous solutions, since the use of top-view images of depth and intensity allows obtaining information to perform the functions of identification and re-identification of people, maintaining their privacy and reducing occlusions. In the procedure of people identification and re-identification, only three frames of intensity and depth are used, so that the first one is obtained when the person enters the scene (frontal view), the second when it is in the central area of the scene (overhead view) and the third one when it leaves the scene (back view). In the implemented method only information from the head and shoulders of people with these three different perspectives is used. From these views three feature vectors are obtained in a simple way, two of them related to depth information and the other one related to intensity data. This increases the robustness of the method against lighting changes. The proposal has been evaluated in two different datasets and compared to other state-of-the-art proposal. The obtained results show a 96,7% success rate in re-identification, with sensors that use different operating principles, all of them obtaining depth and intensity information. Furthermore, the implemented method can work in real time on a PC, without using a GPU.

This paper presents an algorithm that automatically corrects the distortion caused by multipath interference (MpI) in depth measurements obtained with time of flight cameras (ToF cameras). A radiometric model that explains, under some mild simplifications, the working principle of a ToF camera including a model for MpI is proposed. Using this model we demonstrate that all the information needed for compensating the influence of MpI on the scene captured by the camera is self-contained in the measurements (depth and amplitude of infrared signal). We propose an iterative optimization method that, based on the measurements contaminated with MpI, gives depth correction for each pixel. Results are shown in artificially generated time of flight scenes using the radiometric model. In addition, the system has been validated in real scenes using a commercial ToF camera providing good results.