Hepatic steatosis, or non-alcoholic fatty liver disease (NAFLD), affects a significant portion of the global population and can lead to more severe liver conditions, including hepatic fibrosis. Early and accurate risk prediction of fibrosis is crucial for timely intervention. Traditional diagnostic methods are invasive and carry risks, while imaging techniques and blood-based biomarkers have limitations in routine general practice. This study presents a machine learning-based clinical decision support system designed to assess the risk of hepatic fibrosis in patients with NAFLD using routine laboratory tests. The framework is developed using electronic health record data collected over 15 years, initially encompassing 1,272,572 patients from general practice. After applying clinical selection criteria, two cohorts of 12,960 and 25,478 patients were used for model development and evaluation. The proposed approach provides a robust foundation for monitoring fibrosis risk by implementing a novel screening method, which preprocesses predictors by leveraging well-established clinical indicators (e.g., hepatic steatosis index, fibrosis-4 index), alongside a selected minimal number of predictors, making it practical and cost-effective for widespread clinical use. The study’s findings indicate promising results for screening and monitoring fibrosis risk in NAFLD patients, achieving the best AUC of 92.97%, PRAUC of 75.44%, and Sensitivity of 79.63%.

The fashion industry is undergoing a digital transformation, driven by growing demands for sustainability, personalization and immersive experiences. In this paper, we present Made-In (Multimodal and Collaborative Artificial Intelligence for the Design of Inclusive and Sustainable Fashion): an immersive, human-in-the-loop analytics system designed to support fashion professionals in exploring, comparing and contextualizing product data across digital and social platforms. Unlike generative or simulation-based approaches, Made-In provides creative decision support by aggregating real-world data from luxury brand websites and social media. This enables designers and merchandisers to make informed, context-aware choices. The system comprises three core modules: a 3D configurator for visualizing product assortments; a collection grid interface for the comparative analysis of e-commerce data; and a social media trend detector based on deep learning pipelines for image classification, object detection and color clustering. Two curated datasets, one derived from Instagram and the other from fashion e-tailers, provide the system with analytics. A user study with domain experts confirms the platform’s usability and relevance for trend forecasting, sustainability evaluation and visual merchandising strategy. The results demonstrate that Made-In effectively bridges the gap between data analytics and human creativity in fashion, offering a scalable solution that aligns with EU goals for digital sustainability and inclusivity. • Immersive AI system that supports sustainable digital fashion exploration. • Real-time trend detection from Instagram enables geo-localized style insights. • Interactive 3D and collection grids enhance visual merchandising decisions. • AI modules extract product data, dominant colors, and sustainability tags. • Usability study confirms system effectiveness for designers and retailers.

Abstract In an age where consumer preferences are as diverse as they are dynamic, the ability to offer personalized fashion recommendations at scale remains a significant challenge for retailers. Consumers seek a shopping experience that not only understands their unique style preferences but also dynamically adapts to their evolving tastes. The fashion industry is at a crossroads, facing increasing consumer demand for personalization, sustainability and transparency in a rapidly evolving digital marketplace. Traditional retail practices, while rich in tradition and artistry, often struggle to up-to-date with the rapidly, ethically-conscious and technology-driven expectations of today’s consumers. “OutfitAI” is designed to address these challenges by leveraging the power of deep learning to revolutionize the fashion retail experience. By automating the process of background removal in fashion images, using advanced algorithms for personalized product matching, and integrating sustainability filters into the product discovery process, OutfitAI aims to deliver a shopping experience that is not only personalized and engaging, but also aligned with the ethical and environmental values of the contemporary consumer. Unlike existing solutions, OutfitAI uses state-of-the-art semantic segmentation for precise background removal, enabling detailed feature extraction from fashion images. This process enables accurate matching of user-uploaded images with similar fashion items from an extensive database of eco-friendly and ethically produced products sourced from leading e-tailers. Setting itself apart from the current state of the art, OutfitAI places a strong emphasis on ethical data use and privacy, implementing robust measures to ensure user privacy and transparency. It also pioneers the integration of sustainability into the digital fashion discovery process, promoting responsible consumption patterns among users. Through a comprehensive system architecture that combines technical innovation with a commitment to ethics and sustainability, OutfitAI not only addresses the technological needs of the fashion retail industry, but also responds to the growing demand for more responsible and transparent consumer technologies.

Real-world classification problems may disclose different hierarchical levels where the categories are displayed in an ordinal structure. However, no specific deep learning (DL) models simultaneously learn hierarchical and ordinal constraints while improving generalization performance. To fill this gap, we propose the introduction of two novel ordinal-hierarchical DL methodologies, namely, the hierarchical cumulative link model (HCLM) and hierarchical-ordinal binary decomposition (HOBD), which are able to model the ordinal structure within different hierarchical levels of the labels. In particular, we decompose the hierarchical-ordinal problem into local and global graph paths that may encode an ordinal constraint for each hierarchical level. Thus, we frame this problem as simultaneously minimizing global and local losses. Furthermore, the ordinal constraints are set by two approaches [ordinal binary decomposition (OBD) and cumulative link model (CLM)] within each global and local function. The effectiveness of the proposed approach is measured on four real-use case datasets concerning industrial, biomedical, computer vision, and financial domains. The extracted results demonstrate a statistically significant improvement to state-of-the-art nominal, ordinal, and hierarchical approaches.

The convergence of Artificial Intelligence (AI), Computer Vision (CV), Computer Graphics (CG), and Extended Reality (XR) is driving innovation in immersive environments. A key challenge in these environments is the creation of personalized 3D assets, traditionally achieved through manual modeling, a time-consuming process that often fails to meet individual user needs. More recently, Generative AI (GenAI) has emerged as a promising solution for automated, context-aware content generation. In this paper, we present MineVRA (Multimodal generative artificial iNtelligence for contExt-aware Virtual Reality Assets), a novel Human-In-The-Loop (HITL) XR framework that integrates GenAI to facilitate coherent and adaptive 3D content generation in immersive scenarios. To evaluate the effectiveness of this approach, we conducted a comparative user study analyzing the performance and user satisfaction of GenAI-generated 3D objects compared to those generated by Sketchfab in different immersive contexts. The results suggest that GenAI can significantly complement traditional 3D asset libraries, with valuable design implications for the development of human-centered XR environments.

Artificial Intelligence (AI) techniques are becoming more and more sophisticated showing the potential to deeply understand and predict consumer behaviour in a way to boost the retail sector; however, retail-sensitive considerations underpinning their deployment have been poorly explored to date. This paper explores the application of AI technologies in the retail sector, focusing on their potential to enhance decision-making processes by preventing major ethical risks inherent to them, such as the propagation of bias and systems’ lack of explainability. Drawing on recent literature on AI ethics, this study proposes a methodological path for the design and the development of trustworthy, unbiased, and more explainable AI systems in the retail sector. Such framework grounds on European (EU) AI ethics principles and addresses the specific nuances of retail applications. To do this, we first examine the VRAI framework, a deep learning model used to analyse shopper interactions, people counting and re-identification, to highlight the critical need for transparency and fairness in AI operations. Second, the paper proposes actionable strategies for integrating high-level ethical guidelines into practical settings, and particularly, to mitigate biases leading to unfair outcomes in AI systems and improve their explainability. By doing so, the paper aims to show the key added value of embedding AI ethics requirements into AI practices and computer vision technology to truly promote technically and ethically robust AI in the retail domain. • Establishment of AI ethics guidelines for the retail domain. • Design of a detailed framework for AI ethics. • Evaluation of the VRAI framework on new explainability metrics. • Recommendations for AI transparency improvements. • Integration of ethics into the design of AI systems.

Shelf monitoring plays a key role in optimizing retail shelf layout, enhancing the customer shopping experience and maximizing profit margins. The process of automating shelf audit involves the detection, localization and recognition of objects on store shelves, including diverse products with varying attributes in unconstrained environments. This facilitates the assessment of planogram compliance. Accurate product localization within shelves requires the identification of specific shelf rows. To address the current technological challenges, we introduce “Shelf Management”, a deep learning-based system that is carefully tailored to redesign shelf monitoring practices. Our system can navigate the complexities of shelf monitoring by using advanced deep learning techniques and object detection and recognition models. In addition, a complex semantic module enhances the accuracy of detecting and assigning products to their designated shelf rows and locations. In particular, we recognize the lack of finely annotated datasets at the SKU level. As a contribution to the field, we provide annotations for two novel datasets: SHARD (SHelf mAnagement Row Dataset) and SHAPE (SHelf mAnagement Product dataset). These datasets not only provide valuable resources, but also serve as benchmarks for further research in the field of retail. A complete pipeline is designed using a RetinaNet architecture for object detection with 0.752 mAP, followed by a Deep Hough transform to detect shelf rows as semantic lines with an F1 score of 97%, and a product recognition step using a MobileNetV3 architecture trained with triplet loss and used as a feature extractor together with FAISS for fast image retrieval with an accuracy of 93% on top-1 recognition. Localization is achieved using a deterministic approach based on product detection and shelf row detection. Source code and datasets are available at https://github.com/rokopi-byte/shelf_management.

Abstract The early detection of handguns and knives from surveillance videos is crucial to enhance people’s safety. Despite the increasing development of Deep Learning (DL) methods for general object detection, weapon detection from surveillance videos still presents open challenges. Among these, the most significant are: (i) the very small size of the weapons with respect to the camera field of view and (ii) the need of a real-time feedback, even when using low-cost edge devices for computation. Complex and recently-developed DL architectures could mitigate the former challenge but do not satisfy the latter one. To tackle such limitation, the proposed work addresses the weapon-detection task from an edge perspective. A double-step DL approach was developed and evaluated against other state-of-the-art methods on a custom indoor surveillance dataset. The approach is based on a first Convolutional Neural Network (CNN) for people detection which guides a second CNN to identify handguns and knives. To evaluate the performance in a real-world indoor environment, the approach was deployed on a NVIDIA Jetson Nano edge device which was connected to an IP camera. The system achieved near real-time performance without relying on expensive hardware. The results in terms of both COCO Average Precision (AP = 79.30) and Frames per Second (FPS = 5.10) on the low-power NVIDIA Jetson Nano pointed out the goodness of the proposed approach compared with the others, encouraging the spread of automated video surveillance systems affordable to everyone.

Abstract Public space is usually conceived as where people live, perceive, and interact with other people. The environment affects people in several different ways as well. The impact of environmental problems on humans is significant, affecting all human activities, including health and socio-economic development. Thus, there is a need to rethink how space is used. Dealing with the important needs raised by climate emergency, pandemic and digitization, the contributions of this paper consist in the creation of opportunities for developing generative approaches to space design and utilization. It is proposed GREEN PATH, an intelligent expert system for space planning. GREEN PATH uses human trajectories and deep learning methods to analyse and understand human behaviour for offering insights to layout designers. In particular, a Generative Adversarial Imitation Learning (GAIL) framework hybridised with classical reinforcement learning methods is proposed. An example of the classical reinforcement learning method used is continuous penalties, which allow us to model the shape of the trajectories and insert a bias, which is necessary for the generation, into the training. The structure of the framework and the formalisation of the problem to be solved allow for the evaluation of the results in terms of generation and prediction. The use case is a chosen retail domain that will serve as a demonstrator for optimising the layout environment and improving the shopping experience. Experiments were assessed on shoppers’ trajectories obtained from four different stores, considering two years.

Abstract Nowadays, the volume of the multimedia heterogeneous evidence presented for digital forensic analysis has significantly increased, thus requiring the application of big data technologies, cloud-based forensics services, as well as Machine Learning (ML) techniques. In digital forensics domain, ML algorithms have been applied for cybercrime investigation such as child abuse investigations, malware classification, and image forensics. This paper addresses this issues and deals with forensic analysis of digital images and videos. In particular, this work aims at proposing a multimedia classification tool with a parallel software architecture for a fast inspection, which is easy to use (to be used by officers during a search), requires limited hardware resources and it is built on an open-source software to limit its costs. Moreover, this tool must be able to quickly inspect multiple devices at a time. When positives are found in a device, such device will be seized for a deeper analysis later in the lab. It will not be seized otherwise, reducing the inconvenience for the suspect as well as the time required for the next analysis phase. As a case study, we focus on the identification of child pornography images. Experimental results show that the proposed architecture is capable of guaranteeing a high recall, a fast process and high performances in real scenarios.

Abstract Nowadays, decision support systems (DSSs) are widely used in several application domains, from industrial to healthcare and medicine fields. Concerning the industrial scenario, we propose a DSS oriented to the aesthetic quality control (AQC) task, which has quickly established itself as one of the most crucial challenges of Industry 4.0. Taking into account the increasing amount of data in this domain, the application of machine learning (ML) and deep learning (DL) techniques offers great opportunities to automatize the overall AQC process. State-of-the-art is mainly oriented to approach this problem with a nominal DL classification method which does not exploit the ordinal structure of the AQC task, thus not penalizing the error among distant AQC classes (which is a relevant aspect for the real use case). The paper introduces a DL ordinal methodology for the AQC classification. Differently from other deep ordinal methods, we combined the standard categorical cross-entropy with the cumulative link model and we imposed the ordinal constraint via the thresholds and slope parameters. Experimental results were performed for solving an AQC task on a novel image dataset originated from a specific company’s demand (i.e., aesthetic assessment of wooden stocks). We demonstrated how the proposed methodology is able to reduce misclassification errors (up to 0.937 quadratic weight kappa loss) among distant classes while overcoming other state-of-the-art deep ordinal models and reducing the bias factor related to the item geometry. The proposed DL approach was integrated as the main core of a DSS supported by Internet of Things (IoT) architecture that can support the human operator by reducing up to 90% the time needed for the qualitative analysis carried out manually in this specific domain.

Continuous evaluation of preterm infants' spontaneous motility proved to be a decisive tool for timely diagnosing the presence of neurodevelopmental disorders. Automatic infants' limbs pose estimation is a powerful ally to support clinicians in infant's monitoring. This work proposes an end-to-end pipeline for limb-pose estimation based on a region-based convolutional neural network, named Mask R-CNN. The framework was validated on a custom dataset of 6000 depth images from 30 videos of 19 preterm infants acquired in a neonatal intensive care unit during the actual clinical practice. Leave-one-infant-out cross-validation was performed to evaluate the framework performance. Results for joints' detection showed a mean average precision equal to 0.9 with a standard deviation of 0.2. For limb-pose estimation, median root mean square error [pixel] was equal to 6.8 (right arm), 6.7 (left arm), 6.5 (right leg), 6.5 (left leg). The interquartile ranges [pixels] were 1.1, 1.2, 0.6, 1.2 for each limb, respectively. This end - to-end framework represents a step toward embedded monitoring solutions for on-the-edge computation.

BACKGROUND AND OBJECTIVES: During Twin-to-Twin Transfusion Syndrome (TTTS), abnormal vascular anastomoses in the monochorionic placenta can produce uneven blood flow between the fetuses. In the current practice, this syndrome is surgically treated by closing the abnormal connections using laser ablation. Surgeons commonly use the inter-fetal membrane as a reference. Limited field of view, low fetoscopic image quality and high inter-subject variability make the membrane identification a challenging task. However, currently available tools are not optimal for automatic membrane segmentation in fetoscopic videos, due to membrane texture homogeneity and high illumination variability. METHODS: To tackle these challenges, we present a new deep-learning framework for inter-fetal membrane segmentation on in-vivo fetoscopic videos. The framework enhances existing architectures by (i) encoding a novel (instance-normalized) dense block, invariant to illumination changes, that extracts spatio-temporal features to enforce pixel connectivity in time, and (ii) relying on an adversarial training, which constrains macro appearance. RESULTS: We performed a comprehensive validation using 20 different videos (2000 frames) from 20 different surgeries, achieving a mean Dice Similarity Coefficient of 0.8780±0.1383. CONCLUSIONS: The proposed framework has great potential to positively impact the actual surgical practice for TTTS treatment, allowing the implementation of surgical guidance systems that can enhance context awareness and potentially lower the duration of the surgeries.

Kidney Disease (KD) may hide complex causes and is associated with a tremendous socio-economic impact. Timely identification and management from the first level of medical care represent the most effective strategy to address the growing global burden sustainably. Clinical practice guidelines suggest utilizing estimated Glomerular Filtration Rate (eGFR) for routine evaluation within a screening purpose. Accordingly, the analysis of Electronic Health Records (EHRs) using Machine Learning techniques offers great opportunities to monitor and predict the eGFR trend over time. This paper aims to propose a novel Semi-Supervised Multi-Task Learning (SS-MTL) approach for predicting short-term KD evolution on multiple General Practitioners' EHR data. We demonstrated that the SS-MTL approach can (i) capture the eGFR temporal evolution by imposing a temporal relatedness between consecutive time windows and (ii) exploit useful information from unlabeled patients when labeled patients are less numerous with a gain of up to 4.1% in terms of Recall. This situation reflects the real-case scenario, where available labeled samples are limited, but those unlabeled much more abundant. The SS-MTL approach, also given the high level of interpretability, might be the ideal candidate in general practice to get integrated within a decision support system for KD screening purposes.

Objective: In the last years, Robot-assisted partial nephrectomy (RAPN) is establishing as elected treatment for renal cell carcinoma (RCC). Reduced field of view, field occlusions by surgical tools, and reduced maneuverability may potentially cause accidents, such as unwanted vessel resection with consequent bleeding. Surgical Data Science (SDS) can provide effective context-aware tools for supporting surgeons. However, currently no tools have been exploited for automatic vessels segmentation from nephrectomy laparoscopic videos. Herein, we propose a new approach based on adversarial Fully Convolutional Neural Networks (FCNNs) to kidney vessel segmentation from nephrectomy laparoscopic vision. Methods: The proposed approach enhances existing segmentation framework by (i) encoding 3D kernels for spatio-temporal features extraction to enforce pixel connectivity in time, and (ii) perform training in adversarial fashion, which constrains vessels shape. Results: We performed a preliminary study using 8 different RAPN videos (1871 frames), the first in the field, achieving a median Dice Similarity Coefficient of 71.76%. Conclusions: Results showed that the proposed approach could be a valuable solution with a view to assist surgeon during RAPN.

The development of so-called soft-biometrics aims at providing information related to the physical and behavioural characteristics of a person. This paper focuses on body weight estimation based on the observation from a top-view RGB-D camera. In fact, the capability to estimate the weight of a person can be of help in many different applications, from health-related scenarios, to business intelligence and retail analytics. To deal with this issue, a TVWE (Top-View Weight Estimation) framework is proposed with the aim of predicting the weight. The approach relies on the adoption of Deep Neural Networks (DNNs) that have been trained on depth data. Each network has also been modified in their top section to replace classification with prediction inference. The performance of five state-of-art DNNs have been compared, namely VGG16, ResNet, Inception, DenseNet and Efficient-Net. In addition, a convolutional auto-encoder has also been included for completeness. Considering the limited literature in this domain, the TVWE framework has been evaluated on a new publicly available dataset: “VRAI Weight estimation Dataset”, which also collects, for each subject, labels related to weight, gender, and height. The experimental results have demonstrated that the proposed methods are suitable for this task, bringing different and significant insights for the application of the solution in different domains.

In the field of reproductive technology, the biochemical composition of female gametes has been successfully investigated with the use of vibrational spectroscopy. Currently, in assistive reproductive technology (ART), there are no shared criteria for the choice of oocyte, and automatic classification methods for the best quality oocytes have not yet been applied. In this paper, considering the lack of criteria in Assisted Reproductive Technology (ART), we use Machine Learning (ML) techniques to predict oocyte quality for a successful pregnancy. To improve the chances of successful implantation and minimize any complications during the pregnancy, Fourier transform infrared microspectroscopy (FTIRM) analysis has been applied on granulosa cells (GCs) collected along with the oocytes during oocyte aspiration, as it is routinely done in ART, and specific spectral biomarkers were selected by multivariate statistical analysis. A proprietary biological reference dataset (BRD) was successfully collected to predict the best oocyte for a successful pregnancy. Personal health information are stored, maintained and backed up using a cloud computing service. Using a user-friendly interface, the user will evaluate whether or not the selected oocyte will have a positive result. This interface includes a dashboard for retrospective analysis, reporting, real-time processing, and statistical analysis. The experimental results are promising and confirm the efficiency of the method in terms of classification metrics: precision, recall, and F1-score (F1) measures.

The prediction of the risk profile related to the cardiopathy complication is a core research task that could support clinical decision making. However, the design and implementation of a clinical decision support system based on Electronic Health Record (EHR) temporal data comprise of several challenges. Several single task learning approaches consider the prediction of the risk profile related to a specific diabetes complication (i.e., cardiopathy) independent from other complications. Accordingly, the state-of-the-art multi-task learning (MTL) model encapsulates only the temporal relatedness among the EHR data. However, this assumption might be restricted in the clinical scenario where both spatio-temporal constraints should be taken into account. The aim of this study is the proposal of two different MTL procedures, called spatio-temporal lasso (STL-MTL) and spatio-temporal group lasso (STGL-MTL), which encode the spatio-temporal relatedness using a regularization term and a graph-based approach (i.e., encoding the task relatedness using the structure matrix). Experimental results on a real-world EHR dataset demonstrate the robust performance and the interpretability of the proposed approach.

Monitoring children with Autism Spectrum Dis-order (ASD) during the execution of the Applied Behaviour Analysis (ABA) program is crucial to assess the progresses while performing actions. Despite its importance, this monitoring procedure still relies on ABA operators’ visual observation and manual annotation of the significant events. In this work a deep learning (DL) based approach has been proposed to evaluate the autonomy of children with ASD while performing the hand-washing task. The goal of the algorithm is the automatic detection of RGB frames in which the ASD child washes his/her hands autonomously (no-aid frames) or is supported by the operator (aid frames). The proposed approach relies on a pre-trained VGG16 convolutional network (CNN) modified to fulfill the binary classification task. The performance of the fine-tuned VGG16 was compared against that of other CNN architectures. The fine-tuned VGG16 achieved the best performance with a recall of 0.92 and 0.89 for the no-aid and aid class, respectively. These results prompt the possibility of translating the presented methodology into the actual monitoring practice. The integration of the presented tool with other computer-aided monitoring systems into a single framework, will provide fully support to ABA operators during the therapy session.

Abstract In retail environments, understanding how shoppers move about in a store’s spaces and interact with products is very valuable. While the retail environment has several favourable characteristics that support computer vision, such as reasonable lighting, the large number and diversity of products sold, as well as the potential ambiguity of shoppers’ movements, mean that accurately measuring shopper behaviour is still challenging. Over the past years, machine-learning and feature-based tools for people counting as well as interactions analytic and re-identification were developed with the aim of learning shopper skills based on occlusion-free RGB-D cameras in a top-view configuration. However, after moving into the era of multimedia big data, machine-learning approaches evolved into deep learning approaches, which are a more powerful and efficient way of dealing with the complexities of human behaviour. In this paper, a novel VRAI deep learning application that uses three convolutional neural networks to count the number of people passing or stopping in the camera area, perform top-view re-identification and measure shopper–shelf interactions from a single RGB-D video flow with near real-time performances has been introduced. The framework is evaluated on the following three new datasets that are publicly available: TVHeads for people counting, HaDa for shopper–shelf interactions and TVPR2 for people re-identification. The experimental results show that the proposed methods significantly outperform all competitive state-of-the-art methods (accuracy of 99.5% on people counting, 92.6% on interaction classification and 74.5% on re-id), bringing to different and significative insights for implicit and extensive shopper behaviour analysis for marketing applications.

The diagnosis of type 2 diabetes (T2D) at an early stage has a key role for an adequate T2D integrated management system and patient's follow-up. Recent years have witnessed an increasing amount of available electronic health record (EHR) data and machine learning (ML) techniques have been considerably evolving. However, managing and modeling this amount of information may lead to several challenges, such as overfitting, model interpretability, and computational cost. Starting from these motivations, we introduced an ML method called sparse balanced support vector machine (SB-SVM) for discovering T2D in a novel collected EHR dataset (named Federazione Italiana Medici di Medicina Generale dataset). In particular, among all the EHR features related to exemptions, examination, and drug prescriptions, we have selected only those collected before T2D diagnosis from an uniform age group of subjects. We demonstrated the reliability of the introduced approach with respect to other ML and deep learning approaches widely employed in the state-of-the-art for solving this task. Results evidence that the SB-SVM overcomes the other state-of-the-art competitors providing the best compromise between predictive performance and computation time. Additionally, the induced sparsity allows to increase the model interpretability, while implicitly managing high-dimensional data and the usual unbalanced class distribution.

Preterm infants' limb-pose estimation is a crucial but challenging task, which may improve patients' care and facilitate clinicians in infant's movements monitoring. Work in the literature either provides approaches to whole-body segmentation and tracking, which, however, has poor clinical value, or retrieve a posteriori limb pose from limb segmentation, increasing computational costs and introducing inaccuracy sources. In this paper, we address the problem of limb-pose estimation under a different point of view. We proposed a 2D fully-convolutional neural network for roughly detecting limb joints and joint connections, followed by a regression convolutional neural network for accurate joint and joint-connection position estimation. Joints from the same limb are then connected with a maximum bipartite matching approach. Our analysis does not require any prior modeling of infants' body structure, neither any manual interventions. For developing and testing the proposed approach, we built a dataset of four videos (video length = 90 s) recorded with a depth sensor in a neonatal intensive care unit (NICU) during the actual clinical practice, achieving median root mean square distance [pixels] of 10.790 (right arm), 10.542 (left arm), 8.294 (right leg), 11.270 (left leg) with respect to the ground-truth limb pose. The idea of estimating limb pose directly from depth images may represent a future paradigm for addressing the problem of preterm-infants' movement monitoring and offer all possible support to clinicians in NICUs.

Nowadays the inspection of products at the end of line represents a critical phase. At this stage, it is necessary to look for defects in order to prevent the quality check to fail, and to provide information for improving the production as well. This task can be performed by using several sensing technologies, and the contactless optical inspection plays a key role. In this regard, the use of advanced robotic manipulators offers the capability to change the viewpoint of a given object and to inspect its multiple faces. We propose an approach that combines the use of photometric stereo to derive a 3D model of objects, empowered by the super-resolution that is applied on the original dataset (upstream) or on the normal images (downstream) in order to increase the quality of the final 3D model. The vision system is mounted on a robotic manipulator, able to grasp and change the viewpoint, thus offering a more complete view of the object to be inspected. The obtained results show that the developed solution increases the quality of the derived 3D models used for inspection tasks on different faces of the objects; this is achieved by using the manipulation ability offered by the adopted robotic platform.

Augmented Reality (AR) offers the possibility to enrich the real world with digital mediated content, increasing in this way the quality of many everyday experiences. While in some research areas such as cultural heritage, tourism, or medicine there is a strong technological investment, AR for game purposes struggles to become a widespread commercial application. In this article, a novel framework for AR kid games is proposed, already developed by the authors for other AR applications such as Cultural Heritage and Arts. In particular, the framework includes different layers such as the development of a series of AR kid puzzle games in an intermediate structure which can be used as a standard for different applications development, the development of a smart configuration tool, together with general guidelines and long-life usage tests and metrics. The proposed application is designed for augmenting the puzzle experience, but can be easily extended to other AR gaming applications. Once the user has assembled the real puzzle, AR functionality within the mobile application can be unlocked, bringing to life puzzle characters, creating a seamless game that merges AR interactions with the puzzle reality. The main goals and benefits of this framework can be seen in the development of a novel set of AR tests and metrics in the pre-release phase (in order to help the commercial launch and developers), and in the release phase by introducing the measures for long-life app optimization, usage tests and hint on final users together with a measure to design policy, providing a method for automatic testing of quality and popularity improvements. Moreover, smart configuration tools, as part of the general framework, enabling multi-app and eventually also multi-user development, have been proposed, facilitating the serialization of the applications. Results were obtained from a large-scale user test with about 4 million users on a set of eight gaming applications, providing the scientific community a workflow for implicit quantitative analysis in AR gaming. Different data analytics developed on the data collected by the framework prove that the proposed approach is affordable and reliable for long-life testing and optimization.

Detecting and tracking people is a challenging task in a persistent crowded environment (i.e. retail, airport, station, etc.) for human behaviour analysis of security purposes. This paper introduces an approach to track and detect people in cases of heavy occlusions based on CNNs for semantic segmentation using top-view depth visual data. The purpose is the design of a novel U-Net architecture, U-Net3, that has been modified compared to the previous ones at the end of each layer. In particular, a batch normalization is added after the first ReLU activation function and after each max-pooling and up-sampling functions. The approach was applied and tested on a new and public available dataset, TVHeads Dataset, consisting of depth images of people recorded from an RGB-D camera installed in top-view configuration. Our variant outperforms baseline architectures while remaining computationally efficient at inference time. Results show high accuracy, demonstrating the effectiveness and suitability of our approach.

Ambient assisted living and intelligent transportation systems are becoming strongly coupled. There is the necessity of improving the quality of life by developing inclusive mobility solutions for impaired people. In this paper, we focus on a monocular vision-based system to assist people during walking, jogging, and running in outdoor environments. The impaired user is guided along a path represented by a lane or line on a dedicated runway. We developed a set of image processing algorithms to extract lines/lanes to follow. The embedded system is based on a small camera and a board that is responsible for processing the images and communicating with the developed haptic device. The haptic device is formed by a set of two gloves equipped with vibration motors that drive the user to the right direction. The vibration sequences are generated according to a robotic-like controller, considering the user as a two wheel steering robot, where the rotational and translation velocity can be controlled. The results obtained show that the overall system is able to detect the right path and to provide the right stimuli to the user, by means of the gloves, up to a speed over 10 km/h.

Automated recognition of Activities of Daily Living allows to identify possible health problems and apply corrective strategies in Ambient Assisted Living (AAL). Activities of Daily Living analysis can provide very useful information for elder care and long-term care services. This paper presents an automated RGB-D video analysis system that recognises human ADLs activities, related to classical daily actions. The main goal is to predict the probability of an analysed subject action. Thus, abnormal behaviour can be detected. The activity detection and recognition is performed using an affordable RGB-D camera. Human activities, despite their unstructured nature, tend to have a natural hierarchical structure; for instance, generally making a coffee involves a three-step process of turning on the coffee machine, putting sugar in cup and opening the fridge for milk. Action sequence recognition is then handled using a discriminative Hidden Markov Model (HMM). RADiaL, a dataset with RGB-D images and 3D position of each person for training as well as evaluating the HMM, has been built and made publicly available.

New smart objects to improve the quality of life in the ambient assisted living (AAL) scenario are capturing the interest of researchers and companies. In particular, novel assistive technologies are being developed to make accessible street navigation to impaired people. The solution that we propose in this new application domain of intelligent transportation systems is a framework for a safe point-to-point navigation, owing to high-detailed road graphs, including sidewalks, crosswalks, and generic “obstacles.” The system is based on a low-cost modular sensor box (embedded hardware) interfaced with a mobile/phone application that acts as an intelligent navigator. The main novelty is the capability to sense the surrounding area while being able to perform a fast path replanning, owing to a real-time link to a remote server, if an obstacle is detected. The sensing is performed using different sensors, such as ultrasound, lidar, and a 77-GHz mid-range automotive radar (absolutely novel in the AAL context), which are processed and fused in the well-established robot operating system (ROS). We tested the framework by analyzing its performance in two different configurations and environments by using, respectively, a sonar and a laser rangefinder in a building scenario and a radar in an urban environment. Even if in both cases results demonstrated a quite good robustness in the obstacle detection with a quasi-real-time route replanning, we were mainly interested and succeeded in demonstrating the high flexibility and extensibility of our framework.

The importance of finding correct correspondences between two images is the major aspect in problems such as appearance-based robot localization and content-based image retrieval. Local feature matching has become a commonly used method to compare images, despite being highly probable that at least some of the matchings/correspondences it detects are incorrect. In this paper, we describe a novel approach to local feature matching, named Feature Group Matching (FGM), to select stable features and obtain a more reliable similarity value between two images. The proposed technique is demonstrated to be translational, rotational and scaling invariant. Experimental evaluation was performed on large and heterogeneous datasets of images using SIFT and SURF, the actual state-of-the-art feature extractors. Results show that FGM avoids almost 95% of incorrect matchings, reduces the visual aliasing (number of images considered similar) and increases both robotic localization and image retrieval accuracy on the average of 13%.

Traditionally, remote sensing has employed pixel-based classification techniques to deal with land use/land cover (LULC) studies. Generally, pixel-based approaches have been proven to work well with low spatial resolution imagery (e.g. Landsat or System Pour L'Observation de la Terre sensors). Now, however, commercially available high spatial resolution images (e.g. aerial Leica ADS40 and Vexcel UltraCam sensors, and satellite IKONOS, Quickbird, GeoEye and WorldView sensors) can be problematic for pixel-based analysis due to their tendency to oversample the scene. This is driving research towards object-based approaches. This article proposes a hybrid classification method with the aim of incorporating the advantages of supervised pixel-based classification into object-based approaches. The method has been developed for medium-scale (1:10,000) LULC mapping using ADS40 imagery with 1 m ground sampling distance. First, spatial information is incorporated into a pixel-based classification (AdaBoost classifier) by means of additional texture features (Haralick, Gabor, Law features), which can be selected ‘ad hoc’ according to optimal training samples (‘Relief-F’ approach, Mahalanobis distances). Then a rule-based approach sorts segmented regions into thematic CORINE Land Cover classes in terms of membership class percentages (a modified Winner-Takes-All approach) and shape parameters. Finally, ancillary data (roads, rivers, etc.) are exploited to increase classification accuracy. The experimental results show that the proposed hybrid approach allows the extraction of more LULC classes than conventional pixel-based methods, while improving classification accuracy considerably. A second contribution of this article is the assessment of classification reliability by implementing a stability map, in addition to confusion matrices.

The paper presents a novel approach to automate the Change Detection (CD) problem for the specific task of road extraction. Manual approaches to CD fail in terms of the time for releasing updated maps; in the contrary, automatic approaches, based on machine learning and image processing techniques, allow to update large areas in a short time with an accuracy and precision comparable to those obtained by human operators. This work is focused on the road-graph update starting from aerial, multi-spectral data. Georeferenced, ground data, acquired by a GPS and an inertial sensor, are integrated with aerial data to speed up the change detector. After roads extraction by means of a binary AdaBoost classifier, the old road-graph is updated exploiting a particle filter. In particular this filter results very useful to link (track) parts of roads not extracted by the classifier due to the presence of occlusions (e.g., shadows, trees).

During last years Linux started to climb the market of operating systems (OSs), and Ubuntu, derived by Debian OS, has become a good alternative to common OSs like Windows XP or Vista. The mobile robotics scientific community makes use of Linux based OSs to avoid the lack of stability that affects Microsoft OSs, especially when real time conditions must be satisfied. In this paper we present the Linux distribution RoboBuntu, acronym formed by the union of ROBOt and uBUNTU, to overcome the almost totally independent robotic software platforms existing today. The key idea behind RoboBuntu is the integration of different tools for mobile robotics into an embedded Ubuntu distribution. Another important characteristics of RoboBuntu is that every ldquohard steprdquo, like installation and configuration of OS and tools, is hidden to common users. In particular, RoboBuntu can be used either by students or researchers, as LiveCd, permanent installation on standard hard drive or, more interesting, on a USB storage flash disk.

Local feature matching has become a commonly used method to compare images. For mobile robots, a reliable method for comparing images can constitute a key component for localization tasks. In this paper, we address the issues of appearance-based topological and metric localization by introducing a novel group matching approach to select less but more robust features to match the current robot view with reference images. Feature group matching is based on the consideration that feature descriptors together with spatial relations are more robust than classical approaches. Our datasets, each consisting of a large number of omnidirectional images, have been acquired over different day times (different lighting conditions) both in indoor and outdoor environments. The feature group matching outperforms the SIFT in indoor localization showing better performances both in the case of topological and metric localization. In outdoor SURF remains the best feature extraction method, as reported in literature.