Microscopy image analysis is fundamental for different applications, from diagnosis to synthetic engineering and environmental monitoring. Modern acquisition systems have granted the possibility to acquire an escalating amount of images, requiring a consequent development of a large collection of deep learning-based automatic image analysis methods. Although deep neural networks have demonstrated great performance in this field, interpretability — an essential requirement for microscopy image analysis — remains an open challenge.This work proposes a Disentangled Representation Learning (DRL) methodology to enhance model interpretability for microscopy image classification. Exploiting benchmark datasets from three different microscopic image domains (plankton, yeast vacuoles, and human cells), we show how a DRL framework, based on transferring a representation learnt from synthetic data, can provide a good trade-off between accuracy and interpretability in this domain.

This study examines seniors’ creative engagement in group activities using synchronous communication tools and explores automatic assessment methods through behavioral and psychophysiological measurements. Working with a small senior group on collaborative creative tasks, we implemented a comprehensive data collection approach using audio-visual and physiological measurements. Machine learning models were used to evaluate group creative engagement levels using various data subsets. Results show that engagement assessment can be effective with different feature combinations, allowing flexibility across contexts and constraints. The multimodal approach, combining facial, audio, and body analysis, achieved optimal performance and is recommended when conditions permit. Our research provides insights into seniors’ online creative participation and presents an automated system for detecting creative engagement in virtual teams, supporting active participation strategies.

Developing meaningful and efficient representations that separate the fundamental structure of the data generation mechanism is crucial in representation learning. However, Disentangled Representation Learning has not fully shown its potential on real images, because of correlated generative factors, their resolution and limited access to ground truth labels. Specifically on the latter, we investigate the possibility of leveraging synthetic data to learn general-purpose disentangled representations applicable to real data, discussing the effect of fine-tuning and what properties of disentanglement are preserved after the transfer. We provide an extensive empirical study to address these issues. In addition, we propose a new interpretable intervention-based metric, to measure the quality of factors encoding in the representation. Our results indicate that some level of disentanglement, transferring a representation from synthetic to real data, is possible and effective.

Determining the visual focus of attention of people in a scene is a fundamental cue to understand social interactions from videos. Gaze direction is ideal for determining eye contact, a basic cue of non-verbal communication, but it is not always easy to recognise. Head direction is a well-known proxy of gaze direction, more robust to the variability of the scene, thus offering a valuable alternative. In this work, we consider HHP-net, a method for estimating the head direction from single frames based on a heteroscedastic neural network to estimate people’s head pose from a minimal set of head key points. We formulate the problem as a multi-task regression, to predict the pose as a triplet of Euler angles from the output of a 2D pose estimator. HHP-net also provides a measure of the aleatoric heteroscedastic uncertainties associated with the angles, through an ad-hoc loss function we introduce. In a thorough experimental analysis, we show that our model is efficient and effective compared with the state of the art, with only ∼2 degrees of degradation in the worst case counterbalanced by a space occupation ∼12 times smaller. We also show the beneficial effects of uncertainty on interpretability. Finally, we discuss the robustness of our method to input variability, showing that it can be seen as a plug-in to different pose estimators. As a proof-of-concept, we address social interaction analysis, with an algorithm to detect dyadic interactions in images.

When domain knowledge is limited and experimentation is restricted by ethical, financial, or time constraints, practitioners turn to observational causal discovery methods to recover the causal structure, exploiting the statistical properties of their data. Because causal discovery without further assumptions is an ill-posed problem, each algorithm comes with its own set of usually untestable assumptions, some of which are hard to meet in real datasets. Motivated by these considerations, this paper extensively benchmarks the empirical performance of recent causal discovery methods on observational _iid_ data generated under different background conditions, allowing for violations of the critical assumptions required by each selected approach. Our experimental findings show that score matching-based methods demonstrate surprising performance in the false positive and false negative rate of the inferred graph in these challenging scenarios, and we provide theoretical insights into their performance. This work is also the first effort to benchmark the stability of causal discovery algorithms with respect to the values of their hyperparameters. Finally, we hope this paper will set a new standard for the evaluation of causal discovery methods and can serve as an accessible entry point for practitioners interested in the field, highlighting the empirical implications of different algorithm choices.

In this work, we consider the problem of vehicle color recognition and target scenarios with limited computational resources. Indeed, in real traffic monitoring systems running on the field, algorithms must be light in terms of inference time and memory, but also accurate and robust to the scene variability. We employ end-to-end Convolutional Neural Networks to investigate under which conditions the use of such methodologies– that are state-of-the-art in a multitude of vision-based tasks but often lead to a significant computational burden– can provide us a good compromise between efficiency and effectiveness. We reason on the structure and size of the networks, while monitoring the performance in terms of color classification accuracy and computational effort. We provide an extensive experimental analysis comparing the methods using a benchmark and a private dataset acquired on the field, with almost 20K of images covering a variety of scene conditions.

In this paper we introduce a novel method to estimate the head pose of people in single images starting from a small set of head keypoints. To this purpose, we propose a regression model that exploits keypoints computed automatically by 2D pose estimation algorithms and outputs the head pose represented by yaw, pitch, and roll. Our model is simple to implement and more efficient with respect to the state of the art –faster in inference and smaller in terms of memory occupancy –with comparable accuracy.Our method also provides a measure of the heteroscedastic uncertainties associated with the three angles, through an appropriately designed loss function; we show there is a correlation between error and uncertainty values, thus this extra source of information may be used in subsequent computational steps. As an example application, we address social interaction analysis in images: we propose an algorithm for a quantitative estimation of the level of interaction between people, starting from their head poses and reasoning on their mutual positions.

Viewpoint is an essential aspect of how an action is visually perceived, with the motion appearing substantially different for some viewpoint pairs. Data driven action recognition algorithms compensate for this by including a variety of viewpoints in their training data, adding to the cost of data acquisition as well as training. We propose a novel methodology that leverages deeply pretrained features to learn actions from a single viewpoint using domain adaptation for knowledge transfer. We demonstrate the effectiveness of this pipeline on 3 different datasets: IXMAS, MoCA and NTU RGBD+, and compare with both classical and deep learning methods. Our method requires low training data and demonstrates unparalleled cross-view action recognition accuracies for single view learning.

This paper proposes a method based on visual motion primitives to address the problem of action understanding. The approach builds in an unsupervised way a dictionary of kinematic primitives from a set of sub-movements obtained by segmenting the velocity profile of an action on the basis of local minima derived directly from the optical flow. The dictionary is then used to describe each sub-movement as a linear combination of atoms using sparse coding. The descriptive capability of the proposed motion representation is experimentally validated on the MoCA dataset, a collection of synchronized multi-view videos and motion capture data of cooking activities. The results show that the approach, despite its simplicity, has a good performance in action classification, especially when the motion primitives are combined over time. Also, the method is proved to be tolerant to view point changes, and can thus support cross-view action recognition. Overall, the method may be seen as a backbone of a general approach to action understanding, with potential applications in robotics.

In this work, we report an experimental study on an Automatic Licence Plate Recognition system developed and commercialized by a partner company, with the main goals of critically analysing the original system and of devising effective but minimally invasive design changes. From a scientific point of view, ours is an attempt of reducing the gap between the different experimental approaches in academia and industry. The system is organized in layers, with an initial car plate proposal step followed by a OCR step. To cope with the drawbacks of the pre-existing system, we inserted an intermediate CNN binary classification step to discriminate between plates and non plates independently from the OCR module. Our solution incorporates new data available from working installations, in a closed refinement loop. We evaluate the modified system on 8 different installations. With respect to the original performances, we obtained significant improvements with an impact on both false positive (-9.8%) and false negatives (-5%).

The ability to detect potentially interacting agents in the surrounding environment is acknowledged to be one of the first perceptual tasks developed by humans, supported by the ability to recognise biological motion. The precocity of this ability suggests that it might be based on rather simple motion properties, and it can be interpreted as an atomic building block of more complex perception tasks typical of interacting scenarios, as the understanding of non-verbal communication cues based on motion or the anticipation of others’ action goals. In this article, we propose a novel perspective for video analysis, bridging cognitive science and machine vision, which leverages the use of computational models of the perceptual primitives that are at the basis of biological motion perception in humans. Our work offers different contributions. In a first part, we propose an empirical formulation for the Two-Thirds Power Law , a well-known invariant law of human movement, and thoroughly discuss its readability in experimental settings of increasing complexity. In particular, we consider unconstrained video analysis scenarios, where, to the best of our knowledge, the invariant law has not found application so far. The achievements of this analysis pave the way for the second part of the work, in which we propose and evaluate a general representation scheme for biological motion characterisation to discriminate biological movements with respect to non-biological dynamic events in video sequences. The method is proposed as the first layer of a more complex architecture for behaviour analysis and human-machine interaction, providing in particular a new way to approach the problem of human action understanding.

Shearlets are a relatively new directional multi-scale framework for signal analysis, which have been shown effective to enhance signal discontinuities, such as edges and corners at multiple scales even in the presence of a large quantity of noise. In this paper, we consider blob-like features in the shearlets framework. We derive a measure, which is very effective for blob detection, and, based on this measure, we propose a blob detector and a keypoint description, whose combination outperforms the state-of-the-art algorithms with noisy and compressed images. We also demonstrate that the measure satisfies the perfect scale invariance property in the continuous case. We evaluate the robustness of our algorithm to different types of noise, including blur, compression artifacts, and Gaussian noise. Furthermore, we carry on a comparative analysis on benchmark data, referring, in particular, to tolerance to noise and image compression.

In this paper, we propose a sparse coding approach to background modeling. The obtained model is based on dictionaries which we learn and keep up to date as new data are provided by a video camera. We observe that, without dynamic events, video frames may be seen as noisy data belonging to the background. Over time, such background is subject to local and global changes due to variable illumination conditions, camera jitter, stable scene changes, and intermittent motion of background objects. To capture the locality of some changes, we propose a space-variant analysis where we learn a dictionary of atoms for each image patch, the size of which depends on the background variability. At run time, each patch is represented by a linear combination of the atoms learnt online. A change is detected when the atoms are not sufficient to provide an appropriate representation, and stable changes over time trigger an update of the current dictionary. Even if the overall procedure is carried out at a coarse level, a pixel-wise segmentation can be obtained by comparing the atoms with the patch corresponding to the dynamic event. Experiments on benchmarks indicate that the proposed method achieves very good performances on a variety of scenarios. An assessment on long video streams confirms our method incorporates periodical changes, as the ones caused by variations in natural illumination. The model, fully data driven, is suitable as a main component of a change detection system.

In this paper we propose a weighted supervised pooling method for visual recognition systems. We combine a standard Spatial Pyramid Representation which is commonly adopted to encode spatial information, with an appropriate Feature Space Representation favoring semantic information in an appropriate feature space. For the latter, we propose a weighted pooling strategy exploiting data supervision to weigh each local descriptor coherently with its likelihood to belong to a given object class. The two representations are then combined adaptively with Multiple Kernel Learning. Experiments on common benchmarks (Caltech-256 and PASCAL VOC-2007) show that our image representation improves the current visual recognition pipeline and it is competitive with similar state-of-art pooling methods. We also evaluate our method on a real Human-Robot Interaction setting, where the pure Spatial Pyramid Representation does not provide sufficient discriminative power, obtaining a remarkable improvement.

In this paper we consider the problem of classifying people spatial orientation with respect to the camera viewpoint from 2D images. Structured multi-class feature selection allows us to control the amount of redundancy of our input data, while semi-supervised learning helps us coping with the intrinsic ambiguity of output labels. We model the multi-class classification problem with an all-pairs strategy based on the use of a coding matrix. A thorough experimental evaluation on the TUD Multiview Pedestrian benchmark dataset demonstrates the superiority of our approach w.r.t. state-of-the-art.

In this work we consider a machine learning setting where data are represented as graphs. First, we derive a kernel function which evaluates the similarity between graphs, while capturing pair-wise constraints between graph nodes. Second, we apply it to the problem of classifying collective activities: on this respect we first represent groups of people located in a spatial neighborhood as graphs, and then train a multi-class classifier able to capture the behavior of the groups. We evaluate our approach on a benchmark dataset and report a comparative analysis with other state-of-art methods which highlights the benefits of our approach.

In this work we build a model of the background based on dictionary learning. The image is divided into patches of equal size and a background model is obtained as a sparse linear combination of patch prototypes learnt from the image stream and updated when necessary to take into account stable variations. By enforcing sparsity, the obtained reconstruction can be computed and maintained effectively. The proposed method is stable with respect to illumination changes, correctly incorporates stable background changes in the model, and cancels out moving objects. Experiments on benchmark data indicate that the proposed method reaches very good pixel-wise performances even if relatively large patches are used.

In this paper we propose a motion-based people counting algorithm that relies on a weak camera calibration and produces a smooth estimate of the number of people in the scene. The method performs an analysis of the severity of possible occlusions and the integration of instantaneous observations over time. The key features of the algorithm are a simple pipeline, a small computational cost, the use of a model-free approach that does not need complex training procedures and its ability to work in different types of scenarios. We report results on both benchmark and acquired in-house datasets of different degrees of complexity, showing how our solution achieves comparable or superior performances with respect to state-of-art methods, while providing real-time performances.

In this paper we propose a real time face recognition method that combines face matching and identity verification modules in a feedback loop, exploiting the temporal efficiency of matching and the performances of SVM classifiers. Our approach represents an ad-hoc solution for settings characterized by variable quantity, quality and distribution of labeled data among the identities. We assess the procedure on two data sets of different complexities, showing the effectiveness of our solution. For its intrinsic peculiarities and its limited computational cost the method finds application in real time systems, and will be implemented on a wearable device for supporting visually impaired people to localize known faces.

This paper proposes a tracking architecture that finds a trade-off between accuracy and efficiency, via a combined solution of motion and appearance information. We explore the use of color features into a tracking pipeline based on Kalman filtering. The devised architecture is made of simple modules, combined to reach a robust final result, while keeping the computation cost low (we perform 20 fps). The method has been evaluated on three benchmark datasets and is currently under use on real video-surveillance systems, reporting very good tracking results.