• A neuro-symbolic generative model for Automated Planning problems is proposed. • Our method generates valid, diverse and difficult problems, and reduces human effort. • Problem generation is formulated as a Markov Decision Process. • Generative policies are trained with Deep Reinforcement Learning. • Problem constraints are encoded via a novel semi-declarative. language In the field of Automated Planning there is often the need for a set of planning problems from a particular domain, e.g., to be used as training data for Machine Learning methods or as benchmarks in planning competitions. In most cases, these problems are created either by hand or by a domain-specific generator, putting a burden on the human designers. In this paper, we propose NeSIG ( Neuro-Symbolic Instance Generator ), to the best of our knowledge the first domain-independent method for automatically generating typed-STRIPS planning problems that are valid, diverse and difficult to solve. We formulate problem generation as a Markov Decision Process and train two generative policies with Deep Reinforcement Learning to generate problems with the desired properties. We conduct experiments on five classical domains, comparing our approach against handcrafted, domain-specific instance generators and various ablations. Results show NeSIG is able to automatically generate valid and diverse problems of much greater difficulty (6.8 times more on geometric average) than domain-specific generators, while simultaneously reducing human effort when compared to them. Additionally, it can generalize to problems more than twice the size of those seen during training.

Explainability has become one of the leading research topics within Artificial Intelligence (AI) in the last few years, as it has increased the confidence and credibility of “black box” models, such as deep neural networks . However, the evaluation of the explanations provided by different explainability approaches remains a hot research topic. This evaluation enables the possibility of comparing different existing techniques and would play a crucial role in improving the auditability of AI-based systems. The current literature on the subject considers that the evaluation of explainability methods can be approached in two ways: qualitative and quantitative . While qualitative evaluations are based on assumptions induced by human understanding that are hard to develop and prone to introduce certain cognitive biases, quantitative ones avoid these biases by excluding the human expert from the evaluation process. However, the main challenge in quantitatively evaluating an explanation is the lack of ground truth specifying what defines a correct explanation. In this paper, we propose an evaluation measure that quantifies the Level of Strength of an Explanation (LSE), i.e., the extent to which the explanation produced by a post-hoc explainability method supports the class predicted by a classifier. Our proposal is inspired by the semantics underlying the Likelihood Ratio in evaluating forensic evidence, which is defined as the weight to be attributed to a piece of forensic evidence according to the prosecution and defense propositions. To validate our proposal, nine popular explainability techniques are compared across two deep neural architectures dedicated to image classification and three classifiers for binary classification over tabular datasets. In addition, we use MetaQuantus as a meta-evaluation approach. Results from our experimental study reveal that GradCAM and LRP outperform the other explainability methods in terms of the proposed LSE measure.

Opinion Dynamics models in social networks are a valuable tool to study how opinions evolve within a population. However, these models often rely on agent-level parameters that are difficult to measure in a real population. This is the case of the confidence threshold in opinion dynamics models based on bounded confidence, where agents are only influenced by other agents having a similar opinion (given by this confidence threshold). Consequently, a common practice is to apply a universal threshold to the entire population and calibrate its value to match observed real-world data, despite being an unrealistic assumption. In this work, we propose an alternative approach using graph neural networks to infer agent-level confidence thresholds in the opinion dynamics of the Hegselmann-Krause model of bounded confidence. This eliminates the need for additional simulations when faced with new case studies. To this end, we construct a comprehensive synthetic training dataset that includes different network topologies and configurations of thresholds and opinions. Through multiple training runs utilizing different architectures, we identify GraphSAGE as the most effective solution, achieving a coefficient of determination$R^{2}$above 0.7 in test datasets derived from real-world topologies. Remarkably, this performance holds even when the test topologies differ in size from those considered during training.

In the field of Automated Planning there is often the need for a set of planning problems from a particular domain, e.g., to be used as training data for Machine Learning or as benchmarks in planning competitions. In most cases, these problems are created either by hand or by a domain-specific generator, putting a burden on the human designers. In this paper we propose NeSIG, to the best of our knowledge the first domain-independent method for automatically generating planning problems that are valid, diverse and difficult to solve. We formulate problem generation as a Markov Decision Process and train two generative policies with Deep Reinforcement Learning to generate problems with the desired properties. We conduct experiments on three classical domains, comparing our approach against handcrafted, domain-specific instance generators and various ablations. Results show NeSIG is able to automatically generate valid and diverse problems of much greater difficulty (15.5 times more on geometric average) than domain-specific generators, while simultaneously reducing human effort when compared to them. Additionally, it can generalize to larger problems than those seen during training.

Although deep learning models can solve complex prediction problems, they have been criticized for being ‘black boxes’. This implies that their decisions are difficult, if not impossible, to explain by simply inspecting their internal knowledge structures. Explainable Artificial Intelligence has attempted to open the black-box through model-specific and agnostic post-hoc methods that generate visualizations or derive associations between the problem features and the model predictions. This paper proposes a new method, termed REPROT, that explains the decisions of complex deep learning architectures based on local reducts of an image. A ‘reduct’ is a set of sufficiently descriptive features that can fully characterize the acquired knowledge. The created reducts are used to build a ‘prototype image’ that visually explains the inference obtained by a black-box model for an image. We focus on deep learning architectures whose complexity and internal particularities demand adapting existing model-specific explanation methods, making the explanation process more difficult. Experimental results show that the black-box model can detect an object using the prototype image generated from the reduct. Hence, the explanations will be given by “the minimum set of features sufficient for the neural model to detect an object”. The confidence scores obtained by architectures such as Inception, Yolo, and Mask R-CNN are higher for prototype images built from the reduct than those built from the most important superpixels according to the LIME method. Moreover, the target object is not detected on several occasions through the LIME output, thus supporting the superiority of the proposed explanation method.

Image semantic segmentation, a fundamental computer vision task, performs the pixel-wise classification of an image seeking to group pixels that share some semantic content. One of the main issues in semantic segmentation is the creation of fully annotated datasets where each image has one label per pixel. These annotations are highly time-consuming and, the more the labelling increases, the higher the percentage of human-entered errors grows. Segmentation methods based on less supervision can reduce both labelling time and noisy labels. However, when dealing with real-world applications, it is far from trivial to establish a method that minimizes labelling time while maximizing performance. Our main contribution is to present the first comprehensive study of state-of-the-art methods based on different levels of supervision. Image processing baselines, unsupervised, weakly supervised and supervised approaches have been evaluated. We aim to guide anyone approaching a new real-world use case by providing a trade-off between performance and supervision complexity on datasets from different domains, such as street scenes (Camvid), microscopy (MetalDAM), satellite (FloodNet) and medical images (NuCLS). Our experimental results suggest that: (i) unsupervised and weak learning perform well on majority classes, which helps to speed up labelling; (ii) weakly supervised can outperform fully supervised methods on minority classes; (iii) not all weak learning methods are robust to the nature of the dataset, especially those based on image-level annotations; and (iv) among all weakly supervised methods, point-based are the best-performing ones, even competing with fully supervised methods. The code is available at https://github.com/martafdezmAM/lessen_supervision.

Cephalometric landmarks are used in many forensic tasks of great relevance. Nevertheless, the automatic localization of such points is greatly underdeveloped in the scientific literature, especially on in-the-wild images where no published work is available. Inspired by state-of-the-art automatic facial landmark localization research, we present a method based on a cascade of conditional convolutional networks for predicting high-resolution cephalometric landmarks under specific conditions: using a size-limited dataset of in-the-wild images usually handled by forensic anthropologists. Every contribution is thoroughly ablated and validated. We compare our proposal against top-performing standard facial landmark localization methods. Furthermore, we conduct a user study comparing our performance against expert annotators on a different problem-specific dataset. The results show that we outperform competing methods in a cephalometric landmarks dataset by a large margin, two times better than the closest one, and achieve human-like performance in half of the cases. For its good results, our proposal will be included in Skeleton-ID, a commercial solution for forensic identification assisted by artificial intelligence.

Abstract Biological profile estimation, of which sex estimation is a fundamental first stage, is a really important task in forensic human identification. Although there are a large number of methods that address this problem from different bone structures, mainly using the pelvis and the skull, it has been shown that the humerus presents significant sexual dimorphisms that can be used to estimate sex in their absence. However, these methods are often too subjective or costly, and the development of new methods that avoid these problems is one of the priorities in forensic anthropology research. In this respect, the use of artificial intelligence may allow to automate and reduce the subjectivity of biological profile estimation methods. In fact, artificial intelligence has been successfully applied in sex estimation tasks, but most of the previous work focuses on the analysis of the pelvis and the skull. More importantly, the humerus, which can be useful in some situations due to its resistance, has never been used in the development of an automatic sex estimation method. Therefore, this paper addresses the use of machine learning techniques to the task of image classification, focusing on the use of images of the distal epiphysis of the humerus to classify whether it belongs to a male or female individual. To address this, we have used a set of humerus photographs of 417 adult individuals of Mediterranean origin to validate and compare different approaches, using both deep learning and traditional feature extraction techniques. Our best model obtains an accuracy of 91.03% in test, correctly estimating the sex of 92.68% of the males and 89.19% of the females. These results are superior to the ones obtained by the state of the art and by a human expert, who has achieved an accuracy of 83.33% using a state-of-the-art method on the same data. In addition, the visualization of activation maps allows us to confirm not only that the neural network observes the sexual dimorphisms that have been proposed by the forensic anthropology literature, but also that it has been capable of finding a new region of interest.

Computer vision (CV) is a big and important field in artificial intelligence covering a wide range of applications. Image analysis is a major task in CV aiming to extract, analyze and understand the visual content of images. However, image-related tasks are very challenging due to many factors, e.g., high variations across images, high dimensionality, domain expertise requirement, and image distortions. Evolutionary computation (EC) approaches have been widely used for image analysis with significant achievement. However, there is no comprehensive survey of existing EC approaches to image analysis. To fill this gap, this article provides a comprehensive survey covering all essential EC approaches to important image analysis tasks, including edge detection, image segmentation, image feature analysis, image classification, object detection, and others. This survey aims to provide a better understanding of evolutionary CV (ECV) by discussing the contributions of different approaches and exploring how and why EC is used for CV and image analysis. The applications, challenges, issues, and trends associated to this research field are also discussed and summarized to provide further guidelines and opportunities for future research.

Facial biometrics play an essential role in the fields of law enforcement and forensic sciences. When comparing facial traits for human identification in photographs or videos, the analysis must account for several factors that impair the application of common identification techniques, such as illumination, pose, or expression. In particular, facial attributes can drastically change depending on the distance between the subject and the camera at the time of the picture. This effect is known as perspective distortion, which can severely affect the outcome of the comparative analysis. Hence, knowing the subject-to-camera distance of the original scene where the photograph was taken can help determine the degree of distortion, improve the accuracy of computer-aided recognition tools, and increase the reliability of human identification and further analyses. In this paper, we propose a deep learning approach to estimate the subject-to-camera distance of facial photographs: FacialSCDnet. Furthermore, we introduce a novel evaluation metric designed to guide the learning process, based on changes in facial distortion at different distances. To validate our proposal, we collected a novel dataset of facial photographs taken at several distances using both synthetic and real data. Our approach is fully automatic and can provide a numerical distance estimation for up to six meters, beyond which changes in facial distortion are not significant. The proposed method achieves an accurate estimation, with an average error below 6 cm of subject-to-camera distance for facial photographs in any frontal or lateral head pose, robust to facial hair, glasses, and partial occlusion. • Accurate estimation of subject-to-camera distance in portrait photographs. • A novel metric is proposed, based on the effects of perspective in facial distortion. • A new database of facial images at a distance is introduced for human identification. • A transfer learning approach overcomes the limitations of current methods. • Robust to expression, occlusion and pose without requiring anatomical information.

Comparative radiography is the forensic anthropology technique in which ante-mortem (AM) and post-mortem (PM) radiographic materials (e.g., X-ray images or CTs) are compared in order to determine the identity of a deceased human being. One of the most commonly used anatomical structures in comparative radiography are the frontal sinuses. The frontal sinuses are osseous cavities located in the skull, which are used in forensic identification tasks due to their singularity and high identification power. In order to automate the comparison of frontal sinuses in AM and PM materials, it is necessary to perform the registration of these materials (i.e., it is necessary to carry out the alignment of these anatomical regions). However, the manual alignment of these structures is a time-consuming and subjective process. In order to tackle this problem, this paper presents an automatic frontal sinuses registration method in comparative radiography using real-coded evolutionary algorithms (RCEAs). The task is formulated as a 2D-3D image registration problem using a 9 Degrees of Freedom perspective transformation model; two RCEAs (DE and MVMOSH) are compared in the minimization of the registration cost function, and the best of them (MVMO-SH) is applied to an identification scenario including 50X-ray images and 50 CTs. The results obtained show that the proposed automatic identification system is able to filter more than 80% of the sample.

Visual voice activity detection (V-VAD) uses visual features to predict whether a person is speaking or not. V-VAD is useful whenever audio VAD (A-VAD) is inefficient either because the acoustic signal is difficult to analyze or because it is simply missing. We propose two deep architectures for V-VAD, one based on facial landmarks and one based on optical flow. Moreover, available datasets, used for learning and for testing V-VAD, lack content variability. We introduce a novel methodology to automatically create and annotate very large datasets in-the-wild - WildVVAD - based on combining A-VAD with face detection and tracking. A thorough empirical evaluation shows the advantage of training the proposed deep V-VAD models with this dataset1.

Deep learning revolutionized data science, and recently its popularity has grown exponentially, as did the amount of papers employing deep networks. Vision tasks, such as human pose estimation, did not escape from this trend. There is a large number of deep models, where small changes in the network architecture, or in the data pre-processing, together with the stochastic nature of the optimization procedures, produce notably different results, making extremely difficult to sift methods that significantly outperform others. This situation motivates the current study, in which we perform a systematic evaluation and statistical analysis of vanilla deep regression, i.e., convolutional neural networks with a linear regression top layer. This is the first comprehensive analysis of deep regression techniques. We perform experiments on four vision problems, and report confidence intervals for the median performance as well as the statistical significance of the results, if any. Surprisingly, the variability due to different data pre-processing procedures generally eclipses the variability due to modifications in the network architecture. Our results reinforce the hypothesis according to which, in general, a general-purpose network (e.g., VGG-16 or ResNet-50) adequately tuned can yield results close to the state-of-the-art without having to resort to more complex and ad-hoc regression models.

In this paper we address the problems of detecting objects of interest in a video and of estimating their locations, solely from the gaze directions of people present in the video. Objects can be indistinctly located inside or outside the camera field of view. We refer to this problem as extended gaze following. The contributions of the paper are the followings. First, we propose a novel spatial representation of the gaze directions adopting a top-view perspective. Second, we develop several convolutional encoder/decoder networks to predict object locations and compare them with heuristics and with classical learning-based approaches. Third, in order to train the proposed models, we generate a very large number of synthetic scenarios employing a probabilistic formulation. Finally, our methodology is empirically validated using a publicly available dataset.

We investigate deep Bayesian neural networks with Gaussian priors on the weights and a class of ReLU-like nonlinearities. Bayesian neural networks with Gaussian priors are well known to induce an L2, “weight decay”, regularization. Our results indicate a more intricate regularization effect at the level of the unit activations. Our main result establishes that the induced prior distribution on the units before and after activation becomes increasingly heavy-tailed with the depth of the layer. We show that first layer units are Gaussian, second layer units are sub-exponential, and units in deeper layers are characterized by sub-Weibull distributions. Our results provide new theoretical insight on deep Bayesian neural networks, which we corroborate with simulation experiments.

We address the problem of audio-visual gaze control in the specific context of human-robot interaction, namely how controlled robot motions are combined with visual and acoustic observations in order to direct the robot head towards targets of interest. The paper has the following contributions: (i) a novel audio-visual fusion framework that is well suited for controlling the gaze of a robotic head; (ii) a reinforcement learning (RL) formulation for the gaze control problem, using a reward function based on the available temporal sequence of camera and microphone observations; and (iii) several deep architectures that allow to experiment with early and late fusion of audio and visual data. We introduce a simulated environment that enables us to learn the proposed deep RL model without the need of spending hours of tedious interaction. By thoroughly experimenting on a publicly available dataset and on a real robot, we provide empirical evidence that our method achieves state-of-the-art performance.

Convolutional Neural Networks (ConvNets) have become the state-of-the-art for many classification and regression problems in computer vision. When it comes to regression, approaches such as measuring the Euclidean distance of target and predictions are often employed as output layer. In this paper, we propose the coupling of a Gaussian mixture of linear inverse regressions with a ConvNet, and we describe the methodological foundations and the associated algorithm to jointly train the deep network and the regression function. We test our model on the head-pose estimation problem. In this particular problem, we show that inverse regression outperforms regression models currently used by state-of-the-art computer vision methods. Our method does not require the incorporation of additional data, as it is often proposed in the literature, thus it is able to work well on relatively small training datasets. Finally, it outperforms state-of-the-art methods in head-pose estimation using a widely used head-pose dataset. To the best of our knowledge, we are the first to incorporate inverse regression into deep learning for computer vision applications.

We have developed a technique to study how good computers can be at diagnosing gastrointestinal lesions from regular (white light and narrow banded) colonoscopic videos compared to two levels of clinical knowledge (expert and beginner). Our technique includes a novel tissue classification approach which may save clinician's time by avoiding chromoendoscopy, a time-consuming staining procedure using indigo carmine. Our technique also discriminates the severity of individual lesions in patients with many polyps, so that the gastroenterologist can directly focus on those requiring polypectomy. Technically, we have designed and developed a framework combining machine learning and computer vision algorithms, which performs a virtual biopsy of hyperplastic lesions, serrated adenomas and adenomas. Serrated adenomas are very difficult to classify due to their mixed/hybrid nature and recent studies indicate that they can lead to colorectal cancer through the alternate serrated pathway. Our approach is the first step to avoid systematic biopsy for suspected hyperplastic tissues. We also propose a database of colonoscopic videos showing gastrointestinal lesions with ground truth collected from both expert image inspection and histology. We not only compare our system with the expert predictions, but we also study if the use of 3D shape features improves classification accuracy, and compare our technique's performance with three competitor methods.

Artificial Neuron-Glia Networks (ANGNs) are a novel bio-inspired machine learning approach. They extend classical Artificial Neural Networks (ANNs) by incorporating recent findings and suppositions about the way information is processed by neural and astrocytic networks in the most evolved living organisms. Although ANGNs are not a consolidated method, their performance against the traditional approach, i.e. without artificial astrocytes, was already demonstrated on classification problems. However, the corresponding learning algorithms developed so far strongly depends on a set of glial parameters which are manually tuned for each specific problem. As a consequence, previous experimental tests have to be done in order to determine an adequate set of values, making such manual parameter configuration time-consuming, error-prone, biased and problem dependent. Thus, in this paper, we propose a novel learning approach for ANGNs that fully automates the learning process, and gives the possibility of testing any kind of reasonable parameter configuration for each specific problem. This new learning algorithm, based on coevolutionary genetic algorithms, is able to properly learn all the ANGNs parameters. Its performance is tested on five classification problems achieving significantly better results than ANGN and competitive results with ANN approaches.

This paper describes a hybrid level set approach to medical image segmentation. The method combines region-and edge-based information with the prior shape knowledge introduced using deformable registration. A parameter tuning mechanism, based on Genetic Algorithms, provides the ability to automatically adapt the level set to different segmentation tasks. Provided with a set of examples, the GA learns the correct weights for each image feature used in the segmentation. The algorithm has been tested over four different medical datasets across three image modalities. Our approach has shown significantly more accurate results in comparison with six state-of-the-art segmentation methods. The contributions of both the image registration and the parameter learning steps to the overall performance of the method have also been analyzed.

We perform a two-step segmentation of the hippocampus in histological images. First, we maximize the overlap of an empirically-derived parametric Deformable Model with two crucial landmark sub-structures in the brain image using Differential Evolution. Then, the points located in the previous step determine the region where a thresholding technique based on Otsu's method is to be applied. Finally, the segmentation is expanded employing Random Forest in the regions not covered by the model. Our approach showed an average segmentation accuracy of the 92.25% and 92.11% on test sets comprising 15 real and 15 synthetic images, respectively.