Visual place recognition is a critical task in computer vision, especially for localization and navigation systems. Existing methods often rely on contrastive learning: image descriptors are trained to have small distance for similar images and larger distance for dissimilar ones in a latent space. However, this approach struggles to ensure accurate distance-based image similarity representation, particularly when training with binary pairwise labels, and complex re-ranking strategies are required. This work introduces a fresh perspective by framing place recognition as a regression problem, using camera field-of-view overlap as similarity ground truth for learning. By optimizing image descriptors to align directly with graded similarity labels, this approach enhances ranking capabilities without expensive re-ranking, offering data-efficient training and strong generalization across several benchmark datasets.

Visual place recognition (VPR) is a fundamental task of computer vision for visual localization. Existing methods are trained using image pairs that either depict the same place or not. Such a binary indication does not consider continuous relations of similarity between images of the same place taken from different positions, determined by the continuous nature of camera pose. The binary similarity induces a noisy supervision signal into the training of VPR methods, which stall in local minima and require expensive hard mining algorithms to guarantee convergence. Motivated by the fact that two images of the same place only partially share visual cues due to camera pose differences, we deploy an automatic re-annotation strategy to re-label VPR datasets. We compute graded similarity labels for image pairs based on available localization metadata. Furthermore, we propose a new Generalized Contrastive Loss (GCL) that uses graded similarity labels for training contrastive networks. We demonstrate that the use of the new labels and GCL allow to dispense from hard-pair mining, and to train image descriptors that perform better in VPR by nearest neighbor search, obtaining superior or comparable results than methods that require expensive hard-pair mining and re-ranking techniques.

As dimensions of datasets in predictive modelling continue to grow, feature selection becomes increasingly practical. Datasets with complex feature interactions and high levels of redundancy still present a challenge to existing feature selection methods. We propose a novel framework for feature selection that relies on boosting, or sample re-weighting, to select sets of informative features in classification problems. The method uses as its basis the feature rankings derived from fast and scalable tree-boosting models, such as XGBoost. We compare the proposed method to standard feature selection algorithms on 9 benchmark datasets. We show that the proposed approach reaches higher accuracies with fewer features on most of the tested datasets, and that the selected features have lower redundancy.

Simulations and synthetic datasets have historically empower the research in different service robotics-related problems, being revamped nowadays with the utilization of rich virtual environments. However, with their use, special attention must be paid so the resulting algorithms are not biased by the synthetic data and can generalize to real world conditions. These aspects are usually compromised when the virtual environments are manually designed. This article presents Robot@VirtualHome, an ecosystem of virtual environments and tools that allows for the management of realistic virtual environments where robotic simulations can be performed. Here “realistic” means that those environments have been designed by mimicking the rooms’ layout and objects appearing in 30 real houses, hence not being influenced by the designer’s knowledge. The provided virtual environments are highly customizable (lighting conditions, textures, objects’ models, etc.), accommodate meta-information about the elements appearing therein (objects’ types, room categories and layouts, etc.), and support the inclusion of virtual service robots and sensors. To illustrate the possibilities of Robot@VirtualHome we show how it has been used to collect a synthetic dataset, and also exemplify how to exploit it to successfully face two service robotics-related problems: semantic mapping and appearance-based localization.

Semantic maps augment traditional representations of robot workspaces, typically based on their geometry and/or topology, with meta-information about the properties, relations and functionalities of their composing elements. A piece of such information could be: fridges are appliances typically found in kitchens and employed to keep food in good condition. Thereby, semantic maps allow for the execution of high-level robotic tasks in an efficient way, e.g. “Hey robot, Store the leftover salad”. This paper presents ViMantic, a novel semantic mapping architecture for the building and maintenance of such maps, which brings together a number of features as demanded by modern mobile robotic systems, including: (i) a formal model, based on ontologies, which defines the semantics of the problem at hand and establishes mechanisms for its manipulation; (ii) techniques for processing sensory information and automatically populating maps with, for example, objects detected by cutting-edge CNNs; (iii) distributed execution capabilities through a client–server design, making the knowledge in the maps accessible and extendable to other robots/agents; (iv) a user interface that allows for the visualization and interaction with relevant parts of the maps through a virtual environment; (v) public availability, hence being ready to use in robotic platforms. The suitability of ViMantic has been assessed using [email protected], a vast repository of data collected by a robot in different houses. The experiments carried out consider different scenarios with one or multiple robots, from where we have extracted satisfactory results regarding automatic population, execution times, and required size in memory of the resultant semantic maps.

Abstract Convolutional neural networks (CNNs) lack robustness to test image corruptions that are not seen during training. In this paper, we propose a new layer for CNNs that increases their robustness to several types of corruptions of the input images. We call it a ‘push–pull’ layer and compute its response as the combination of two half-wave rectified convolutions, with kernels of different size and opposite polarity. Its implementation is based on a biologically motivated model of certain neurons in the visual system that exhibit response suppression, known as push–pull inhibition. We validate our method by replacing the first convolutional layer of the LeNet, ResNet and DenseNet architectures with our push–pull layer. We train the networks on original training images from the MNIST and CIFAR data sets and test them on images with several corruptions, of different types and severities, that are unseen by the training process. We experiment with various configurations of the ResNet and DenseNet models on a benchmark test set with typical image corruptions constructed on the CIFAR test images. We demonstrate that our push–pull layer contributes to a considerable improvement in robustness of classification of corrupted images, while maintaining state-of-the-art performance on the original image classification task. We released the code and trained models at the url http://github.com/nicstrisc/Push-Pull-CNN-layer .

Developing tools to understand and visualize lifestyle is of high interest when addressing the improvement of habits and well-being of people. Routine, defined as the usual things that a person does daily, helps describe the individuals’ lifestyle. With this paper, we are the first ones to address the development of novel tools for automatic discovery of routine days of an individual from his/her egocentric images. In the proposed model, sequences of images are firstly characterized by semantic labels detected by pre-trained CNNs. Then, these features are organized in temporal-semantic documents to later be embedded into a topic models space. Finally, Dynamic-Time-Warping and Spectral-Clustering methods are used for final day routine/non-routine discrimination. Moreover, we introduce a new EgoRoutine-dataset, a collection of 104 egocentric days with more than 100.000 images recorded by 7 users. Results show that routine can be discovered and behavioural patterns can be observed.

Extraction of depth from images is of great importance for various computer vision applications. Methods based on convolutional neural networks are very accurate but have high computation requirements, which can be achieved with GPUs. However, GPUs are difficult to use on devices with low power requirements like robots and embedded systems. In this light, we propose a stereo matching method appropriate for applications in which limited computational and energy resources are available. The algorithm is based on a hierarchical representation of image pairs which is used to restrict disparity search range. We propose a cost function that takes into account region contextual information and a cost aggregation method that preserves disparity borders. We tested the proposed method on the Middlebury and KITTI benchmark data sets and on the TrimBot2020 synthetic data. We achieved accuracy and time efficiency results that show that the method is suitable to be deployed on embedded and robotics systems.

Audio surveillance is gaining in the last years wide interest. This is due to the large number of situations in which this kind of systems can be used, either alone or combined with video-based algorithms. In this paper we propose a deep learning method to automatically recognize events of interest in the context of audio surveillance (namely screams, broken glasses and gun shots). The audio stream is represented by a gammatonegram image. We propose a 21-layer CNN to which we feed sections of the gammatonegram representation. At the output of this CNN there are units that correspond to the classes. We trained the CNN, called AReN, by taking advantage of a problem-driven data augmentation, which extends the training dataset with gammatonegram images extracted by sounds acquired with different signal to noise ratios. We experimented it with three datasets freely available, namely SESA, MIVIA Audio Events and MIVIA Road Events and we achieved 91.43%, 99.62% and 100% recognition rate, respectively. We compared our method with other state of the art methodologies based both on traditional machine learning methodologies and deep learning. The comparison confirms the effectiveness of the proposed approach, which outperforms the existing methods in terms of recognition rate. We experimentally prove that the proposed network is resilient to the noise, has the capability to significantly reduce the false positive rate and is able to generalize in different scenarios. Furthermore, AReN is able to process 5 audio frames per second on a standard CPU and, consequently, it is suitable for real audio surveillance applications.

Recent studies have shown that the environment where people eat can affect their nutritional behavior [1]. In this paper, we provide automatic tools for personalized analysis of a person's health habits by the examination of daily recorded egocentric photo-streams. Specifically, we propose a new automatic approach for the classification of food-related environments, that is able to classify up to 15 such scenes. In this way, people can monitor the context around their food intake in order to get an objective insight into their daily eating routine. We propose a model that classifies food-related scenes organized in a semantic hierarchy. Additionally, we present and make available a new egocentric dataset composed of more than 33 000 images recorded by a wearable camera, over which our proposed model has been tested. Our approach obtains an accuracy and F-score of 56% and 65%, respectively, clearly outperforming the baseline methods.

Sound analysis research has mainly been focused on speech and music processing. The deployed methodologies are not suitable for analysis of sounds with varying background noise, in many cases with very low signal-to-noise ratio (SNR). In this paper, we present a method for the detection of patterns of interest in audio signals . We propose novel trainable feature extractors, which we call COPE (Combination of Peaks of Energy). The structure of a COPE feature extractor is determined using a single prototype sound pattern in an automatic configuration process , which is a type of representation learning. We construct a set of COPE feature extractors, configured on a number of training patterns. Then we take their responses to build feature vectors that we use in combination with a classifier to detect and classify patterns of interest in audio signals . We carried out experiments on four public data sets: MIVIA audio events, MIVIA road events, ESC-10 and TU Dortmund data sets. The results that we achieved (recognition rate equal to 91.71% on the MIVIA audio events, 94% on the MIVIA road events, 81.25% on the ESC-10 and 94.27% on the TU Dortmund) demonstrate the effectiveness of the proposed method and are higher than the ones obtained by other existing approaches. The COPE feature extractors have high robustness to variations of SNR. Real-time performance is achieved even when the value of a large number of features is computed.

Automatic interpretation of human actions gained strong interest among researchers in patter recognition and computer vision because of its wide range of applications, such as in social and home robotics, elderly people health care, surveillance, among others. In this paper, we propose a method for recognition of human actions by analysis of skeleton poses. The method that we propose is based on novel trainable feature extractors, which can learn the representation of prototype skeleton examples and can be employed to recognize skeleton poses of interest. We combine the proposed feature extractors with an approach for classification of pose sequences based on string kernels. We carried out experiments on three benchmark data sets (MIVIA-S, MSRSDA and MHAD) and the results that we achieved are comparable or higher than the ones obtained by other existing methods. A further important contribution of this work is the MIVIA-S dataset, that we collected and made publicly available.

Delineation of curvilinear structures in images is an important basic step of several image processing applications, such as segmentation of roads or rivers in aerial images, vessels or staining membranes in medical images, and cracks in pavements and roads, among others. Existing methods suffer from insufficient robustness to noise. In this paper, we propose a novel operator for the detection of curvilinear structures in images, which we demonstrate to be robust to various types of noise and effective in several applications. We call it RUSTICO, which stands for RobUST Inhibition-augmented Curvilinear Operator. It is inspired by the push-pull inhibition in visual cortex and takes as input the responses of two trainable B-COSFIRE filters of opposite polarity. The output of RUSTICO consists of a magnitude map and an orientation map. We carried out experiments on a data set of synthetic stimuli with noise drawn from different distributions, as well as on several benchmark data sets of retinal fundus images, crack pavements, and aerial images and a new data set of rose bushes used for automatic gardening. We evaluated the performance of RUSTICO by a metric that considers the structural properties of line networks (connectivity, area, and length) and demonstrated that RUSTICO outperforms many existing methods with high statistical significance. RUSTICO exhibits high robustness to noise and texture.

The European Union (EU)'s keen concern about citizens' health and well-being advancement has been expressed at all levels. It has been understood that at present, these can only be achieved through coordinated actions at the individual member states' level based on EU directives, as well as through promoting and funding R&D and expanding the use of eHealth technologies. Despite the diversities and particularities among member states, common values such as universal access to good quality healthcare, equity, and solidarity have been widely accepted across EU. That demanded the adoption of policies and follow directives, which streamlined actions to bridge healthcare gaps, and facilitate cross-border healthcare. This paper articulates a framework for deriving a national healthcare system, based on interoperable Electronic Health Record (EHR) with safeguarding healthcare quality, enabling quadruple helix (Public, Academia, Industry, NGOs) driven R&D and guided by a patient-centered approach. A methodology to develop an integrated EHR at National level is proposed as a prerequisite for eHealth and put into perspective. Recommendations are given for the steps needed, from the managerial, legal, technical, and financial concerns in developing an open access, patient-centered national healthcare system based on the context and constraints of a country. The example of a small country to apply the proposed methodology is demonstrated. Stakeholders, including citizens, healthcare professionals, academia, and the industry are mobilized, enabled, and incentivized for implementing the methodology. Experiences are aspired to be offered as lessons learned for other countries to adapt on their environment.

We propose an architecture for real-time audio source localization based on the integration of localization methodologies within a framework that employs a cheap acquisition sensor. The architecture that we present takes as input the audio signals from two calibrated microphones. Then, it computes biological-inspired features of the sound signal and estimates its direction by means of a Gaussian Mixture Model estimator. We carried out an extensive experimental analysis on four data sets, one of which we realized and made publicly available. We evaluated several characteristics of the sound localization architecture and its use in real scenarios.

In this work we explore the relevance of markers that are used for the early detection of fetal chromosomal abnormalities. For medical applications, it is important to optimize the number of used markers with respect to the number of necessary clinical examinations. We use the Generalized Matrix Learning Vector Quantization (GMLVQ) method to identify the most relevant markers from a set of 18 clinical examinations. We cross-validated our results using ten different training and test sets and we repeated our experiments using different parameters of GMLVQ. We identified the seven most relevant markers and we found that with these seven markers we obtain results that are comparable with the results that can be achieved with the full set of 18 markers. The results are in line with previous work that is found in the literature.

Most object recognition methods rely on contour-defined features obtained by edge detection or region segmentation. They are not robust to diffuse region boundaries. Furthermore, such methods do not exploit region color information. We propose color-blob-based COSFIRE (Combination of Shifted Filter Responses) filters to be selective for combinations of diffuse circular regions (blobs) in specific mutual spatial arrangements. Such a filter combines the responses of a certain selection of Difference-of-Gaussians filters, essentially blob detectors, of different scales, in certain channels of a color space, and at certain relative positions to each other. Its parameters are determined/learned in an automatic configuration process that analyzes the properties of a given prototype object of interest. We use these filters to compute features that are effective for the recognition of the prototype objects. We form feature vectors that we use with an SVM classifier. We evaluate the proposed method on a traffic sign (GTSRB) and a butterfly data sets. For the GTSRB data set we achieve a recognition rate of 98.94%, which is slightly higher than human performance and for the butterfly data set we achieve 89.02%. The proposed color-blob-based COSFIRE filters are very effective and outperform the contour-based COSFIRE filters. A COSFIRE filter is trainable, it can be configured with a single prototype pattern and it does not require domain knowledge.

Visual place recognition is the task of automatically recognizing a previously visited location through its appearance, and plays a key role in mobile robotics and autonomous driving applications. The difficulty of recognizing a revisited location increases with appearance variations caused by weather, illumination or point of view changes. In this paper we present a convolutional neural network (CNN) embedding to perform place recognition, even under severe appearance changes. The network maps images to a low dimensional space where images from nearby locations map to points close to each other, despite differences in visual appearance caused by the aforementioned phenomena. In order for the network to learn the desired invariances, we train it with triplets of images selected from datasets which present a challenging variability in visual appearance. Our proposal is validated through extensive experimentation that reveals better performance than state-of-the-art methods. Importantly, though the training phase is computationally demanding, its online application is very efficient.

We propose a sound analysis system for the detection of audio events in surveillance applications. The method that we propose combines short- and long-time analysis in order to increase the reliability of the detection. The basic idea is that a sound is composed of small, atomic audio units and some of them are distinctive of a particular class of sounds. Similarly to the words in a text, we count the occurrence of audio units for the construction of a feature vector that describes a given time interval. A classifier is then used to learn which audio units are distinctive for the different classes of sound. We compare the performance of different sets of short-time features by carrying out experiments on the MIVIA audio event data set. We study the performance and the stability of the proposed system when it is employed in live scenarios, so as to characterize its expected behavior when used in real applications.

The recently proposed trainable COSFIRE filters are highly effective in a wide range of computer vision applications, including object recognition, image classification, contour detection and retinal vessel segmentation. A COSFIRE filter is selective for a collection of contour parts in a certain spatial arrangement. These contour parts and their spatial arrangement are determined in an automatic configuration procedure from a single user-specified pattern of interest. The traditional configuration, however, does not guarantee the selection of the most distinctive contour parts. We propose a genetic algorithm-based optimization step in the configuration of COSFIRE filters that determines the minimum subset of contour parts that best characterize the pattern of interest. We use a public dataset of images of an edge milling head machine equipped with multiple cutting tools to demonstrate the effectiveness of the proposed optimization step for the detection and localization of such tools. The optimization process that we propose yields COSFIRE filters with substantially higher generalization capability. With an average of only six COSFIRE filters we achieve high precision P and recall R rates (P = 91.99%; R = 96.22%). This outperforms the original COSFIRE filter approach (without optimization) mostly in terms of recall. The proposed optimization procedure increases the efficiency of COSFIRE filters with little effect on the selectivity.

Visual localization is the process of finding the location of a camera from the appearance of the images it captures. In this work, we propose an observation model that allows the use of images for particle filter localization. To achieve this, we exploit the capabilities of Gaussian Processes to calculate the likelihood of the observation for any given pose, in contrast to methods which restrict the camera to a graph or a set of discrete poses. We evaluate this framework using different visual features as input and test its performance against laser-based localization in an indoor dataset, showing that our method requires smaller particle filter sizes while having better initialization performance.

enjoys a high international visibility and attracts participants from all over the world.

The shape and meaning of an object can radically change with the addition of one or more contour parts. For instance, a T-junction can become a crossover. We extend the COSFIRE trainable filter approach which uses a positive prototype pattern for configuration by adding a set of negative prototype patterns. The configured filter responds to patterns that are similar to the positive prototype but not to any of the negative prototypes. The configuration of such a filter comprises selecting given channels of a bank of Gabor filters that provide excitatory or inhibitory input and determining certain blur and shift parameters. We compute the response of such a filter as the excitatory input minus a fraction of the maximum of inhibitory inputs. We use three applications to demonstrate the effectiveness of inhibition: the exclusive detection of vascular bifurcations (i.e., without crossovers) in retinal fundus images (DRIVE data set), the recognition of architectural and electrical symbols (GREC’11 data set) and the recognition of handwritten digits (MNIST data set).

The inspection of retinal fundus images allows medical doctors to diagnose various pathologies. Computer-aided diagnosis systems can be used to assist in this process. As a first step, such systems delineate the vessel tree from the background. We propose a method for the delineation of blood vessels in retinal images that is effective for vessels of different thickness. In the proposed method, we employ a set of B -COSFIRE filters selective for vessels and vessel-endings. Such a set is determined in an automatic selection process and can adapt to different applications. We compare the performance of different selection methods based upon machine learning and information theory. The results that we achieve by performing experiments on two public benchmark data sets, namely DRIVE and STARE, demonstrate the effectiveness of the proposed approach.

Finding the best vector autoregression model for any dataset, medical or otherwise, is a process that, to this day, is frequently performed manually in an iterative manner requiring a statistical expertize and time. Very few software solutions for automating this process exist, and they still require statistical expertize to operate. We propose a new application called Autovar, for the automation of finding vector autoregression models for time series data. The approach closely resembles the way in which experts work manually. Our proposal offers improvements over the manual approach by leveraging computing power, e.g., by considering multiple alternatives instead of choosing just one. In this paper, we describe the design and implementation of Autovar, we compare its performance against experts working manually, and we compare its features to those of the most used commercial solution available today. The main contribution of Autovar is to show that vector autoregression on a large scale is feasible. We show that an exhaustive approach for model selection can be relatively safe to use. This study forms an important step toward making adaptive, personalized treatment available and affordable for all branches of healthcare.

In the last decades, several systems based on video analysis have been proposed for automatically detecting accidents on roads to ensure a quick intervention of emergency teams. However, in some situations, the visual information is not sufficient or sufficiently reliable, whereas the use of microphones and audio event detectors can significantly improve the overall reliability of surveillance systems. In this paper, we propose a novel method for detecting road accidents by analyzing audio streams to identify hazardous situations such as tire skidding and car crashes. Our method is based on a two-layer representation of an audio stream: at a low level, the system extracts a set of features that is able to capture the discriminant properties of the events of interest, and at a high level, a representation based on a bag-of-words approach is then exploited in order to detect both short and sustained events. The deployment architecture for using the system in real environments is discussed, together with an experimental analysis carried out on a data set made publicly available for benchmarking purposes. The obtained results confirm the effectiveness of the proposed approach.

In the last years, video surveillance has been employed for roads monitoring in order to detect abnormal events and improve the safety procedures in case of emergency. Certain events, such as car crashes or tire skidding, are difficult or impossible to detect when only the visual information is considered. In this paper we describe a preliminary system to detect events in roads by means of audio analysis. The system that we propose combines short- and long-time analysis of the audio signal in order to detect both impulsive and sustained events. We present the preliminary results achieved by the proposed system on a data set specifically made for roads surveillance, which we made publicly available. We also discuss the architectural deployment of such system in real environments with respect to a model of the noise of road traffic. The achieved results are promising and confirm the effectiveness of the system.

Due to the economic recession in the recent years, more and more mortgage customers default on the payments. This brings tremendous losses to banks and forces their arrear management departments to develop more efficient processes. In this paper, we propose a classification system to predict the outcome of a mortgage arrear. Each customer who delays a monthly mortgage rate payment is assigned a label with two possible values: a delayer, who will pay the rate before the end of the month, and a defaulter, who will fail to do so. In this way, the arrear management department only needs to treat defaulters intensively. We use arrear history records obtained from a data warehouse of one Dutch bank. We apply basic classifiers, ensemble methods and sampling techniques to this classification problem. The obtained results show that sampling techniques and ensemble learning improve the performance of basic classifiers considerably. We choose balanced random forests to build the ultimate classification system. The resulting system has already been deployed in the daily work of the arrear management department of the concerned bank, and this leads to huge cost savings.

UNLABELLED: The results of routine patient assessments in psychiatric healthcare in the Northern Netherlands are primarily used to support clinicians. We developed Wegweis, a web-based advice platform, to make this data accessible and understandable for patients. OBJECTIVE: We show that a fully automated explanation and interpretation of assessment results for schizophrenia patients, which prioritizes the information in the same way that a clinician would, is possible and is considered helpful and relevant by patients. The goal is not to replace the clinician but rather to function as a second perspective and to enable patient empowerment through knowledge. METHODS: We have developed and implemented an ontology-based approach for selecting and ranking information for schizophrenia patients based on their routine assessment results. Our approach ranks information by severity of associated schizophrenia-related problems and uses an ontology to decouple problems from advice, which adds robustness to the system, because advice can be inferred for problems that have no exact match. RESULTS: We created a problem ontology, validated by a group of experts, to combine and interpret the results of multiple schizophrenia-specific questionnaires. We designed and implemented a novel ontology-based algorithm for ranking and selecting advice, based on questionnaire answers. We designed, implemented, and illustrated Wegweis, a proof of concept for our algorithm, and, to the best of our knowledge, the first fully automated interpretation of assessment results for patients suffering from schizophrenia. We evaluated the system vis-à-vis the opinions of clinicians and patients in two experiments. For the task of identifying important problems based on MANSA questionnaires (the MANSA is a satisfaction questionnaire commonly used in schizophrenia assessments), our system corresponds to the opinion of clinicians 94% of the time for the first three problems and 72% of the time, overall. Patients find two out of the first three advice topics selected by the system to be relevant and roughly half of the advice topics overall. CONCLUSIONS: Our findings suggest that an approach that uses problem severities to identify important problems for a patient corresponds closely to the way a clinician thinks. Furthermore, after applying a severity threshold, the majority of advice units selected by the system are considered relevant by the patients. Our findings pave the way for the development of systems that facilitate patient-centered care for chronic illnesses by automating the sharing of assessment results between patient and clinician.

We propose trainable filters, which we call COSFIRE (Combination Of Shifted FIlter REsponses), and use to detect vascular bifurcations in retinal images. We configure a COSFIRE filter to be selective for a bifurcation that is specified by a user in a single-step training phase. The automatic configuration comprises the selection of channels of a bank of Gabor filters and the determination of certain blur and shift parameters. A COSFIRE filter response is computed as the geometric mean of the blurred and shifted responses of the selected Gabor filters. The proposed filters share similar properties with some shape-selective neurons in visual cortex. With only five filters we achieve a recall of 98.57% at a precision of 95.37% on the 40 binary retinal images (from DRIVE), containing more than 5000 bifurcations.

A new diadic family of eigenfunctions of the 2-D Fourier transform has been discovered. Specifically, new wavelets are derived by steering the elongated Hermite-Gauss filters with respect to rotations, thus obtaining a natural generalization of the Laguerre-Gauss harmonics. Interestingly, these functions are also proportional to their 2-D Fourier transform. Their analytical expression is provided in a compact and treatable form, by means of a new ad hoc matrix notation in which the cases of even and odd orders of the Hermite polynomials are unified. Moreover, these functions can be efficiently implemented by means of a recursive formula that is derived in this paper. The proposed filters are applied to the problem of gradient estimation to improve the theoretical Canny tradeoff of position accuracy versus noise rejection that occurs in edge detection. Experimental results show considerable improvements in using the new wavelets over both isotropic Gaussian derivatives and other elongated steerable filters more recently introduced. Finally, being the proposed wavelets a set of Fourier eigenfunctions, they can be of interest in other fields of science, such as optics and quantum mechanics.

We provide a closed form, both in the spatial and in the frequency domain, of a family of wavelets which arise from steering elongated Hermite-Gauss filters. These wavelets have interesting mathematical properties, as they form new dyadic families of eigenfunctions of the 2D Fourier transform, and generalize the well known Laguerre-Gauss harmonics. A special notation introduced here greatly simplifies our proof and unifies the cases of even and odd orders. Applying these wavelets to edge detection increases the performance of about 12.5% with respect to standard methods, in terms of the Pratt's figure of merit, both for noisy and noise-free input images.

In this paper, we perform approximate steering of the elongated 2D Hermite-Gauss functions with respect to rotations and provide a compact analytical expressions for the related basis functions. A special notation introduced here considerably simplifies the derivation and unifies the cases of even and odd indices. The proposed filters are applied to edge detection. Quantitative analysis shows a performance increase of about 12.5% in terms of the Pratt's figure of merit with respect to the well-established Gaussian gradient proposed by Canny.

Glass patterns have been exhaustively studied both in the vision literature and from a purely mathematical point of view. We extend the related formalism to the continuous case and we show that continuous Glass patterns can be used for artistic imaging applications. The general idea is to replace natural texture present in an input image with synthetic painterly texture that is generated by means of a continuous Glass pattern, whose geometrical structure is controlled by the gradient orientation of the input image. The behavior of the proposed algorithm is analytically interpreted in terms of the theory of dynamical systems. Experimental results on a broad range of input images validate the effectiveness of the proposed method in terms of lack of undesired artifacts, which are present with other existing methods, and easy interpretability of the input parameters.

We consider the problem of detecting object contours in natural images. In many cases, local luminance changes turn out to be stronger in textured areas than on object contours. Therefore, local edge features, which only look at a small neighborhood of each pixel, cannot be reliable indicators of the presence of a contour, and some global analysis is needed. We introduce a new morphological operator, called adaptive pseudo-dilation (APD), which uses context dependent structuring elements in order to identify long curvilinear structure in the edge map. We show that grouping edge pixels as the connected components of the output of APD results in a good agreement with the gestalt law of good continuation. The novelty of this operator is that dilation is limited to the Voronoi cell of each edge pixel. An efficient implementation of APD is presented. The grouping algorithm is then embedded in a multithreshold contour detector. At each threshold level, small groups of edges are removed, and contours are completed by means of a generalized reconstruction from markers. The use of different thresholds makes the algorithm much less sensitive to the values of the input parameters. Both qualitative and quantitative comparison with existing approaches prove the superiority of the proposed contour detector in terms of larger amount of suppressed texture and more effective detection of low-contrast contours.

Two important visual properties of paintings and painting-like images are the absence of texture details and the increased sharpness of edges as compared to photographic images. Painting-like artistic effects can be achieved from photographic images by filters that smooth out texture details, while preserving or enhancing edges and corners. However, not all edge preserving smoothers are suitable for this purpose. We present a simple nonlinear local operator that generalizes both the well known Kuwahara filter and the more general class of filters known in the literature as "criterion and value filter structure." This class of operators suffers from intrinsic theoretical limitations which give rise to a dramatic instability in presence of noise, especially on shadowed areas. Such limitations are discussed in the paper and overcome by the proposed operator. A large variety of experimental results shows that the output of the proposed operator is visually similar to a painting. Comparisons with existing techniques on a large set of natural images highlight conditions on which traditional edge preserving smoothers fail, whereas our approach produces good results. In particular, unlike many other well established approaches, the proposed operator is robust to degradations of the input image such as blurring and noise contamination.

In natural images, luminance changes occur both on object contours and on textures. Often, the latter are stronger than the former, thus standard edge detectors fail in isolating object contours from texture. To overcome this problem, we propose a multiresolution contour detector motivated by biological principles. At each scale, texture is suppressed by using a bipolar surround inhibition process. The binary contour map is obtained by a contour selection criterion that is more effective than the classical hysteresis thresholding. Robustness to noise is achieved by Bayesian gradient estimation.

Psychophysical researches on the human visual system have shown that the points of high curvature on the contour of an object play an important role in the recognition process. Inspired by these studies we propose: (i) a novel algorithm to select points of high curvature on the contour of an object which can be used to construct a recognizable polygonal approximation, (ii) a test which evaluates the effect of deletion of contour segments containing such points on the performance of contour based object recognition algorithms. We use complete contour representations of objects as a reference (training) set. Incomplete contour representations of the same objects are used as a test set. The performance of an algorithm is reported using the recognition rate as a function of the percentage of contour retained. We consider two types of contour incompleteness obtained by deletion of contour segments of high or low curvature. We illustrate the test procedure using two shape recognition algorithms that deploy a shape context and a distance multiset as local shape descriptors. Both algorithms qualitatively mimic human visual perception in that the deletion of segments of high curvature has a stronger performance degradation effect than the deletion of other parts of the contour. This effect is more pronounced in the performance of the shape context method.

With inspiration from psychophysical researches of the human visual system, we propose a novel aspect and a method for performance evaluation of contour-based shape recognition algorithms regarding their robustness to incompleteness of contours. We use complete contour representations of objects as a reference (training) set. Incomplete contour representations of the same objects are used as a test set. The performance of an algorithm is reported using the recognition rate as a function of the percentage of contour retained. We call this evaluation procedure the ICR test. We consider three types of contour incompleteness, viz. segment-wise contour deletion, occlusion, and random pixel depletion. As an illustration, the robustness of two shape recognition algorithms to contour incompleteness is evaluated. These algorithms use a shape context and a distance multiset as local shape descriptors. Qualitatively, both algorithms mimic human visual perception in the sense that recognition performance monotonously increases with the degree of completeness and that they perform best in the case of random depletion and worst in the case of occluded contours. The distance multiset method performs better than the shape context method in this test framework.

With inspiration from psychophysical researches of the human visual system we propose a novel method for performance evaluation of contour based shape recognition algorithms. We use complete contour representations of objects as a training set. Incomplete contour representations of the same objects are used as a test set and the recognition performance of two shape based methods is investigated. The amount of incompleteness in test cases is quantified using the percentage of contour pixels retained. The performances of the methods are reported using the recognition rate as a function of the degree of incompleteness. We consider three types of incomplete contour representations, viz. segment-wise deletion, occlusion and random pixel depletion. The methods compared in this framework use shape context and distance multiset as local shape descriptors. Qualitatively, both methods mimic human visual perception in the sense that they perform best in the case of random depletion and worst in the case of occluded contours. Quantitatively, the distance multiset method performs better than the shape context method in this test framework.

We propose a biologically motivated computational step, called non-classical receptive field (non-CRF) inhibition, to improve contour detection in images of natural scenes. We augment a Gabor energy operator with non-CRF inhibition. The resulting contour operator responds strongly to isolated lines, edges, and contours but exhibits a weaker or no response to edges that make part of texture. As such, the contour operator is more useful for contour-based object recognition tasks than traditional edge detectors, which do not make such a distinction. The contour operator consistently outperforms the Canny edge detector on natural images with associated ground truth contour maps.

We introduce a novel rich local descriptor of an image point, we call the (labeled) distance set, which is determined by the spatial arrangement of image features around that point. We describe a two-dimensional (2D) visual object by the set of (labeled) distance sets associated with the feature points of that object. Based on a dissimilarity measure between (labeled) distance sets and a dissimilarity measure between sets of (labeled) distance sets, we address two problems that are often encountered in object recognition: object segmentation, for which we formulate a distance sets shape filter, and shape matching. The use of the shape filter is illustrated on printed and handwritten character recognition and detection of traffic signs in complex scenes. The shape comparison procedure is illustrated on handwritten character classification, COIL-20 database object recognition and MPEG-7 silhouette database retrieval.

We propose a biologically motivated method, called nonclassical receptive field (non-CRF) inhibition (more generally, surround inhibition or suppression), to improve contour detection in machine vision. Non-CRF inhibition is exhibited by 80% of the orientation-selective neurons in the primary visual cortex of monkeys and has been shown to influence human visual perception as well. Essentially, the response of an edge detector at a certain point is suppressed by the responses of the operator in the region outside the supported area. We combine classical edge detection with isotropic and anisotropic inhibition, both of which have counterparts in biology. We also use a biologically motivated method (the Gabor energy operator) for edge detection. The resulting operator responds strongly to isolated lines, edges, and contours, but exhibits weak or no response to edges that are part of texture. We use natural images with associated ground truth contour maps to assess the performance of the proposed operator for detecting contours while suppressing texture edges. Our method enhances contour detection in cluttered visual scenes more effectively than classical edge detectors used in machine vision (Canny edge detector). Therefore, the proposed operator is more useful for contour-based object recognition tasks, such as shape comparison, than traditional edge detectors, which do not distinguish between contour and texture edges. Traditional edge detection algorithms can, however, also be extended with surround suppression. This study contributes also to the understanding of inhibitory mechanisms in biology.

Texture features that are based on the local power spectrum obtained by a bank of Gabor filters are compared. The features differ in the type of nonlinear post-processing which is applied to the local power spectrum. The following features are considered: Gabor energy, complex moments, and grating cell operator features. The capability of the corresponding operators to produce distinct feature vector clusters for different textures is compared using two methods: the Fisher (1923) criterion and the classification result comparison. Both methods give consistent results. The grating cell operator gives the best discrimination and segmentation results. The texture detection capabilities of the operators and their robustness to nontexture features are also compared. The grating cell operator is the only one that selectively responds only to texture and does not give false response to nontexture features such as object contours.

Graph-based features, such as the number of connected components, edges of a given orientation and vertices per unit area, and the number of vertices and pixels per connected component, are proposed for the analysis of textures which consist of structural elements. The proposed set of features is compared with features obtained by a typical filter-based scheme which makes use of Gabor filters. The discrimination properties of the two types of features are assessed by evaluating the separability of sets of feature vectors which are derived from different types of texture using the Mahalanobis distance. The graph-based features are shown to be superior to the filter-based features for the class of concerned textures. They are particularly suited for discrimination between textures which have the same spatial and orientation regularity but consist of elements of different forms.

Texture feature extraction operators, which comprise linear filtering, eventually followed by post-processing, are considered. The filters used are Laws' masks (1980), filters derived from well-known discrete transforms, and Gabor filters. The post-processing step comprises nonlinear point operations and/or local statistics computation. The performance is measured by means of the Mahalanobis distance between clusters of feature vectors derived from different textures. The results show that post-processing improves considerably the performance of filter based texture operators.

The performance of two well-known texture operators (based on Gabor energy and the cooccurrence matrix) is compared with the performance of a new, biologically motivated texture operator, the dot-pattern selective cell operator. The comparison is made using a quantitative method based on the Mahalanobis distance. Together with some classification experiments the comparison shows a clear superiority of the new operator in dot-pattern texture problems.

An algorithm is introduced which is related to matching pursuit but allows updating of more than one coding coefficient per iteration: the updated coefficients correspond to mutually orthogonal elements of the dictionary. Coding experiments on natural images show that the method achieves the same trade-off as matching pursuit between number of coding coefficients and reconstruction error, but significantly faster convergence.

Texture is an important part of the visual world of animals and humans and their visual systems successfully detect, discriminate, and segment texture. Relatively recently progress was made concerning structures in the brain that are presumably responsible for texture processing. Neurophysiologists reported on the discovery of a new type of orientation selective neuron in areas V1 and V2 of the visual cortex of monkeys which they called grating cells. Such cells respond vigorously to a grating of bars of appropriate orientation, position and periodicity. In contrast to other orientation selective cells, grating cells respond very weakly or not at all to single bars which do not make part of a grating. Elsewhere we proposed a nonlinear model of this type of cell and demonstrated the advantages of grating cells with respect to the separation of texture and form information. In this paper, we use grating cell operators to obtain features and compare these operators in texture analysis tasks with commonly used feature extracting operators such as Gabor-energy and co-occurrence matrix operators. For a quantitative comparison of the discrimination properties of the concerned operators a new method is proposed which is based on the Fisher (1923) linear discriminant and the Fisher criterion. The operators are also qualitatively compared with respect to their ability to separate texture from form information and their suitability for texture segmentation.

The performance of two well-known texture operators (based on Gabor-energy and the cooccurrence matrix) is compared with the performance of a new, biologically motivated texture operator, the grating cell operator, previously proposed by the authors (1995, 1997). The comparison is made using a new quantitative method, based on the Fisher criterion (1923). The results show the clear superiority of the new operator in oriented texture problems.