Scene parsing has attracted a lot of attention in computer vision. While parametric models have proven effective for this task, they cannot easily incorporate new training data. By contrast, nonparametric approaches, which bypass any learning phase and directly transfer the labels from the training data to the query images, can readily exploit new labeled samples as they become available. Unfortunately, because of the computational cost of their label transfer procedures, state-of-the-art nonparametric methods typically filter out most training images to only keep a few relevant ones to label the query. As such, these methods throw away many images that still contain valuable information and generally obtain an unbalanced set of labeled samples. In this paper, we introduce a nonparametric approach to scene parsing that follows a sample-andfilter strategy. More specifically, we propose to sample labeled superpixels according to an image similarity score, which allows us to obtain a balanced set of samples. We then formulate label transfer as an efficient filtering procedure, which lets us exploit more labeled samples than existing techniques. Our experiments evidence the benefits of our approach over state-of-the-art nonparametric methods on two benchmark datasets.

Exploiting multiple modalities for semantic scene parsing has been shown to improve accuracy over the single modality scenario. Existing methods, however, assume that corresponding regions in two modalities have the same label. In this paper, we address the problem of data misalignment and label inconsistencies, e.g., due to moving objects, in semantic labeling, which violate the assumption of existing techniques. To this end, we formulate multimodal semantic labeling as inference in a CRF, and introduce latent nodes to explicitly model inconsistencies between two domains. These latent nodes allow us not only to leverage information from both domains to improve their labeling, but also to cut the edges between inconsistent regions. To eliminate the need for hand tuning the parameters of our model, we propose to learn intra-domain and inter-domain potential functions from training data. We demonstrate the benefits of our approach on two publicly available datasets containing 2D imagery and 3D point clouds. Thanks to our latent nodes and our learning strategy, our method outperforms the state-of-the-art in both cases.

In this paper, we introduce a multi-modal graphical model to address the problems of semantic segmentation using 2D-3D data exhibiting extensive many-to-one correspondences. Existing methods often impose a hard correspondence between the 2D and 3D data, where the 2D and 3D corresponding regions are forced to receive identical labels. This results in performance degradation due to misalignments, 3D-2D projection errors and occlusions. We address this issue by defining a graph over the entire set of data that models soft correspondences between the two modalities. This graph encourages each region in a modality to leverage the information from its corresponding regions in the other modality to better estimate its class label. We evaluate our method on a publicly available dataset and beat the state-of-the-art. Additionally, to demonstrate the ability of our model to support multiple correspondences for objects in 3D and 2D domains, we introduce a new multi-modal dataset, which is composed of panoramic images and LIDAR data, and features a rich set of many-to-one correspondences.

The focus of this work is addressing the challenges of performing object recognition in real world scenes as captured by a commercial, state-of-the-art, surveying vehicle equipped with a 360° panoramic camera in conjunction with a 3D laser scanner (LIDAR). Even with state-of-the-art surveying equipment, there is colour saturation and very dark regions in images, as well as some degree of time-varying misalignment between the point cloud data and imagery due to, for instance, imperfect tracking of sensor pose. Moreover, there are frequent occlusions due to both static and moving objects. These issues are inherently difficult to avoid and therefore need to be dealt with in a more robust fashion. This is where the contribution of the paper is; that is, the development of a consensus method that can intelligently incorporate feature responses from multiple views and reject those that are not very descriptive. It is shown that the overall performance in a ten class problem is increased from 70.5% for a simple 2D-3D classification system, to 77.5%. Subsequently, an enhanced CRF which has become robust using the misclassifications of training data and equipped with the probabilities of the adjacent points, was applied to the system and further improved its performance to 82.9%. The experiments were performed on a challenging dataset captured both in summer and winter.

In this paper, a new enhanced CRF for discriminating between different materials in natural scenes using terrestrial multi spectral imaging is established. Most of the existing formulations of the CRF often suffer from over smoothing and loss of small detail, thereby deteriorating the information from the underlying unary classifier in areas with a high spatial frequency. This work specifically addresses this issue by incorporating a new pairwise potential that is better at taking local context into account. Certain materials are very unlikely to appear next to each other in the scene and such configurations are penalised by employing the confusion matrix of the unary classifier. Similarly, horizontal as well as vertical configurations, which may be more or less likely for certain combinations of materials, are regarded in this formulation. Furthermore, the proposed pairwise potential also considers the length of boundaries between regions to account for the segmentation granularity issues and also uses class probabilities of the neighbouring regions to make up for the uncertainty of the unary classifier results. Seven band terrestrial multi spectral imaging were used due to its potential in distinguishing between different materials and objects. The proposed approach was evaluated using cross-validation, resulting in an average accuracy of 88.9% which is about 17% more than the accuracy of a standard CRF, which demonstrates the superiority of our approach in preserving local details.

In this paper, a method suitable for distinguishing between different materials occurring in natural scenes using a multi-spectral camera is devised. Such a capability is useful in autonomous robot applications to help negotiating the environment as well as, e.g. applications intended to create large scale inventories of assets in the proximity of roads. The utilised sensor records a seven band multi-spectral image, of which six bands are in the visible range and one in the NIR (near-infrared) range. Many materials appearing similar if viewed by a common RGB camera, will show discriminating properties if viewed by a camera capturing a greater number of separated wavelengths. The approach in this paper is to combine the discriminating strength of the multi-spectral signature in each pixel and the corresponding nature of the surrounding texture. Local features, considering seven bands in each pixel and texture features such as GLCM and Fourier spectrum features are exploited to make the system more robust to different lighting conditions. Then classifiers built using SVM and AdaBoost are evaluated with very promising results, an average classification accuracy of 91.9% and 89.1%, respectively for a ten class problem.

Computer-aided diagnosis (CAD) systems for recognition of pulmonary nodules are of great importance for early diagnosis of lung cancer. In this paper, we propose a CAD system for automated detection and classification of pulmonary nodules in 3D computerized-tomography (CT) images. First, we segment lung parenchyma from the CT data using a thresholding method. Afterwards, we apply Gaussian filters for noise reduction and nodule enhancement. Then, we use intensity and volumetric shape Index (SI) for detecting suspicious nodule candidates that include both nodules and vessels. Besides, by using SI, we can recognize nodules that are attached to vessels, pleural wall or mediastinal surface. Next, features such as sphericity, mean and variance of the gray level, elongation and border variation of potential nodules are extracted to classify detected nodules to malignant and benign groups. Fuzzy KNN is employed to classify potential nodules as non-nodule or nodule with different degree of malignancy. To assess our proposed method, 63 thoracic CT scans, from the Lung Image Database Consortium (LIDC), are recruited. Our method achieved sensitivity of 88% for nodule detection with approximately 10.3 False-Positive (FP)/subject; also the achieved classification of nodules is concordant with radiologists' opinion. Considering nodules of small size, as well as those with irregular shape, the results of nodule detection and classification are reasonable.