In this paper we present a heuristic method to pro- vide individual explanations for those elements in a dataset (data points) which are wrongly predicted by a given classifier. Since the general case is too difficult, in the present work we focus on faulty data from an underfitted model. First, we project the faulty data into a hand-crafted, and thus human readable, intermediate representation (meta-representation, profile vectors), with the aim of separating the two main causes of miss-classification: the classifier is not strong enough, or the data point belongs to an area of the input space where classes are not separable. Second, in the space of these profile vectors, we present a method to fit a meta-classifier (decision tree) and express its output as a set of interpretable (human readable) explanation rules, which leads to several target diagnosis labels: data point is either correctly classified, or faulty due to a too weak model, or faulty due to mixed (overlapped) classes in the input space. Experimental results on several real datasets show more than 80% diagnosis label accuracy and confirm that the proposed intermediate representation allows to achieve a high degree of invariance with respect to the classifier used in the input space and to the dataset being classified, i.e. we can learn the meta- classifier on a dataset with a given classifier and successfully predict diagnosis labels for a different dataset or classifier (or both).

This paper introduces a new efficient autopre-coder (AP) based deep learning approach for massive multiple-input multiple-output (mMIMO) downlink systems in which the base station is equipped with a large number of antennas with energy-efficient power amplifiers (PAs) and serves multiple user terminals. We present AP-mMIMO, a new method that jointly eliminates the multi-user interference and compensates the severe nonlinear (NL) PA distortions. Unlike previous works, AP-mMIMO has a low computational complexity, making it suitable for a global energy-efficient system. Specifically, we aim to design the PA-aware precoder and the receive decoder by leveraging the concept of autoprecoder, whereas the end-to-end massive multi-user (MU)-MIMO downlink is designed using a deep neural network (NN). Most importantly, the proposed AP-mMIMO is suited for the varying block fading channel scenario. To deal with such scenarios, we consider a two-stage precoding scheme: 1) a NN-precoder is used to address the PA non-linearities and 2) a linear precoder is used to suppress the multi-user interference. The NN-precoder and the receive decoder are trained off-line and when the channel varies, only the linear precoder changes on-line. This latter is designed by using the widely used zero-forcing precoding scheme or its low-complexity version based on matrix polynomials. Numerical simulations show that the proposed AP-mMIMO approach achieves competitive performance with a significantly lower complexity compared to existing literature.

Finding content in large video archives has so far required textual annotation to enable search by keywords. Our aim is to support retrieval from such archives using queries based on the example video clips that contain meaningful human actions. We propose a solution for the scalable search of actions in large-scale archives by leveraging the complementarity between the description at the frame level and the aggregation in time of descriptors. To permit fast search, we introduce a two-level cascade. The inexpensive first level employs aggregation to filter out a large part of the video. At the second level, aided by feature selection, a more discriminative comparison by frame alignment ranks the remaining video sequences. We improve upon the state of the art on popular data sets, and we introduce and show the results on a novel video archive data set that is significantly larger than previous ones.

Considering the idea that objects in images have a higher local structural integrity than the background they lie into, we propose a method that learns a supervised distance characterizing the membership of a pair of elements to the target structure. We test our ideas by applying them to the task of extracting semantic structures in high resolution Earth Observation images. The results show that the method works well in many situations when there is no training dataset. The limits of the method are also discussed.

To detect and locate complex human actions in video, one trains a detector for each target class and applies it to the video content. This approach can scale to large video databases if the application of the detector can be made sublinear in the size of the database. Sublin-ear retrieval methods have been successfully explored for query-by-example but few were devised for these more challenging queries by detector. We put forward here a novel approximate search method that relies on LSH to support query-by-detector. We evaluate our method on a recent large action localization dataset and show it has significantly better efficiency than linear search.

In this paper, we address the task of semantic labeling of multisource earth-observation (EO) data. Precisely, we benchmark several concurrent methods of the last 15 years, from expert classifiers, spectral support-vector classification and high-level features to deep neural networks. We establish that (1) combining multisensor features is essential for retrieving some specific classes, (2) in the image domain, deep convolutional networks obtain significantly better overall performances and (3) transfer of learning from large generic-purpose image sets is highly effective to build EO data classifiers.

A method for identifying shape features of local nature on the shape's boundary, in a way that is facilitated by the presence of noise is presented. The boundary is seen as a real function. A study of a certain distance function reveals, almost counter-intuitively, that vertices can be defined and localized better in the presence of noise, thus the concept of noising, as opposed to smoothing, is conceived and presented. The method works on both smooth and noisy shapes, the presence of noise having an effect of improving on the results of the smoothed version. Experiments with noise and a comparison to state of the art validate the method.

Data from satellite and aerial images are now widely used by everyone. These images contain information from different frequency bands that help to characterize areas of interest. In this paper we study a framework for object detection in aerial image based on discriminatively-trained models trained on multimodal data. Specifically, we investigate a method to merge outputs of large margin classifiers trained on images from different sensors: we use the ranking ability of these classifiers to learn a probabilistic model.

In this paper we apply the deformable part model by Felzenszwalb et al., which is at this moment the state of the art in many computer vision related tasks, to detect different types of man made structures in very high resolution aerial images — a reputedly difficult problem in our field. We test the framework on a database of crops of aerial images at a definition of 10 cm/pixel and investigate how the model performs on several classes of objects. The results show that the model can achieve reasonable performance in this context. However, depending on the type of object, there are specific issues which will have to be taken into account to build an effective semi-supervised annotation tool based on this model.

In the context of image search and classification, we describe an active learning strategy that relies on the intrinsic data distribution modeled as a mixture of Gaussians to speed up the learning of the target class using an interactive relevance feedback process. The contributions of our work are twofold: First, we introduce a new form of a semi-supervised C-SVM algorithm that exploits the intrinsic data distribution by working directly on equiprobable envelopes of Gaussian mixture components. Second, we introduce an active learning strategy which allows to interactively adjust the equiprobable envelopes in a small number of feedback steps. The proposed method allows the exploitation of the information contained in the unlabeled data and does not suffer from the drawbacks inherent to semi-supervised methods, e.g. computation time and memory requirements. Tests performed on a database of high-resolution satellite images and on a database of color images show that our system compares favorably, in terms of learning speed and ability to manage large volumes of data, to the classic approach using SVM active learning.

In this paper, we describe an active learning scheme which performs coarse to fine testing using a multiscale patch-based representation of images to retrieve objects in large satellite image repositories. The proposed hierarchical top-down approach reduces step by step the size of the analysis window, eliminating each time large parts of the images considered as non-relevant. Unlike most object detection methods which requires large training sets and costly offline training, we use an active learning strategy to build a classifier at each level of the hierarchy and we propose an algorithm to propagate automatically the training examples from one level to the other.

The increasing number and resolution of earth observation (EO) imaging sensors has had a significant impact on both the acquired image data volume and the information content in images. There is consequently a strong need for highly efficient search tools for EO image databases and for search methods to automatically identify and recognize structures within EO images. In this paper, we present a concept for an earth observation image data mining system mixing an auto-annotation component with a category search engine which combines a generic image class search and an object detection feature. The proposed concept relies thus on three distinct components which are detailed successively: in the first part, we describe the auto-annotation component, in the second part, the generic category search engine and in the third part, the object detection tool. In the concluding part of the paper, we provide an insight into how these three components can be related to each other and used in a complementary way to arrive at a system which combines the advantages of both the auto-annotation systems and the category search engines.

Multi-view tracking of objects in video surveillance consists in segmenting and automatically following them through different camera views. This may be achieved using geometric methods, e.g. by calibrating camera sensors and using their transformation matrices. However, in practice the precision of calibration is a major issue when trying to achieve this task robustly. In this paper, we present an alternative framework for multi-view object matching and tracking based on canonical correlation analysis. Our method is purely statistical and encodes intrinsic object appearances while being view-point invariant. We will show that our technique is (i) easy-to-set (ii) theoretically well grounded and (iii) provides robust matching and tracking results for traffic surveillance.

Starting from a member of an image database designated the "query image," traditional image retrieval techniques, for example, search by visual similarity, allow one to locate additional instances of a target category residing in the database. However, in many cases, the query image or, more generally, the target category, resides only in the mind of the user as a set of subjective visual patterns, psychological impressions, or "mental pictures." Consequently, since image databases available today are often unstructured and lack reliable semantic annotations, it is often not obvious how to initiate a search session; this is the "page zero problem." We propose a new statistical framework based on relevance feedback to locate an instance of a semantic category in an unstructured image database with no semantic annotation. A search session is initiated from a random sample of images. At each retrieval round, the user is asked to select one image from among a set of displayed images-the one that is closest in his opinion to the target class. The matching is then "mental." Performance is measured by the number of iterations necessary to display an image which satisfies the user, at which point standard techniques can be employed to display other instances. Our core contribution is a Bayesian formulation which scales to large databases. The two key components are a response model which accounts for the user's subjective perception of similarity and a display algorithm which seeks to maximize the flow of information. Experiments with real users and two databases of 20,000 and 60,000 images demonstrate the efficiency of the search process.

Traditional image retrieval methods require a "query image" to initiate a search for members of an image category. However, when the image database is unstructured, and when the category is semantic and resides only in the mind of the user, there is no obvious way to begin (the "page zero " problem). We propose a new mathematical framework for relevance feedback based on mental matching and starting from a random sample of images. At each iteration the user declares which of several displayed images is closest to his category; performance is measured by the number of iterations necessary to display an instance. Our core contribution is a Bayesian formulation which scales to large databases with no semantic annotation. The two key components are a response model which accounts for the user's subjective perception of similarity and a display algorithm which seeks to maximize the flow of information. Experiments with real users and a database with 20,000 images demonstrate the efficiency of the search process.

As the resolution of remote-sensing imagery increases, the full complexity of the scenes becomes increasingly difficult to approach. User-defined classes in large image databases are often composed of several groups of images and span very different scales in the space of low-level visual descriptors. The interactive retrieval of such image classes is then very difficult. To address this challenge, we evaluate here, in the context of satellite image retrieval, two general improvements for relevance feedback using support vector machines (SVMs). First, to optimize the transfer of information between the user and the system, we focus on the criterion employed by the system for selecting the images presented to the user at every feedback round. We put forward an active-learning selection criterion that minimizes redundancy between the candidate images shown to the user. Second, for image classes spanning very different scales in the low-level description space, we find that a high sensitivity of the SVM to the scale of the data brings about a low retrieval performance. We argue that the insensitivity to scale is desirable in this context, and we show how to obtain it by the use of specific kernel functions. Experimental evaluation of both ranking and classification performance on a ground-truth database of satellite images confirms the effectiveness of our approach

When using relevance feedback for the interactive categorization of images, the strategy employed by the system to select images to be presented to the user is of paramount importance for overall performance. Using SVM-based relevance feedback, we present a new selection criterion, based on the active learning principle, that minimizes redundancy between the candidate images shown to the user at every round. We also emphasize the fact that insensitivity to the scale of the target classes in the description space is an important quality of the learner in the interactive categorization context and we propose specific kernel functions to achieve this. Experimental results on several image databases confirm the attractiveness of our suggestions.