Most non-invasive gaze estimation methods regress gaze directions directly from a single face or eye image. However, due to important variabilities in eye shapes and inner eye structures amongst individuals, universal models obtain limited accuracies and their output usually exhibit high variance as well as subject dependent biases. Thus, increasing accuracy is usually done through calibration, allowing gaze predictions for a subject to be mapped to her actual gaze. In this article, we introduce a novel approach, which works by directly training a differential convolutional neural network to predict gaze differences between two eye input images of the same subject. Then, given a set of subject specific calibration images, we can use the inferred differences to predict the gaze direction of a novel eye sample. The assumption is that by comparing eye images of the same user, annoyance factors (alignment, eyelid closing, illumination perturbations) which usually plague single image prediction methods can be much reduced, allowing better prediction altogether. Furthermore, the differential network itself can be adapted via finetuning to make predictions consistent with the available user reference pairs. Experiments on 3 public datasets validate our approach which constantly outperforms state-of-the-art methods even when using only one calibration sample or those relying on subject specific gaze adaptation.

As an indicator of human attention gaze is a subtle behavioral cue which can be exploited in many applications. However, inferring 3D gaze direction is challenging even for deep neural networks given the lack of large amount of data (groundtruthing gaze is expensive and existing datasets use different setups) and the inherent presence of gaze biases due to person-specific difference. In this work, we address the problem of person-specific gaze model adaptation from only a few reference training samples. The main and novel idea is to improve gaze adaptation by generating additional training samples through the synthesis of gaze-redirected eye images from existing reference samples. In doing so, our contributions are threefold:(i) we design our gaze redirection framework from synthetic data, allowing us to benefit from aligned training sample pairs to predict accurate inverse mapping fields; (ii) we proposed a self-supervised approach for domain adaptation; (iii) we exploit the gaze redirection to improve the performance of person-specific gaze estimation. Extensive experiments on two public datasets demonstrate the validity of our gaze retargeting and gaze estimation framework.

Change detection is a key problem for many remote sensing applications. In this paper, we present a novel unsupervised method for change detection between two high-resolution remote sensing images possibly acquired by two different sensors. This method is based on keypoints matching, evaluation, and grouping, and does not require any image co-registration. It consists of two main steps. First, global and local mapping functions are estimated through keypoints extraction and matching. Second, based on these mappings, keypoint matchings are used to detect changes and then grouped to extract regions of changes. Both steps are defined through an a contrario framework, simplifying the parameter setting and providing a robust pipeline. The proposed approach is evaluated on synthetic and real data from different optic sensors with different resolutions, incidence angles, and illumination conditions.

Texture characterization is a key problem in image understanding and pattern recognition. In this paper, we present a flexible shape-based texture representation using shape co-occurrence patterns. More precisely, texture images are first represented by a tree of shapes, each of which is associated with several geometrical and radiometric attributes. Then, four typical kinds of shape co-occurrence patterns based on the hierarchical relationships among the shapes in the tree are learned as codewords. Three different coding methods are investigated for learning the codewords, which can be used to encode any given texture image into a descriptive vector. In contrast with existing works, the proposed approach not only inherits the shape-based method's strong ability to capture geometrical aspects of textures and high robustness to variations in imaging conditions but also provides a flexible way to consider shape relationships and to compute high-order statistics on the tree. To the best of our knowledge, this is the first time that co-occurrence patterns of explicit shapes have been used as a tool for texture analysis. Experiments on various texture and scene data sets demonstrate the efficiency of the proposed approach.

This paper presents a significant improvement for the synthesis of texture images using convolutional neural networks (CNNs), making use of constraints on the Fourier spectrum of the results. More precisely, the texture synthesis is regarded as a constrained optimization problem, with constraints conditioning both the Fourier spectrum and statistical features learned by CNNs. In contrast with existing methods, the presented method inherits from previous CNN approaches the ability to depict local structures and fine scale details, and at the same time yields coherent large scale structures, even in the case of quasi-periodic images. This is done at no extra computational cost. Synthesis experiments on various images show a clear improvement compared to a recent state-of-the art method relying on CNN constraints only.

The segmentation of synthetic aperture radar (SAR) images is a long-standing yet challenging task, not only because of the presence of speckle but also due to the variations of surface backscattering properties in the images. Tremendous investigations have been made to suppress the speckle effects for the segmentation of SAR images, whereas few works are devoted to dealing with the variations of backscattering intensities in the images. To overcome the two difficulties, this paper presents a novel SAR image segmentation method by exploiting a multiscale active contour model based on the nonlocal processing principle. More precisely, we first formulize the SAR segmentation problem with an active contour model by integrating the nonlocal interactions between pairs of patches inside and outside the segmented regions. Second, a multiscale strategy is proposed to speed up the nonlocal active contour segmentation procedure and to avoid falling into a local minimum for achieving more accurate segmentation results. Experimental results on simulated and real SAR images demonstrate the efficiency and feasibility of the proposed method: It can not only achieve precise segmentations for images with heavy speckle and nonlocal intensity variations but also be used for SAR images from different types of sensors.

This paper presents a flexible shape-based texture analysis method by investigating the co-occurrence patterns of shapes. More precisely, a texture image is represented by a tree of shapes, each of which is associated with several attributes. The modeling of texture is thus converted to characterize the tree of shapes. To this aim, we first learn a set of co-occurrence patterns of shapes from texture images, then establish a bag-of-words model on the learned shape co-occurrence patterns (SCOPs), and finally use the resulting SCOPs distributions as features for texture analysis. In contrast with existing work, the proposed method not only inherits the strong ability to depict geometrical aspects of textures and the high robustness to variations of imaging conditions from the shape-based texture analysis method, but also provides a more flexible way to model shape relationships (high-order statistics) on the tree. To our knowledge, this is the first time to use co-occurrence patterns of explicit shapes as a tool for texture analysis. Experiments of texture retrieval and classification on various databases report state-of-the-art results and demonstrate the efficiency of the proposed method.

This paper presents a method for SAR image segmentation by relying on active contour model with the non-local processing principle [1]. The idea is to partition a SAR image via computing the patch similarity in the SAR image non-locally, and formulize the segmentation problem with an active contour model. More precisely, after computing the statistical features of SAR images, non-local comparisons between feature patches are used to calculate the active contour energy, which is defined by integrating the interactions between pairs of patches inside and outside the segmented region. A level set method is finally used to minimize the non-local energy. Compared with existing approaches for SAR image segmentation, the only requirement of this method is a local similarity between patches, and it is less sensitive to initial segmentation. The experimental results show the effectiveness and feasibility of the proposed method.

This paper addresses the problem of automatic extraction of built-up areas from high-resolution remote sensing images. We propose a new building presence index from the point view of perception. We argue that built-up areas usually result in significant corners and junctions in high-resolution satellite images, due to the man-made structures and occlusion, and thus can be measured by the geometrical structures they contained. More precisely, we first detect corners and junctions by relying on a perception-inspired corner detector, called an a-contrario junction detector. Each detected corner is associated with a perceptual significance, which measures the structural saliency of the corner in the image and is independent of the contrast and scale. All these detected corners together with their significance are then used to compute the building index. The proposed approach is evaluated on a high-resolution satellite image set, including 15 big images from GeoEye-1, QuickBird and IKONOS. The results demonstrated that our method achieves the state-of-the-art results and can be used in practical applications.

Recently, the combination of topic models and random fields has been frequently and successfully applied to image classification due to their complementary effect. However, the number of classes is usually needed to be assigned manually. This letter presents an efficient unsupervised semantic classification method for high-resolution satellite images. We add label cost, which can penalize a solution based on a set of labels that appear in it by optimization of energy, to the random fields of latent topics, and an iterative algorithm is thereby proposed to make the number of classes finally be converged to an appropriate level. Compared with other mentioned classification algorithms, our method not only can obtain accurate semantic segmentation results by larger scale structures but also can automatically assign the number of segments. The experimental results on several scenes have demonstrated its effectiveness and robustness.

Detecting cars in high-resolution aerial images has attracted particular attention in recent years. However, scene complexity, large illumination change and occlusions make the task very challenging. In this paper, we propose a robust and effective framework for car detection from high-resolution aerial imagery. More specifically, we first incorporate multiple diverse and complementary image descriptors, Histogram of Oriented Gradients (HOG), Local Binary Pattern (LBP) and Opponent Histogram. Subsequently taking computational efficiency and runtime complexity into account, we adopt an interactive bootstrapping approach to collect hard negatives for training an intersection kernel support vector machine (IKSVM). After training, detection is performed by exhaustive search. Finally for post-processing, we employ a greedy procedure for eliminating repetitive detections via non-maximum suppression. Furthermore, contextual information is utilized to refine the detections. Experimental results on Vaihingen dataset have demonstrated that the proposed method can achieve state-of-the-art performance in various real scenes.