We propose an approach for enhancing pedestrian detection in thermal infrared images using paired visible-thermal images in training. Recently, approaches that retrieve the corresponding visible features from thermal features using a key-value memory network have been proven effective for improving detection results. However, for memory networks storing thermal-visible features, random initialization and end-to-end training may not be ideal, as this can reduce the diversity of memory slots. Also, the retrieved visible features have different reliability as the overall similarities between key slots in the memory network and thermal features differ. These motivate us to propose a DIscrepancy Preserving (DIP) Memory that is updated manually to prevent convergence of key-value memory slots. We also evaluate the reliability of each retrieved visible feature and adjust the training protocol of the detection head. Experiment results on two visible-infrared pedestrian detection datasets demonstrate the superiority of our framework.

It has been proved that the fusion of hyperspectral and Li-DAR data can effectively improve the performance of land-use classification. Most recent models have novel architectures which treat hyperspectral and LiDAR data equally and convolutional neural networks are widely used for extracting features of hyperspectral data. We argue that we should pay more attention to hyperspectral data and improve feature extraction tools. This paper proposes a simple yet effective model with parallel transformers. Transformers are powerful in feature extraction and feature fusion. One transformer acts as a hyperspectral image feature extractor, while the other transformer is responsible for capturing cross-modal interactions. Experiments on Houston dataset and MUUFL Gulfport dataset demonstrate that the proposed model has significantly better performance than other state-of-the-art models.

Aerial Image Segmentation is a particular semantic segmentation problem and has several challenging characteristics that general semantic segmentation does not have. There are two critical issues: The one is an extremely foreground-background imbalanced distribution, and the other is multiple small objects along with the complex background. Such problems make the recent dense affinity context modeling perform poorly even compared with baselines due to over-introduced background context. To handle these problems, we propose a point-wise affinity propagation module based on the Feature Pyramid Network (FPN) framework, named PointFlow. Rather than dense affinity learning, a sparse affinity map is generated upon selected points between the adjacent features, which reduces the noise introduced by the background while keeping efficiency. In particular, we design a dual point matcher to select points from the salient area and object boundaries, respectively. Experimental results on three different aerial segmentation datasets suggest that the proposed method is more effective and efficient than state-of-the-art general semantic segmentation methods. Especially, our methods achieve the best speed and accuracy trade-off on three aerial benchmarks. Further experiments on three general semantic segmentation datasets prove the generality of our method. Code and models are made available (https://github.com/lxtGH/PFSegNets).

Glass-like objects such as windows, bottles, and mirrors exist widely in the real world. Sensing these objects has many applications, including robot navigation and grasping. However, this task is very challenging due to the arbitrary scenes behind glass-like objects. This paper aims to solve the glass-like object segmentation problem via enhanced boundary learning. In particular, we first propose a novel refined differential module that outputs finer boundary cues. We then introduce an edge-aware point-based graph convolution network module to model the global shape along the boundary. We use these two modules to design a decoder that generates accurate and clean segmentation results, especially on the object contours. Both modules are lightweight and effective: they can be embedded into various segmentation models. In extensive experiments on three recent glass-like object segmentation datasets, including Trans10k, MSD, and GDD, our approach establishes new state-of-the-art results. We also illustrate the strong generalization properties of our method on three generic segmentation datasets, including Cityscapes, BDD, and COCO Stuff. Code and models will be available for further research.

The state-of-the-art object detection networks for natural images have recently demonstrated impressive performances. However the complexity of ship detection in high resolution satellite images exposes the limited capacity of these networks for strip-like rotated assembled object detection which are common in remote sensing images. In this paper, we embrace this observation and introduce the rotated region based CNN (RR-CNN), which can learn and accurately extract features of rotated regions and locate rotated objects precisely. RR-CNN has three important new components including a rotated region of interest (RRoI) pooling layer, a rotated bounding box regression model and a multi-task method for non-maximal suppression (NMS) between different classes. Experimental results on the public ship dataset HRSC2016 confirm that RR-CNN outperforms baselines by a large margin.

Institute of Automation Chinese Academy of Sciences, 95 Zhongguancun East Road, 100190, Beijing, China

Extracting ships from complex backgrounds is the bottleneck of ship detection in high-resolution optical satellite images. In this letter, we propose a nearly closed-form ship rotated bounding box space used for ship detection and design a method to generate a small number of highly potential candidates based on this space. We first analyze the possibility of accurately covering all ships by labeling rotated bounding boxes. Moreover, to reduce search space, we construct a nearly closed-form ship rotated bounding box space. Then, by scoring for each latent candidate in the space using a two-cascaded linear model followed by binary linear programming, we select a small number of highly potential candidates. Moreover, we also propose a fast version of our method. Experiments on our data set validate the effectiveness of our method and the efficiency of its fast version, which achieves a close detection rate in near real time.

Generating object proposals before object detection has become a common way. In this paper, we present a novel method to measure the objectness of bounding boxes using edges. The contours play an important role in object localization and detection. The number of edges that are close to the boundary of a box has strong relationship with the likelihood of the box covering an object. In our method, we adopt a two-step scheme to generate object proposals. In the first step, we count the number of contours close to the box, where we use the proposed “Tile Algorithm” to wipe off the inner edges of a box. In the second step we re-rank the object proposals with a linear SVM classifier across all aspect-ratios for calibration. Experiments on the VOC2007 dataset show that we achieve 96.47% object detection rate with 1000 proposals.

In hyperspectral image (HSI) classification, feature extraction is one important step. Traditional methods, e.g., principal component analysis (PCA) and locality preserving projection, usually neglect the information of within-class similarity and between-class dissimilarity, which is helpful to the improvement of classification. On the other hand, most of these methods, e.g., PCA and linear discriminative analysis, consider that the HSI data lie on a low-dimensional manifold or each class is on a submanifold. However, some class data of HSI may lie on a multimanifold. To avoid these problems, we propose a method for feature extraction in HSIs, assuming that a local region resides on a submainfold. In our method, we deal with the data region by region by taking into account the different discriminative locality information. Then, under the metric learning framework, a robust distance metric is learned. It aims to learn a subspace in which the samples in the same class are as near as possible while the samples in different classes are as far as possible. Encouraging experimental results on two available hyperspectral data sets indicate that our proposed algorithm outperforms many existing feature extract methods for HSI classification.