Building change detection (BCD) is a hot topic in geoscience and remote sensing (RS) with widespread applications. However, most existing BCD methods only focus on areas where changes have occurred, but ignore the change statuses. To address this problem, a building fine-grained change detection (BFCD) task is further explored in this work, which aims to judge the time-related “disappeared”, “appeared”, and “rebuilt” change types of buildings. Meanwhile, a scale- and shape-aware network (S2Net) with prediction decoupling is designed. Firstly, a prediction decoupling framework with dual decoders is built to ensure the prediction consistency with the temporal order of bi-temporal images. Secondly, considering the rebuilt type is the changes between building instances, which are often reflected in the scale and shape differences of the buildings. Thereby, a scale-aware module (ScAM) and a shape-aware module (ShAM) are designed. These two modules help extract the discriminative features of buildings with different scales and shapes for subsequent change detection (CD). In addition, two BCD datasets widely used, LEVIR-CD+ and WHU-CD, are relabeled in this work to support the study of BFCD. Experimental results show that S2Net achieves competitive performance, and its effectiveness is confirmed. The code and datasets will be publicly available at https://github.com/ptdoge/S2Net.

Non-negative Matrix Factorization (NMF) is an effective way to solve the redundancy of non-negative high-dimensional data. Most of the traditional probability-based NMF methods use Gaussian distribution to model the differences between the matrices before and after decomposition. However, the Gaussian distribution is strongly affected by outliers, and it may not fit all datasets accurately when there are no outliers in the data. In this article, we propose a novel Bayesian NMF with the Student’s t-distribution, i.e., TNMF. specifically, in order to reduce the impact of outliers on the algorithm, we use the Student’s t-distribution to fit the data points instead of the Gaussian distribution. In addition, it is possible to adjust the Degree of Freedom (DF) to make the Student’s t-distribution more flexible than the Gaussian distribution to fit data points when there are no outliers. Next, we combine the Automatic Relevance Determination (ARD) prior in our algorithm to simplify the model and allow for better performance of the algorithm. Finally, the article used 10 datasets to design two kinds of experiments, outlier removal and data clustering. The outlier removal results of this proposed algorithm are significantly better than the other methods, and it performs better in clustering compared to the other methods in the majority of cases.

Considering a wide range of applications of nonnegative matrix factorization (NMF), many NMF and their variants have been developed. Since previous NMF methods cannot fully describe complex inner global and local manifold structures of the data space and extract complex structural information, we propose a novel NMF method called dual-graph global and local concept factorization (DGLCF). To properly describe the inner manifold structure, DGLCF introduces the global and local structures of the data manifold and the geometric structure of the feature manifold into CF. The global manifold structure makes the model more discriminative, while the two local regularization terms simultaneously preserve the inherent geometry of data and features. Finally, we analyze convergence and the iterative update rules of DGLCF. We illustrate clustering performance by comparing it with latest algorithms on four real-world datasets.

With the success of ViT (Vision Transformer), Transformer is being increasingly used for hyperspectral image (HSI) classification given its ability to extract global context dependencies. However, existing methods based on transformers tend to classify HSI in the traditional patch-wise manner. Thus, these methods cannot obtain true global features because the inputs of the model are local patches. To solve these problems, a hybrid convolution and ViT network (HCVN) is proposed for HSI classification. HCVN realizes the classification task from the perspective of semantic segmentation, and its input is the entire HSI, which makes it possible to obtain truly meaningful global features. By improving the original ViT, an HCV module is proposed, which enhances the ability of local structure characterization while extracting global features. The HCVN hybrid convolution layer and HCV module realize the extraction and fusion of local and global features. Finally, the dual branch network architecture is used to integrate the spatial and spectral features. Extensive experiments on two datasets verify the effectiveness of the proposed method.

Existing two-dimensional principal component analysis methods can only handle second-order tensors (i.e., matrices). However, with the advancement of technology, tensors of order three and higher are gradually increasing. This brings new challenges to dimensionality reduction. Thus, a multilinear method called MPCA was proposed. Although MPCA can be applied to all tensors, using the square of the F-norm makes it very sensitive to outliers. Several two-dimensional methods, such as Angle 2DPCA, have good robustness but cannot be applied to all tensors. We extend the robust Angle 2DPCA method to a multilinear method and propose Cosine Multilinear Principal Component Analysis (CosMPCA) for tensor representation. Our CosMPCA method considers the relationship between the reconstruction error and projection scatter and selects the cosine metric. In addition, our method naturally uses the F-norm to reduce the impact of outliers. We introduce an iterative algorithm to solve CosMPCA. We provide detailed theoretical analysis in both the proposed method and the analysis of the algorithm. Experiments show that our method is robust to outliers and is suitable for tensors of any order.

In order to address problems, such as radiation and intensity differences in multi-modal images, this letter proposes a novel idea that integrates maximal indices into the construction of a local self-similarity (LSS) descriptor. The LSS vectors at the same angles but different radial intervals are added to construct the max-index similarity map (MISM) and form the proposed descriptor. This novel descriptor is named max-index-based local self-similarity (MLSS). The MLSS descriptor not only captures the shape similarity between images but is also robust to radiation distortions. Furthermore, a fast and robust algorithm is introduced based on the MLSS descriptor. Comprehensive analysis of accuracy, precision, and computational efficiency shows that the proposed method outperforms five other state-of-the-art methods with stable and better performance on nine pairs of multi-modal test images.

With the increasing popularity of Unmanned Aerial Vehicles (UAVs), the accuracy of detecting small objects in large-view images is also expected to increase. However, accurate small object detection is still a challenging problem. Currently, Image Pyramid Network, Feature Pyramid Network (FPN), rich training strategies and data augmentation are widely used to address this problem. To accurately detect small objects, the most important thing is to mine for more feature information. We propose Widened Residual Block (WRB) to break through the bottleneck of residual information gain to extract more feature information. The second is to emphasize or suppress features to prevent small objects from being overwhelmed by a broad background. We introduce an attention mechanism into PANet and propose Enhanced Attention PANet (EA-PANet), which consists of two parts: Context Attention Module (COAM) and Attention Enhancement Module (AEM). COAM outputs attention heatmaps with context, and AEM fuses features from the channel attention module (CAM) and COAM to avoid distraction from a vast background. In addition, we design a lightweight Decoupled Attention Head (DA-head) to dynamically compute important regions for specific tasks and achieve reliable predictions. Experiments show that our method outperforms state-of-the-art (SOTA) detectors. The source code for this work is available at https://github.com/liuxiaolei111/FindSmallObjects.

Transfer learning is an effective way to alleviate the problem of insufficient samples in a hyperspectral image (HSI) classification. However, the present transfer learning-based methods usually transfer knowledge from a single source domain, such as the natural image domain. Therefore, these methods cannot simultaneously transfer spectral and spatial knowledge to the target domain in HSIs. Generally, the natural image has rich spatial structure and texture information, while the HSI has abundant spectral information. To better utilize the knowledge learned from natural image datasets and HSI datasets, we proposed a multimodal transfer feature fusion network (MTFFN) for HSI classification. In MTFFN, a dual-branch network structure is designed to transfer the two-modal knowledge from the natural image domain and the source HSI domain to the target domain in two branches, respectively. A multitask learning strategy is adopted to achieve feature fusion. The fused features are used to generate the final classification result. Moreover, a local attention mechanism is designed to extract more meaningful spectral features. Experiments on two public datasets show that the proposed method is effective (https://github.com/HuaipYan/MTFFN).

Many methods have been developed to improve the performance of image registration. In this letter, we introduce a novel method based on a local affine constraint for remote sensing image registration, which can be widely used in image processing and pattern recognition. Our algorithm has three components. First, we exploit the scale invariant feature transform (SIFT) method to extract feature points and calculate the gradient magnitude to establish feature descriptors with a circular instead of square neighborhood. Second, an initial matching is implemented by the nearest neighbor distance ratio (NNDR) and the fast sample consensus (FSC) algorithm. Finally, fine registration is established using more correct matches obtained by the local affine transformation circular region search algorithm. Experimental results show that the proposed method achieves subpixel accuracy. In addition, both the correct matching rate and registration demonstrate the effectiveness and efficiency of our method.

Recently, it is generally acknowledged that the receptive field size of visual cortical neurons are regulated by the stimulus in the neuroscience community. Thus, once the global receptive field is obtained, the network performance can be greatly improved. Unfortunately, the larger receptive field method has been rarely considered in constructing CNNs. Recent studies on network design have demonstrated that the key to improving model performance is channel attention. However, they usually neglect the location information. Hence, it is difficult to capture the long-term dependency of location information. In particular, the location information is important for generating spatially selective attention blocks. Therefore, in this paper, we propose a novel attention neural network termed “GEA” by embedding global context information into channel attention. Firstly, instead of channel attention, which is transformed from feature tensor to single feature vector via 2D pooling, our method decomposes the channel attention into two 1D feature encoding processes that aggregate features along two spatial directions. In particular, the long-term dependency can be captured by using one spatial direction as well as preserving accurate location information in the opposite direction. Then, we concatenate the results of the two directions, while carry out batch normalization and use relu activation function. In addition, the cross channel soft attention is used to adaptively select different spatial scales of the information. Finally, our method is simple and can be flexibly inserted into classic networks, such as ResNet and EfficientNet, with limited computational overhead. A large number of experiments show that our method is not only conducive to the classification of COCO and ImageNet, but also demonstrates promising performance for 3D features, such as Magnetic Resonance Imaging(MRI).

We propose a novel NMF algorithm, named Total Variation constrained Graph-regularized Convex Non-negative Matrix Factorization (TV-GCNMF), to incorporate total variation and graph Laplacian with convex NMF. In this model, the feature details of the data are preserved by a diffusion coefficient based on the gradient information. The graph regularization and convex constraints reveal the intrinsic geometry and structure information of the features; thereby, obtaining sparse and parts-based representations. Furthermore, we give the multiplicative update rules and prove convergence of the proposed algorithm. The results of clustering experiments on multiple datasets, under various noise conditions, show the effectiveness and robustness of the proposed method compared to state-of-the-art clustering methods and other related work.

Summary Face anti‐spoofing has attracted many attentions in security applications, such as mobile payment and entrance guard. Until now, face anti‐spoofing technique is still a challenging task. Mainstream image‐based spoofing algorithms usually use global motion or texture information to distinguish whether an input face is live or fake. However, the performance of these methods are sensitive in light changes, or images acquired from different sensors. The main reason is that spoofed face image always has slight different texture in local areas, such as landmark or salient region of face. To this end, this paper proposes a novel multi‐patches feature extraction strategy to detect spoofing. First, a set of patches with specific combination scheme is selected to cover the face image. Second, features such as hand‐crafted Gray Level Co‐occurrence Matrix (GLCM), Local Binary Patterns (LBP), or deep features are extracted from these patches. Third, all features are combined as the global descriptor of the face image, then fed into an SVM classifier to verify the anti‐spoofing detection. Experimental results show that the proposed strategy can effectively enhance the performance, concerning with the accuracy of spoofed face detection in four widely used anti‐spoofing databases.

This paper presents a novel, fast image segmentation method based on normalized Gaussian distance on nodes in conjunction with normalized graph cuts. We review the equivalence between kernel k-means and normalized cuts. Then we extend the framework of efficient spectral clustering and avoid choosing weights in the weighted graph cuts approach. Experiments on synthetic data sets and real-world images demonstrate that the proposed method is effective and accurate.

This paper presents a novel perspective on characterizing the spectral correspondence between the nodes of weighted graphs for image matching applications. The algorithm is based on the principal feature components obtained by stochastic perturbation of a graph. There are three areas of contributions in this paper. First, a stochastic normalized Laplacian matrix of a weighted graph is obtained by perturbing the matrix of a sensed graph model. Second, we obtain the eigenvectors based on an eigen-decomposition approach, where representative elements of each row of this matrix can be considered to be the feature components of a feature point. Third, correct correspondences are determined in a low-dimensional principal feature component space between the graphs. In order to further enhance image matching, we also exploit the random sample consensus algorithm, as a post-processing step, to eliminate mismatches in feature correspondences. The experiments on synthetic and real-world images demonstrate the effectiveness and accuracy of the proposed method.