Multimodal change detection (MCD) has important applications in disaster assessment, but the nonlinear distortion of features and spatial misalignment caused by sensor imaging differences make it difficult to obtain changes through direct comparison. To overcome the above problems, this study aims to realize MCD by capturing the modality-independent structural commonality features between Multimodal Remote Sensing Images (MRSIs). To achieve this, we devise a basic Graph Kolmogorov-Arnold Network (GKAN) to excavate spatial structural relationships and cross-modal nonlinear mappings simultaneously. Based on this, we propose a Dual-branch GKAN (DGKAN) for unsupervised MCD, which can capture spatial-spectral structural commonality features and compare them directly to detect changes. Concretely, the GKAN is used within the DGKAN to build two autoencoders consisting of a Siamese encoder and two independent decoders to learn spatial-spectral structural commonality features through feature reconstruction. Besides, we introduce a Covariance Structural Commonality Loss (CSCL), which guides the network in extracting spatial-spectral structural commonality features between MRSIs by unsupervised constraints on the distributional consistency of cross-modal features. Experiments on several MCD datasets show that the proposed DGKAN can achieve convincing results, and ablation studies verify the effectiveness of the GKAN and CSCL.

Due to the inherent inductive bias of operations, convolutional neural networks (CNN) cannot model global information of remote sensing (RS) images. In contrast, Transformer-based methods can establish long-range dependencies of images through self-attention (SA) mechanism, but it faces the challenges of computational complexity and memory requirements, but also ignores the exploration on the feature redundancy removal of RS images. To address these two issues, we propose a network based on dynamic sparse encoding and cross-temporal collaborative attention (DSECTCA-Net) for RS image change detection (CD). First, we implement dynamic sparse encoding (DSE) by designing hierarchical sparse Transformer module (HSTM), which decreases the correlation calculation of the SA mechanism and effectively reduces the computational complexity and parameter amount of Transformer. Secondly, we propose cross-temporal collaborative attention (CTCA) to model RS images in time series and fully explore the interactivity between dual-temporal RS images, so as to better extract the global understanding of visual scenes. Extensive experiments on two large-scale public RS datasets show that the proposed method not only provides higher detection accuracy, but also achieves lower computational complexity and required storage space than most popular CD networks.

Contrastive learning plays an important role in the semi-supervised medical image segmentation. However, existing contrastive learning methods struggle to capture the correlation of global and local features and improve feature discrimination for complex medical scenes, resulting in poor segmentation performance in challenging polyp segmentation. To overcome these limitations, we propose a semi-supervised polyp segmentation method using Hierarchical Granularity Contrastive Learning (HGCL). HGCL has two advantages. First, we design a hierarchical spatial contrastive learning module to divide the feature maps into large and small regions and perform different region-level contrastive learning, which can effectively capture the correlation of global and local information and improve the intra-class cohesion and inter-class separation. Second, we design a fine-granularity contrastive learning module, which can perform finer pixel-level contrastive learning to capture finer subtle local features and improve the generalization capacity of HGCL for complex medical scenes. Extensive experiments on three publicly available polyp datasets demonstrate that HGCL can achieve the better segmentation performance than existing popular semi-supervised methods. The code is available at https://github.com/Milk-White/HGCL.

Underwater images suffer from light absorption and scattering, impairs their visibility and applications. Existing underwater image restoration (UIR) methods based on generative models struggle are difficult to adapt to the complex and dynamic underwater environments characterized by illumination interference, low-light conditions, and non-uniform turbidity. To address these issues, we propose Water-CDM, a novel Adaptive Double-Branch Fusion Conditional Diffusion Model for underwater image restoration. Specifically, an adaptive double-branch fusion conditional diffusion model is presented utilizing a U-shaped full-attention network and Guided Multi-Scale Retinex with Brightness Correction (GMSRBC) to restore the challenging regions within underwater images. More precisely, to correct color casts and enhance the sharpness of underwater images, a U-shaped full-attention network incorporating Attention Blocks is designed for noise estimation during the reverse process of the conditional diffusion model. Concurrently, to mitigate overexposure during the enhancement of low-light underwater images under illumination interference, the GMSRBC method, featuring an Adaptive Brightness Correction Module, is proposed to efficiently adjust the brightness of underwater images. Experimental results demonstrate that the proposed Water-CDM significantly improves the quality of underwater images in challenging scenarios. Encouragingly, our proposed Water-CDM yields superior restoration outcomes compared to current state-of-the-art methods on three challenging publicly available datasets. Our codes will be released at: https://github.com/HKandWJJ/Water-CDM.

Road change detection (RCD) is crucial for intelligent transportation, disaster assessment, and urban planning. However, current general change detection (CD) methods focus on various targets, such as buildings, while less attention is paid to the CD of narrow and elongated roads. Compared with general CD, RCD may still be limited by the following two aspects: On the one hand, the road usually occupies a small proportion of pixels in remote sensing images (RSIs) and is often easily blocked by buildings, trees, etc., making it difficult to ensure the integrity and connectivity of road structural features in RCD. On the other hand, RCD may easily be confused with the semantic information of similar material backgrounds (such as parking lots and building roofs) due to the lack of salient road features. To overcome the above limitations, we propose a skeleton-guided large-kernel attention Transformer (SLAFormer) for RCD, which can focus on salient road structural and semantic features to enhance its performance. In the proposed SLAFormer, we construct a novel skeleton-guided large-kernel attention module (SLKAM) and a frequency-guided cross spatial-channel difference module (FSCDM) to achieve the above goals. The SLKAM is used to make the model focus on road-specific skeleton features, which preserve the overall structure and morphology of roads to enhance the continuity and integrity of road features. In addition, the FSCDM is devised to better capture small-scale road changes and reduce semantic confusion with similar backgrounds, thereby enhancing change regions and extracting highly discriminative road difference information. Extensive experiments on two public RCD datasets show that ours achieves better RCD accuracy compared with several state-of-the-art (SOTA) approaches. The code will be available at https://github.com/TongfeiLiu/SLAFormer-for-RCD.

Multimodal change detection (MCD) has garnered significant interest due to its capacity to address a variety of emergencies in a timely and effective manner. However, discrepancies in sensors and imaging techniques often hinder the direct comparison of heterogeneous remote sensing images (HRSIs), making it difficult to extract change information. To overcome this challenge, we propose a novel superpixel-based AutoEncoder Kolmogorov-Arnold Network (AEKAN) for unsupervised MCD. The primary objective of AEKAN is to excavate the latent commonality features between HRSIs. Notably, commonality features in unchanged regions are generally more pronounced than those in changed regions, which can be leveraged to assess change magnitude. To achieve this, the proposed method utilizes the Kolmogorov-Arnold Network (KAN), renowned for its capability to model data distributions, to extract these commonality features between HRSIs. Concretely, the proposed AEKAN consists of a Siamese KAN encoder and dual KAN decoders. The Siamese encoder aims to map HRSIs and extract latent commonality features, while the dual decoders reconstruct original bitemporal images from these features. In addition, we incorporate a hierarchical commonality loss function within the Siamese encoder to train AEKAN. This loss function is designed to intentionally guide the network in capturing commonality features by minimizing the discrepancies in features extracted from HRSIs at each layer of the Siamese encoder. The extracted commonality features are then adopted to quantify the change magnitude between images through mean square error (MSE). Extensive experiments on five MCD datasets demonstrate that the proposed AEKAN outperforms existing methods. The source code is available at:https://github.com/TongfeiLiu/AEKAN-for-MCD.

Sample augmentation is crucial for improving land cover classification performance when the samples are limited. However, the traditional sample augmentation approach concentrates on enlarging the quantity of sample via generation and synthetic technique directly, the sample quality is usually neglected. In this article, we propose a novel sample augmentation approach with threshold estimation (SATE) to improve both the quantity and quality of samples for hyperspectral remotely sensed image (HRSI) classification. Firstly, a threshold estimation algorithm (TEA) is proposed to identify high-confidence potential samples from the initial classification map by utilizing the prediction probabilities of different classes. Second, a semi-variational model is employed to detect and correct pseudo-labels in the spatial domain, further enhancing the quality of selected potential samples. Finally, a farthest point sampling (FPS) algorithm optimizes sample distribution in the spectral domain, improving representation for intra-class heterogeneity. Experimental results based on four real HRSIs and compared with eight state-of-the-art few-shot-based methods verify the feasibility and superiority of the proposed SATE approach. The improvement achieved by our proposed approach is about 0.79% ~ 4.31% in terms of the overall accuracy. Code is available at https://github.com/ImgSciGroup/SATE.

Visual foundation models (VFMs) have been widely applied in the field of remote sensing (RS). However, they still face two main challenges when applied to precise remote sensing change detection (RSCD) tasks in complex scenes. Firstly, the nonnegligible domain shift between natural scene and RS scene limits the direct application of VFMs to the RSCD task. Second, most of existing RSCD methods may suffer from the boundary displacement problem due to the inadequate exploration of temporal differences for bi-temporal features. To address the above issues, this study proposes a SAM2-based domain adaptive and spatial difference aggregation network (DA2-Net) for RSCD. The proposed DA2-Net has two main advantages. First, a hierarchical low-rank adaptation (LoRA) strategy is presented by introducing low-rank matrices at key positions of SAM2, which can inject inductive biases from the RS domain into the network and alleviate the domain shift problem. Second, a difference adaptive enhancement module (DAEM) is designed to explore temporal differences for hierarchical bi-temporal features. The DAEM provides respective attention weights for different information through a dual branch of global difference awareness and local detail optimization. Experimental results on SYSU-CD, WHU-CD, and LEVIR-CD datasets demonstrate the superiority of DA2-Net. Code is available at https://github.com/xuptheqi-hash/ DA2Net.

The following three factors restrict the application of existing low-light image enhancement methods: corruptions induced by the light-up process, color distortion, and a restricted generalization capacity due to limited paired training data. To address these limitations, we first combine HSV theory and Transformer, proposing a robust unsupervised low-light image enhancement framework, named HSVFormer. Secondly, we introduce brightness disturbance and design an unsupervised value enhancement network, which estimates brightness information and restores degraded brightness information to obtain enhanced reflectance. Finally, we utilize the V-subspace and devise a value-guided multi-head channel self-attention to capture brightness representations of regions with different brightness conditions and guide the modeling of non-local interactions. Experiment results on publicly available datasets demonstrate that HSVFormer can achieve superior performance compared with state-of-the-art approaches. The code is available at https://github.com/m0fig/HSVFormer.

Popular Transformer networks have been successfully applied to remote sensing (RS) image change detection (CD) identifications and achieved better results than most convolutional neural networks (CNNs), but they still suffer from two main problems. First, the computational complexity of the Transformer grows quadratically with the increase of image spatial resolution, which is unfavorable to RS images. Second, these popular Transformer networks tend to ignore the importance of fine-grained features, which results in poor edge integrity and internal tightness for largely changed objects and leads to the loss of small changed objects. To address the above issues, this letter proposes a lightweight structure-aware Transformer (LSAT) network for RS image CD. The proposed LSAT has two advantages. First, a cross-dimension interactive self-attention (CISA) module with linear complexity is designed to replace the vanilla self-attention (SA) in the visual Transformer, which effectively reduces the computational complexity while improving the feature representation ability of the proposed LSAT. Second, a structure-aware enhancement module (SAEM) is designed to enhance difference features and edge detail information, which can achieve double enhancement by difference refinement and detail aggregation to obtain fine-grained features of bi-temporal RS images. Experimental results show that the proposed LSAT achieves significant improvement in detection accuracy and offers a better tradeoff between accuracy and computational costs than most state-of-the-art (SOTA) CD methods for RS images.

Supervised classification with hyperspectral remote-sensing images (HRSIs) plays an important role in practical applications. However, labeling samples with HRSIs for supervised classification is time-consuming and labor-intensive. In this letter, we propose a new sample enhancement approach to improve the classification performance of HRSIs. First, the uncertainty and representativeness of the sample are defined to achieve sample possibility measurement for each pixel, and some pixels with high possibility can be selected as candidate samples. Then, two rules related to label correlation analysis and spectral similarity are defined to further refine the candidate samples used for generating the final sample set. Finally, the above-mentioned steps are fused into an iterative algorithm to enhance and balance the training samples for each class. The feasibility of the proposed approach was verified by applying it to classification with two real HRSIs. A comparison with some typical traditional sample enhancement methods and widely used few-shot deep-learning methods indicated the advantages of the proposed approach for improving classification accuracies. The improvement achieved by our proposed approach is about 0.79% ~ 2.31% in terms of the overall accuracy (OA). The code of the proposed approach is available athttps://github.com/ImgSciGroup/2023-GRSL-SIEA.

Landslide is one of the most serious geological disasters around the world, and acquiring landslide inventory mapping (LIM) with remote sensed images (RSIs) plays an important role in disaster relief. However, various external imaging conditions of bitemporal RSIs usually cause pseudo-changes and challenges for achieving satisfied LIMs. In this article, a pioneering change magnitude measured distance named Spectral-Spatial Similarity based on Adaptive Region (S3AR) is proposed for achieving LIMs with bitemporal RSIs. First, an adaptive region is proposed to utilize the spatial-contextual information around each pixel, because the shapes and size of a landslide site are usually irregular and unpredictable. Then, a shape description algorithm is proposed for constructing a shape description vector, which aims at measuring the spatial difference of adaptive regions. Finally, to improve the separability between the landslide area and the background, brightness is suggested to couple with the shape description vector of an adaptive region to generate spatial-spectral similarity to measure the change magnitude between pairwise adaptive regions from the bitemporal RSIs. When the entire bitemporal RSIs are scanned and calculated via these steps, a change magnitude image between bitemporal RSIs can be generated, and then binary LIMs are obtained by a binary threshold. Experiments based on comparing eight state-of-the-art approaches demonstrated the feasibility and superiorities of the proposed S3AR for achieving LIMs with bitemporal RSIs. For example, the improvements on the four datasets are 5.81%, 14.06%, 6.03%, and 20.51% in terms of total error.

Sample augmentation is effective for improving the supervised performance of land-cover classification with hyperspectral remote sensed image (HRSI) when the training samples are limited. However, numerous existing methods have neglected, considering the interclass-imbalance problem in the process of sample augmentation. In this work, new sample generation and sample balance strategies were promoted and simultaneously combined into an iteration for balancing and improving classification performance with HRSI. First, a sample augmentation with superpixel’s constraint (SASC) is designed to augment the initial training samples set to avoid the overfitting of a sample generation neural network. Second, sample generation based on generative adversarial network (SGGAN) was proposed to generate samples for each class. Then, the proposed SASC, SGGAN, and a pattern recognition neural network named 3 dimensions-convolutional neural network (3-D-CNN) are combined into an iterative classification process called iterative sample generation and balance (ISGB) for balancing the user’s accuracy for each class and optimizing the classification performance. Experiments on four widely used HRSIs are performed. The results when compared with eight state-of-the-art methods based on few-shot learning and generative adversarial network (GAN) efficiently demonstrate the feasibility and superiorities of the proposed approach for improving land-cover classification performance when the initial samples are limited. Moreover, the comparisons of the standard deviation of the user’s accuracies (SDUA) demonstrated the balancing ability of the proposed approach. The code of the proposed approach is available athttps://github.com/ImgSciGroup/ISGBA.

Remote sensing building change detection (RSBCD) detects changes in the spatial distribution of buildings which is of great significance for urban planning and construction. Existing deep learning-based RSBCD methods usually suffer from low object completeness and erroneous detection problem, mainly due to insufficient utilization of difference information between bi-temporal images. To address the above issues, this article proposed a new Siamese ensemble difference attention network (SEDANet) for RSBCD tasks in very-high resolution (VHR) images. Firstly, the key module ensemble difference attention module (EDAM) is designed to effectively extract difference representation between the bi-temporal features and filter out irrelevant changes. EDAM calculates difference map of bi-temporal features and transforms the extracted change information into trainable difference attention weights. The output weights from EDAM works as a guidance for both spatial and channel visual attention process, which enables the network to focus on foreground building changes and further resolve erroneous attention problems in existing RSBCD methods. The Siamese structure is adopted to better represent bi-temporal features, and convolutional blocks are replaced with residual convolution blocks (RCBs) to speed up network fitting and prevent gradient explosion or descent. We conduct comprehensive experiments on three benchmark datasets. Both visual and quantitative results show that our proposed SEDANet is superior to other eight state-of-the-art networks. Especially on GZ-CD dataset, SEDANet outperforms other comparison methods by 3%-8%. In addition, the effectiveness of EDAM module is also discussed through a series of ablation studies.

The popular networks for change detection (CD) in very-high-resolution (VHR) remote sensing (RS) images usually suffer from two problems. First, it is difficult for these networks to model simultaneously the local and global features of changed targets, which leads to the limited feature representation ability of popular CD networks. Second, these networks often have a large number of parameters and high computational costs due to complex network architecture. To address the above issues, we propose a local-global siamese network (LGS-Net) for CD in VHR RS images. First, we design an encoder with a parallel dual-branch structure consisting of convolutional neural networks (CNNs) and Transformer to extract rich features from bi-temporal images. Furthermore, we design a local-global feature enhancement (LGFE) module to help our encoder improve its feature representation ability. Second, we design a compact and efficient convolution module called inter-scale separable convolution (ISSConv). This module first divides feature maps into multiple groups, and then performs depthwise separable convolution in each group using atrous convolution with different dilation rates, which can not only capture changed targets across scales but also effectively reduce the number of model parameters. Experiments demonstrate that the proposed LGS-Net is superior to the state-of-the-art CD networks in terms of parameters, computational costs, and detection accuracy.

Polyp segmentation is of great importance for the diagnosis and treatment of colorectal cancer. However, it is difficult to segment polyps accurately due to a large number of tiny polyps and the low contrast between polyps and the surrounding mucosa. To address this issue, we design an Adaptive Tiny-object Enhanced Network (ATENet) for tiny polyp segmentation. The proposed ATENet has two advantages: First, we design an adaptive tiny-object encoder containing three parallel branches, which can effectively extract the shape and position features of tiny polyps and thus improve the segmentation accuracy of tiny polyps. Second, we design a simple enhanced feature decoder, which can not only suppress the background noise of feature maps, but also supplement the detail information to improve further the polyp segmentation accuracy. Extensive experiments on three benchmark datasets demonstrate that the proposed ATENet can achieve the state-of-the-art performance while maintaining low computational complexity.

The hybrid architecture of convolutional neural networks (CNNs) and Transformer are very popular for medical image segmentation. However, it suffers from two challenges. First, although a CNNs branch can capture the local image features using vanilla convolution, it cannot achieve adaptive feature learning. Second, although a Transformer branch can capture the global features, it ignores the channel and cross-dimensional self-attention, resulting in a low segmentation accuracy on complex-content images. To address these challenges, we propose a novel hybrid architecture of convolutional neural networks hand in hand with vision Transformers (CiT-Net) for medical image segmentation. Our network has two advantages. First, we design a dynamic deformable convolution and apply it to the CNNs branch, which overcomes the weak feature extraction ability due to fixed-size convolution kernels and the stiff design of sharing kernel parameters among different inputs. Second, we design a shifted-window adaptive complementary attention module and a compact convolutional projection. We apply them to the Transformer branch to learn the cross-dimensional long-term dependency for medical images. Experimental results show that our CiT-Net provides better medical image segmentation results than popular SOTA methods. Besides, our CiT-Net requires lower parameters and less computational costs and does not rely on pre-training. The code is publicly available at https://github.com/SR0920/CiT-Net.

Land cover change detection (LCCD) with bitemporal remote sensing images has been widely used in practical applications. However, when the bitemporal images are multimodal remote sensing images (MRSIs) which are acquired with different sensors, the change detection performance may be unsatisfactory, because MRSIs cannot be compared directly to generate a change magnitude and obtain a change detection map. Here a novel approach is proposed to overcome this problem, i.e., the Enhanced UNet (E-UNet) which learns deep shared features from MRSIs to achieve change detection with MRSIs. First, apre-event image to post-eventimage (P2P) transformation module based on classical Cycle-consistent Generative Adversarial Network (CGAN) is suggested to embed at the head of the proposed E-UNet to translate the pre-event image to a post-event image one. Then, multi-scale convolutions are added at each encoding layer to capture the various shapes and sizes of ground targets. Finally, a Polarized Self-Attention (PSA) module is employed before beginning the decoding progress of E-UNet with an aim to pay extra attention to changed areas. Compared with five typical state-of-the-art methods, experimental results based on two pairs of MRSIs well demonstrated the feasibility and advantages of the proposed E-UNet for LCCD with MRSIs in terms of visual observations and quantitative evaluations. For example, the improvement is 4.19% and 4.75% in terms of the overall accuracy for the Sardinia dataset and California dataset, respectively. The code of the proposed approach can be found at https://github.com/ImgSciGroup/E-UNet.

Deep convolutional neural networks have achieved much success in remote sensing image change detection (CD) but still suffer from two main problems. First, existing multi-scale feature fusion methods often employ redundant feature extraction and fusion strategies, which often leads to high computational costs and memory usage. Second, the regular attention mechanism in CD is difficult to model spatial-spectral features and generate 3D attention weights at the same time, ignoring the cooperation between spatial features and spectral features. To address the above issues, an efficient ultra-lightweight spatial-spectral feature cooperation network (USSFC-Net) is proposed for CD in this paper. The proposed USSFC-Net has two main advantages. First, a multi-scale decoupled convolution (MSDConv) is designed, which is clearly different from the popular atrous spatial pyramid pooling (ASPP) module and its variants since it can flexibly capture the multi-scale features of changed objects by using cyclic multi-scale convolution. Meanwhile, the design of MSDConv can greatly reduce the number of parameters and computational redundancy. Second, an efficient spatial-spectral feature cooperation strategy (SSFC) is introduced to obtain richer features. The SSFC differs from existing 2D attention mechanisms since it learns 3D spatial-spectral attention weights without adding any parameters. The experiments on three datasets for remote sensing image CD demonstrate that the proposed USSFC-Net achieves better CD accuracy than most convolutional neural networks-based methods and requires lower computational costs and fewer parameters, even it is superior to some Transformer-based methods. The code is available at https://github.com/SUST-reynole/USSFC-Net.

Deep learning techniques have become popular in land cover change detection (LCCD) with remote sensing images (RSIs). However, many existing networks mostly concentrate on learning deep features but without considering the effect of different features’ attention and fusion strategy on detection performance. In this paper, a novel hierarchical attention feature fusion (HAFF)-based network for LCCD with RSIs is proposed. In the proposed HAFF-based network, novel multi-scale convolution fusion filters (MCFFs) explore the global semantic feature of the interested targets from multi-perspectives ways. To achieve that objective, the proposed MCFFs are composed by a well-known position attention module (PAM) and a novel multi-perspectives feature filter block with different kernel sizes. In addition, a compound loss function was proposed for balancing the impact from the features at different levels in terms of backpropagation error. Experiments conducted on six pairs of real RSIs, including three pairs of homogeneous images and three pairs of heterogeneous images, confirmed the superiority of the proposed HAFF network over other cognate methods. Moreover, the ablation experiments further confirmed the feasibility and superiority of the proposed MCFFs, whereas quantitative observations indicated that competitive improvements are achieved by the proposed MCFFs in terms of all the evaluation indicators. The code for the proposed approach will be available at https://github.com/ImgSciGroup/HAFF.

Samples play a crucial role in the supervised classification of remote sensing images. However, labeling large samples for training a classifier or deep learning network is not only time-consuming but also labor-intensive. In this paper, a novel land cover classification with nonparametric sample augmentation is proposed to improve the performance of hyperspectral remote sensing images (HRSIs) classification. First, initial samples with limited quantity are selected randomly from the ground truth map. Second, based on the gray image, a nonparametric adaptive region generation (NARG) algorithm is developed for utilizing the contextual information around each sample. Then, an nonparametric sample augmentation algorithm is developed with NARG to explore reliable samples iteratively around each initial sample. Finally, the above steps are fused into an iterative progress to obtain the final classification map. Compared with some typical traditional methods and some widely used deep learning methods based on four real HRSIs, our proposed approach exhibits some advantages in improving the visual performance and quantitative accuracies of HRSIs classification, such as the improvement is about 2.0% ~ 10.34% for four real HRSIs in term of the overall accuracy.

Land cover change detection (LCCD) using bitemporal remote sensing images is a crucial task for identifying the change areas on the Earth’s surface. However, the utilization of hyperspectral remote sensing images (HRSIs) introduces challenges as the detection performance is affected by the spectral noise and deducing change detection accuracies. In this work, we concentrated on utilizing spatial-contextual information to improve the change detection performance while using HRSIs. First, a band selection approach is used to minimize the spectral redundancy of HRSIs. Second, an iterative spatial-adaptive filter is proposed to smooth the noise of HRSIs. Thereafter, the change magnitude between bitemporal HRSIs is measured by coupling change vector analysis and the adaptive region around each pixel, resulting in a change magnitude image (CMI). Subsequently, the CMI is divided into a binary change detection map by using an Ostu threshold method. The experimental results on three pairs of real HRSIs efficiently demonstrated the feasibility and superiorities of the proposed approach compared with six state-of-art methods. For example, the improvement rates are approximately 0.43%-11.83% and 1.05%-15.41% for overall accuracy and average accuracy, respectively. The code of our proposed approach will be available at: https://github.com/ImgSciGroup/2023-HSICD.

Although deep learning-based change detection (CD) methods achieve great success in remote sensing images, they still suffer from two main challenges. First, popular Convolutional Neural Networks (CNNs) are weak in extracting discriminated features focusing on changed regions, since most methods ignore the multi-frequency components of bi-temporal images. Second, although existing CD methods employ the Transformer structure to capture long-range dependency for global feature representation, it is difficult for them to simultaneously take into account the long-range dependency of changed objects at various scales. To address the above issues, we propose a triple change detection network (TCD-Net) via joint multi-frequency and full-scale Swin-Transformer. The proposed TCD-Net has two main advantages. First, we propose a multi-frequency channel attention (MFCA) module to boost the ability of modeling the channel correlation, which can compensate for the problem of insufficient feature representation caused by only performing global average pooling (GAP). Furthermore, a joint multi-frequency difference feature enhancement (JM-DFE) guiding block is proposed to improve the boundary quality and the position awareness of truly changed objects, which can effectively extract channel features of multi-frequency information and thus improve the discriminative ability of features. Second, unlike Siamese-based structures, we propose a full-scale Swin-Transformer (FST) module as the third branch to model and aggregate the long-range dependency of multi-scale changed objects, which can alleviate the missed detections of small objects and achieve more compact changed regions effectively. Experiments on three public CD datasets exhibit that the proposed TCD-Net achieves better CD accuracy with smaller model complexity than state-of-the-art methods. The code is publicly available at https://github.com/RSCD-mz/TCD-Net.

Convolutional neural networks (CNNs) have achieved significant success in medical image segmentation. However, they also suffer from the requirement of a large number of parameters, leading to a difficulty of deploying CNNs to low-source hardwares, e.g., embedded systems and mobile devices. Although some compacted or small memory-hungry models have been reported, most of them may cause degradation in segmentation accuracy. To address this issue, we propose a shape-guided ultralight network (SGU-Net) with extremely low computational costs. The proposed SGU-Net includes two main contributions: it first presents an ultralight convolution that is able to implement double separable convolutions simultaneously, i.e., asymmetric convolution and depthwise separable convolution. The proposed ultralight convolution not only effectively reduces the number of parameters but also enhances the robustness of SGU-Net. Secondly, our SGU-Net employs an additional adversarial shape-constraint to let the network learn shape representation of targets, which can significantly improve the segmentation accuracy for abdomen medical images using self-supervision. The SGU-Net is extensively tested on four public benchmark datasets, LiTS, CHAOS, NIH-TCIA and 3Dircbdb. Experimental results show that SGU-Net achieves higher segmentation accuracy using lower memory costs, and outperforms state-of-the-art networks. Moreover, we apply our ultralight convolution into a 3D volume segmentation network, which obtains a comparable performance with fewer parameters and memory usage.

To investigate the diversified technologies in Internet of Vehicles (IoVs) under intelligent edge computing, brain-inspired computing techniques are proposed in this study, which is a promising biologically inspired method by using brain cognition mechanism for various applications. A neuromorphic approach in a scalable and fault-tolerant framework is presented, targeting to realize the navigation function for the edge computing in IoV applications. A novel fault-tolerant address event representation approach is proposed for the spike information routing, which makes the presented model both scalable and fault-tolerant. Experimental results reveal that the proposed approaches can enhance the communication distance, the load balancing and the maximum throughput of the neuromorphic system accordingly. Based on the proposed neuromorphic model, the effects of the dopamine level are investigated. Besides, the results show that the proposed work can realize the accurate obstacle avoidance for the edge IoV computing, and the performance of the proposed network is superior to the network without the proposed scalable and fault-tolerant design. Therefore, the proposed IoV model provides an experimental basis for the improvement of the IoV system.

Popular semi-supervised medical image segmentation networks often suffer from error supervision from unlabeled data since they usually use consistency learning under different data perturbations to regularize model training. These networks ignore the relationship between labeled and unlabeled data, and only compute single pixel-level consistency leading to uncertain prediction results. Besides, these networks often require a large number of parameters since their backbone networks are designed depending on supervised image segmentation tasks. Moreover, these networks often face a high over-fitting risk since a small number of training samples are popular for semi-supervised image segmentation. To address the above problems, in this paper, we propose a novel adversarial self-ensembling network using dynamic convolution (ASE-Net) for semi-supervised medical image segmentation. First, we use an adversarial consistency training strategy (ACTS) that employs two discriminators based on consistency learning to obtain prior relationships between labeled and unlabeled data. The ACTS can simultaneously compute pixel-level and image-level consistency of unlabeled data under different data perturbations to improve the prediction quality of labels. Second, we design a dynamic convolution-based bidirectional attention component (DyBAC) that can be embedded in any segmentation network, aiming at adaptively adjusting the weights of ASE-Net based on the structural information of input samples. This component effectively improves the feature representation ability of ASE-Net and reduces the overfitting risk of the network. The proposed ASE-Net has been extensively tested on three publicly available datasets, and experiments indicate that ASE-Net is superior to state-of-the-art networks, and reduces computational costs and memory overhead. The code is available at: https://github.com/SUST-reynole/ASE-Nethttps://github.com/SUST-reynole/ASE-Net.

The popular convolutional neural networks (CNNs) have been successfully used in very high-resolution remote sensing image semantic segmentation. However, these networks often suffer from performance limitations. First, although deeper networks usually provide better feature representation, they may cause parameter redundancy and the inefficient use of prior knowledge. Secondly, attention-based networks often only focus on weighting different features of a single sample but ignore the correlation of all samples in training set, thus leading to the loss of global information. To address above issues, we propose two simple yet effective global evolution strategies. The first is knowledge enhancement. This strategy can reactivate invalid convolutional kernels through convergence of different models and make full use of prior knowledge from the network to improve its feature representation. The second is a dict-attention module that greatly enhances the generalization of networks by learning and inferring the global relationship among different samples through the dictionary unit. As a result, a novel global evolution network (GENet) is designed based on knowledge enhancement and dict-attention for remote sensing image semantic segmentation. Experiments demonstrate that the proposed GENet is not only superior to popular networks in segmentation accuracy.

Popular semi-supervised image segmentation networks suf-fer from two problems: firstly, supervision is only performed on the last layer of the decoder, resulting in the network's weak generalization ability; secondly, the geometry shape constraints of targets are frequently disregarded in these net-works, leading to poor segmentation results. To address these issues, we propose a novel shape-guided dual consistency semi-supervised learning framework for 3D medical image segmentation. The proposed framework makes two contri-butions. Initially, we introduce a shape constraint to learn the shape representation, which converts the difference be-tween two networks into an unsupervised loss and lets the model learn the boundary information of targets. Addition-ally, we develop a deep-supervised knowledge transfer strat-egy that improves the generalization ability of the network without increasing extra computation costs. Experiments demonstrate that the proposed framework outperforms state-of-the-art semi-supervised methods due to the strong ability of knowledge mining on unlabeled data.

Abstract Deep learning has been widely used for medical image segmentation and a large number of papers has been presented recording the success of deep learning in the field. A comprehensive thematic survey on medical image segmentation using deep learning techniques is presented. This paper makes two original contributions. Firstly, compared to traditional surveys that directly divide literatures of deep learning on medical image segmentation into many groups and introduce literatures in detail for each group, we classify currently popular literatures according to a multi‐level structure from coarse to fine. Secondly, this paper focuses on supervised and weakly supervised learning approaches, without including unsupervised approaches since they have been introduced in many old surveys and they are not popular currently. For supervised learning approaches, we analyse literatures in three aspects: the selection of backbone networks, the design of network blocks, and the improvement of loss functions. For weakly supervised learning approaches, we investigate literature according to data augmentation, transfer learning, and interactive segmentation, separately. Compared to existing surveys, this survey classifies the literatures very differently from before and is more convenient for readers to understand the relevant rationale and will guide them to think of appropriate improvements in medical image segmentation based on deep learning approaches.

Fuzzy c-means (FCM) algorithms with spatial information have been widely applied in the field of image segmentation. However, most of them suffer from two challenges. One is that the introduction of fixed or adaptive single neighboring information with narrow receptive field limits contextual constraints leading to clutter segmentations. The other is that the incorporation of superpixels with wide receptive field enlarges spatial coherency leading to block effects. To address these challenges, we propose fuzzy STUDENT’S t-distribution model based on richer spatial combination (FRSC) for image segmentation. In this article, we make two significant contributions. The first is that both the narrow and wide receptive fields are integrated into the objective function of FRSC, which is convenient to mine image features and distinguish local difference. The second is that the rich spatial combination under STUDENT’S t-distribution ensures that spatial information is introduced into the updated parameters of FRSC, which is helpful in finding a balance between the noise-immunity and detail-preservation. Experimental results on synthetic and publicly available images further demonstrate that the proposed FRSC addresses successfully the limitations of FCM algorithms with spatial information, and provides better segmentation results than state-of-the-art clustering algorithms.

The popular Siamese convolutional neural networks (CNNs) for remote sensing (RS) image change detection (CD) often suffer from two problems. First, they either ignore the original information of bitemporal images or insufficiently utilize the difference information between bitemporal images, which leads to the low tightness of the changed objects. Second, Siamese CNNs always employ dual-branch encoders for CD, which increases computational cost. To address the above issues, this article proposes a network based on difference enhancement and spatial–spectral nonlocal (DESSN) for CD in very-high-resolution (VHR) images. This article makes threefold contributions. First, we design a difference enhancement (DE) module that can effectively learn the difference representation between foreground and background to reduce the impact of irrelevant changes on the detection results. Second, we present a spatial–spectral nonlocal (SSN) module that is different from vanilla nonlocal because multiscale spatial global features are incorporated to model the large-scale variation of objects during CD. The module can be used to strengthen the edge integrity and internal tightness of changed objects. Third, the asymmetric double convolution with Ghost (ADCG) module is exploited instead of standard convolution. The ADCG can not only refine the edge information of the changed objects, since horizontal and vertical convolutional kernels have good contour preservation advantages, but also greatly reduce the computational complexity of the proposed model. The experiments on two public VHR CD datasets demonstrate that the proposed network can provide higher detection accuracy and requires smaller memory usage than state-of-the-art networks.

Land cover change detection (LCCD) using remote sensing images (RSIs) plays an important role in natural disaster evaluation, forest deformation monitoring, and wildfire destruction detection. However, bitemporal images are usually acquired at different atmospheric conditions, such as sun height and soil moisture, which usually cause pseudo and noise change into the change detection map. Changed areas on the ground also generally have various shapes and sizes, consequently making the utilization of spatial contextual information a challenging task. In this paper, we design a novel neural network with spatial-spectral attention mechanism and multi-scale dilation convolution modules. This work is based on the previously demonstrated promising performance of convolutional neural network for LCCD with RSIs and attempts to capture more positive changes and further enhance the detection accuracies. The learning of the proposed neural network is guided with a change magnitude image. The performance and feasibility of the proposed network are validated with four pairs of RSIs that depict real land cover change events on the Earth’s surface. Comparison of the performance of the proposed approach with that of five state-of-art methods indicates the superiority of the proposed network in terms of 10 quantitative evaluation metrics and visual performance. Such as, the proposed network achieved an improvement about 0.08%~14.87% in terms of OA for Dataset-A.

Aiming at the problem of depression recognition, this paper proposes a computer-aided recognition framework based on decision-level multimodal fusion. In Song Dynasty of China, the idea of multimodal fusion was contained in "one gets different impressions of a mountain when viewing it from the front or sideways, at a close range or from afar" poetry. Objective and comprehensive analysis of depression can more accurately restore its essence, and multimodal can represent more information about depression compared to single modal. Linear electroencephalography (EEG) features based on adaptive auto regression (AR) model and typical nonlinear EEG features are extracted. EEG features related to depression and graph metric features in depression related brain regions are selected as the data basis of multimodal fusion to ensure data diversity. Based on the theory of multi-agent cooperation, the computer-aided depression recognition model of decision-level is realized. The experimental data comes from 24 depressed patients and 29 healthy controls (HC). The results of multi-group controlled trials show that compared with single modal or independent classifiers, the decision-level multimodal fusion method has a stronger ability to recognize depression, and the highest accuracy rate 92.13% was obtained. In addition, our results suggest that improving the brain region associated with information processing can help alleviate and treat depression. In the field of classification and recognition, our results clarify that there is no universal classifier suitable for any condition.

The popular deep convolutional networks used for image super-resolution (SR) reconstruction often increase the network depth and employ attention mechanism to improve image reconstruction effect. However, these networks suffer from two problems. The first is the deeper network easily causes higher computational cost and more GPU memory usage. The second is traditional attention mechanism often misses the spatial information of images leading the loss of image detail information. To address these issues, we propose a lightweight non-local network (LNLN) for image super resolution in this paper. The proposed network makes two contributions. First, we use non-local module instead of normal attention module to obtain larger receptive field and extract more comprehensive feature information, which is helpful for improving image SR reconstruction results. Secondly, we use the depthwise separable convolution (DSC) instead of the vanilla convolution to reconstruct the residual block, which greatly reduces the number of parameters and computational cost. The proposed LNLN and comparative networks are evaluated on five commonly public datasets, and experiments demonstrate that the proposed LNLN is superior to state-of-the-art networks in terms of reconstruction performance, the number of parameters and storage space.

U-Net and a large number of variants of U-Net have been successfully used for liver and liver-tumor segmentation. In this paper, we propose a novel network called quartet attention U-Net (QAU-Net). First, QAU-Net employs quartet attention including four branches to capture inner and cross-dimensional interactions between channels and spatial locations. Secondly, QAU-Net employs long-short skip-connection to instead of the vanilla skip-connection, which avoids the duplicate process of low-resolution information and improves the feature fusion of low-resolution and high-resolution information. We evaluate the proposed method on the public LITS dataset. Experiments demonstrate that QAU-Net has better feature representation and higher liver and liver-tumor segmentation accuracy. The available code of QAU-Net we proposed is opened at https://github.com/15029257158/QAU-Net.

Abstract Many biomedical applications require accurate non‐rigid image registration that can cope with complex deformations. However, popular diffeomorphic Demons registration algorithms suffer from difficulties for complex and serious distortions since they only use image greyscale and gradient information. To address these difficulties, a new diffeomorphic Demons registration algorithm is proposed using hierarchical neighbourhood spectral features namely HNSF Log‐Demons in this paper. In view of three important properties of hierarchical neighbourhood spectral features based on line graph such as rotation invariance, invariance of linear changes of brightness, and robustness to noise, the hierarchical neighbourhood spectral features of a reference image and a moving image is first extracted and these novel spectral features are incorporated into the energy function of the diffeomorphic registration framework to improve the capability of capturing complex distortions. Secondly, the Nystrm approximation based on random singular value decomposition is employed to effectively enhance the computational efficiency of HNSF Log‐Demons. Finally, the hybrid multi‐resolution strategy based on wavelet decomposition in the registration process is utilised to further improve the registration accuracy and efficiency. Experimental results show that the proposed HNSF Log‐Demons not only effectively ensures the generation of smooth and reversible deformation field, but also achieves better performance than state‐of‐the‐art algorithms.

Abstract Although deep learning has been widely used for dense crowd counting, it still faces two challenges. Firstly, the popular network models are sensitive to scale variance of human head, human occlusions, and complex background due to repeated utilization of vanilla convolution kernels. Secondly, the vanilla feature fusion often depends on summation or concatenation, which ignores the correlation of different features leading to information redundancy and low robustness to background noise. To address these issues, a multi‐scale feature pyramid network (MFP‐Net) for dense crowd counting is proposed in this paper. The proposed MFP‐Net makes two contributions. Firstly, the feature pyramid fusion module is designed that adopts rich convolutions with different depths and scales, not only to expand the receptive field, but also to improve the inference speed of models by using parallel group convolution. Secondly, a feature attention‐aware module is added in the feature fusion stage. The module can achieve local and global information fusion by capturing the importance of the spatial and channel domains to improve model robustness. The proposed MFP‐Net is evaluated on five publicly available datasets, and experiments show that the MFP‐Net not only provides better crowd counting results than comparative models, but also requires fewer parameters.

The V-Net based 3D fully convolutional neural networks have been widely used in liver volumetric data segmentation. However, due to the large number of parameters of these networks, 3D FCNs suffer from high computational cost and GPU memory usage. To address these issues, we design a lightweight V-Net (LV-Net) for liver segmentation in this paper. The proposed network makes two contributions. The first is that we design an inverted residual bottleneck block (IRB block) and a 3D average pooling block and apply them to the proposed LV-Net. Compared with vanilla convolution, depth-wise convolution and point-wise convolution employed by the IRB block can not only reduce the number of parameters significantly, but also extract features sufficiently well by decoupling cross-channel corrections and spatial correlations. The second is that the LV-Net employs 3D deep supervision to improve the final loss function in training phase, which makes the proposed LV-Net acquire a more powerful discrimination capability between liver areas and non-liver areas. The proposed LV-Net is evaluated on public LiTS dataset, and experiments demonstrate that the proposed LV-Net is superior to popular 2D and 3D networks in terms of segmentation performance, parameter quantity and computational cost.

Clustering algorithms by minimizing an objective function share a clear drawback of having to set the number of clusters manually. Although density peak clustering is able to find the number of clusters, it suffers from memory overflow when it is used for image segmentation because a moderate-size image usually includes a large number of pixels leading to a huge similarity matrix. To address this issue, here we proposed an automatic fuzzy clustering framework (AFCF) for image segmentation. The proposed framework has threefold contributions. First, the idea of superpixel is used for the density peak (DP) algorithm, which efficiently reduces the size of the similarity matrix and thus improves the computational efficiency of the DP algorithm. Second, we employ a density balance algorithm to obtain a robust decision-graph that helps the DP algorithm achieve fully automatic clustering. Finally, a fuzzy c-means clustering based on prior entropy is used in the framework to improve image segmentation results. Because the spatial neighboring information of both the pixels and membership are considered, the final segmentation result is improved effectively. Experiments show that the proposed framework not only achieves automatic image segmentation, but also provides better segmentation results than state-of-the-art algorithms.

In this paper, we propose a novel approach based on a symmetric fully convolutional network within pyramid pooling (FCN-PP) for landslide mapping (LM). The proposed approach has three advantages. Firstly, this approach is automatic and insensitive to noise because multivariate morphological reconstruction (MMR) is used for image preprocessing. Secondly, it is able to take into account features from multiple convolutional layers and explore efficiently the context of images, which leads to a good tradeoff between wider receptive field and the use of context. Finally, the selected pyramid pooling module addresses the drawback of single-scale pooling employed by convolutional neural network (CNN), fully convolutional network (FCN), U-Net, etc. Experimental results show that the proposed FCN-PP is effective for LM, and it outperforms state-of-the-art approaches in terms of four metrics, Precision, Recall, F -score, and Accuracy.

Most of the approaches used for Landslide inventory mapping (LIM) rely on traditional feature extraction and unsupervised classification algorithms. However, it is difficult to use these approaches to detect landslide areas because of the complexity and spatial uncertainty of landslides. In this letter, we propose a novel approach based on a fully convolutional network within pyramid pooling (FCN-PP) for LIM. The proposed approach has three advantages. First, this approach is automatic and insensitive to noise because multivariate morphological reconstruction is used for image preprocessing. Second, it is able to take into account features from multiple convolutional layers and explore efficiently the context of images, which leads to a good tradeoff between wider receptive field and the use of context. Finally, the selected PP module addresses the drawback of global pooling employed by convolutional neural network, FCN, and U-Net, and, thus, provides better feature maps for landslide areas. Experimental results show that the proposed FCN-PP is effective for LIM, and it outperforms the state-of-the-art approaches in terms of five metrics, $Precision$ , $Recall$ , $Overall~Error$ , $F$ -$score$ , and $Accuracy$ .

A great number of improved fuzzy c-means (FCM) clustering algorithms have been widely used for grayscale and color image segmentation. However, most of them are time-consuming and unable to provide desired segmentation results for color images due to two reasons. The first one is that the incorporation of local spatial information often causes a high computational complexity due to the repeated distance computation between clustering centers and pixels within a local neighboring window. The other one is that a regular neighboring window usually breaks up the real local spatial structure of images and thus leads to a poor segmentation. In this work, we propose a superpixel-based fast FCM clustering algorithm that is significantly faster and more robust than stateof-the-art clustering algorithms for color image segmentation. To obtain better local spatial neighborhoods, we first define a multiscale morphological gradient reconstruction operation to obtain a superpixel image with accurate contour. In contrast to traditional neighboring window of fixed size and shape, the superpixel image provides better adaptive and irregular local spatial neighborhoods that are helpful for improving color image segmentation. Second, based on the obtained superpixel image, the original color image is simplified efficiently and its histogram is computed easily by counting the number of pixels in each region of the superpixel image. Finally, we implement FCM with histogram parameter on the superpixel image to obtain the final segmentation result. Experiments performed on synthetic images and real images demonstrate that the proposed algorithm provides better segmentation results and takes less time than state-of-the-art clustering algorithms for color image segmentation.

Detecting land cover change through very-high-resolution (VHR) remote sensing images is helpful in supporting urban sustainable development, natural disaster evaluation, and environmental assessment. However, the intraclass spectral variance in VHR remote sensing images is usually larger than that of median-low remote sensing images. Furthermore, the bitemporal images are usually acquired under different atmospheric conditions, sun height, soil moisture, and other factors. Consequently, in practical applications, many pseudo changes are presented in the detected map. In this paper, an adaptive histogram trend (AHT) similarity approach is promoted to quantitatively measure the magnitude between the corresponding pixels in bitemporal images in terms of change semantic. In the proposed approach, to reduce the phenological effect on the bitemporal images of land cover change detection (LCCD), we first define the quantitative description of AHT. Second, the change magnitudes between pairwise pixels are quantitatively measured by an improved bin-to-bin (B2B) distance between the corresponding AHTs. Then, the change magnitudes between two entire bitemporal images are measured AHT-by-AHT. Finally, binary threshold methods, such as the Otsu method or the double-window flexible pace search (DFPS) method, are used to divide the change magnitude image into binary change detection maps and obtain the final change detection map. The performance of the AHT-based LCCD approach is verified by four pairs of VHR remote-sensing images that correspond to two types of real land cover change cases. The detected results based on the four pairs of bitemporal VHR images outperformed the compared state-of-the-art LCCD methods.

Morphological reconstruction (MR) is often employed by seeded image segmentation algorithms such as watershed transform and power watershed, as it is able to filter out seeds (regional minima) to reduce over-segmentation. However, the MR might mistakenly filter meaningful seeds that are required for generating accurate segmentation and it is also sensitive to the scale because a single-scale structuring element is employed. In this paper, a novel adaptive morphological reconstruction (AMR) operation is proposed that has three advantages. First, AMR can adaptively filter out useless seeds while preserving meaningful ones. Second, AMR is insensitive to the scale of structuring elements because multiscale structuring elements are employed. Finally, the AMR has two attractive properties: monotonic increasingness and convergence that help seeded segmentation algorithms to achieve a hierarchical segmentation. Experiments clearly demonstrate that the AMR is useful for improving performance of algorithms of seeded image segmentation and seed-based spectral segmentation. Compared to several state-of-the-art algorithms, the proposed algorithms provide better segmentation results requiring less computing time.

It is a challenge to recognize holoscopic 3D (H3D) micro-gesture based on general vision techniques because images captured by H3D imaging system are unclear, i.e., the captured images include a large number of blurred grids. Many feature extraction methods can not be directly used for H3D images because the edge information of the grids will be captured. In this paper, we propose a fast and robust preprocessing method for H3D image reconstruction. The reconstructed images are clear and can be used directly for feature extraction or feature learning. Two contributions are presented in this paper. Firstly, we propose a bi-directional morphological filter used for enhancing the grids in an H3D image. Secondly, we propose a fast clustering algorithm with spatial information to extract grids from the H3D image. Because bi-directional morphological filter is able to incorporate local spatial information to the objective function of the fast clustering algorithm, the grids in H3D images are removed completely. Moreover, because the fast clustering algorithm perform clustering on gray levels of H3D images, a small computational cost is required. The proposed method is used to reconstruct H3D images to obtain multiple images with low resolution captured for 3D gesture recognition. Experiments show that the proposed preprocessing method is not only able to obtain better images that are clear and suitable for feature extraction or feature learning, but also is able to improve recognition accuracy in the micro-gesture recognition based on H3D imaging systems.

As fuzzy c-means clustering (FCM) algorithm is sensitive to noise, local spatial information is often introduced to an objective function to improve the robustness of the FCM algorithm for image segmentation. However, the introduction of local spatial information often leads to a high computational complexity, arising out of an iterative calculation of the distance between pixels within local spatial neighbors and clustering centers. To address this issue, an improved FCM algorithm based on morphological reconstruction and membership filtering (FRFCM) that is significantly faster and more robust than FCM is proposed in this paper. First, the local spatial information of images is incorporated into FRFCM by introducing morphological reconstruction operation to guarantee noise-immunity and image detail-preservation. Second, the modification of membership partition, based on the distance between pixels within local spatial neighbors and clustering centers, is replaced by local membership filtering that depends only on the spatial neighbors of membership partition. Compared with state-of-the-art algorithms, the proposed FRFCM algorithm is simpler and significantly faster, since it is unnecessary to compute the distance between pixels within local spatial neighbors and clustering centers. In addition, it is efficient for noisy image segmentation because membership filtering are able to improve membership partition matrix efficiently. Experiments performed on synthetic and real-world images demonstrate that the proposed algorithm not only achieves better results, but also requires less time than the state-of-the-art algorithms for image segmentation.

Among all image representation and classification methods, sparse representation has proven to be an extremely powerful tool. However, a limited number of training samples are an unavoidable problem for sparse representation methods. Many efforts have been devoted to improve the performance of sparse representation methods. In this study, the authors proposed a novel framework to improve the classification accuracy of sparse representation methods. They first introduced the concept of the approximations of all training samples (i.e., virtual training samples). The advantage of this is that the application of virtual training samples can allow noise in original training samples to be partially reduced. Then they proposed an efficient and competent objective function to disclose more discriminant information between different classes, which is very significant for obtaining a better classification result. The devised sparse representation method employs both the original and virtual training samples to improve the classification accuracy since the two kinds of training samples makes sample information to be fully exploited in a good way, also satisfactory robustness to be obtained. The experimental results on the JAFFE, ORL, Columbia Object Image Library (COIL‐100) AR and CMU PIE databases show that the proposed method outperforms the state‐of‐art image classification methods.

Hue component is generally denoted by angle value, so the conventional edge detection operators are incapable to accurately detect edges of hue component. As a result, the popular methods of colour image edge detection usually omit the role of hue component, thus missing some edges caused by hue changes. The authors propose a novel colour edge detection method based on the fusion of hue component and principal component analysis to solve the above problems. First, a novel computational method of hue difference is defined, and then it is applied to classical gradient operators to obtain accurate edges for hue component. Moreover, complete object edges can be obtained by using the edge fusion of the first principal component and hue component of colour image with low‐computational complexity. Experimental results show that the proposed approach not only can act on hue component directly and obtain accurate edges caused by hue changes, but also is effective and easy to implement.

The existing lexicographical ordering approaches respect the total ordering properties, thus making this approach a very robust solution for multivariate ordering. However, different marginal components derived from various representations of a colour image will lead to different results of multivariate ordering. Moreover, the output of lexicographical ordering only depends on the first component leading to the followed components taking no effect. To address these issues, three new marginal components are obtained by means of quaternion decomposition, and they are employed by fuzzy lexicographical ordering, and thus a new fuzzy extremum estimation algorithm (FEEA) based on quaternion decomposition is proposed in this study. The novel multivariate mathematical morphological operators are also defined according to FEEA. Comparing with the existing solutions, experimental results show that the proposed FEEA performs better results on multivariate extremum estimation, and the presented multivariate mathematical operators can be easily handled and can provide better results on multivariate image filtering.