Pansharpening under thin cloudy conditions is a practically significant yet rarely addressed task, challenged by simultaneous spatial resolution degradation and cloud-induced spectral distortions. Existing methods often address cloud removal and pansharpening sequentially, leading to cumulative errors and suboptimal performance due to the lack of joint degradation modeling. To address these challenges, we propose a Unified Pansharpening Model with Thin Cloud Removal (Pan-TCR), an end-to-end framework that integrates physical priors. Motivated by theoretical analysis in the frequency domain, we design a frequency-decoupled restoration (FDR) block that disentangles the restoration of multispectral image (MSI) features into amplitude and phase components, each guided by complementary degradation-robust prompts: the near-infrared (NIR) band amplitude for cloud-resilient restoration, and the panchromatic (PAN) phase for high-resolution structural enhancement. To ensure coherence between the two components, we further introduce an interactive inter-frequency consistency (IFC) module, enabling cross-modal refinement that enforces consistency and robustness across frequency cues. Furthermore, we introduce the first real-world thin-cloud contaminated pansharpening dataset (PanTCR-GF2), comprising paired clean and cloudy PAN-MSI images, to enable robust benchmarking under realistic conditions. Extensive experiments on real-world and synthetic datasets demonstrate the superiority and robustness of Pan-TCR, establishing a new benchmark for pansharpening under realistic atmospheric degradations.

Recently, video-language models (VidLMs) have gained widespread attention and adoption. However, existing works primarily focus on terrestrial scenarios, overlooking the highly demanding application needs of underwater observation. To overcome this gap, we introduce UVLM, an under water observation benchmark which is build through a collaborative approach combining human expertise and AI models. To ensure data quality, we have conducted in-depth considerations from multiple perspectives. First, to address the unique challenges of underwater environments, we selected videos that represent typical underwater challenges including light variations, water turbidity, and diverse viewing angles to construct the dataset. Second, to ensure data diversity, the dataset covers a wide range of frame rates, resolutions, 419 classes of marine animals, and various static plants and terrains. Next, for task diversity, we adopted a structured design where observation targets are categorized into two major classes: biological and environmental. Each category includes content observation and change/action observation, totaling 20 subtask types. Finally, we designed several challenging evaluation metrics to enable quantitative comparison and analysis of different methods. Experiments on two representative VidLMs demonstrate that fine-tuning VidLMs on UVLM significantly improves underwater world understanding while also showing potential for slight improvements on existing in-air VidLM benchmarks.

Whilst Convolutional Neural Network (CNN)-based object tracking methods can achieve promising results on traditional well-lit datasets, it is challenging to accurately locate targets in low-light images taken in nighttime scenes, even for state-of-the-art (SOTA) trackers. Existing solutions often disregard potential image features beneficial for object tracking or focus solely on improving human perception, making it difficult to balance image enhancement and object tracking tasks. To address this issue and attain reliable nighttime unmanned aerial vehicle (UAV) tracking, we propose a lightweight Pyramid Attention-based low-light image enhancer, which serve as a plug-and-play solution before the trackers. In addition, we introduce a Pyramid Attention Module (PAM) to enhance the capability for multi-scale feature representation of images as image features are difficult to distinguish under low-light conditions. Experimental results reflect the effectiveness of our method in dealing with poor illumination situations.

Ship detection using remote sensing imagery is a crucial research area with both military and civilian applications. However, it remains challenging due to limitations in current ship datasets, such as insufficient volume, incomplete annotations, and inaccuracies. Additionally, ships often exhibit arbitrary orientations, dense clustering, varying aspect ratios, and significant dimensional changes. To address these issues, this paper advances ship detection from both data and methodological perspectives. First, a new dataset, ORSISOD, is introduced. This dataset includes seven finely categorized ship types, annotated with rotated bounding boxes, which are more appropriate for ship detection than traditional horizontal boxes. Second, a novel rotated ship detection method is proposed, incorporating a Dynamic IOU Threshold Selection (DITS) module and a Positive Sample Quality Assessment (PSQA) module. DITS adjusts the IOU threshold based on ship size and shape, while PSQA assesses sample quality using ship aspect ratio and angle information. The ORSISOD dataset was tested on 12 object detection algorithms, providing benchmarks for ship detection. Furthermore, the proposed method was evaluated on both ORSISOD and DOTA datasets, demonstrating superior performance.

Hyperspectral Image (HSI) cross-scene classification is a challenging task in remote sensing, particularly when real-time processing of Target Domain (TD) HSI is required, and data cannot be reused for training. While deep learning methods have shown promising results, the generalization ability of HSI representations remains limited, mainly due to class label imbalance. This paper introduces a dual-stage learning framework based on transfer learning to enhance classification accuracy in the TD. The framework includes a self-supervised learning stage and a supervised fine-tuning stage. The self-supervised stage focuses on learning robust representations by leveraging inherent structures within HSI data, while the fine-tuning stage uses training labels to extract semantic information. A masked diffusion model predicts masked tokens from unmasked ones, capturing both high-level structures and fine details in HSI data. An efficient spatiospectral Transformer, which removes self-attention from the decoder, is proposed to enhance the self-supervised process. This design allows mask tokens to obtain information from visible tokens without interacting with each other, reducing sequence length and computational costs. By decoding each mask token conditionally independently, only a subset of masked tokens is processed. Extensive experiments on two public HSI datasets demonstrate that the proposed method outperforms state-of-the-art techniques.

Few-shot object detection (FSOD) within remote sensing imagery has achieved great advancements in recent years. However, most existing methods are facing one key challenge while handling remote sensing images: many unlabeled instances in few-shot images are treated as background, which tends to degrade the generalization of the trained model severely. This paper presents HPMF, a Hypergraph-guided Prototype Mining Framework that addresses the challenge through joint optimization from three perspectives. The first is Hierarchical Reference Mining (HRM) which constructs a class-instance dual-driven prototype space that enables mining the unlabeled instances via cross-hierarchical similarity fusion. The second is a Robust Pseudo-box Estimator (RPE) that generates high-quality pseudo bounding boxes for the HRM-mined instances via adaptive density clustering and multi-statistic aggregation. The third is a Hypergraph-Guided Decoder (HGD) that introduces hypergraphs into the transformer decoder for group semantic modeling, enhancing high-order semantic association and similarity of instance features, thereby further improving the mining performance of the HRM module. Extensive experiments under various settings show that the proposed HPMF outperforms state-of-the-art methods consistently across multiple widely adopted remote-sensing FSOD benchmarks such as DIOR, NWPU-VHR10 v2, and HRRSD.

Stable diffusion, a generative model used in text-to-image synthesis, frequently encounters resolution-induced composition problems when generating images of varying sizes. This issue primarily stems from the model being trained on pairs of single-scale images and their corresponding text descriptions. Moreover, direct training on images of unlimited sizes is unfeasible, as it would require an immense number of text-image pairs and entail substantial computational expenses. To overcome these challenges, we propose a two-stage pipeline named Any-Size-Diffusion (ASD), designed to efficiently generate well-composed HD images of any size, while minimizing the need for high-memory GPU resources. Specifically, the initial stage, dubbed Any Ratio Adaptability Diffusion (ARAD), leverages a selected set of images with a restricted range of ratios to optimize the text-conditional diffusion model, thereby improving its ability to adjust composition to accommodate diverse image sizes. To support the creation of images at any desired size, we further introduce a technique called Fast Seamless Tiled Diffusion (FSTD) at the subsequent stage. This method allows for the rapid enlargement of the ASD output to any high-resolution size, avoiding seaming artifacts or memory overloads. Experimental results on the LAION-COCO and MM-CelebA-HQ benchmarks demonstrate that ASD can produce well-structured images of arbitrary sizes, cutting down the inference time by 2X compared to the traditional tiled algorithm. The source code is available at https://github.com/ProAirVerse/Any-Size-Diffusion.

Artifact-free super-resolution (SR) aims to translate low-resolution images into their high-resolution counterparts with a strict integrity of the original content, eliminating any distortions or synthetic details. While traditional diffusion-based SR techniques have demonstrated remarkable abilities to enhance image detail, they are prone to ar-tifact introduction during iterative procedures. Such arti-facts, ranging from trivial noise to unauthentic textures, de-viate from the true structure of the source image, thus chal-lenging the integrity of the super-resolution process. In this work, we propose Self-Adaptive Reality-Guided Diffusion (SARGD), a training-free method that delves into the latent space to effectively identify and mitigate the propagation of artifacts. Our SARGD begins by using an artifact detector to identify implausible pixels, creating a binary mask that highlights artifacts. Following this, the Reality Guidance Refinement (RGR) process refines artifacts by integrating this mask with realistic latent representations, improving alignment with the original image. Nonetheless, initial realistic-latent representations from lower-quality images result in over-smoothing in the final output. To address this, we introduce a Self-Adaptive Guidance (SAG) mechanism. It dynamically computes a reality score, enhancing the sharpness of the realistic latent. These alternating mechanisms collectively achieve artifact-free super-resolution. Extensive experiments demonstrate the superiority of our method, delivering detailed artifact-free high-resolution images while reducing sampling steps by 2 x. We release our code at https://github.com/ProAirVerse/Self-Adaptive-Guidance-Diffusion.git.

Object tracking in satellite videos poses a significant challenge for existing trackers due to the typical involvement of small objects, multiple similar disruptors, and occlusions. To improve the performance of remote sensing tracking, a novel motion-aware correlation filter (MACF) algorithm is developed in this study. The proposed approach provides the following improvements: 1) a motion estimation module based on the historical trajectory of the target is embedded into an enhanced spatial–temporal regularized correlation filter (CF)-based tracking framework, to suppress distractions caused by similar objects (SOs); and 2) a failure correction module is employed to deal with the occlusion problem, thereby further enhancing the tracking robustness. Extensive experiments are conducted on the publicly available VISO and SatSOT datasets, with the experimental results demonstrating that the proposed MACF algorithm achieves superior accuracy in comparison to state-of-the-art trackers. Particularly, the present approach has offered the first-place solution for the single object tracking task given in the ICPR 2022 challenge, on moving object detection and tracking in satellite videos (SatVideoDT).

Despite the rapid advance in multispectral (MS) pansharpening, existing convolutional neural network (CNN)-based methods require training on separate CNNs for different satellite datasets. However, such a single-task learning (STL) paradigm often leads to overlooking any underlying correlations between datasets. Aiming at this challenging problem, a multitask network (MTNet) is presented to accomplish joint MS pansharpening in a unified framework for images acquired by different satellites. Particularly, the pansharpening process of each satellite is treated as a specific task, while MTNet simultaneously learns from all data obtained from these satellites following the multitask learning (MTL) paradigm. MTNet shares the generic knowledge between datasets via task-agnostic subnetwork (TASNet), utilizing task-specific subnetworks (TSSNets) to facilitate the adaptation of such knowledge to a certain satellite. To tackle the limitation of the local connectivity property of the CNN, TASNet incorporates Transformer modules to derive global information. In addition, band-aware dynamic convolutions (BDConvs) are proposed that can accommodate various ground scenes and bands by adjusting their respective receptive field (RF) size. Systematic experimental results over different datasets demonstrate that the proposed approach outperforms the existing state-of-the-art (SOTA) techniques.

Deep convolutional neural networks have delivered remarkable aptitude in performing Synthetic Aperture Radar (SAR) image speckle removal tasks. Such approaches are nevertheless constrained in balancing speckle removal and preservation of spatial information, particularly with respect to strong speckle noise. In this paper, a novel residual-in-residual dense generative adversarial network is proposed to effectively suppress SAR image speckle while retaining rich spatial information. A despeckling sub-network composed of residual-in-residual dense blocks with an encoder-decoder structure is devised to learn end-to-end mapping of noisy images onto noise-free images, where the combination of residual-in-residual structure and dense connection significantly enhances the feature representation capability. In addition, a discriminator sub-network with a fully convolutional structure is introduced, and the adversarial learning strategy is adopted to continuously refine the quality of despeckled results. Systematic experimental results on simulated and real SAR images demonstrate that the novel approach offers superior performance in both quantitative and visual evaluation as compared to state-of-the-art methods.

Despite the proven significance of hyperspectral images (HSIs) in performing various computer vision tasks, its potential is adversely affected by the low-resolution (LR) property in the spatial domain, resulting from multiple physical factors. Inspired by recent advancements in deep generative models, we propose an HSI Super-resolution (SR) approach with Conditional Diffusion Models (HSR-Diff) that merges a high-resolution (HR) multispectral image (MSI) with the corresponding LR-HSI. HSR-Diff generates an HR-HSI via repeated refinement, in which the HR-HSI is initialized with pure Gaussian noise and iteratively refined. At each iteration, the noise is removed with a Conditional Denoising Transformer (CDFormer) that is trained on denoising at different noise levels, conditioned on the hierarchical feature maps of HR-MSI and LR-HSI. In addition, a progressive learning strategy is employed to exploit the global information of full-resolution images. Systematic experiments have been conducted on four public datasets, demonstrating that HSR-Diff outperforms state-of-the-art methods.

Open-World Instance Segmentation (OWIS) is an emerging research topic that aims to segment class-agnostic object instances from images. The mainstream approaches use a two-stage segmentation framework, which first locates the candidate object bounding boxes and then performs instance segmentation. In this work, we instead promote a single-stage transformer-based framework for OWIS. We argue that the end-to-end training process in the single-stage framework can be more convenient for directly regularizing the localization of class-agnostic object pixels. Based on the transformer-based instance segmentation framework, we propose a regularization model to predict foreground pixels and use its relation to instance segmentation to construct a cross-task consistency loss. We show that such a consistency loss could alleviate the problem of incomplete instance annotation - a common problem in the existing OWIS datasets. We also show that the proposed loss lends itself to an effective solution to semi-supervised OWIS that could be considered an extreme case that all object annotations are absent for some images. Our extensive experiments demonstrate that the proposed method achieves impressive results in both fully-supervised and semi-supervised settings. Compared to SOTA methods, the proposed method significantly improves the AP_100 score by 4.75% in UVO dataset →UVO dataset setting and 4.05% in COCO dataset →UVO dataset setting.

Deep learning reveals excellent potential for accomplishing change detection in SAR imagery. Yet, it suffers from the problem of requiring large amounts of labeled samples, whilst labeling SAR imagery for change detection requires experts to label individual images at the pixel level , which is extremely tedious and time-consuming. Also, sample imbalance continues to present a serious challenge for the existing change detection techniques. To tackle these problems, in this study, a Deep Collaborative semi-supervised learning Framework with Class-Rebalancing (DCF-CRe) is proposed for SAR imagery change detection, by exploiting Convolutional Neural Network (CNN) and deep clustering. In particular, a Siamese Difference Fusion Network (SDFNet) is devised to implement change detection while effectively reducing the information loss due to the generation of difference images and highlighting features of the changed regions.In so doing, only a tiny batch of labeled samples is utilized to train SDFNet in order to obtain predicted change map and deep features. In addition, the Approximate Rank-Order Clustering (AROC) algorithm is employed to cluster the deep features, generating pseudo-labels for abundant unlabeled samples . DCF-CRe is then applied to select appropriate pseudo-labels and to add labeled samples to train SDFNet. Experimental results evaluated on six challenging datasets show that this proposed approach can achieve performance superior to state-of-the-art change detection methods for SAR imagery.

The complex Wishart distribution is a widely used statistical model for multilook PolSAR image data, of which the equivalent number of looks (ENL) is a critical parameter. Over the past decades, various estimators have been developed to estimate the ENL of complex Wishart distribution, of which the maximum likelihood (ML) estimator is important since it is asymptotically unbiased and has small variance. However, this estimator is very time-consuming since it has no analytical solution and is usually solved numerically. To address this problem, this letter proposes an efficient ML estimator of ENL by deriving an approximate closed-form solution. Moreover, to estimate the ENL map of a PolSAR image, we also develop an efficient way to compute the local sample statistics parallelly. The experimental results on two PolSAR images show that our method yields highly approximate ENL values as the traditional ML estimator while is much more efficient. It costs less than 0.8 seconds on a general laptop to estimate the ENL map of a PolSAR image with 900×1024 pixels.

The key to hyperspectral image (HSI) and multispectral image (MSI) fusion is to take advantage of the properties of interspectra self-similarities of HSIs and spatial correlations of MSIs. However, leading convolutional neural network (CNN)-based methods show shortcomings in capturing long-range dependencies and self-similarity prior. To this end, we propose a simple yet efficient Transformer-based network, hyperspectral and multispectral image fusion (HMF)-Former, for the HSI/MSI fusion. The HMF-Former adopts a U-shaped architecture with a spatio-spectral Transformer block (SSTB) as the basic unit. In the SSTB, embedded spatial-wise multihead self-attention (Spa-MSA) and spectral-wise multihead self-attention (Spe-MSA) effectively capture interactions of spatial regions and interspectra dependencies, respectively. They are consistent with the properties of spatial correlations of MSIs and interspectra self-similarities of HSIs. In addition, specially designed SSTB enables the HMF-Former to capture both local and global features while maintaining linear complexity. Extensive experiments on four benchmark datasets show that our method significantly outperforms state-of-the-art methods.

Salient object detection is a binary pixel-wise classification to distinguish objects in an image, and also have attracted many research interests in the optical Remote Sensing Images (RSIs). The existing state-of-the-art method exploits the full encoder-decoder architecture to predict salient objects in the optical RSIs, suffering from the problem of unsmooth edges and incomplete structures. To address these problems, in this paper, we propose a Boundary-Aware Network (BANet) with two-stage partial decoders sharing the same encoders for salient object detection in RSIs. Specifically, a Boundary-Aware Partial Decoder (BAD) is introduced at the first stage to focus on learning clear edges of salient objects. To solve the pixel-imbalance problem between boundary and background, an edge-aware loss is proposed to guide learning the BAD network. The resulting features are then employed in turn to enhance high-level features. Afterwards, the Structure-Aware Partial Decoder (SAD) is further introduced at the second stage to improve the structure integrity of salient objects. To alleviate the problem of incomplete structures, the structural similarity loss is further proposed to supervise learning the SAD network. In a consequence, our proposed BANet can predict salient objects with clear edges and complete structure, while reducing model parameters due to the discardment of low-level features. Besides, training a deep neural network requires a large amount of images, and the current benchmark datasets for optical remote sensing images are not large enough. Therefore, we also create a large-scale challenging dataset for salient object detection in RSIs. Extensive experiments demonstrate that our proposed BANet outperforms previous RSI SOD models on all existing benchmark datasets and our new presented dataset available at https://github.com/QingpingZheng/RSISOD.

This paper probes intrinsic factors behind typical failure cases (e.g. spatial inconsistency and boundary confusion) produced by the existing state-of-the-art method in face parsing. To tackle these problems, we propose a novel Decoupled Multi-task Learning with Cyclical Self-Regulation (DML-CSR) for face parsing. Specifically, DML-CSR designs a multi-task model which comprises face parsing, binary edge, and category edge detection. These tasks only share low-level encoder weights without high-level interactions between each other, enabling to decouple auxiliary modules from the whole network at the inference stage. To address spatial inconsistency, we develop a dynamic dual graph convolutional network to capture global contextual information without using any extra pooling operation. To handle boundary confusion in both single and multiple face scenarios, we exploit binary and category edge detection to jointly obtain generic geometric structure and fine-grained semantic clues of human faces. Besides, to prevent noisy labels from degrading model generalization during training, cyclical self-regulation is proposed to self-ensemble several model instances to get a new model and the resulting model then is used to self-distill subsequent models, through alternating iterations. Experiments show that our method achieves the new state-of-the-art performance on the Helen, CelebAMask-HQ, and Lapa datasets. The source code is available at https://github.com/deepinsight/insightface/tree/master/parsing/dml_csr.

With the rapid development of various satellite sensor techniques, remote sensing imagery has been an important source of data in change detection applications. This paper aims to propose an unsupervised change detection method based on Object-based Markov Random Filed (OMRF) and Inception UNet (IUNet). Our method first utilizes a difference image (DI) obtained from two bi-temporal images as the initial feature, and proposes the OMRF algorithm based on homogeneous region to pre-classify the DI thus derive the coarse change map. The IUNet is then constructed to extract the points with high confidence from the coarse change map for training. Eventually, the trained model is fed to classify the original feature, then the final change map is obtained. Experimental results indicate that our method yields great detection results even without supervision.

In this paper, we briefly summarize the first challenge on moving object detection and tracking in satellite videos (SatVideoDT). This challenge has three tracks related to satellite video analysis, including moving object detection (Track 1), single object tracking (Track 2), and multiple-object tracking (Track 3). 123, 89, and 70 participants successfully registered, while 37, 42, and 29 teams submitted their final results on the test datasets for Tracks 1-3, respectively. The top-performing methods and their results in each track are described with details. This challenge establishes a new benchmark for satellite video analysis.

Ship detection has been an active and vital topic in the field of remote sensing for a decade, but it is still a challenging problem due to the large-scale variations, the high aspect ratios, the intensive and rotated arrangement, and the background clutter disturbance. In this letter, we propose a locality-aware rotated ship detection (LARSD) framework based on a multiscale convolutional neural network (CNN) to tackle these issues. The proposed framework applies a UNet-like multiscale CNN to generate multiscale feature maps with high-level semantic information in high resolution. Then, an anchor-based rotated bounding box regression is applied for directly predicting the probability, the edge distances, and the angle of ships. Finally, a locality-aware score alignment (LASA) is proposed to fix the mismatch between classification results and location results caused by the independence of each subnet. Furthermore, to enlarge the data sets of ship detection, we build a new high-resolution ship detection (HRSD) data set, where 2499 images and 9269 instances were collected from Google Earth with different resolutions. Experiments based on public data set high-resolution ship collection 2016 (HRSC2016) and our HRSD data set demonstrate that our detection method achieves the state-of-the-art performance.

Multispectral (MS) pansharpening aims to improve the spatial resolution of MS images (MSI) using the spatial details of panchromatic (PAN) images. Due to the gap of prior knowledge between the simulated data and real-world cases, unsupervised learning-based approaches have grown increasing interest. However, some key hyper-parameters, such as the initial weights of the networks, are set manually, which significantly impacts the fusion performance. To tackle this problem, we propose a novel unsupervised adaptation method with meta-learning for MS pansharpening (MetaPan), in which the meta-learning aims to automatically learn the initial parameters of a three-stream fusion network (TSFNet) for unsupervised adaptation learning (UAL). Specifically, the TSFNet consists of a PAN stream, an MS stream, and a fusion stream, where the fusion stream implicitly leverages domain-specific knowledge of input image pairs while the other two streams explicitly inject spatial details and spectral information into the fusion stream. The MetaPan consists of a pre-training stage, a meta-learning stage, and a UAL stage. At the pre-training stage, the TSFNet is trained with the supervision of simulated ground truth such that it is universal for all image pairs. Then, the process of meta-learning optimizes for an internal representation of network parameters that can adapt to a specific image pair with UAL through only a few steps. Finally, the learned internal representation is fine-tuned to a real-world image pair (a test image pair) with UAL. Experiments on two datasets show that our method performs better than state-of-the-art methods in both quantitative metrics and visual appearance.

Discriminative correlation filter (DCF) has contributed tremendously to address the problem of object tracking benefitting from its high computational efficiency. However, it has suffered from performance degradation in unmanned aerial vehicle (UAV) tracking. This article presents a novel semantic-aware real-time correlation tracking framework (SARCT) for UAV videos to enhance the performance of DCF trackers without incurring excessive computing cost. Specifically, SARCT first constructs an additional detection module to generate ROI proposals and to filter any response regarding the target irrelevant area. Then, a novel semantic segmentation module based on semantic template generation and semantic coefficient prediction is further introduced to capture semantic information, which can provide precise ROI mask, thereby effectively suppressing background interference in the ROI proposals. By sharing features and specific network layers for object detection and semantic segmentation, SARCT reduces parameter redundancy to attain sufficient speed for real-time applications. Systematic experiments are conducted on three typical aerial datasets in order to evaluate the performance of the proposed SARCT. The results demonstrate that SARCT is able to improve the accuracy of conventional DCF-based trackers significantly, outperforming state-of-the-art deep trackers.

An adaptive-network-based fuzzy inference system (ANFIS) offers a popular and powerful fuzzy inference mechanism. As with many other advanced data-driven techniques, developing an effective ANFIS typically requires sufficient training data. However, in many real-world applications, it is not always straightforward to obtain a large amount of representative data that cover the entire problem space to accomplish the required training, seriously restricting the performance of a learned ANFIS. This article introduces a new ANFIS learning approach through an evolutionary process, which is able to generate an ANFIS with only a small amount of training data in a certain problem region, by interpolating well-trained ANFISs in the neighboring regions. Such a process works by first producing an initial population of candidate fuzzy rules in the region of data shortage, through interpolating a rule dictionary constructed from trained ANFISs in the neighborhood regions. The crossover and mutation operations over these candidate rules are then executed in an effort to attain candidates of improved performance. When this genetic learning process terminates, the chromosomes in the final population either collectively form or each individually represents a learned ANFIS, depending on whether a single fuzzy rule or a set of fuzzy rules representing an entire ANFIS is implemented with a chromosome within the evolving population. Comparative experimental evaluations on both synthetic and real-world datasets are carried out, demonstrating that in spite of data shortage, the proposed interpolation approach is able to produce ANFIS models that significantly outperform those trained using existing learning mechanisms.

Multispectral (MS) pan-sharpening aims at producing high-resolution (HR) MS images in both spatial and spectral domains, by merging single-band panchromatic (PAN) images and corresponding MS images with low spatial resolution. The intuitive way to accomplish such MS pan-sharpening tasks, or to reconstruct ideal HR-MS images, is to extract feature pairs from the given PAN and MS images and to fuse the results. Therefore, feature extraction and feature fusion are two key components for MS pan-sharpening. This article presents a novel MS pan-sharpening network (MPNet), including a heterogeneous pair of feature extraction pathways (FEPs) and a convolutional long short-term memory (ConvLSTM)-based hierarchical feature fusion module (HFFM). Specifically, we design a PAN FEP to extract 2-D feature maps via 2-D convolutions and dual attention, while an MS FEP is introduced in an effort to obtain 3-D representations of MS image by 3-D convolutions and triple attention. To merge the resulting hierarchical features, the ConvLSTM-based HFFM is developed, leveraging intralevel fusion, interlevel fusion, and information exchange within one single framework. Here, the interlevel fusion is implemented with the ConvLSTM to capture the dependencies among hierarchical features, reduce redundant information, and effectively integrate them via its recurrent architecture. The information exchange between different FEPs helps enhance the representations for subsequent processing. Systematic comparative experiments have been conducted on three publicly available datasets at both reduced resolution and full resolution, demonstrating that the proposed MPNet outperforms state-of-the-art methods in the literature.

Hyperspectral image (HSI) classification has been a hot topic for decides, as hyperspectral images have rich spatial and spectral information and provide strong basis for distinguishing different land-cover objects. Benefiting from the development of deep learning technologies, deep learning based HSI classification methods have achieved promising performance. Recently, several neural architecture search (NAS) algorithms have been proposed for HSI classification, which further improve the accuracy of HSI classification to a new level. In this paper, NAS and Transformer are combined for handling HSI classification task for the first time. Compared with previous work, the proposed method has two main differences. First, we revisit the search spaces designed in previous HSI classification NAS methods and propose a novel hybrid search space, consisting of the space dominated cell and the spectrum dominated cell. Compared with search spaces proposed in previous works, the proposed hybrid search space is more aligned with the characteristic of HSI data, that is, HSIs have a relatively low spatial resolution and an extremely high spectral resolution. Second, to further improve the classification accuracy, we attempt to graft the emerging transformer module on the automatically designed convolutional neural network (CNN) to add global information to local region focused features learned by CNN. Experimental results on three public HSI datasets show that the proposed method achieves much better performance than comparison approaches, including manually designed network and NAS based HSI classification methods. Especially on the most recently captured dataset Houston University, overall accuracy is improved by nearly 6 percentage points. Code is available at: https://github.com/Cecilia-xue/HyT-NAS.

Hyperspectral images (HSIs) provide abundant spectral and spatial information, playing an irreplaceable role in land-cover classification. Recently, based on deep learning (DL) technologies, an increasing number of HSI classification approaches have been proposed, which demonstrate promising performance. However, previous studies suffer from two major drawbacks: 1) the architecture of most DL models is manually designed, relies on specialized knowledge, and is relatively tedious. Moreover, in HSI classifications, datasets captured by different sensors have different physical properties. Correspondingly, different models need to be designed for different datasets, which further increases the workload of designing architectures and 2) the mainstream framework is a patch-to-pixel framework. The overlap regions of patches of adjacent pixels are calculated repeatedly, which increases computational cost and time cost. In addition, the classification accuracy is sensitive to the patch size, which is artificially set based on extensive investigation experiments. To overcome the issues mentioned above, we first propose a 3-D asymmetric neural network search algorithm and leverage it to automatically search for efficient architectures for HSI classifications. By analyzing the characteristics of HSIs, we specifically build a 3-D asymmetric decomposition search space, where spectral and spatial information is processed with different decomposition convolutions. Furthermore, we propose a new fast classification framework, i.e., pixel-to-pixel classification framework, which has no repetitive operations and reduces the overall cost. Experiments on three public HSI datasets captured by different sensors demonstrate the networks designed by our 3-D asymmetric neural architecture search (3-D-ANAS) achieve competitive performance compared to several state-of-the-art methods, while having a much faster inference speed. Code is available at:https://github.com/hkzhang91/3D-ANAS.

Multispectral (MS) pan-sharpening aims at producing a high spatial resolution (HR) MS image by fusing a single-band HR panchromatic (PAN) image and a corresponding MS image with low spatial resolution. In this paper, we propose a heterogeneous two-stream network (HTSNet) with hierarchical feature prefusion for MS pan-sharpening. The HTSNet employs a heterogeneous group of spatial and spectral streams for spatial and spectral information extraction, respectively. The spatial stream utilizes a 2D CNN for spatial information extraction from the PAN images, and the spectral stream obtains spectral feature cubes from the MS images by a 3D CNN. At the same time, a prefusion module is introduced to prefuse the spatial details with spectral information and transfer information between different streams, which can enhance later processing. In the experiment, the Gaofen-2 satellite dataset is utilized to compare the proposed method with the state-of-the-art MS pan-sharpening methods. Experimental results demonstrate the superiority of our HTSNet in terms of visual effect and quantitative qualities.

While achieving remarkable success in remote sensing (RS) scene classification for the past few years, convolutional neural network (CNN) based methods suffer from the demand for large amounts of training data. The bottleneck in prediction accuracy has shifted from data processing limits toward a lack of ground truth samples, usually collected manually by experienced experts. In this work, we provide a metalearning framework for few-shot classification of RS scene. Under the umbrella of meta-learning, we show it is possible to learn much information about a new category from only 1 or 5 samples. The proposed method is based on Prototypical Networks with a pre-trained stage and a learnable similarity metric. The experimental results show that our method outperforms three state-of-the-art few-shot algorithms and one typical CNN-based method, D-CNN, on two challenging datasets: NWPU-RESISC45 and RSD46-WHU.

A major assumption for constructing an effective adaptive-network-based fuzzy inference system (ANFIS) is that sufficient training data are available. However, in many real-world applications, this assumption may not hold, thereby requiring alternative approaches. In light of this observation, this article focuses on automated construction of ANFISs in an effort to enhance the potential of the Takagi-Sugeno fuzzy regression models for situations where only limited training data are available. In particular, the proposed approach works by interpolating a group of fuzzy rules in a certain given domain with the assistance of existing ANFISs in its neighboring domains. The construction process involves a number of computational mechanisms, including a rule dictionary which is created by extracting the rules from the existing ANFISs; a group of rules which are interpolated by exploiting the local linear embedding algorithm to build an intermediate ANFIS; and a fine-tuning method which refines the resulting intermediate ANFIS. The experimental evaluation on both synthetic and real-world datasets is reported, demonstrating that when facing the data shortage situations, the proposed approach helps significantly improve the performance of the original ANFIS modeling mechanism.

In recent years, deep convolutional neural networks (CNNs) have demonstrated impressive ability to represent hyperspectral images (HSIs) and achieved encouraging results in HSI classification. However, the existing CNN-based models operate at the patch level, in which a pixel is separately classified into classes using a patch of images around it. This patch-level classification will lead to a large number of repeated calculations, and it is hard to identify the appropriate patch size that is beneficial to classification accuracy. In addition, the conventional CNN models operate convolutions with local receptive fields, which cause the failure of contextual spatial information modeling. To overcome these aforementioned limitations, we propose a novel end-to-end, pixel-to-pixel, fully convolutional spatial propagation network (FCSPN) for HSI classification. Our FCSPN consists of a 3-D fully convolution network (3D-FCN) and a convolutional spatial propagation network (CSPN). Specifically, the 3D-FCN is first introduced for reliable preliminary classification, in which a novel dual separable residual (DSR) unit is proposed to effectively capture spectral and spatial information simultaneously with fewer parameters. Moreover, the channel-wise attention mechanism is adapted in the 3D-FCN to grasp the most informative channels from redundant channel information. Finally, the CSPN is introduced to capture the spatial correlations of HSIs via learning a local linear spatial propagation, which allows maintaining the HSI spatial consistency and further refining the classification results. Experimental results on three HSI benchmark data sets demonstrate that the proposed FCSPN achieves state-of-the-art performance on HSI classification.

Recently, neural architecture search (NAS) methods have attracted much attention and outperformed manually designed architectures on a few high-level vision tasks. In this paper, we propose HiNAS (Hierarchical NAS), an effort towards employing NAS to automatically design effective neural network architectures for image denoising. HiNAS adopts gradient based search strategies and employs operations with adaptive receptive field to build an flexible hierarchical search space. During the search stage, HiNAS shares cells across different feature levels to save memory and employ an early stopping strategy to avoid the collapse issue in NAS, and considerably accelerate the search speed. The proposed HiNAS is both memory and computation efficient, which takes only about 4.5 hours for searching using a single GPU. We evaluate the effectiveness of our proposed HiNAS on two different datasets, namely an additive white Gaussian noise dataset BSD500, and a realistic noise dataset SIM1800. Experimental results show that the architecture found by HiNAS has fewer parameters and enjoys a faster inference speed, while achieving highly competitive performance compared with state-of-the-art methods. We also present analysis on the architectures found by NAS. HiNAS also shows good performance on experiments for image de-raining.

We propose a semi-tensor product attention network model as a visual question answering tool for complex interaction over image features. Proposed model performs matrix multiplication of two arbitrary dimensions, which is used to overcome possible dimensional limitations and improve recognition flexibility. In used block-wise operation we preserve spatial and temporal information but reduce the number of parameters by using low-rank pooling scheme. Applied BERT pre-train model is tuned to recognize question features. The proposed model is evaluated on the VQA2.0 dataset. Research results show that our model has good accuracy and easy reconfiguration for future research.

Fuzzy rule interpolation (FRI) facilitates approximate reasoning in fuzzy rule-based systems only with sparse knowledge available, remedying the limitation of conventional compositional rule of inference working with a dense rule base. Most of the existing FRI work assumes equal significance of the conditional attributes in the rules while performing interpolation. Recently, interesting techniques have been reported for achieving weighted interpolative reasoning. However, they are either particularly tailored to perform classification problems only or employ attribute weights that are obtained using additional information (rather than just the given rules), without integrating them with the associated FRI procedure. This paper presents a weighted rule interpolation scheme for performing prediction tasks by the use of fuzzy sparse knowledge only. The weights of rule conditional attributes are learned from a given rule base to discriminate the relative significance of each individual attribute and are integrated throughout the internal mechanism of the FRI process. This scheme is demonstrated using the popular scale and move transformation-based FRI for resolving prediction problems, systematically evaluated on 12 benchmark prediction tasks. The performance is compared with the relevant state-of-the-art FRI techniques, showing the efficacy of the proposed approach.

Fuzzy rule interpolation (FRI) enables sparse fuzzy rule-based systems to derive an interpolated conclusion using neighboring rules, when presented with an observation that matches none of the given rules. The efficacy of FRI has been further empowered by the recent development of weighted FRI techniques, particularly the one that introduces attribute weights of rule antecedents from the given rule base, removing the conventional assumption of antecedent attributes having equal weighting or significance. However, such work was carried out within the specific transformation-based FRI mechanism. This short paper reports the results of generalizing it through enhancing two alternative representative FRI methods. The resultant weighted FRI algorithms facilitate the individual attribute weights to be integrated throughout the corresponding procedures of the conventional unweighted methods. With systematical comparative evaluations over benchmark classification problems, it is empirically demonstrated that these algorithms work effectively and efficiently using just two nearest neighboring rules.

In this article, we proposed a thick cloud removal method for remote-sensing imagery based on multisource estimation. A convolutional-mapping-deconvolutional (CMD) network is proposed to estimate the cloud-free image directly from multisource reference images. Synthetic aperture radar (SAR) image and low-resolution heterogeneous (LRH) image, namely, image from a different optical sensor with lower spatial resolution, are used as reference images to recover the missing information in the cloud-contaminated high-resolution (HR) image. The CMD net is composed of three functional components: the convolutional layers for encoding, the mapping layer for feature transferring, and the deconvolutional layers for decoding. In the training procedure, HR images from cloud-free regions and their corresponding LRH and SAR reference images are used to train the CMD net. When the CMD net is fully trained, it is able to estimate the HR images with their corresponding LRH and SAR reference images. The LRH and SAR reference images are first encoded by the convolutional layers before being transferred to the feature space at HR by the mapping layer. The transferred features are then decoded into cloud-free HR image by the deconvolutional layers. Cloud-free regions in the cloud-contaminated HR image are used to further improve the estimated image via intensity normalization. At last, the cloudy pixels are replaced by their corresponding pixels from the estimated cloud-free HR image. Comparisons with several recently proposed multisource cloud removal methods show that our proposed method is superior as validated by quantitative indexes and visual inspections.

Accuracy of depth estimation from static images has been significantly improved recently, by exploiting hierarchical features from deep convolutional neural networks (CNNs). Compared with static images, vast information exists among video frames and can be exploited to improve the depth estimation performance. In this work, we focus on exploring temporal information from monocular videos for depth estimation. Specifically, we take the advantage of convolutional long short-term memory (CLSTM) and propose a novel spatial-temporal CSLTM (ST-CLSTM) structure. Our ST-CLSTM structure can capture not only the spatial features but also the temporal correlations/consistency among consecutive video frames with negligible increase in computational cost. Additionally, in order to maintain the temporal consistency among the estimated depth frames, we apply the generative adversarial learning scheme and design a temporal consistency loss. The temporal consistency loss is combined with the spatial loss to update the model in an end-to-end fashion. By taking advantage of the temporal information, we build a video depth estimation framework that runs in real-time and generates visually pleasant results. Moreover, our approach is flexible and can be generalized to most existing depth estimation frameworks. Code is available at: https://tinyurl.com/STCLSTM

Thin cloud removal is very important for optical remote sensing imagery. Different from thick cloud removal, the pixels contaminated by thin cloud still preserve some surface information. Therefore, thin cloud removal methods usually focus on suppressing the cloud influence instead of replacing the cloudy pixels. In this paper, we proposed a deep residual symmetrical concatenation network (RSC-Net) to make end-to-end cloud removal. The RSC-Net is based on an encoding-decoding framework consisting of multiple residual convolutional layers and residual deconvolutional layers. The feature maps of each convolutional layer are copied and concatenated to their symmetrical deconvolutional layers. Using supervised training with real Landsat-8 data, input samples include one cloudy image and one cloud-free reference image, and the cloud-free reference image also serves as the target. When the RSC-Net is fully trained, it is able to take cloudy image as input and produce cloud-free image as output. Experimental results show that our method has significant advantages in removing thin cloud contaminations in different bands when compared with other traditional and state-of-the-art methods.

Deep learning (DL) has proven to be a promising technique for hyperspectral image (HSI) classification. However, due to complex network structure and massive parameters, it is challenging to achieve satisfying classification accuracy with only a small number of training samples. In this letter, we propose a novel HSI classification method by collaborating the 3-D separable ResNet (3-D-SRNet) with cross-sensor transfer learning. The 3-D-SRNet replaces 3-D convolutions with spatial and spectral separable 3-D convolutions, thus showing much less parameters than models that use standard 3-D convolutions. First, we pretrain a classification model with the proposed 3-D-SRNet on the source HSI data set with sufficient training samples compared with the target HSI data set. Then, the pretrained model is transferred to the target HSI data set for fine-tuning to finish the classification task. It is worth noting that the source data for pretraining can be captured by the different sensor with the target data. Compared with the conventional 3-D-ResNet, the proposed 3-D-SRNet has less parameters involving lower computation cost while achieving better classification performance. Experimental results on three benchmark data sets show that our method outperforms several state-of-the-art methods in HSI classification with small training samples.

Recently, hyperspectral image (HSI) classification approaches based on deep learning (DL) models have been proposed and shown promising performance. However, because of very limited available training samples and massive model parameters, DL methods may suffer from overfitting. In this paper, we propose an end-to-end 3-D lightweight convolutional neural network (CNN) (abbreviated as 3-D-LWNet) for limited samples-based HSI classification. Compared with conventional 3-D-CNN models, the proposed 3-D-LWNet has a deeper network structure, less parameters, and lower computation cost, resulting in better classification performance. To further alleviate the small sample problem, we also propose two transfer learning strategies: 1) cross-sensor strategy, in which we pretrain a 3-D model in the source HSI data sets containing a greater number of labeled samples and then transfer it to the target HSI data sets and 2) cross-modal strategy, in which we pretrain a 3-D model in the 2-D RGB image data sets containing a large number of samples and then transfer it to the target HSI data sets. In contrast to previous approaches, we do not impose restrictions over the source data sets, in which they do not have to be collected by the same sensors as the target data sets. Experiments on three public HSI data sets captured by different sensors demonstrate that our model achieves competitive performance for HSI classification compared to several state-of-the-art methods.

This paper presents a novel fuzzy rule-based interpolative reasoning system for mammographic mass shape classification that is interpretable to medical professionals. In particular, a feature ranking-guided fuzzy rule interpolation (FRI) method is embedded in the proposed system to make inference possible given a sparse rule base, which may occur in dealing with insufficient mammographic image data (and indeed in coping with many other computer-aided medical diagnostic problems). The rule base for inference is learned from a set of labelled morphological features which are extracted from mass shapes. A classical FRI mechanism is integrated with a procedure for feature selection to score the individual rule antecedents in the inducted sparse rule base for more accurate interpolative reasoning. The work is evaluated on a real-world mammographic image data base with promising results, demonstrating the efficacy of the proposed fuzzy rule-based interpolative classification system.

Fuzzy rule interpolation (FRI) makes inference possible when dealing with a sparse and imprecise rule base. However, the rule antecedents are commonly assumed to be of equal significance in most FRI approaches in the implementation of interpolation. This may lead to a poor performance of interpolative reasoning due to inaccurate or incorrect interpolated results. In order to improve the accuracy by minimising the disadvantage of the equal significance assumption, this paper presents a novel inference system where an information gain (IG)-guided fuzzy rule interpolation method is embedded. In particular, the rule antecedents in FRI are weighted using IG to evaluate the relative importance given the consequent for decision making. The computation of antecedent weights is enabled by introducing an innovative reverse engineering process that artificially converts fuzzy rules into training samples. The antecedent weighting scheme is integrated with scale and move transformation-based interpolation (though other FRI techniques may be improved in the same manner). An illustrative example is used to demonstrate the execution of the proposed approach, while systematic comparative experimental studies are reported to demonstrate the potential of the proposed work.

Using fuzzy rule interpolation (FRI) interpolative reasoning can be effectively performed with a sparse rule base where a given system observation does not match any fuzzy rules. While offering a potentially powerful inference mechanism, in the current literature, typical representation of fuzzy rules in FRI assumes that all attributes in the rules are of equal significance in deriving the consequents. This is a strong assumption in practical applications, thereby, often leading to less accurate interpolated results. To address this challenging problem, this paper employs feature selection (FS) techniques to adjudge the relative significance of individual attributes and therefore, to differentiate the contributions of the rule antecedents and their impact upon FRI. This is feasible because FS provides a readily adaptable mechanism for evaluating and ranking attributes, being capable of selecting more informative features. Without requiring any acquisition of real observations, based on the originally given sparse rule base, the individual scores are computed using a set of training samples that are artificially created from the rule base through an innovative reverse engineering procedure. The attribute scores are integrated within the popular scale and move transformation-based FRI algorithm (while other FRI approaches may be similarly extended following the same idea), forming a novel method for attribute ranking-supported fuzzy interpolative reasoning. The efficacy and robustness of the proposed approach is verified through systematic experimental examinations in comparison with the original FRI technique over a range of benchmark classification problems while utilizing different FS methods. A specific and important outcome that is supported by attribute ranking, only two (i.e., the least number of) nearest adjacent rules are required to perform accurate interpolative reasoning, avoiding the need of searching for and computing with multiple rules beyond the immediate neighborhood of a given observation.

Fuzzy rule interpolation (FRI) provides an alternative means to make inference with a sparse rule base, rather than directly resulting in failed reasoning when no rules can be fired for an input observation. However, existing approaches to FRI typically assume that rule antecedents are of equal significance in the implementation of interpolation, thereby often leading to less accurate interpolated results. Having taken notice of feature selection (FS) techniques being capable of selecting (subsets of) informative features, providing a mechanism of evaluating and ranking features, this work employs FS to score the individual rule antecedents in a given rule base. In particular, the computation of individual scores is enabled by the introduction of an innovative reverse engineering technique that artificially creates a set of training samples from a given sparse rule base. The antecedent scores are integrated within the scale and move transformation-based FRI algorithm (though other FRI approaches may employ the same idea), forming a novel feature ranking-guided FRI method. The work is systematically examined, by utilising six different FS techniques and comparing over eight benchmark classification problems, demonstrating improved classification performance.

This paper proposes a new approach for single frame image super resolution using multiple ANFIS (Adaptive Network-based Fuzzy Inference System) mappings. It presents an implemented learning system that captures the relationship between a low resolution (LR) image patch space and a high resolution (HR) one given an external image database. In particular, a collected large number of LR and HR image patch pairs are divided into different groups with a clustering method. For each clustered group of the training samples, an ANFIS mapping is learned for super resolution (SR). The non-local means filter is subsequently employed to suppress the displeasing artefacts of the resulting reconstructed HR image. The proposed approach is evaluated on a range of natural images and compared with a number of existing state-of-the-art SR algorithms, demonstrating its effectiveness.

Fuzzy rule interpolation forms an important approach for performing inference with systems comprising sparse rule bases. Even when a given observation has no overlap with the antecedent values of any existing rules, fuzzy rule interpolation may still derive a useful conclusion. Unfortunately, very little of the existing work on fuzzy rule interpolation can conjunctively handle more than one form of uncertainty in the rules or observations. In particular, the difficulty in defining the required precise-valued membership functions for the fuzzy sets that are used in conventional fuzzy rule interpolation techniques significantly restricts their application. In this paper, a novel rough-fuzzy approach is proposed in an attempt to address such difficulties. The proposed approach allows the representation, handling and utilisation of different levels of uncertainty in knowledge. This allows transformation-based fuzzy rule interpolation techniques to model and harness additional uncertain information in order to implement an effective fuzzy interpolative reasoning system. Final conclusions are derived by performing rough-fuzzy interpolation over this representation. The effectiveness of the approach is illustrated by a practical application to the prediction of diarrhoeal disease rates in remote villages. It is further evaluated against a range of other benchmark case studies. The experimental results confirm the efficacy of the proposed work.

The task of handwritten Chinese character recognition is one of the most challenging areas of human handwriting classification. The main reason for this is related to the writing system itself which encompasses thousands of characters, coupled with high levels of diversity in personal writing styles and attributes. Much of the existing work for both online and off-line handwritten Chinese character recognition has focused on methods which employ feature extraction and segmentation steps. The preprocessed data from these steps form the basis for the subsequent classification and recognition phases. This paper proposes an approach for handwritten Chinese character recognition and classification using only an image alignment technique and does not require the aforementioned steps. Rather than extracting features from the image, which often means building models from very large training data, the proposed method instead uses the mean image transformations as a basis for model building. The use of an image-only model means that no subjective tuning of the feature extraction is required. In addition by employing a fuzzy-entropy-based metric, the work also entails improved ability to model different types of uncertainty. The classifier is a simple distance-based nearest neighbour classification system based on template matching. The approach is applied to a publicly available real-world database of handwritten Chinese characters and demonstrates that it can achieve high classification accuracy and is robust in the presence of noise.

Fuzzy rule interpolation is an important technique for performing inference with sparse rule bases. Even when a given observation has no overlap with the antecedent values of any existing rules, fuzzy rule interpolation may still derive a conclusion. In particular, the recently proposed rough-fuzzy rule interpolation offers greater flexibility in handling different levels of uncertainty that may be present in sparse rule bases and observations. Nevertheless, in practical applications with inter-connected subsets of rules, situations may arise where a crucial antecedent of observation is absent, either due to human error or difficulty in obtaining data, while the associated conclusion may be derived according to alternative rules or even observed directly. If such missing antecedents were involved in the subsequent interpolation process, the final conclusion would not be deduced using a forward rule interpolation technique alone. However, missing antecedents may be related to certain intermediate conclusions and therefore, may be interpolated us- ing the known antecedents and these conclusions. Following this idea, a novel backward rough-fuzzy rule interpolation approach is proposed in this paper, allowing missing observations which are indirectly related to the final conclusion to be interpolated from the known antecedents and intermediate conclusions. As illustrated experimentally, the resulting backward rough-fuzzy rule interpolation system is able to deal with uncertainty, in both data and knowledge, with more flexibility.

This letter presents a cloud removal method for reconstructing the missing information in cloud-contaminated regions of a high-resolution (HR) optical satellite image (HRI) using two types of auxiliary images, i.e., a low-resolution (LR) optical satellite composite image (LRI) and a synthetic aperture radar (SAR) image. The LRI contributes low-frequency information, and the SAR image contributes high-frequency information for restoring the HRI. The approach is implemented using structure correspondences established by sparse representation. Specifically, two dictionary pairs are trained jointly: One pair is generated from the HRI and LRI gradient image patches, and the other is generated from the HRI and SAR gradient image patches. Experimental reconstructions of cloud-contaminated regions in HR Thematic Mapper images are performed using three types of auxiliary images, i.e., MODIS 16-day composite only, SAR only, and both MODIS composite and SAR, respectively. It is shown that the MODIS composite or the SAR data alone are not sufficient to restore the missing HR information, whereas the combination of the two types of data can provide both low- and high-frequency information. The proposed approach can achieve a highly accurate result and has potential in areas where land-cover change may occur.

Small sample space target recognition is a difficult problem in applications because the limited training samples cannot lead to satisfactory recognition accuracy. Combined with novel compression perception theory, we propose a new space target recognition method based on compressive sensing. This method avoids the sophisticated image preprocessing and feature extraction process. Firstly, a sparse representation dictionary is constructed according to the training samples. Secondly, the linear measurements of test samples are obtained by measurement matrix. Finally, the classification and recognition are done through solving an optimization problem. Simulation experiment results show that the proposed method has good recognition performance and stability.

Digital image degradation normally arises during image acquisition and processing, which has a direct influence on the visual quality of the image. This paper proposes a combined method for enhancement of noisy image by using the mirror-extended curvelet transform and nonlinear anisotropic diffusion. First, an improved enhancement function is proposed to nonlinearly shrink and stretch the curvelet coefficients. Then, the enhanced results are further processed by the nonlinear diffusion where only the nonsignificant, i.e., nonthresholded, curvelet coefficients are changed by means of a diffusion process in order to reduce the pseudo-Gibbs artifacts. Experimental results indicate the proposed method has better performances to enhance the shape of edges and important detailed features as well as suppress noise, in comparison to the curvelet-based enhancement method without diffusion and the wavelet-based enhancement methods with/without diffusion.

This paper proposes an adaptive method based on the mirror-extended curvelet transform and the improved particle swarm optimization (PSO) algorithm, which reduce speckle noise and enhance edge features and contrast of synthetic aperture radar (SAR) images. First, an improved gain function, which integrates the speckle reduction with the feature enhancement, is introduced to nonlinearly shrink and stretch the curvelet coefficients. Then, a novel objective criterion for the quality of the despeckled and enhanced images is proposed in order to adaptively obtain the optimal parameters in the gain function. Finally, the PSO algorithm is employed as a global search strategy for the best despeckled and enhanced image. In order to increase the convergence speed and avoid the premature convergence, two further improvements for the classic PSO algorithm are presented. That is, a new learning scheme and a mutation operator are introduced. Experimental results demonstrate that the proposed method can efficiently reduce the speckle and enhance the edge features and the contrast of SAR images and outperforms the wavelet- and curvelet-based nonadaptive despeckling and enhancement methods.

Speckle and low contrast can cause image degradation, which reduces the detectability of targets and impedes further investigation of synthetic aperture radar (SAR) images. This paper presents an adaptive enhancement method with speckle reduction for SAR images using mirror-extended curve let (ME-curve let) transform and particle swarm optimization (PSO). First, an improved enhancement function is proposed to nonlinearly shrink and stretch the curve let coefficients. Then, a novel objective evaluation criterion is introduced to adaptively obtain the optimal parameters in the enhancement function. Finally, a PSO algorithm with two improvements is used as a global search strategy for the best enhanced image. Experimental results indicate that the proposed method can reduce the speckle and enhance the edge features and the contrast of SAR images better with comparison to the wavelet-based and curve let-based non-adaptive enhancement methods.

The influence of noises is obviously in the image transform method based on the pixels elements representation of images. To some extent the method of image transform based on linear elements representation can solution the problem. The Beamlet transform is an effective method of line segment extraction. This work improved the traditional Beamlet transform by considering the directional information of lines. The improved method transformed a digital image to a coefficient matrix. This method can embody the linear singularity of some linear targets and can be used for edge detection and extracting other useful targets in noisy images. The experimental results on manual images and SAR images demonstrate the effectiveness of this method.

This paper presents a compound neural network model, i.e., adaptive neurofuzzy network (ANFN), which can be used for identifying the complicated nonlinear system. The proposed ANFN has a simple structure and exploits a hybrid algorithm combining supervised learning and unsupervised learning. In addition, ANFN is capable of overcoming the error of system identification due to the existence of some changing points and improving the accuracy of identification of the whole system. The effectiveness of the model and its algorithm is tested on the identification results of missile attacking area.