Source-free object detection (SFOD) transfers a source-trained model to a target domain using only unlabeled data. Most SFOD methods adopt a mean-teacher framework, filtering pseudo labels by teacher confidence and alternately updating student and teacher models. However, this may discard high-quality low-confidence pseudo labels, as confidence alone does not reflect label quality. To better exploit these labels, we propose Decoupled Pseudo-label Learning (DPL), which disentangles classification and localization to identify high-quality pseudo labels. DPL comprises a double confirmation class mechanism and jitter-based localization evaluation to handle low-confidence labels in terms of category and localization. After obtaining high-quality pseudo labels, mixed contrastive learning further enhances target-domain representation. Extensive experiments demonstrate that DPL achieves state-of-the-art performance.

Graph Neural Networks (GNNs) have been widely used across various fields under the homophily assumption that connected nodes are similar. However, in heterophilic graphs, where connected nodes tend to have dissimilar features, existing GNNs still face some limitations. From the perspective of structure, shallow GNNs could not capture the high-order node information, whereas deep GNNs may suffer from the over-smoothing problem. From the perspective of feature, the useful information of high-order similar nodes is often weakened by low-order dissimilar nodes in the feature update phase. To address the above problems, we propose a Global Structure-aware and Feature-augmented Graph Neural Network (GSF-GNN) to alleviate the limitations from the perspectives of structure and feature. Specifically, from the structure perspective, we design a Structure-based Global Propagation (SGP) module to establish global connections among nodes and adaptively adjust edge weights for message propagation. From the feature perspective, we introduce a Feature-augmented Compensatory Update (FCU) module, which employs a multi-view feature updating mechanism to enhance node features from different perspectives. Our theoretical analysis formally demonstrates the effectiveness of GSF-GNN in heterophilic graphs. Experiments on heterophilic and homophilic benchmark datasets validate the effectiveness of GSF-GNN across various graph structures. Moreover, GSF-GNN achieves stable performance across multiple layers and effectively alleviates the over-smoothing problem. Our codes are available on https://github.com/huijieliu2023/GSF-GNN .

Pan-sharpening is a commonly employed strategy to obtain high-resolution multispectral (HRMS) images. Existing deep unfolding networks for pan-sharpening suffer from ineffectively establishing the relationship between panchromatic (PAN) images and generated noisy HRMS (GN-HRMS) images in PAN-guided image denoising, lacking the support of physical models. In this paper, we first design a degradation-fusion-aware unfolding framework (DF-UF) to separate the processing of PAN-prior in PAN-guided image denoising into an individual module, PAN-prior processor, for better integrating physical models. Then, we derive a flexible intensity-hue-saturation (F-IHS) to act as the PAN-prior processor, which models the relationship between PAN images and GN-HRMS images in terms of intensity components through the intensity-hue-saturation (IHS) theory. Finally, plugging F-IHS into DF-UF, we propose a degradation-intensity-aware unfolding transformer (DIUT) to address the problem of incomplete utilization of PAN images in the denoising process. Extensive experiments on diverse scenes show that the performance of DIUT surpasses existing state-of-the-art methods.

3D Gaussian Splatting (3DGS) has been widely discussed due to its impressive ability to provide fast and high-fidelity reconstruction. However, the training efficiency of 3D Gaussian Splatting limits its application in real-time tasks such as Simultaneous Localization and Mapping (SLAM). In this paper, we present a low-resolution hierarchical training method for 3DGS. We show that the computational complexity of 3DGS can be dramatically reduced by down-sampling the training images. Thus, we use a low-resolution hierarchical training method to accelerate the training process, which optimizes 3D Gaussians from coarse to fine. In addition, we employ anti-alias filters to overcome the aliasing effect due to multi-resolution training. Experiments show that the proposed method outperforms existing methods in terms of training efficiency, which reduces the time consumption of the original 3DGS by 57.81% without obvious impact on the reconstruction quality.

Multispectral remote sensing images, with their richer spectral information, can achieve better scene classification performance compared to RGB images. However, high annotation costs remain a significant challenge. To reduce these costs, we propose a spatial-spectral consistency (SSC) semi-supervised learning method that fully leverages abundant unlabeled data and effectively exploits spectral information from multispectral images. Our method employs two branches to extract spatial and spectral features, respectively. The predictions from the two branches for the weakly augmented input are first fused to generate pseudo-labels, which are then used to supervise the branches in predicting the strongly augmented input. Additionally, we introduce a spectral attention module into the network to enhance its ability to extract spectral information. We conduct extensive experiments on the EuroSAT and SEN12MS datasets, demonstrating that our method outperforms other semi-supervised approaches, achieving state-of-the-art (SOTA) performance.

This paper proposes a moment-constrained marginal distributionally robust Kalman filter (MC-MDRKF) for centralized state estimation in multi-sensor systems with unknown sensor noise correlations. We first derive a robust static estimator and then extend it to dynamic systems for the MC-MDRKF algorithm. The static estimator defines a marginal distributional uncertainty set using moment constraints and formulates a minimax optimization problem to robustly address unknown correlations. We prove that this minimax problem admits an equivalent convex optimization formulation, enabling efficient numerical solutions. The resulting MC-MDRKF algorithm recursively updates state estimates in dynamic state-space models. Simulation results demonstrate the superiority and robustness of the proposed method in a multi-sensor target tracking scenario.

Remote sensing scene classification is a crucial task for remote sensing image interpretation. Existing multispectral scene classification methods have overlooked the interrelationships between different spectral bands, which limits the mining of complementary information within the images. Addressing this issue, we propose a grouped cross-band fusion (GCBF) network for remote sensing multispectral scene classification to take full advantage of complementary information between various spectral bands. Firstly, we separate the various bands of the given multispectral image into different groups to better capture the characteristics of each spectral band. Then, we use the existing UniFormer as a feature extractor to learn the representations of red, green, and blue (RGB) bands. For the spectral bands other than RGB, we propose a new network called multi-stage grouped spectral feature extraction (MGSFE) network to learn discriminative representations. We also draw inspiration from the band combination in the field of remote sensing and introduce a cross-band attention fusion (CBAF) module designed to adaptively merge features from both the RGB bands and other spectral bands. Extensive experiments on three widely used remote sensing multispectral scene classification datasets of BigEarthNet, SEN12MS, and EuroSAT demonstrate the superiority of our proposed method compared with several state-of-the-art (SOTA) methods.

A key issue of existing deep-learning-based object detection methods in remote sensing images is that they often struggle to differentiate the background and small object regions due to multi-level downsampling operations therein. Downsampling operations help extract high-level semantic features but result in excessive loss of spatial features of small objects. In this paper, we propose a new small object detector using multispectral remote sensing images, named content-aware downsampling-based detection network (CADDN), where we newly design a content-aware downsampling-based module (CADM). Unlike conventional downsampling operations that apply uniform downsampling parameters across the entire feature map, CADM adaptively assigns higher weights to feature elements that are critical for distinguishing objects from the background, and this assignment is guided by the contextual awareness of object locations during the downsampling process. Experiments based on multispectral remote sensing images with small ships and vehicles demonstrate that CADM can accurately identify and preserve the locations of important object-related features, and CADDN correspondingly achieves superior small object detection performance than state-of-the-art methods.

Infrared few-shot object detection (IFSOD) aims to detect infrared objects with limited labeled examples. Current infrared datasets, however, suffer from limited diversity in object types and classes, hindering robust evaluation of model generalization on novel classes. To systematically assess dataset quality, we propose metrics for class diversity, instance variability, and object density. By integrating three widely used infrared datasets, we construct the first dataset specifically tailored for IFSOD, increasing instance density to 4.8 (a 1.1 improvement) and expanding the number of classes to 18 (a 5-class increase) compared to the source datasets. Furthermore, frequency analysis of spatial features reveals that sparse annotations introduce spectral bias in the frequency domain. Directly transforming spatial features to the frequency domain, however, mixes background noise with object features, causing spectral leakage and impairing the learning of discriminative features for novel classes. To address these issues, we propose the frequency compression few-shot detection (FC-fsd) method, which incorporates a frequency compression (FC) module. The FC module leverages Discrete Cosine Transform (DCT) within localized windows to reduce spectral leakage and enhance feature clarity. With minimal additional computational overhead, FC-fsd significantly outperforms state-of-the-art methods, achieving nAP50 scores of 28.57 (+13.37) and 35.63 (+2.59) in 1-shot and 2-shot settings, respectively. Our dataset is published athttps://github.com/RuihengZhang/IFSOD-dataset.

Imperfect information and multiple players are the two common features of real-world games. However, few of the existing game-theoretic methods are applicable to multiplayer imperfect information games (IIGs) when it comes to finding Nash equilibria. Moreover, the commonly used methods that rely on average-iterate convergence are not conducive to deep reinforcement learning (DRL), which is widely applied to large-scale problems, as it is costly to preserve average policies under function approximation. To deal with these problems, we construct a continuous-time dynamic named imperfect-information exponential-decay score-based learning (IESL) by considering the concept of Nash distribution [a type of quantal response equilibrium (QRE)] in IIGs. Theoretically, we prove the last-iterate convergence of IESL to approximate Nash equilibria in multiplayer IIGs under the assumption of individual concavity. Empirically, we verify that IESL converges in six poker scenarios, with the ultimate NashConv lower than that of the comparative methods (including counterfactual regret minimization (CFR), replicator dynamics (RDs), and their variants) in multiplayer Leduc hold'em. When compared with the existing equilibrium-finding algorithms in multiplayer normal-form games (NFGs), IESL also demonstrates a more stable performance. In addition, we observe a trade-off between the difficulty of IESL's last-iterate convergence and the NashConv of the convergent policies, which aligns with our convergence analysis based on the hypomonotonicity of the game.

Collaborative perception allows for information sharing between multiple agents, such as vehicles and infrastructure, to obtain a comprehensive view of the environment through communication and fusion. Current research on multi-agent collaborative perception systems often assumes ideal communication and perception environments and neglects the effect of real-world noise such as pose noise, motion blur, and perception noise. To address this gap, in this paper, we propose a novel motion-aware robust communication network (MRCNet) that mitigates noise interference and achieves accurate and robust collaborative perception. MRCNet consists of two main components: multi-scale robust fusion (MRF) addresses pose noise by developing cross-semantic multi-scale enhanced aggregation to fuse features of different scales, while motion enhanced mechanism (MEM) captures motion context to compensate for information blurring caused by moving objects. Experimental results on popular collaborative 3D object detection datasets demonstrate that MRCNet outperforms competing methods in noisy scenarios with improved perception performance using less bandwidth. Our code will be released at https://github.com/IndigoChildren/collaborative-perception-MRCNet.

The fusion module of RGB and infrared (IR) remote sensing images is the key of multispectral ship detection. Existing works have shown that the cross-attention-based feature fusion can achieve good performance by extracting the complementary information of RGB and IR modalities. However, the existing commonly used cross-attention mechanisms introduce lots of redundancy parameters and mainly focus on global feature interaction of multispectral images, ignoring local detail information that is also important for small ship detection. In this paper, we propose a novel multispectral ship detection approach named LoGFusion. In LoGFusion, we design the cross stage partial module with partial convolution (CSPMPC) to reduce feature redundancy and utilize the local cross-modal fusion module (LoCFM) and global cross-modal fusion module (GCFM) to capture both local and global cross-modal features. Furthermore, we introduce a Multispectral Small Ship Dataset (MSSD) containing over 5k ship targets for small target detection. Experiments on MSSD validate the effectiveness of our method in terms of small ship detection in multispectral images.

The diffusion model has demonstrated impressive performance in offline reinforcement learning. However, non-deterministic sampling in diffusion models can lead to unstable performance. Furthermore, the lack of confidence measurements makes it difficult to evaluate the reliability and trustworthiness of the sampled decisions. To address these issues, we present ReDiffuser, which utilizes confidence estimation to ensure reliable decision-making. We achieve this by learning a confidence function based on Random Network Distillation. The confidence function measures the reliability of sampled decisions and contributes to quantitative recognition of reliable decisions. Additionally, we integrate the confidence function into task-specific sampling procedures to realize adaptive-horizon planning and value-embedded planning. Experiments show that the proposed ReDiffuser achieves state-of-the-art performance on standard offline RL datasets.

Existing convolution neural network (CNN)-based synthetic aperture radar (SAR) ship detectors often suffer from poor performance to small-scale ship targets due to the scarcity of extractable features and the bottleneck of local receptive field in the CNN framework. To address the challenges, we propose a novel end-to-end transformer-based detection network for small-scale ship targets in SAR images, named DINO with Refined Denoising and Box (R2DB-DINO). First, we propose a complete contrastive denoising (CCD) training technique which can reconstruct and exploit various types of noisy queries to alleviate the confusion between small ships and background. Second, a look twice towards maximum (LTTM) algorithm for iterative box refinement is devised to acquire abundant features of prediction boxes for small ships by enhancing gradient information. Experiments conducted on measured dataset demonstrate the superiority of the proposed method in small-scale ship detection compared with existing methods.

As the number of remote sensing (RS) images increases, it is crucial to retrieval ship targets according to specific demands. The existing ship image retrieval methods only extract features from the image modality, which may not fully utilize the rich text information available and ignore the high-level hierarchical relations between ship classes. In this paper, we propose a language-assisted siamese contrastive framework, namely LASCF, for fine-grained ship retrieval in RS images. In the new LASCF, the siamese vision models are employed to measure the similarity between images. Moreover, a label text encoder with a pretrained language model is designed to extract the high-level semantic information from labels, and thus the information of the hierarchical relations between ship classes are fused in LASCF. Finally, the multimodal similarity measurement module based on contrastive learning is proposed to optimize the siamese vision models. The experimental results show that the proposed LASCF outperforms several existing state-of-the-art methods.

Existing deep neural network (DNN)-based target detectors in remote sensing images (RSIs) often face challenges in distinguishing small targets from the background. This is mainly because the downsampling process in DNN-based target detectors results in excessive loss of small-target-related features. This paper proposes a new small target detector in RSIs named content-perception downsampling-based target detector (CPDTD), where a novel content-perception downsampling module (CPDM) is designed to replace standard downsampling methods (e.g. pooling and convolution with stride greater than 1). CPDM encodes the input feature map and predicts the location of important features that distinguish targets from backgrounds, assigning larger weights to critical features according to the perception of the position of targets in the content during the downsampling process. Experiments on measured multispectral RSIs regarding small ship and vehicle targets demonstrate the superiorities of our proposed CPDTD in comparison with existing methods.

Deep networks require a large number of samples for optimization, so few-shot segmentation in remote sensing scenes is still an open problem. However, this challenge is exacerbated by the feature blurring and aliasing of bodies (low frequency) and boundaries (high frequency). The existing methods usually only focus on the body part of the class, that is, the low-frequency part, and ignore the critical role of boundary information, that is, high-frequency details, on feature representation. In this letter, we propose a novel body joint boundary prototype match (B2PM) approach that aims to enable prior learning of low- and high-frequency information by explicitly modeling the body and boundary features of objects. First, body-aware prototype learning (BodyPL) realizes the adaptive modeling of the body part of the object through a precise farthest point sampling (FPS) initialization algorithm and an adaptive part shift (APS) strategy, which alleviates the feature ambiguity of the body. Second, boundary-aware prototype learning (BoundPL) explicitly models boundary prototypes by building a patch division and assignment strategy to alleviate feature aliasing at boundaries. Finally, prototype match performs prior knowledge aggregation by computing the affinity between query features and support prototypes. Extensive experiments on commonly used benchmarks (iSAID and PASCAL VOC) demonstrate that B2PM improves the state of the art by significant margins.

Change detection involves analysis of discrepancies between two phases. However, when the unchanged elements are known, the changed features to be identified become straightforward. In addition, remote sensing image is constrained by limited spectral information, which leads to blurred boundaries between different semantics. Based on these two prior knowledge, in this artical, we introduce a novel change detection framework, named the mutually reinforcing similarity network (MutSimNet). This architecture aims to minimize false alarms along changing boundaries and reduce misjudgment rates among outliers. First, similarity learning is applied to change detection. The relationship between the two phases is considered when deriving the change feature maps. Second, we devise a mutually reinforcing loss function that integrates initial features with final features. Third, a self-attention module is connected in the feature pyramid network. This design mitigates information loss during the down-sampling process. Fourth, an attention feature fusion strategy is proposed for the integration of multi-layer features. This strategy takes into account the interaction between layer-by-layer features. Fifth, experimental results validate MutSimNet’s efficiency, particularly its ability to focus on edge contour learning. The MutSimNet also achieves superior performance on two benchmark datasets and predicts positive samples with higher probability. The codebase is accessible at https://github.com/ly-yu/MutSimNet.

Long-tailed distribution of remote sensing data generally limits the object recognition performance of deep neural networks. We notice that too many samples from head class will induce the neural network to learn features of tail class samples being biased towards the head. To solve this, we propose a novel center-wise feature consistency learning (CFCL) mechanism for long-tailed remote sensing object recognition. Firstly, we implement a head-tail center feature generation procedure that builds two teacher models to extract the knowledge from the head class and tail class samples respectively, so as to avoid the extracted tail class features being affected by the head classes. Secondly, a center-wise feature consistency learning strategy is introduced, which distills the central feature of each class to a student model, thereby making the classification boundaries more prominent. Especially, the central feature is estimated by referring to the features which are correctly classified by the teacher models, thus the inaccurate knowledge is abandoned. Extensive experiments on widely-adopted remote sensing recognition datasets including FGSC-23, DIOR, xView and HRSC2016 demonstrate that our method achieves superior performance compared to the state-of-the-art approaches.Code and data are available at: https://github.com/wdzhao123/CWFC.

Previous remote sensing recognition approaches predominantly perform well on the training-testing dataset. However, due to large style discrepancies not only among multidomain datasets but also within a single domain, they suffer from obvious performance degradation when applied to unseen domains. In this paper, we propose a style-content metric learning framework to address the generalizable remote sensing object recognition issue. Specifically, we firstly design an inter-class dispersion metric to encourage the model to make decision based on content rather than the style, which is achieved by dispersing predictions generated from the contents of both positive sample and negative sample and the style of input image. Secondly, we propose an intra-class compactness metric to force the model to be less style-biased by compacting classifier's predictions from the content of input image and the styles of positive sample and negative sample. Lastly, we design an intra-class interaction metric to improve model's recognition accuracy by pulling in classifier's predictions obtained from the input image and positive sample. Extensive experiments on four datasets show that our style-content metric learning achieves superior generalization performance against the state-of-the-art competitors. Code and model are available at: https://github.com/wdzhao123/TSCM.

In high-resolution automotive radar tracking systems, vehicle targets are often regarded as extended targets, which means multiple measurements originated from scattering centers of vehicle targets can be detected at each scan and thus the traditional point target tracking schemes are unsuitable. Meanwhile, vehicle maneuvers, e.g., braking and swerving, cause serious degradation of the classical extended target tracking methods. In this paper, a novel method is proposed for maneuvering extended vehicle tracking with automotive radar. The data-region association (DRA) strategy is adopted to handle the vehicle extension effect, which is superior in describing the complex spatial distribution of vehicle target measurements. The interacting multiple model (IMM) method is combined with this DRA strategy to describe the evolution of target motion models. Accordingly, the proposed DRA-IMM method achieves satisfying tracking performance of extended vehicles and also guarantees the robustness in case of maneuvers. Furthermore, in view of the correlation between vehicle extension and its kinematic state, a ray-based strategy is devised to improve the prior distribution of the data-region association of the basic DRA-IMM, and accordingly an enhanced DRA-IMM (EDRA-IMM) method is proposed. Simulation result validates the effectiveness of the proposed DRA-IMM method for maneuvering extended vehicle tracking and the further improvement of the proposed EDRA-IMM method.

State-of-the-art lane detection methods often rely on specific knowledge about lanes – such as straight lines and parametric curves – to detect lane lines. While the specific knowledge can ease the modeling process, it poses challenges in handling lane lines with complex topologies (e.g., dense, forked, curved, etc.). Recently, dynamic convolution-based methods have shown promising performance by utilizing the features from some key locations of a lane line, such as the starting point, as convolutional kernels, and convoluting them with the whole feature map to detect lane lines. While such methods reduce the reliance on specific knowledge, the kernels computed from the key locations fail to capture the lane line’s global structure due to its long and thin structure, leading to inaccurate detection of lane lines with complex topologies. In addition, the kernels resulting from the key locations are sensitive to occlusion and lane intersections. To overcome these limitations, we propose a transformer-based dynamic kernel generation architecture for lane detection. It utilizes a transformer to generate dynamic convolutional kernels for each lane line in the input image, and then detect these lane lines with dynamic convolution. Compared to the kernels generated from the key locations of a lane line, the kernels generated with the transformer can capture the lane line’s global structure from the whole feature map, enabling them to effectively handle occlusions and lane lines with complex topologies. We evaluate our method on three lane detection benchmarks, and the results demonstrate its state-of-the-art performance. Specifically, our method achieves an F1 score of 63.40 on OpenLane and 88.47 on CurveLanes, surpassing the state of the art by 4.30 and 2.37 points, respectively.

The change detection task of remote sensing images is a basic scientific problem, and has been further widely used in real life. Recently, transformer model has shown strong learning and representation abilities in visual interpretation. In this article, Inspired by the success of the Vision Transformer and its variants, we propose a novel change detection model for remote sensing images, named Swin ResNet Transformers (Swin ResNet). Different from other methods, the proposed Swin ResNet architecture uses a Swin transform encoder, which extracts feature representations of multiple resolutions through a shift window mechanism to calculate self-attention. On three datasets, the proposed model showed good performance, and demonstrate that the Swin transformer has a strong ability to learn long-term dependencies of multi-scale context representation.

Diffusion models have emerged as a powerful paradigm in video synthesis tasks including prediction, generation, and interpolation. Due to the limitation of the computational budget, existing methods usually implement conditional diffusion models with an autoregressive inference pipeline, in which the future fragment is predicted based on the distribution of adjacent past frames. However, only the conditions from a few previous frames can't capture the global temporal coherence, leading to inconsistent or even outrageous results in long-term video prediction. In this paper, we propose a Local-Global Context guided Video Diffusion model (LGC-VD) to capture multi-perception conditions for producing high-quality videos in both conditional/unconditional settings. In LGC-VD, the UNet is implemented with stacked residual blocks with self-attention units, avoiding the undesirable computational cost in 3D Conv. We construct a local-global context guidance strategy to capture the multi-perceptual embedding of the past fragment to boost the consistency of future prediction. Furthermore, we propose a two-stage training strategy to alleviate the effect of noisy frames for more stable predictions. Our experiments demonstrate that the proposed method achieves favorable performance on video prediction, interpolation, and unconditional video generation. We release code at https://github.com/exisas/LGC-VD.

Accurate estimation of the sizes of ship targets plays a critical role in the task of ship classification in synthetic aperture radar (SAR) images. Existing deep neural networks (DNNs)-based methods for SAR ship size estimation (SSE) often adopt a fully connected structure that has limited capability in accurately modeling the relationships of features extracted from SAR images, leading to degraded performance of size estimation. It has been demonstrated that capsule networks provide new guidelines to capture relationships of image features by replacing traditional neurons with capsules, where the dynamic routing strategy is used to calculate correlations among capsules. In this letter, we propose an improved method for SAR SSE based on the capsule network named Caps-SSE network (SSENet). In our Caps-SSENet, a capsule-neural-mixing size mapping module is designed to transform the extracted image features into capsules and complete the estimation of ship sizes using informative feature correlations from dynamic routing. In addition, an average scaled mean square error (ASMSE) loss is proposed to improve the size estimation performance of small ships. Experimental results based on measured SAR data show that the proposed method reduces the estimation error of ship sizes in SAR images in comparison with the existing state-of-the-art method.

Convolutional neural networks (CNNs) have been widely applied in the context of ship detection in synthetic aperture radar (SAR) images, but the detection performance is still not ideal in scenarios with clutter interference. Mining frequency-domain information to suppress the sea clutter in SAR ship detection has attracted wide attention. However, existing frequency-domain ship detection methods do not process frequency-domain information adaptively, which results in the degradation of ship detection performance. To overcome this problem, this article proposes a novel deep learning network called YOLO-FA. YOLO-FA contains the proposed frequency attention module (FAM), which can process frequency-domain information of SAR images adaptively. The proposed method can suppress the sea clutter in the SAR images with the help of frequency-domain information. We evaluate the proposed method YOLO-FA on two datasets, i.e., the high-resolution SAR images’ dataset (HRSID) and SAR ship detection dataset (SSDD). Compared with the baseline method YOLOv5 and the existing commonly used methods, YOLO-FA achieves state-of-the-art detection performance on both the datasets.

Remote sensing object labels require high specialization, resulting in a limited number of labeled samples. Without large labeled samples to support training, general remote sensing object recognition models have limited accuracy. Addressing this issue, this paper proposes a weakly correlated distillation learning framework for remote sensing object recognition with small number of samples. Benefitting from large-scale natural image datasets, many recognition models achieve superior feature extraction capabilities. Thus, we use them as backbones to build teacher models, and then fine-tune the teacher models with a small-scale remote sensing dataset. However, due to the limited number of remote sensing samples, the teacher models may produce noisy features that reduce the performance of the student model. Therefore, we propose a weakly correlated distillation method that selects the weakly correlated features from teacher models to distill the student. Since the weakly correlated features contain different noise distributions which can be mutually suppressed, thereby improving the performance of the student. Extensive experiments on three widely-used datasets of DOTA, HRRSD and NWPU VHR-10 demonstrate the superior performance of our method compared with the state of the arts. Code is available at: https://github.com/wdzhao123/WCD.

Homogeneous regions, which are smooth areas that lack blur clues to discriminate if they are focused or non-focused. Therefore, they bring a great challenge to achieve high accurate multi-focus image fusion (MFIF). Fortunately, we observe that depth maps are highly related to focus and defocus, containing a preponderance of discriminative power to locate homogeneous regions. This offers the potential to provide additional depth cues to assist MFIF task. Taking depth cues into consideration, in this paper, we propose a new depth-distilled multi-focus image fusion framework, namely D2MFIF. In D2MFIF, depth-distilled model (DDM) is designed for adaptively transferring the depth knowledge into MFIF task, gradually improving MFIF performance. Moreover, multi-level fusion mechanism is designed to integrate multi-level decision maps from intermediate outputs for improving the final prediction. Visually and quantitatively experimental results demonstrate the superiority of our method over several state-of-the-art methods.

In this study, we presented the prospects for multi-agent system research with a special focus on agent collaboration and gaming tasks. We briefly introduced some open issues and task challenges from three major perspectives: the multi-agent environment, collaboration, and gaming. Then we provided a related outlook for the technology directions that may create some research challenge insights. Finally, we discussed the outlook for the multi-agent collaboration and gaming application areas.

Image matching is a primary technology for optical and synthetic aperture radar (SAR) image fusion but often shows limited performance due to the highly nonlinear differences between optical and SAR modalities. Recently, deep neural networks (DNNs) have been investigated to effectively extract nonlinear features for image matching tasks, where DNNs are trained based on the elaborated design of loss functions and a low loss value is often expected to obtain better image matching performance. In this letter, we first theoretically demonstrate that when the value of a state-of-the-art loss function decreases, the corresponding matching performance may not consistently improve due to the imbalanced effect of positive and negative samples. To tackle this issue, we proposed an improved loss function to train DNNs for image matching of SAR and optical images. We theoretically prove that the improved loss function ensures the improvement of the matching performance when the loss value decreases based on Taylor’s series expansion analysis. Experimental results on an open dataset with extensive optical and SAR image pairs show that 1) the proposed loss function is better than the original one in terms of image matching performance and 2) the combination of our loss function and existing multiscale convolutional gradient feature (MCGF)-based network provides better matching performance than other state-of-art approaches.

Existing crowd counting approaches predominantly perform well on the training-testing protocol. However, due to large style discrepancies not only among images but also within a single image, they suffer from obvious performance degradation when applied to unseen domains. In this paper, we aim to design a generalizable crowd counting framework which is trained on a source domain but can generalize well on the other domains. To reach this, we propose a gated ensemble learning framework. Specifically, we first propose a diverse fine-grained style attention model to help learn discriminative content feature representations, allowing for exploiting diverse features to improve generalization. We then introduce a channel-level binary gating ensemble model, where diverse feature prior, input-dependent guidance and density grade classification constraint are implemented, to optimally select diverse content features to participate in the ensemble, taking advantage of their complementary while avoiding redundancy. Extensive experiments show that our gating ensemble approach achieves superior generalization performance among four public datasets. Codes are publicly available athttps://github.com/wdzhao123/DCSL.

Remote sensing data distribution generally exposes the long-tail characteristic. This will limit the object recognition performance of existing deep models when they are trained with such unbalanced data. In this paper, we propose a novel hierarchical distillation framework to address the long-tailed object recognition in aerial images. Firstly, we notice that not only student model should learn feature representations from teachers, but also teacher models should learn feature representations from each other. Therefore, we build hierarchical teacher-wise distillation to improve the feature representations of the teacher models trained with middle and tail data, which is achieved by distilling the feature representations of the teacher model trained with head data. Secondly, we notice that the feature representations of the middle and tail classes can not be effectively distilled from the teacher to the student, since too little middle and tail data can be used to learn. Thus, we propose self-calibrated sampling learning that enforces the student to strengthen the learning of the middle and tail data, thereby improving the student’ feature learning ability. Extensive experiments on two widely-used DOTA and FGSC-23 datasets demonstrate superior performance of the proposed method compared with state-of-the-art methods. Model and code are publicly available at: https://github.com/wdzhao123/T2FTS.

Fine-grained object classification (FGOC) focuses on identifying subcategories of objects, which is crucial in military and civilian. Existing FGOC methods primarily focus on high-resolution aerial images, limiting their application on low-resolution (LR) FGOC that is a more realistic setting, especially on resource-constrained satellite devices. It is more challenging to deal with LR FGOC since objects’ details are blurred or missing. Addressing this issue, we make the first attempt to explore LR FGOC and propose a novel pipeline based on two technical insights: 1) feature balance strategy discriminatively integrates super-resolution weak and strong detailed presentations into coarse features of LR aerial images, achieving a feature balance to avoid that the weak detailed presentations are inhibited by the strong ones and 2) iterative interaction mechanism alternately refines feature details of the discriminative ship regions and optimizes the performance of FGOC. Moreover, we build a low-resolution fine-grained object (LFS) dataset to promote further study and evaluation. Extensive experiments on the proposed LFS dataset and the other three object datasets of DOTA, FS23, and HRSC2016 demonstrate that our method outperforms state-of-the-art algorithms. Dataset and code are publicly available athttps://github.com/wdzhao123/FBNet.

Model uncertainty is a serious challenge for robustness of tracking algorithms in radar systems. The smooth variable structure filter (SVSF) achieves error-bounded estimations for target state by scaling the magnitude of kinematic modeling error and accordingly performing a flexible switching strategy for the correction gain. However, the SVSF, without any smoothing functions, suffers from undesired chattering phenomenon since the measurement noise causes random disturbance to the identification of actual level of uncertainties, leading to obvious deterioration of tracking accuracy. In this paper, we present a new switching function for SVSF, i.e. the hyperbolic tangent function, for effective chattering suppression. Then we propose a new algorithm named as the Tanh-SVSF, which reformulates the correction gain with the new switching function, to improve the estimation accuracy for target state. A mathematical definition of SVSF chattering is proposed to quantify the chattering amplitude. It is demonstrated that the new switching function exerts a nonlinear compressing effect on the likelihood of measurement innovation and substantially reduces the disturbance of measurement noise, leading to elimination of the chattering problem. The stability of the Tanh-SVSF is analyzed, based on a proposed stability theorem and the numerical exhaustion strategy. Finally, the proposed method is tested on a simulated vehicle tracking scenario and real-world radar data from the Oxford Radar RobotCar Dataset, and shows superior performance over existing SVSF formulations and the Kalman filter, in view of tracking accuracy, track continuity and the proposed chattering indicator.

This paper concerns the robust state estimation of automotive radar targets in presence of model uncertainty. Smooth variable structure filter (SVSF) achieves error-bounded estimation for target state, even with an inaccurate description of target kinematic model. However, it suffers the undesired chattering phenomenon especially in case of a high model uncertainty level, and its performance is sensitive to a preset smoothing boundary layer parameter. In this paper, we propose a novel hybrid SVSF algorithm to handle these two problems simultaneously. First, we derive a nonlinear generalized variable smoothing boundary layer (NGVBL) parameter based on the conventional Tanh-SVSF method by minimizing the pseudo posterior estimation error covariance. Then this NGVBL is employed to realize an adaptive two-module switching strategy with respect to the uncertainty level to calculate the correction gain. If the uncertainty level is high, the undesired chattering is effectively suppressed by the standard Tanh-SVSF gain. In case of a low uncertainty level, the NGVBL is utilized to replace the preset smoothing boundary layer parameter and reformulate the correction gain. Furthermore, it is demonstrated that the NGVBL-based gain is quasi-optimal in the mean square error (MSE) sense. Accordingly, this novel NGVBL-based hybrid SVSF (NGVBL-SVSF) algorithm improves the estimation performance by avoiding parameter sensitivity in a low uncertainty level case, and maintains effective chattering suppression and robustness to increasing uncertainties. Simulation and real-world automotive radar data experiment results show that, the proposed NGVBL-SVSF outperforms existing SVSFs and the classical Kalman filter in terms of tracking accuracy and track continuity.

Evidence theory, also called belief function theory, provides an efficient tool to represent and combine uncertain information for pattern classification. Evidence combination can be interpreted, in some applications, as classifier fusion. The sources of evidence corresponding to multiple classifiers usually exhibit different classification qualities, and they are often discounted using different weights before combination. In order to achieve the best possible fusion performance, a new credal belief redistribution (CBR) method is proposed to revise such evidence. The rationale of CBR consists of transferring belief from one class not just to other classes, but also to the associated disjunctions of classes (i.e., meta-classes). As classification accuracy for different objects in a given classifier can also vary, the evidence is revised according to prior knowledge mined from its training neighbors. If the selected neighbors are relatively close to the evidence, a large amount of belief will be discounted for redistribution. Otherwise, only a small fraction of belief will enter the redistribution procedure. An imprecision matrix estimated based on these neighbors is employed to specifically redistribute the discounted beliefs. This matrix expresses the likelihood of misclassification (i.e., the probability of a test pattern belonging to a class different from the one assigned to it by the classifier). In CBR, the discounted beliefs are divided into two parts. One part is transferred between singleton classes, whereas the other is cautiously committed to the associated meta-classes. By doing this, one can efficiently reduce the chance of misclassification by modeling partial imprecision. The multiple revised pieces of evidence are finally combined by the Dempster-Shafer rule to reduce uncertainty and further improve classification accuracy. The effectiveness of CBR is extensively validated on several real datasets from the UCI repository and critically compared with that of other related fusion methods.

Regularization method is an effective tool for synthetic aperture radar (SAR) image despeckling. Design of the effective regularization terms describing the image priors plays a vital role in this kind of method. In this article, a new combinational regularization model for speckle reduction (CRM-SR) is proposed, in which a regularization term is elaborately designed to contain both a fractional-order total variation (FrTV) regularization and a nonlocal low rank (NLR) regularization. The new regularization model inherits both the advantages of FrTV and NLR and improves the performance of SAR despeckling and, therefore, better preserves the edges and geometrical features of the images during the despeckling process. An efficient algorithm based on alternating direction optimization is derived to solve the proposed combinational regularization model. Experimental results show that the proposed model can effectively remove SAR image speckle and preserve the geometrical features of images according to both subjective visual assessment of image quality and objective evaluation.

Homogeneous transformation receives considerable attention in recent years as it is essential for change detection in heterogeneous images. However, most existing methods perform the homogeneous transformation based on low-level features. It leads to inaccurate homogeneous representations of the heterogeneous images and accordingly causes unsatisfied performance of change detection. To solve this problem, this paper presents a new model that utilizes deep- level features for homogeneous transformation instead of low-level features. Experimental results on real remote sensing data show that, the proposed method achieves an overall change detection accuracy of 95.91%, providing better performance than the existing methods based on low- level features.

Conventional compressive sensing (CS) aims at sparse signal recovery from the measurements with continuous values. Quantized CS (QCS) methods arise in digital implementations where quantization of the receiver data is performed prior to signal processing. The extreme case of QCS is the so-called 1-bit CS where each real-valued measurement maintains only the sign information with one bit. The 1-bit CS alleviates the burden of storage and transmission of large data volumes and reduces the cost of the analog-to-digital converter. Recently, the 1-bit CS has been successfully applied to inverse scattering and radar imaging. In high-resolution radar imaging scenarios, targets assume spatial extent and occupy clustering pixels. The real and imaginary components of a complex sparse signal are the projections of the same complex value onto two orthogonal axes and, therefore, share a joint sparsity pattern. In this paper, a new 1-bit CS algorithm, referred to as enhanced-binary iterative hard thresholding (E-BIHT), is proposed to improve quality of 1-bit radar imaging by exploiting the two-level block sparsity exhibited in the two properties of clustering and the joint sparsity pattern of the real and imaginary parts of the target image. Simulations and experimental results demonstrate that compared to commonly used 1-bit CS algorithms, the proposed E-BIHT provides more informative imaging resulting in higher target-to-clutter ratio.

Since the width of range swath of synthetic aperture radar (SAR) is restricted by the pulse repetition frequency, there exists a tradeoff between the azimuth resolution and the range swath width. As a result, conventional SAR imaging methods based on the Nyquist sampling theorem can hardly achieve the high resolution and wide swath simultaneously. In this paper, we propose an algorithm of high-resolution and wide-swath SAR imaging based on the combination of Poisson disk sampling and iterative shrinkage thresholding. Poisson disk sampling adopted in the azimuth direction can ensure that the interval between any two adjacent pulses is longer than the Nyquist sampling interval, which provides the potential to widen SAR imaging swath in the range direction. The imaging formation is carried out by performing the inverse operator of the chirp scaling algorithm and the shrinkage thresholding in an iterative fashion. Compared with the existing SAR imaging methods, the proposed method can realize high-resolution and wide-swath SAR imaging simultaneously with affordable computational cost. Simulations and experiments on real SAR data demonstrate the effectiveness of the proposed method.

This paper deals with the problem of maneuvering target tracking with networked multiple sensors. To avoid linearization of nonlinear function, and obtain more accurate estimate for maneuvering target, a novel distributed maneuvering target tracking method based on interacting multiple model with unscented information consensus protocol is proposed. The pseudo measurement matrix is computed according to unscented transform, based on which the information form of measurements is calculated and local estimate is updated. To unify estimation in different sensors and improve the maneuvering target tracking accuracy throughout the whole network, the weighted information consensus protocol is applied for each model in all sensors. With multiple models interacting, the posterior estimate in each sensor is acquired with weighted combination of the model-conditioned estimates. Experimental results demonstrate that the proposed algorithm outperforms the existing methods in the aspect of tracking accuracy and agreement of estimates in all sensors.

To address radar/ESM track association problem in formation in the presence of systematic biases, an anti-bias track association algorithm based on hierarchical clustering analysis is proposed. The influence of formation and systematic biases on association is analyzed first. In order to eliminate the effect of biases, the relative bearing bias between radar and ESM is estimated by hierarchical clustering for distance vectors in MPC. Finally, anti-bias track association is achieved based on the global optimal assignment. Simulation results indicate the proposed algorithm outperforms the state-of-the-art approaches.

In this paper, we propose a two-level block matching pursuit (TLBMP) algorithm based on a probabilistic graph model for polarimetric through-wall radar imaging (TWRI). In typical L-band to X-band TWRI, indoor targets assume a spatial extent and occupy clustered pixels. When polarimetric sensing is used to obtain independent observations, radar images of clustered targets can be enhanced within the joint sparsity framework. Toward this objective, TLBMP is devised to exploit both the clustered property and the joint sparsity pattern of multiple polarimetric through-wall radar images. Simulations and experimental results based on polarimetric through-wall radar data demonstrate that compared to commonly used algorithms for solving the same underlying problem, TLBMP provides more informative imaging with higher target-to-clutter ratio.

Considering the convergence rate is a very important issue as distributed sensors networks usually consist of low-powered wireless devices and speeding up the consensus convergence rate is also important to reduce the number of messages exchanged among neighbors, a new adaptive method for weight assignment of communication links between sensor nodes is proposed based on the dynamic network topology. Based on the adaptive weight assignment method, an improved Kalman consensus filter (KCF) named IKCF is tailored in this letter for distributed state estimation in sensor networks with cluster structure. Furthermore, the experiments demonstrate the adaptive weight assignment method is effective for distributed state estimation when the sensor network is sparsely deployed. In addition, the simulation results also validate the superior performance of the new algorithm and show that IKCF is an excellent algorithm for multi-clusters sensor networks. And there is no additional communication overhead in IKCF because only some local knowledge is used to autonomously calculate the adaptive consensus rate parameter for each node.

This paper proposes a combinational regularization model for synthetic aperture radar (SAR) image despeckling. In contrast to most of the well-known regularization methods that only use one image prior property, the proposed combinational regularization model includes both fractional-order total variation (FrTV) regularization term and nonlocal low rank (NLR) regularization term. By characterizing the smoothness and nonlocal self-similarity property of the SAR image simultaneously, the proposed model, on the one hand, can better remove the noise in homogeneous regions of a noisy image, and on the other hand, can better preserve edges and geometrical features of the images during the despeckling process. Afterwards, an alternating direction method (ADM) is derived to efficiently solve the optimization problem in the proposed model. Experimental results demonstrate the good performance of the proposed model, both in removing SAR image speckles and preserving image texture and details.

Researches on office building energy consumption have been hot in these years, but few researchers consider the classification of office energy consumption performance which can evaluate user behaviors in order to offer a clear analysis of energy consumption and improve their energy saving consciousness. In this paper, we propose a novel hierarchical classification algorithm for evaluating energy consumption behaviors at a real energy management system, which combines fuzzy c-means clustering with GA (genetic algorithm)-based SVM (support vector machine) to fully utilize collected samples. The experiment results with real energy consumption data show that the proposed algorithm works well to distinguish the abnormal behaviors and classify energy consumption behaviors accurately on normal offices.