KalmanNet demonstrates potential for nonlinear state estimation, but it directly concatenates heterogeneous components with disparate semantics into a composite vector. This approach ignores capturing semantic component correlations, limiting the network's ability to utilize this information when computing the Kalman gain. To address this, we propose CNPE-KalmanNet, a novel learning-aided Kalman filter that models semantic component correlations by integrating element-wise rotary encoding with multilayer perceptrons in the complex domain. CNPE serves as a plug-and-play module compatible with two baseline KalmanNet architectures. Experiments on the Michigan NCLT dataset demonstrate that CNPE-KalmanNet significantly improves estimation accuracy. Compared to the existing KalmanNet, our method reduces the RMSE by 20%-25% in state estimation tasks and by 15%-20% in sensor fusion tasks.

The extended Kalman filter has been widely used in sensor fusion to achieve integrated navigation and localization. Efficiently integrating multiple sensors requires prior knowledge about their errors for setting the filter. The recently emerged KalmanNet managed to use recurrent neural networks to learn prior knowledge from data and carry out state estimation for problems under non-linear dynamics with partial information. In this letter, the KalmanNet is implemented for integrated navigation using data from GPS/Wheels and the Inertial Measurement Unit. Therein, a practical strategy for the training algorithm of truncated backpropagation through time is presented by taking advantage of the first-order Markov property of the system state of the Kalman filter, which improves the training robustness and performance of the existing KalmanNet. Experimental results on the Michigan NCLT dataset show that our fusion KalmanNet significantly outperforms the conventional EKF-based fusion algorithm with an improvement of 20%$\sim$40% in average RMSE.

Camouflaged object detection (COD) aims to identify objects that are perfectly concealed in their surroundings and has attracted increasing attention in recent years. The challenge with COD is the intrinsic similarity between camouflaged objects and background, as well as the weak boundary that often accompanies camouflaged objects. In this paper, a Progressive Refinement Network called PRNet is proposed based on human perception of camouflaged images. Specifically, we develop a position-aware module to roughly locate the position of camouflaged objects by reverse-guiding with high-level semantic information. Moreover, an edge-guided fusion module is designed to simultaneously refine the boundaries and regions of camouflaged objects by using edge features as a guide in cross-level feature fusion. Benefited from the utility of the above two modules, our PRNet is able to identify camouflaged objects accurately and quickly. Numerous experiments on four widely used benchmark datasets demonstrate that the proposed PRNet is an efficient COD model, outperforming 14 state-of-the-art algorithms significantly and running at a real-time

Due to the lack of color and texture information and the fuzzy boundary of infrared (IR) small targets, the pixel-level mask annotation process consumes a lot of manual cost and is difficult to achieve accurate annotation. To further reduce the annotation burden, we propose an IR small target mask generation algorithm based on single-point supervised multi-scale chain growth clustering (MCGC). The core of this work is the adaptive generation of IR small-target pseudo mask maps under the supervision of randomly given single-point labels, sequentially through the strategies of multi-scale chain growth, Euclidean coefficient decay, K-Means clustering, and eight-neighborhood clustering. On the four public datasets, ablation experiments, qualitative and quantitative comparison experiments demonstrate that the MCGC algorithm has an efficient and accurate IR small target pseudo mask generation capability, which can be adapted to different numbers, scales, shapes, and intensities of targets in complex backgrounds. In addition, IR-Labelmask, an IR small target mask annotation software designed based on the MCGC algorithm, is publicly available on kourenke/IR-Labelmask-software (github.com). To our knowledge, this is the first mask annotation software designed for IR small target.

Arbitrary-oriented object detection is vital for improving UAV sensing and has promising applications. However, challenges persist in detecting objects under extreme conditions like low-illumination and strong occlusion. Cross-modal feature fusion enhances detection in complex environments but current methods do not adequately learn the features of each modality for the current environment, resulting in degraded performance. To tackle this, we propose the CRSIOD network that effectively learns diverse sensor image features to capture distinct scenarios and target characteristics. Firstly, we design an illumination perception module to guide the object detection network in performing various feature processing tasks. Secondly, to leverage the respective advantages of two modalities and mitigate their negative impacts, we introduce an uncertainty aware module to quantify the uncertainties present in each modality as weights to motivate the network to learn in a direction favorable for optimal object detection. Moreover, in the object detection network, we design a two-stream backbone network based on the attention mechanism to enhance the learning of difficult samples, utilize the CMAFF module to fully extract the shared and complementary features between the two modalities, and design a three-branch feature enhancement network to enhance the learning of the three modal features separately. Finally, to optimize detection results, we design light perception non-maximum suppression and improve the horizontal detection head to a rotating one to preserve object orientation. We evaluate the proposed method CRSIOD on the Drone Vehicle dataset of public UAV aerial images. Compared with the existing commonly used methods, CRSIOD achieves state-of-the-art detection performance.

The aircrafts and ships in visible remote sensing (RS) images are of different scales. They are difficult to detect as they may be easily obscured by complex weather conditions such as snow and cloud. Therefore, it is important to eliminate the interference of complex weather conditions in order to detect these multi-scale objects accurately. This letter proposes an improved robust stacked multi-scale feature fusion network RSMNet to address this problem from two aspects. First, a stacked dilated convolution is used to enlarge the receptive fields of high-resolution images and improve the ability to extract multi-scale information. Second, the maps of extracted features are resized and integrated to refine the connection among different layers. Compared to the original Faster R-CNN model, RSMNet provides a 2% and 3.9% higher AP in the detection of aircrafts and ships, respectively. RSMNet also shows much more robust performance than the original model in detection under cloudy and snowy conditions.

Ship detection is a significant and challenging task in remote sensing. At present, anchor-based ship detectors have achieved remarkable results, but they require introducing additional parameters and their performance is easily affected by the size of anchor boxes. In this paper, we propose an anchor-free rotated detector (OFCOS) for ship detection based on FCOS, which can be trained end-to-end. Specifically, a feature-enhanced feature pyramid network (FE-FPN) is proposed, in which the structure of the feature pyramid is optimized and an attention mechanism is introduced during fusion to enhance the significance of object features. Then, to better describe the orientation of objects, a regression branch with orientation characterization capability is constructed, and a center-to-corner bounding box prediction strategy is used to improve the accuracy of object localization. Moreover, the calculation of center-ness is optimized so that the assignment of center weights is orientation-aware and adaptive to down-weight low-quality predictions. A new remote sensing ship dataset, named RS-Ship, is constructed to further verify the effectiveness and robustness of OFCOS. Our experiments show that OFCOS achieves AP values of 91.07% and 97.05% on the publicly available dataset HRSC2016 and our self-built RS-Ship dataset, respectively, which are 13.01% and 9.84% higher than FCOS. OFCOS outperforms other mainstream detection methods in terms of both detection speed and detection accuracy.

To solve the problem of infrared (IR) small target tracking loss or error caused by factors such as scale changes, motion blur, occlusion, etc., this paper proposes a multi-strategy fusion tracking algorithm using an IR small target segmentation network as the detection head, which mainly includes six strategies: target pixel clustering, target feature threshold adjustment, large area search, small area tracking, gate tracking, and coordinate solution. First, candidate targets are obtained through the IR small target segmentation network and pixel clustering strategy. Second, the range of candidate targets is further reduced through threshold adjustment strategies. Then, real-time tracking of IR small targets is achieved through large area search, small area tracking, and wave gate tracking strategies. Finally, the longitude, latitude, and altitude of the tracked target are obtained through coordinate calculation strategies. Both qualitative and quantitative experiments based on real IR small target sequences verify that our algorithm can achieve more satisfactory performances in terms of success rate, precision, and robustness compared with other typical visual trackers. In addition, we have deployed tracking algorithms on the Orange Pi 5 embedded platform, and the tracking speed meets the real-time requirements.

Efficiently and accurately separating infrared (IR) small targets from complex backgrounds presents a significant challenge. Numerous studies in the literature have proposed various feature fusion modules designed specifically to enhance the extraction of IR small target features. While these designs offer some incremental improvement to the accuracy of IR small target detection, they come at a steep cost of significantly increasing network parameters and FLOPs. Striving for a balance between computational efficiency and model accuracy, we decided to forgo these complex feature fusion modules. Instead, we developed a new lightweight encoding and decoding structure known as the Lightweight IR Small Target Segmentation Network (LW-IRSTNet). This structure integrates regular convolutions, depthwise separable convolutions, atrous convolutions, and asymmetric convolutions modules. In addition, we devised post-processing modules including an eight-neighborhood clustering algorithm and an online target feature adjustment strategy. Experimental results indicate that: 1) the segmentation accuracy metrics of LW-IRSTNet match the best results of 14 state-of-the-art comparative baselines; 2) the parameters and FLOPs of LW-IRSTNet, at only 0.16M and 303M respectively, are significantly smaller in comparison to these baselines; and 3) the post-processing modules enhance both user-friendliness and the robustness of algorithm deployment. Moreover, LW-IRSTNet has been successfully implemented on both embedded platforms and websites, expanding its range of applications. Utilizing the ONNX framework, NPU acceleration, and CPU multi-threaded resource allocation, we have been able to achieve high-performance inference capabilities, as well as online dynamic threshold adjustment with the LW-IRSTNet. The source codes for this project can be accessed at https://github.com/kourenke/LW-IRSTNet.

Since existing detectors are often sensitive to the complex background, a novel detection pattern based on generative adversarial network (GAN) is proposed to focus on the essential features of infrared small target in this article. Motivated by the fact that the infrared small targets have their unique distribution characteristics, we construct a GAN model to automatically learn the features of targets and directly predict the intensity of targets. The target is recognized and reconstructed by the generator, built upon U-Net, according the data distribution. A five-layer discriminator is constructed to enhance the data-fitting ability of generator. Besides, the L2 loss is added into adversarial loss to improve the localization. In general, the detection problem is formulated as an image-to-image translation problem implemented by GAN, namely the original image is translated to a detected image with only target remained. By this way, we can achieve reasonable results with no need of specific mapping function or hand-engineering features. Extensive experiments demonstrate the outstanding performance of proposed method on various backgrounds and targets. In particular, the proposed method significantly improve intersection over union (IoU) values of the detection results than state-of-the-art methods.