With the extensive use of multisensors in uncrewed aerial vehicles (UAVs), multimodality information processing has become the research focus. In academic research pertaining to object detection and tracking tasks in UAVs, researchers often align visible-infrared image pairs as a preprocessing step. However, in actual tasks, the dual-modality image pair acquired by UAVs is unaligned, which significantly limits the application of downstream tasks. At present, there are no publicly available multimodality image alignment datasets for UAVs. In this article, we present a large-scale benchmark for the dual-modality image alignment task in UAVs, including 81000 training image pairs and 15000 testing image pairs. Meanwhile, we propose a transformer-based dual-modality image alignment network as the baseline for this benchmark. First, the algorithm extracts multiscale features for image representation to address unaligned image pairs with varying resolutions. Second, a transformer-based alignment network is proposed to improve the fusion of features from heterogeneous modalities. Finally, deformable attention is adopted to alleviate the problem of memory explosion. Numerous experiments on this dual-modality image alignment benchmark are conducted to demonstrate the effectiveness of our algorithm. Source codes are available athttps://github.com/gaozhinanjiu/UAVmatch.

In general, target localization in distributed multiple-input multiple-output (MIMO) radar systems requires synchronization among the sensors, which may not be feasible when the sensors are widely separated. In this paper, the target localization problem is addressed under asynchronous distributed MIMO radar systems in the presence of clock and frequency offsets with the aid of a cooperative target, whose position and velocity are known with measurement errors. Specifically, the influence of the cooperative target position and velocity errors on the unknown target localization accuracy is analyzed through the Cramer-Rao lower bound (CRLB) derivation firstly. Then two methods, namely, the offset estimation based localization method and the offset elimination based localization method, are proposed to estimate the unknown target location. The former estimates the clock and frequency offsets firstly and then locates the unknown target based on the offset estimates, and the latter achieves the unknown target location estimate based on the differential measurements of the two targets. Through the theoretical derivation and numerical simulations, the two proposed methods are demonstrated to be approximately unbiased and can attain the CRLB under mild noise. Moreover, the simulation results show that the proposed methods enjoy higher localization accuracy than the state-of-the-art algorithms.

Semantic context plays a significant role in image segmentation. However, few prior works have explored semantic contexts for 3D point cloud segmentation. In this paper, we propose a simple yet effective Point Context Encoding (PointCE) module to capture semantic contexts of a point cloud and adaptively highlight intermediate feature maps. We also introduce a Semantic Context Encoding loss (SCE-loss) to supervise the network to learn rich semantic context features. To avoid hyperparameter tuning and achieve better convergence performance, we further propose a geometric mean loss to integrate both SCE-loss and segmentation loss. Our PointCE module is general and lightweight, and can be integrated into any point cloud segmentation architecture to improve its segmentation performance with only marginal extra overheads. Experimental results on the ScanNet, S3DIS and Semantic3D datasets show that consistent and significant improvement can be achieved for several different networks by integrating our PointCE module.

Global contextual dependency is important for semantic segmentation of 3D point clouds. However, most existing approaches stack feature extraction layers to enlarge the receptive field to aggregate more contextual information of points along the spatial dimension. In this paper, we propose a Point Global Context Reasoning (PointGCR) module to capture global contextual information along the channel dimension. In PointGCR, an undirected graph representation (namely, ChannelGraph) is used to learn channel independencies. Specifically, channel maps are first represented as graph nodes and the independencies between nodes are then represented as graph edges. PointGCR is a plug-andplay and end-to-end trainable module. It can easily be integrated into an existing segmentation network and achieves a significant performance improvement. We conduct extensive experiments to evaluate the proposed PointGCR module on both indoor and outdoor datasets. Experimental results show that our PointGCR module efficiently captures global contextual dependencies and significantly improve the segmentation performance of several existing networks.

Discriminative correlation filter (DCF)-based trackers have recently exhibited high efficiency and impressive robustness to challenging factors, such as illumination change and partial occlusion. However, in cases with fast motion and full occlusion, these trackers drift off soon and can hardly re-detect the target from the restricted search region due to the boundary effect. On the contrary, recent work using a fully convolutional Siamese network (Siamfc) locates the exemplar image within a large search image but suffers from coarse location and distractors. In this paper, we propose a real-time complementary tracker (RCT) by integrating DCF and Siamfc into a two-stage tracking framework where DCF and Siamfc share mutual advantages and complement each other. In the first stage of this framework, RCT locates the target coarsely but robustly with Siamfc. In the second stage, the derived coarse location is refined by DCF for higher accuracy. For efficiency reasons, Siamfc in the first stage is activated occasionally based on the tracking status inferred from the correlation response map of DCF in the second stage. Comprehensive experiments are performed on three popular benchmark datasets: OTB2013, OTB2015, and VOT2016. On OTB2013, RCT runs with over 40 f/s and achieves an absolute gain of 4.8% and 5.2% in mean overlap precision compared with two base trackers (Staple and Siamfc). On VOT2016, RCT makes a good balance between performance and efficiency, ranking fifth in EAO and first in EFO compared with the top five trackers.

The task of ground-to-aerial image geo-localization can be achieved by matching a ground view query image to a reference database of aerial/satellite images. It is highly challenging due to the dramatic viewpoint changes and unknown orientations. In this paper, we propose a novel in-batch reweighting triplet loss to emphasize the positive effect of hard exemplars during end-to-end training. We also integrate an attention mechanism into our model using feature-level contextual information. To analyze the difficulty level of each triplet, we first enforce a modified logistic regression to triplets with a distance rectifying factor. Then, the reference negative distances for corresponding anchors are set, and the relative weights of triplets are computed by comparing their difficulty to the corresponding references. To reduce the influence of extreme hard data and less useful simple exemplars, the final weights are pruned using upper and lower bound constraints. Experiments on two benchmark datasets show that the proposed approach significantly outperforms the state-of-the-art methods.

Numerous hyperspectral anomaly detection (AD) methods suffer from complex background compositions and subpixel objects due to their inadequate Gaussian-distributed representations for nonhomogeneous backgrounds or their low discrimination between subpixel anomalies and the background. To alleviate these issues, a novel hyperspectral AD weighting strategy based on tensor decomposition and cluster weighting is proposed in this letter. Equipped with this simple but effective strategy as a postprocess, the detection performances of generic AD methods can be significantly boosted. In this strategy, Tucker decomposition is adopted to remove the major background information. A parameter-adaptive k -means clustering method is then applied on the decomposed anomaly/noise data cube to assemble homogeneous regions. After segmenting the clustering result into a number of nonoverlapping eight-connected domains, corresponding weights are assigned to large domains according to an improved Gaussian weight function. Finally, the resulting weight matrix is multiplied by the results of the detectors to achieve a performance boost. Experiments on two hyperspectral data sets validate the effectiveness of the proposed strategy.

Recently, Discriminative Correlation Filters (DCF) have achieved enormous popularity in the tracking community due to high efficiency and fair robustness. With the circular structure, DCF transform computationally consuming spatial correlation into efficient element-wise operation in the Fourier domain. In this paper, we argue that this element-wise solution can be derived only in the case of single-channel features. In terms of tracking with multi-channel features, this element-wise solution trains each feature dimension independently and fails to learn a joint correlation filter. To tackle this problem, we propose a rigorous solution to closed-form correlation filter tracking. This rigorous solution can be computed pixel by pixel from a small linear equation system. Experimental results demonstrate that our rigorous pixel-wise solution achieves better tracking performance than the baseline element-wise solution.

In our previous work, a tensor decomposition-based anomaly detection algorithm has been proposed. However, determining K1, K2, and K3(i.e., the major principal component numbers along the three modes of hyperspectral data) has not been settled satisfactorily. In this letter, a fast and adaptive method for determining K1, K2, and K3is proposed. In the proposed method, the determination problem is converted into an optimization problem by constructing the energy function by maximizing the anomalous degree of the reconstructed anomaly data in both spectral and spatial domains. In order to reduce the computational complexity, a fast initialization strategy is introduced to initialize those parameters in the feature space directly. In addition, to avoid the problem of parameter selection, an adaptive strategy is utilized. Furthermore, K1and K2are considered to be independent, making the degree of freedom of the three parameters conform with the actual. Experiments with three hyperspectral data sets reveal that the proposed method works effectively.

Recently, the performance advancement of discriminative correlation filter (DCF) based trackers is predominantly driven by the use of deep convolutional features. As convolutional features from multiple layers capture different target information, existing works integrate hierarchical convolutional features to enhance target representation. However, these works separate feature integration from DCF learning and hardly benefit from end-to-end training. In this letter, we incorporates feature integration and DCF learning in a unified convolutional neural network. This network reformulates feature integration as a differential module that concatenates features from the shallow and deep layers. A channel attention mechanism is introduced to adaptively impose channel-wise weight on the integrated features. Experimental results on OTB100 and UAV123 demonstrate that our method achieves significant performance improvement while running in real-time.

This paper proposes a robust synthetic aperture radar (SAR) automatic target recognition method based on the 3-D scattering center model. The 3-D scattering center model is established offline from the CAD model of the target using a forward method, which can efficiently predict the 2-D scattering centers as well as the scattering filed of the target at arbitrary poses. For the SAR images to be classified, the 2-D scattering centers are extracted based on the attributed scattering center model and matched with the predicted scattering center set using a neighbor matching algorithm. The selected model scattering centers are used to reconstruct an SAR image based on the 3-D scattering center model, which is compared with the test image to reach a robust similarity. The designed similarity measure comprehensively considers the image correlation between the test image and the model reconstructed image and the model redundancy as for describing the test image. As for target recognition, the model with the highest similarity is determined to the target type of the test SAR image when it is denied to be an outlier. Experiments are conducted on both the data simulated by an electromagnetic code and the data measured in the moving and stationary target acquisition recognition program under standard operating condition and various extended operating conditions to validate the effectiveness and robustness of the proposed method.

Automatic target recognition (ATR) of synthetic aperture radar (SAR) images is performed on either global or local features. The global features can be extracted and classified with high efficiency. However, they lack the reasoning capability thus can hardly work well under the extended operation conditions (EOCs). The local features are often more difficult to extract and classify but they provide reasoning capability for EOC target recognition. To combine the efficiency and robustness in an ATR system, a hierarchical fusion of the global and local features is proposed for SAR ATR in this paper. As the global features, the random projection features can be efficiently extracted and effectively classified by sparse representation-based classification (SRC). The physically relevant local descriptors, i.e., attributed scattering centers (ASCs), are employed for local reasoning to handle various EOCs like noise corruption, resolution variance, and partial occlusion. A one-to-one correspondence between the test and template ASC sets is built by the Hungarian algorithm. Then, the local reasoning is performed by evaluating individual matched pairs as well as the false alarms and missing alarms. For the test image to be recognized, it is first classified by the global classifier, i.e., SRC. Once a reliable decision is made, the whole recognition process terminates. When the decision is not reliable enough, it is passed to the local classifier, where a further classification by ASC matching is carried out. Therefore, by the hierarchical fusion strategy, the efficiency of global features and the robustness of local descriptors to various EOCs can be maintained jointly in the ATR system. Extensive experiments on the moving and stationary target acquisition and recognition data set demonstrate that the proposed method achieves superior effectiveness and robustness under both SOC and typical EOCs, i.e., noise corruption, resolution variance, and partial occlusion, compared with some other SAR ATR methods.

The quality of synthetic aperture radar (SAR) images and the completeness of the template database are two important factors in template-based SAR automatic target recognition. This letter gives a solution to the two factors by multilevel reconstruction of SAR targets using attributed scattering centers (ASCs). The ASCs of original SAR images are extracted to reconstruct the target's image, which not only reduces the noise and background clutters but also keeps the electromagnetic characteristics of the target. Template database are reconstructed at multilevels to simulate various extents of ASC absence in the extended operation conditions. Therefore, the quality of SAR images as well as the completeness of the template database is augmented. Features are extracted from the augmented SAR images, and the classifier is trained by the augmented database for target recognition. Experimental results on the moving and stationary target acquisition and recognition data set demonstrate the validity of the proposed method.

Partial configuration model (PCM) is an occluded object detection method in high-resolution remote sensing images (HR-RSIs) based on the deformable part-based model (DPM). However, it needs extra category predefinition, considerable partlevel annotation, and repeated multimodel training. In this letter, an automatic and fast PCM generation method is proposed based on a novel part sharing mechanism. We propose to share parts from one trained DPM model (tDPM) among different models of partial configurations (PCs) to address the above problems. PCs are first designed according to part anchors of tDPM. The model is then generated through corresponding parts selection, root coverage cropping, and elements reweighing. This method avoids the need for manual category predefinition and partlevel annotation, while largely reducing the computation of PCM training. Experimental results on three HR-RSI data sets show that the proposed method obtains a training speedup of 6.7× and 2× for each PC of airplane and ship categories, while achieving a comparable accuracy compared with PCM.

Recognizing targets in synthetic aperture radar (SAR) images is an important, yet challenging problem in SAR image interpretation. In traditional methods, the 2-D image data is rearranged into vectors and regressed to its label by a vector where the structure information is lost. Multiple rank regression (MRR) method directly manipulation on matrix data by applying a multiple-rank left projecting vectors and right projecting vectors, and the matrix data can be regressed to its label for each category. In this paper, a multiple rank regression (MRR) based method is developed for the classification of SAR images. Firstly, SAR image samples are preprocessed to weak the nonideal factors, then train image samples are used to train the regression vectors for each class. In the test stage, the test image is regression by the trained vectors and is classified to the class with the highest regression label. Experimental results based on the moving and stationary target acquisition and recognition database verified the effectiveness of the proposed method.

The pose of a rotational symmetry target in wideband radar systems is the angle between its symmetry axis and the radar line of sight (LOS). It is one of the important parameters in target attitude measurement. Pose estimation has been applied in many fields, such as target tracking and target recognition. On the goal of estimating the target pose from the wideband radar measurements, a pose estimation algorithm based on the three-dimensional electromagnetic model (3-D em-model) is proposed. The 3-D em-model is an effective way to feature target's electromagnetic scattering behavior. It can recover radar echo of the target fast and accurately. The proposed algorithm utilizes the 3-D em-model of the target to generate high resolution range profiles and then matches the profiles generated by 3-D em-model with profile obtained from the high-resolution radar to estimate the pose information. Experiments using data predicted by a high-frequency electromagnetic code prove the effectiveness of the proposed algorithm.

In wideband stepped-frequency radar systems, relative motions between radar and target can induce high distortions of high-resolution range profiles (HRRPs), such as range migration, shape deformation, and signal-to-noise-ratio (SNR) loss. These distortions, if ignored, can lead to unacceptable performance deterioration in detection. To solve this problem, a new algorithm for detecting moving range-spread targets (RSTs) is proposed in this letter. The proposed detector utilizes the waveform contrast of the HRRP to perform both motion compensation and constant false-alarm rate target detection, and it is simple and robust even for low-SNR scenarios. Simulated experiments are carried out to verify the effectiveness and advantages of the proposed detector.

The pose of a rotational symmetry target in a wideband radar is the angle between its symmetry axis and the radar line of sight. It is a required parameter for target absolute attitude measurement using the bistatic radar measuring system. In this letter, a 3-D electromagnetic model (3-D EM model)-based method is proposed for estimating the target pose from fully polarimetric wideband radar measurements. The 3-D EM model is established offline based on the target geometric structure. Scattering features at different target poses can be accurately predicted by this model. In the proposed algorithm, a synthetic high-resolution range profile (HRRP), which is proved to be only related to the target pose, is first generated from the sum of echoes in radar copolarization channels. Then, by matching this synthetic HRRP with the synthetic HRRPs recovered by the 3-D EM model, the target pose is obtained. Experiments using both data predicted by a high-frequency electromagnetic code and data measured in an anechoic chamber verify the validity of the method.

Anomalies usually refer to targets with a spot of pixels (even subpixels) that stand out from their neighboring background clutter pixels in hyperspectral imagery (HSI). Compared to backgrounds, anomalies have two main characteristics. One is the spectral anomaly, i.e., their spectral signatures are different from those associated to their surrounding backgrounds; another is the spatial anomaly, i.e., anomalies occur as few pixels (even subpixels) embedded in the local homogeneous backgrounds. However, most of the existing anomaly detection algorithms for HSI only employed the spectral anomaly. If the two characteristics are exploited in a detection method simultaneously, better performance may be achieved. The third-order (two modes for space and one mode for spectra) tensor representation of HSI has been proved to be an effective tool to describe the spatial and spectral information equivalently; therefore, tensor representation is convenient for exhibiting the two characteristics of anomalies simultaneously. In this paper, a new anomaly detection method based on tensor decomposition is proposed and divided into three steps. Three factor matrices and a core tensor are first estimated from the third-order tensor that is constructed from the HSI data cube by using the Tucker decomposition, and their major and minor principal components (PCs) are more likely to correspond to the spectral signatures of the backgrounds and the anomalies, respectively. In the second step, a reconstruction-error-based method is presented to find the first largest PCs along each mode to eliminate the spectral signatures of the backgrounds as much as possible, and thus, the remaining data may be modeled as the spectral signatures of the anomalies with a Gaussian noise. Finally, a CFAR test is implemented to detect the anomalies from the remaining data. Experiments with simulated, synthetic, and real HSI data sets reveal that the proposed method outperforms those spectral-anomaly-based methods with better detection probability and less false alarm rate.

In this letter, a novel algorithm for attitude measurement based on a 3-D electromagnetic model (3-D em-model) is proposed. The 3-D em-model is established offline based on the geometric structure of the target, and it can be used to predict the scattering features at different target attitudes. In order to measure the attitude of the air target, we design a bistatic step frequency radar system. The directions of the two radars' lines of sight (LOSs) relative to the target are acquired by matching the high-resolution range profiles (HRRPs) from the target echoes to the HRRPs generated from the 3-D em-model. Since the directions of two radars' LOSs relative to the Earth are already known, the absolute attitude of the target can be acquired. The innovative contributions of this letter are as follows: 1) A comprehensive theoretical analysis of air target attitude measurement based on its own 3-D em-model is proposed; 2) the method can be applied to different kinds of air targets such as aircraft, satellite, missile, etc.; 3) the proposed attitude measurement method does not require target motion model in advance; and 4) the proposed algorithm can be applied to any kind of step frequency waveforms. Experiments using both data predicted by a high-frequency electromagnetic code and data measured in the chamber verify the validity of the method.

A new partially occluded target location method based on straight line is proposed. It is divided into four steps: firstly, we label the straight lines of concerned target in the history image artificially and store the line points together with the grads orientation. The labeled lines, the length of which is restricted, should distribute symmetrically. Then, transform the stored lines using the transformation model whose parameters are derived from geometry calibration result of the real-time image. Afterwards, construct pyramid structure of real-time image and search the optimal match position. The geometry coherence rule is used to gain holistic optimal match result. Lastly, compare the matching measure with the threshold to decide whether need to perform the same match process using the higher solution image, and output the match result. The experiment results, tested by real-time remote sensing images especially when part of them are occluded, are shown that the proposed algorithm for target location is accurate and effective.