Partial multi-label image classification (PMLIC) learns from typical weak supervision, where each image is labeled with a set of candidate labels, only some of which are correct. We find that noisy labels generate conflicting gradient signals that disrupt the learning of latent true labels, causing the model to prefer learning clean negative labels that provide consistent supervisory signals, thereby hindering disambiguation. Meanwhile, noisy labels cause the model to activate misattributed pixel regions, which interfere with feature pattern extraction, leading to inaccurate label correlation. In this paper, we propose a PMLIC framework that constructs a correlation-induced negative suppression disambiguation loss (CoNeS). First, we exploit the property that networks tend to learn clean labels first by extracting class activation maps to identify and screen misattributed pixel regions. Meanwhile, we aggregate noise-disturbed feature patterns into more expressive representations via k-means clustering and construct accurate label correlations to aid disambiguation. In addition, we design the negative suppression disambiguation loss to focus the model on disambiguation by introducing a weight distribution to suppress the contribution of negative labels. This weighting distribution can be adaptively inferred by a closed-form solution. Extensive experiments demonstrate that the CoNeS framework achieves significant advantages over current state-of-the-art methods. Specifically, it achieves average mAP improvements of 1.26%, 2.74%, 0.85%, and 0.33% on the VOC 2007, MS-COCO, VG-256, and CUB-200 datasets at different resolutions and noise rates. Code has been made available at https://github.com/zhongjingyu1/CoNeS.

Synthetic Aperture Radar (SAR) image registration is an essential upstream task in geoscience applications, in which pre-detected keypoints from two images are employed as observed objects to seek matched-point pairs. In general, the registration is regarded as a typical closed-set classification, which forces each keypoint to be classified into the given classes, but ignoring an essential issue that numerous redundant keypoints are beyond the given classes, which unavoidably results in capturing incorrect matched-point pairs. Based on this, we propose a Cross-Rejective Open-set SAR Image Registration (CroR-OSIR) method. In this work, these redundant keypoints are regarded as out-of-distribution (OOD) samples, and we formulate the registration as a special open-set task with two modules: supervised contrastive feature-tuning and cross-rejective open-set recognition (CroR-OSR). Unlike traditional open-set recognition, all samples, including OOD samples, are available in the CroR-OSR module. CroR-OSR conducts the closed-set classifications in individual open-set domains from two images, meanwhile employing the cross-domain rejection during training, to exclude these OOD samples based on confidence and consistency. Moreover, a new supervised contrastive tuning strategy is incorporated for feature-tuning. Especially, the cross-domain estimation labels obtained by CroR-OSR are fed back to the feature-tuning module for feature-tuning, to enhance feature discriminability. The experimental results illustrate that the proposed method achieves more precise registration than the state-of-the-art methods. The code is released at https://github.com/XDyaoshi/CroR-OSIR-main.

Multiple-instance learning (MIL) was initially proposed to identify key instances within a set (bag) of instances when only one bag-level label is provided. Current deep MIL models mostly solve multi-instance problem in feature space. Nevertheless, with the increasing complexity of data, we found this paradigm faces significant risks in representation learning stage, which could lead to algorithm degradation in deep MIL models. We speculate that the degradation issue stems from the persistent drift of instances in feature space during learning. In this paper, we propose a novel Probability-Space MIL network (PSMIL) as a countermeasure. In PSMIL, a self-training alignment strategy is introduced in probability space to cope with the drift problem in feature space, and the alignment target objective is proven mathematically optimal. Furthermore, we reveal that the widely-used attention-based pooling mechanism in current deep MIL models is easily affected by the perturbation in feature space and further introduce an alternative called probability-space attention pooling. It effectively captures the key instance in each bag from feature space to probability space, and further eliminates the impact of selection drift in the pooling stage. To summarize, PSMIL seeks to solve a MIL problem in probability space rather than feature space. Experimental results illustrate that PSMIL could potentially achieve performance close to supervised learning level in complex tasks (gap within 5\%), with the incremental alignment in propability space bring more than 19\% accuracy improvements for current existing mainstream models in simulated CIFAR datasets. For existing publicly available MIL benchmarks/datasets, attention in probability space also achieves competitive performance to the state-of-the-art deep MIL models. Codes are available at \url{https://github.com/LMBDA-design/PSAMIL}.

Masked autoencoders (MAEs) have emerged as a powerful self-supervised approach for point cloud analysis. Nevertheless, existing methods often separately focus on global structures or multi-scale features, ignoring their complementary potential. In this paper, we propose a novel dual-Siamese masked autoencoder (DS-MAE) framework that explores integrating global and hierarchical feature learning in a unified architecture for point cloud analysis. In particular, we introduce a consistent dual-branch patch embedding strategy to partition the point cloud into patches using shared group centers, ensuring both global and hierarchical branches process point patches centered at the same spatial locations. Each branch employs dual-branch Siamese encoders to process original and augmented point patches, learning representations that capture both local details and global context. In addition, we have designed cross-attention Siamese decoders to reconstruct masked point patches and align features both within and between branches with cross-attention mechanisms. Comprehensive experiments demonstrate our method consistently achieves superior results to prior methods. Code is available at https://github.com/shaoandy1211/DS-MAE.git.

Due to labeling subjectivity, label noises have become a critical issue that is addressed in facial expression recognition. From the view of human visual perception, the facial exhibited emotion characteristic should be unaltered corresponding to its truth expression, rather than the noise label, whereas most methods ignore the emotion consistency during FER, especially from different networks. Based on this, we propose a new FER method based heterogeneous dual-branch emotional consistency constrains, to prevent the model from memorizing noise samples based on features associated with noisy labels. In the proposed method, the emotion consistency from spatial transformation and heterogeneous networks are simultaneously considered to guide the model to perceive the overall visual features of expressions. Meanwhile, the confidence of the given label is evaluated based on emotional attention maps of original and transformed images, which effectively enhances the classification reliability of two branches to alleviate the negative effect of noisy labels in the learning process. Additionally, the weighted ensemble strategy is used to unify two branches. Experimental results illustrate that the proposed method achieves better performance than the state-of-the-art methods for 10%, 20% and 30% label noises.

Millimeter-wave body screening technology has gained significant attention at inspection sites due to its noncontact and safety. However, existing concealed object detection methods still face challenges in real-world security scenarios, especially the missed detection of dim-small objects posing security risks. The challenge stems primarily from insufficient discrimination of features between small concealed objects and backgrounds. To this end, we propose a collaborative knowledge injection detection network (CKID-Net). It injects the object semantic knowledge learned from an external object database into the concealed object detection model, which forces the model to push background representations apart from the object prior knowledge, and pull together concealed object representations and the prior knowledge, thereby improving the model's discrimination. Our method collaboratively learns representations of prior objects and objects to be detected via excavating their semantic relation. Experiments on active millimeter-wave (AMMW) and terahertz (THz) human datasets show that the CKID-Net outperforms state-of-the-art methods, especially on detection rate.

Recently, there has been significant progress in automatic pain assessment based on facial expression analysis. However, the performance of pain assessment remains unsatisfactory, due to a lack of analysis on local pain-related action units and emotional ambiguity. In particular, ambiguous pain expressions complicate the estimation of pain. It is argued that certain facial local regions related to pain should receive more attention while estimating pain intensities. Based on this, we propose a multi-task hybrid Conv-Transformer method for facial pain assessment, which utilizes the self-attention mechanism to explore facial local features related to pain intensities and constructs a multi-task joint optimizing module to mitigate facial emotional ambiguity. In particular, the proposed method modifies the network structure of the vision transformer model to better estimate continuous pain intensities. Meanwhile, a multi-task module is constructed to jointly optimize the classification and the regression tasks of pain assessment, which effectively regularizes the extracted features and facilitates a better fit of the regressed prediction to the given label. Finally, experimental results on the UNBC Pain dataset illustrate that the proposed method performs better with pain assessment compared with state-of-the-art methods.

The long-tailed data distribution frequently occurs in the real-world scenarios, whereas deep learning is not effective enough for such distribution. In order to improve the effectiveness for the long-tailed data, data augmentation is widely used to balance the distribution of classes by generating new samples. However, most existing studies are designed from the perspective of the class-independence assumption by default, ignoring the effect of interrelation among classes for data augmentation, which causes that some generated samples may be unrepresentative and useless for balancing the class-distribution. Inspired by this, we propose a new data augmentation method based the sparse class-correlation exploitation in this paper, which can generate more representative samples by utilizing the class-correlation, to effectively balance the class-distribution for the long-tailed data. In the proposed method, a sparse class-correlation exploration module is first proposed to explore the potential correlations among multiple classes for boosting the classification performance. Based on the class-correlations, the pivotal seed-samples are generated by maximizing the sparse representation of challenging samples. Meanwhile, an ambiguity-filtered translation module is designed to generate more representative new samples for the target classes based the obtained seed-samples by enhancing the class-consistency and suppressing the deviation from the target classes. In addition, we introduce the self-supervised feature and fuse it with the discriminative feature to explore more accurate class-correlations. Experimental results illustrate that the proposed method obtains better performance only with a small number of generated samples than the state-of-the-art methods.

The facial expression recognition is very important for human-computer interaction. Therefore, a large number of researchers are focusing on this topic research and have acquired many valuable research achievements. However, there still exist many problems that need to be solved for practical applications, such as the impact of identity and appearance differences, posture change etc. In this work, a dual-branch residual disentangled adversarial learning network is proposed to learn more accurate expression features by disentangling the non-expression features from basic features through a novel combinatorial loss function. In the proposed method, dual-branch network structure is designed, one branch with a D-Net module is utilized to explore non-expression features and another branch just uses subtraction operation to obtain expression features. Based on the above network structure, a novel loss function is constructed to guide the two branches to learn different type features, which contains expression recognition loss, adversarial loss and cosine similarity loss. The main highlight of this work is that the proposed method could achieve the disentanglement of expression features and non-expression features just based on a low-complexity network and expression datasets without other auxiliary data. Finally, abundant experimental results on multiple expression datasets have confirmed the proposed method could obtain better expression recognition results than other state-ofthe-art methods.

Vehicle behavior analysis has gradually developed by utilizing trajectories and motion features to characterize on-road behavior. However, the existing methods analyze the behavior of each vehicle individually, ignoring the interaction between vehicles. According to the theory of interactive cognition, vehicle-to-vehicle interaction is an indispensable feature for future autonomous driving, just as interaction is universally required for traditional driving. Therefore, we place the vehicle behavior analysis in the context of the vehicle interaction scene, where the self-vehicle should observe the behavior category and degree of the other-vehicle that is about to interact with itself, in order to predict whether the other-vehicle will pass through the intersection first or later, and then decide to pass through or wait. Inspired by the interactive cognition, we develop a general framework of Structured Vehicle Behavior Analysis (StruVBA) and derive a new model of Structured Fully Convolutional Networks (StruFCN). Moreover, both Intersection over Union (IoU) and False Negative Rate (FNR) are adopted to measure the similarity between the predicted behavior degree and the ground truth. Experimental results illustrate that the proposed method achieves higher prediction accuracy than most existing methods, while predicting vehicle behavior with richer visual meaning. In addition, it also provides an example of modeling the interaction between vehicles and a verification for interaction cognition theory as well.

For GAN-based Person Re-identification (Re-ID), the key is to generate pedestrian images with higher identity consistency and meanwhile larger intra-class diversity. Generally, the main discriminative parts focus on some local regions from the foreground of each pedestrian image for Re-ID, and they should be irrelevant to the background. Whereas, most existing methods generate pedestrian images only based on global features, which difficultly achieves emphasizing crucial local regions and weakening the background. Based on this, we propose a CAM-enhancing generative Re-ID method in which the global and local features are jointly used. In the proposed method, an adaptive CAM-enhancing local encoder is designed to explore the significance of local appearances and enhance the effect of crucial local features in generations, where the foreground is divided into multiple local parts and separated from the background by pedestrian segmentation. Moreover, a new generation loss is proposed to supervise the identity consistency by reducing the inconsistency of crucial regions in foregrounds and meanwhile enrich the intra-class diversity by generating variant backgrounds. Experimental results indicate that the proposed method obtains better generation images and Re-ID performance than other methods.

In video analysis, an important challenge is insufficient annotated data due to the rare occurrence of the critical patterns, and we need to provide discriminative frame-level representation with limited annotation in some applications. Multiple Instance Learning (MIL) is suitable for this scenario. However, many MIL models paid attention to analyzing the relationships between instance representations and aggregating them, but neglecting the critical information from the MIL problem itself, which causes difficultly achieving ideal instance-level performance compared with the supervised model. To address this issue, we propose the $\textbf{\textit{Regressor-Guided MIL network} (RGMIL)}$, which effectively produces discriminative instance-level representations in a general multi-classification scenario. In the proposed method, we make full use of the $\textit{regressor}$ through our newly introduced $\textit{aggregator}$, $\textbf{\textit{Regressor-Guided Pooling} (RGP)}$. RGP focuses on simulating the correct inference process of humans while facing similar problems without introducing new parameters, and the MIL problem can be accurately described through the critical information from the $\textit{regressor}$ in our method. In experiments, RGP shows dominance on more than 20 MIL benchmark datasets, with the average bag-level classification accuracy close to 1. We also perform a series of comprehensive experiments on the MMNIST dataset. Experimental results illustrate that our $\textit{aggregator}$ outperforms existing methods under different challenging circumstances. Instance-level predictions are even possible under the guidance of RGP information table in a long sequence. RGMIL also presents comparable instance-level performance with S-O-T-A supervised models in complicated applications. Statistical results demonstrate the assumption that a MIL model can compete with a supervised model at the instance level, as long as a structure that accurately describes the MIL problem is provided. The codes are available on $\url{https://github.com/LMBDA-design/RGMIL}$.

The concealed object detection in millimeter-wave human body images is a challenging task due to the noise and dim-small objects. Exploiting the spatial dependencies to mine the difference between the object and the noise is vital for the discrimination of objects. However, most approaches ignore the context around the object. In this paper, a concealed object detection framework based on structural context is proposed to suppress noise interference and refine localizable semantic features. The framework consists of two subnetworks, structural region-based multi-scale weakly supervised feature refinement and local context-based concealed object detection. The multi-scale weakly supervised feature refinement is constructed to learn position-aware semantics of objects of various sizes while suppressing background noises in structural regions. Specifically, a multi-scale pooling method is proposed to better localize objects of different sizes, and an object-activated region enhancement module is designed to strengthen object semantic representations and suppress the background interference. Moreover, an adaptive local context aggregation module is designed to integrate the local context around the bounding box in the concealed object detection, which improves the discrimination of the model for the dim-small objects. Experimental results on the AMMW and the PMMW datasets demonstrate that the proposed approach improves detection performance with lower false alarm rates.

Considering that deep learning achieves the prominent performance, it has been applied to synthetic aperture radar (SAR) image registration to improve the registration accuracy. In most methods, a deep registration model is constructed to classify matched points and unmatched points, in which SAR image registration is regarded as a supervised two-classification problem. However, it is difficult to annotate massive matched points manually in practice, which limits the performance of deep networks. Besides, inevitable differences among SAR images easily cause that some training and testing samples are inconsistent, which probably brings negative effects for training a robust registration model. To address these problems, we propose an adaptive self-supervised SAR image registration method, where SAR image registration is regarded as a self-supervised task rather than the supervised two-classification task. Inspired by self-supervised learning, we consider each point on SAR images as a category-independent instance, which mitigates the requirement of manual annotations. Based on key points from images, a self-supervised model is constructed to explore the latent feature of each key point, and then, pairs of match points are sought via evaluating similarities among key points and used to calculate the alignment transformation matrix. Meanwhile, to enhance the consistency of samples, we design a new strategy that constructs multiscale samples by transforming key points from one image into another, which avoids inevitable diversities between two images effectively. In particular, the constructed samples feeding to the self-supervised model are adaptively updated with the modification of the transformation matrix in iterations. Moreover, the similarity of maximal public areas (MPAS) indicator is proposed to assist in estimating the transformation. Finally, experimental results illustrate that the proposed method achieves more accurate registrations than other compared methods.

The active millimeter-wave (AMMW) scanner has been widely used for inspecting human security in public places in recent years owing to its ability to detect all kinds of objects under the clothes and be harmless to the body. However, it is really challenging to detect all concealed objects automatically and accurately due to inherent imaging noise, unknown object kind, and uncertain position. Recently, many existing methods, especially deep learning-based, have achieved good performances on concealed object detection. These methods work well for detecting a few kinds of large objects, but fail to perform on dim and incomplete hard objects. To address this task, a concealed object detection model with self-paced feature attention fusion network (SPFAFN) is proposed in this article. To be specific, the features with different scales are fused in a top-down manner to integrate details and global semantics to better detect small objects. During fusing multi-scale features, a hierarchical pyramid attention mechanism composed of channel and spatial attention is developed to perceive the object. Moreover, boosting self-paced learning is exploited to guide the model to learn hard samples that are difficultly detected. The proposed method is validated on two real-world datasets: an AMMW dataset and a publicly available passive millimeter-wave (PMMW) dataset. Experimental results demonstrate that the proposed approach is superior to the state-of-the-art methods, and achieves better performances on the two datasets with Average Precision (AP).

Template matching of multi-modal image has been a challenge to image matching, and it is difficult to balance the speed and the accuracy, especially for images with large sizes. Based on this, we propose a stepwise image matching method to achieve a precise location from the coarse-to-fine image matching by utilizing cascaded networks. In the proposed method, a coarse-grained matching network is firstly constructed to locate a rough matching position based on cross-correlating features of optical and SAR images. Specially, to enhance the credible matching position, a suppression network is designed to evaluate for the obtained cross-correlation feature and added into the coarse-grained network as a feedback. Secondly, a fine-grained matching network is constructed based on the obtained rough matching result to gain a more precise matching. In this part, ternary groups are utilized to construct the training samples. Interestingly, we apply the region with a few pixels offset as the negative class, which effectively distinguishes similar neighbourhoods of the rough matching position. Moreover, a modified Siamese network is used to extract features of SAR and optical images, respectively. Finally, experimental results illustrate that the proposed method obtains more precise matching compared with the state-of-the-art methods.

Ensemble learning performs better than a single classifier in most tasks due to the diversity among multiple classifiers. However, the enhancement of the diversity is at the expense of reducing the accuracies of individual classifiers in general and, thus, how to balance the diversity and accuracies is crucial for improving the ensemble performance. In this paper, we propose a new ensemble method which exploits the correlation between individual classifiers and their corresponding weights by constructing a joint optimization model to achieve the tradeoff between the diversity and the accuracy. Specifically, the proposed framework can be modeled as a shallow network and efficiently trained by the end-to-end manner. In the proposed ensemble method, not only can a high total classification performance be achieved by the weighted classifiers but also the individual classifier can be updated based on the error of the optimized weighted classifiers ensemble. Furthermore, the sparsity constraint is imposed on the weight to enforce that partial individual classifiers are selected for final classification. Finally, the experimental results on the UCI datasets demonstrate that the proposed method effectively improves the performance of classification compared with relevant existing ensemble methods.

An equivalent range equation with a monostatic equivalent component and a bistatic synthetic aperture radar (BiSAR) compensation component was proposed. There were five monostatic equivalent parameters (MEPs) in the both components. With the MEPs, a BiSAR range equation can be expressed in a form similar to the monostatic SAR (MoSAR) expression. Then, by using the range equation and the principle of stationary phase (POSP), we analytically derive the 2-D spectrum of a point target, and this 2-D spectrum is implemented into the range Doppler algorithm (RDA) that processes the translational invariant (TI) BiSAR data. In our RDA, spatially variant range cell migration (RCM) correction and azimuth compression (AC), both of which are related to the spatially variant MEPs, are required to focus all targets in the whole scene of different range cells. Simulations under a wide range of imaging parameters are conducted to assess the equivalent range equation and the 2-D spectrums. Satisfactory results are obtained. In addition, traditional methods for motion compensation can be directly applied to our equivalent range equation. Finally, the RDA is evaluated through the analysis of acquired raw BiSAR data. Well-focused images are obtained. Therefore, the proposed equivalent range equation, derived 2-D spectrum, and RDA have been validated for forming imagery using raw BiSAR data.

Riemannian optimization has been widely used to deal with the fixed low-rank matrix completion problem, and Riemannian metric is a crucial factor of obtaining the search direction in Riemannian optimization. This paper proposes a new Riemannian metric via simultaneously considering the Riemannian geometry structure and the scaling information, which is smoothly varying and invariant along the equivalence class. The proposed metric can make a tradeoff between the Riemannian geometry structure and the scaling information effectively. Essentially, it can be viewed as a generalization of some existing metrics. Based on the proposed Riemanian metric, we also design a Riemannian nonlinear conjugate gradient algorithm, which can efficiently solve the fixed low-rank matrix completion problem. By experimenting on the fixed low-rank matrix completion, collaborative filtering, and image and video recovery, it illustrates that the proposed method is superior to the state-of-the-art methods on the convergence efficiency and the numerical performance.

The diversity and the accuracy are two important ingredients for ensemble generalization error in an ensemble classifiers system. Nevertheless enhancing the diversity is at the expense of decreasing the accuracy of classifiers, thus balancing the diversity and the accuracy is crucial for constructing a good ensemble method. In the paper, a new ensemble method is proposed that selecting classifiers to ensemble via the transformation of individual classifiers based on diversity and accuracy. In the proposed method, the transformation of classifiers is made to produce new individual classifiers based on original classifiers and the true labels, in order to enhance diversity of an ensemble. The transformation approach is similar to principal component analysis (PCA), but it is essentially different between them that the proposed method employs the true labels to construct the covariance matrix rather than the mean of samples in PCA. Then a selecting rule is constructed based on two rules of measuring the classification performance. By the selecting rule, some available new classifiers are selected to ensemble in order to ensure the accuracy of the ensemble with selected classifiers. In other words, some individuals with poor or same performance are eliminated. Particularly, a new classifier produced by the transformation is equivalent to a linear combination of original classifiers, which indicates that the proposed method enhances the diversity by different transformations instead of constructing different training subsets. The experimental results illustrate that the proposed method obtains the better performance than other methods, and the kappa-error diagrams also illustrate that the proposed method enhances the diversity compared against other methods.

In this letter, a new classification method for fully polarimetric synthetic aperture radar (PolSAR) data based on three novel parameters is presented. The three parameters are derived from the eigenspace of the coherency matrix as linear combinations of its three eigenvalues. In the proposed classification method, the maximum value out of the three parameters is determined to assign a label to each image pixel, and the PolSAR image is classified into three classes accordingly. Experimental results based on NASA/JPL AIRSAR L-band data and CSA RADARSAT-2 C-band data illustrate the validity and efficacy of the procedure.