In multimodal learning, multimodal coordinated representation is an important yet challenging issue, which establishes the interaction between different modalities to describe multimodal data more effectively. Existing coordinated representation methods are implemented in the deep feature space (or encoding space) of each modality. In this paper, based on the framework of evidence theory, we propose a novel coordinated representation method, where multimodal data is described as the basic belief assignment (BBA), and coordinated learning is implemented in the evidential space (i.e., the BBA-based space). That is, the information interaction between different modalities is implemented at the level of evidence modeling (or uncertainty modeling). To use the intra-class and inter-class difference information of multimodal data, we design an evidential coordinated constraint. Furthermore, to represent each modality clearly, we introduce an ambiguity constraint. Experimental results of multimodal classification show that our proposed method is rational and effective.

Dempster-Shafer evidence theory (DST) is a theoretical framework for uncertainty modeling and reasoning, with modeling the basic belief assignment (BBA) as one of its most crucial and challenging tasks. The prevailing BBA determination methods have their own pros and cons, and the joint use of them is expected to provide a better BBA. To realize an end-to-end BBA modeling without explicitly using various prevailing BBA modeling methods, a learning-based BBA modeling approach with multi-method fusion (LBMMF) is proposed in this paper. Deep learning is used to train a deep network which learns the mapping from the training samples to the comprehensive BBAs obtained by jointly using the prevailing BBA modeling methods as the generalized training labels. Given a test sample, the corresponding BBA can be obtained in an end-to-end manner, which is the output of the trained deep neural network. Experimental results show that to use the BBA obtained by our method can achieve better classification performance.

Background Aware Correlation Filter (BACF) tracker achieves accurate tracking result in visual object tracking by mitigating boundary effects, yet is limited in challenging scenarios especially in viewpoint change and illumination variation, which are frequently encountered in Unmanned Aerial Vehicle (UAV) tracking tasks. To address the shortcomings, we propose a Foreground Aware Correlation Filter with adaptive feature response fusion (FACF). In this paper, we use saliency detection to generate foreground prior knowledge in training phase for suppressing potential noise. Furthermore, recognizing the limitation of BACF, which relies on a single feature, a novel adaptive fusion strategy is designed to fuse multiple feature responses during the detection phase. This strategy aims to enhance the robustness of the tracker. Extensive experiments have been conducted on three challenging benchmarks. The tracking results show that the proposed tracker performs accurate and robust tracking result and satisfies real-time requirement with 48.28fps.

Traditional pattern recognition systems, tasked with categorizing inputs into known classes, often struggle when they encounter samples they haven’t been trained to recognize. This introduces the need for the open set recognition—enhancing models to reject unidentified samples effectively. The Openmax method represents a significant breakthrough in this field by leveraging deep learning to spot and handle these new, unseen classes, broadening the traditional Softmax layer to accommodate an “unknown” class and employing a single threshold to separate the known from the unknown. However, the reliance of the original Openmax method on a single threshold may result in incorrect classifications if the parameters are not selected appropriately. To address this, we introduce a dual-threshold fusion mechanism based on Dempster-Shafer evidence theory in this paper. This approach releases the difficulty of finding a precise threshold in the complex and dynamic real-world environments. By integrating deep networks with a novel evidence-based system, the refined approach can bolster the robustness of rejecting undefined classes.

In this article, a new variational pansharpening method based on kernel estimation and regional extended low rank is proposed, which aims to generate a high resolution multispectral (HRMS) image by fusing the panchromatic (PAN) and multispectral (MS) image. First, an estimated blurring kernel is generated for the spectral constraint term, which can build the relationship between the MS and HRMS image more accurately and improve the spectral quality of the HRMS image. Second, a spatial constrain term is designed by adopting the proportional relationship of the PAN and HRMS image in gradient domain, which preserves the geometric information of the PAN image well. Third, according to sensor imaging principle, a prior constraint term is proposed based on regional extended low rank, which can improve the spatial clarity of HRMS image. The above three constraint terms are combined to form the proposed variational pansharpening method, and the ADMM method is applied for solving it. Finally, experiments show the effectiveness of the proposed method through comparing with other state-of-art pansharpening methods.

Dempster-Shafer theory (DST) is an important theory for information fusion. However, in DST how to determinate the basic belief assignment (BBA) is still an open issue. The interval number based BBA determination method is simple and effective, where the features of different classes' samples are modeled using the interval numbers, i.e., an interval number model is constructed for each focal element. Then, the distances of interval numbers are used for measuring the similarity degrees between the testing sample and each focal element, and the similarity degrees are used for determinating the BBA. The definition of interval numbers' distance is crucial for the effectiveness of the interval number based BBA determination methods. In this paper, we use different interval numbers' distances for determinating BBAs. By using the artificial data set and the Iris date set of open UCI data base, respectively, we compare and analyze the determination of BBAs with different distances.

Edge detection is one of the most important tasks in image processing and pattern recognition. Edge detector with multiple color channels can provide more edge information. However, the uncertainty occurring with the edge detection in each single channel and the discordance existing in the fusion of multiple channels edge detectors make the detection difficult. In this paper, we propose a new edge detection method in color images based on information fusion. We show that obtaining final edge through fusing the edge information in each channel is a challenging problem to make decision in the framework of Multi-Criteria decision making (MCDM). In this work, we propose to detect edges in color images using Cautious OWA with evidential reasoning (COWA-ER) and Fuzzy-Cautious OWA with evidential reasoning (FCOWA-ER) to handle the uncertainty and discordance. Experimental results show that the proposed approaches achieve better edge detection performance compared with the original edge detector.

Support vector machine (SVM) is a popular machine learning method and has been widely applied in many real-world applications. Since SVM is sensitive to noises, fuzzy SVM (FSVM) has been proposed to relieve the over-fitting problem caused by noises through assigning a fuzzy membership to each sample. Then, different samples make different contributions to the learning of classification hyperplane. However, standard fuzzy SVM only concerns on the information within the classes. It can't make full use of the prior information of the samples. Therefore, we propose a new method called weighted SVM based on the theory of belief functions (BFW-SVM). The main idea is to associate the samples with corresponding weights before training according to the theory of belief functions, which can describe the information more detailed. Experiments on benchmark datasets show that our proposed BFW-SVM can handle the classification problems with noises and outliers more effectively.

Dempster-Shafer evidence theory (DST) is a theoretical framework for uncertainty modeling and reasoning. The determination of basic belief assignment (BBA) is crucial in DST, however, there is no general theoretical method for BBA determination. In this paper, a method of generating BBA using fuzzy numbers is proposed. First, the training data are modeled as fuzzy numbers. Then, the dissimilarities between each test sample and the training data are measured by the distance between fuzzy numbers. In the final, the BBAs are generated from the normalized dissimilarities. The effectiveness of this method is demonstrated by an application of classification problem. Experimental results show that the proposed method is robust to outliers.

In the sense of likelihood ratio test (LRT), a new type of distributed constant false alarm rate (CFAR) scheme-CCAWCA(censored cell-averaging -R-weighted cell averaging) CFAR detector is presented. Its characteristic is that censored cell-averaging (CCA) CFAR algorithms are used in local processors to form the estimation of SNR of local observations, and then the estimation transmitted to the data fusion center (DFC). Finally, the fusion center makes the final decision based on the weighted cell averaging (WCA). Since the weights are adjusted according to different SNR adaptively, the proposed detector can be available in the case that the target echo and noise/clutter have different level for every sensor. In addition, it does not need a priori knowledge about the interference in order to perform well. Furthermore, unlike the OS-CFAF the tolerance of interfering targets is restricted in appointed k value’s. Under Swerling 2 assumption, the analytic expression of detection probability and false alarm probability are derived.

Due to the presence of speckle in synthetic aperture radar (SAR) image, the existing registration algorithms, which are successfully used in optical remote sensing image, are usually not applicable to it directly. An automatic SAR image registration algorithm is proposed in this paper. Firstly, the element triangles are constructed from the point targets detected from the SAR images; then they are matched by integrating triangle moment invariance proposed in this paper and region invariant moments; finally, the LMSE algorithm is used to estimate the affine transformation parameters, thus the SAR images can be registered automatically. The proposed algorithm is evaluated and compared with the existing methods by means of invariant moments (IM) and affine moment invariants (AMI). It is shown from Monte-Carlo simulations that the proposed algorithm is robust to detection error and partial correspondence of control points (CPs), and has higher ratio of correct matching than the methods using IM or AMI. Experimental results show that the proposed new algorithm is not only valid in the automatic registration of SAR images, but also can avoid the influence caused by speckle in feature detection and feature matching process