Pansharpening involves extracting spectral information from multispectral images and structural details from panchromatic images, then fusing them to produce high-resolution multispectral remote sensing images. However, high-resolution multispectral images often suffer from spectral or structural information loss. In this paper, we introduce a pansharpening algorithm based on a Progressive Feature Interaction Visual State Space Network. It enables interaction between local and global features of multispectral and panchromatic images and facilitates the injection of spectral and spatial details through distinct attention modules. This approach effectively preserves both spectral characteristics and spatial structure through inter-branch information interaction and complementation. Additionally, by integrating a visual state space network, the proposed model achieves deep reconstruction of multi-scale global information, enhancing robustness and generalization. Extensive experimental results demonstrate that the proposed network achieves highly competitive performance in both visual assessments and objective metric evaluations.

In this paper, we present the Magnetic Resonance Image (MRI) denoising method via nonlocal multidimensional low rank tensor transformation constraint (NLRT). We first design a nonlocal MRI denoising method by non-local low rank tensor recovery framework. Furthermore, a multidimensional low rank tensor constraint is used to obtain low-rank prior information combined with 3-dimensional structure feature of MRI image cubes. Our NLRT can achieve denoising by retaining more image detail information. The optimization and updating process of the model is solved via the alternating direction method of multipliers (ADMM) algorithm. Several state-of-the-art denoising methods are selected for comparative experiments. In order to reflect the performance of the denoising method, Rician noise with different levels is added to the experiment to analyze the results. The experimental results prove that our NLTR has more outstanding denoising ability and can obtain better MRI images.

Recently, a multimodal sentiment analysis of social media has attracted increasing attention, and its core idea is to discovery heuristic fusion strategy to analyze the sentiment orientations over heterogeneous multimodal source from a learned compact multimodal representation. The existing multimodal fusion techniques not only struggle to achieve full heterogeneous data interaction, but also they are unable to dynamically assess the quality of various modal data to determine predictability. In this article, we present a novel deep tensor evidence fusion (DTEF) network for multimodal sentiment classification. First, we propose a common view evaluation network that uses a long short-term memory (LSTM) network and a tensor-based neural network to extract rich intermodal and intramodal information. Then, we propose a unique time cue evaluation network that takes advantage of the temporal granularity associated with numerous pattern sequences. To make reliable decisions, we finally incorporate uncertainty through the trusted fusion layer, which improves the accuracy and robustness of sentimental classification. Our model is validated using the CMU Multimodal Opinion Sentiment and Emotion Intensity (CMU-MOSEI) and CMU Multimodal Corpus of Sentiment Intensity (CMU-MOSI) datasets, and the experimental findings demonstrate the superior performance of the proposed network in terms of accuracy compared with the state-of-the-art methods.

The rapid development of the digital age has led to a qualitative leap in social media. To meet the cognitive needs of users, social media platforms have been mining users’ private information and disseminating information through various means. However, these platforms lack effective management of information release and various forms of emotional expressions make public propaganda increasingly diverse and complex. Therefore, accurately identifying the relationships between multimodal data poses a challenge. An effective modal representation must consider both the consistency of multimodal data and the complementarity of single-modal data. However, existing methods focus on fusing different modal features into a unified feature representation, while neglecting to evaluate the reliability of prediction results. In this article, we disentangle the consistency and complementarity in the fused representation problem of multimodal data. We construct the modal private task (unique) by using the Dirichlet distribution and evidence theory to solve the uncertainty of each modal prediction. The model can output the uncertainty of prediction and learn complementary information through the fusion of decision layers. At the same time, we construct the modal common task using a low-rank tensor fusion model to learn consistent features. Finally, we compare the model with the current mainstream methods on three public datasets, and the experimental results show that the performance of our method reaches the level of current advanced algorithms.

Cyber-Manufacturing combines industrial big data with intelligent analysis to find and understand the intangible problems in decision-making, which requires a systematic method to deal with rich signal data. With the development of spectral detection and photoelectric imaging technology, spectral blind deconvolution has achieved remarkable results. However, spectral processing is limited by one-dimensional signal, and there is no available structural information with few training samples. Moreover, in the majority of practical applications, it is entirely feasible to gather unpaired spectrum dataset for training. This training method of unpaired learning is practical and valuable. Therefore, a two-stage deconvolution scheme combining self supervised learning and feature extraction is proposed in this paper, which generates two complementary paired sets through self supervised learning to extract the final deconvolution network. In addition, a new deconvolution network is designed for feature extraction. The spectrum is pre-trained through spectral feature extraction and noise estimation network to improve the training efficiency and meet the assumed noise characteristics. Experimental results show that this method is effective in dealing with different types of synthetic noise.

With the development of spectral detection and photoelectric imaging, multiband spectrum is always degraded by the random noise and band overlap during the acquisition of spectrum devices. Owing to the fixed spectrum degradation model, the existing spectrum deconvolution technologies are sensitive to the handcrafted model designed and manually selected parameters. The fundamental cause of these limitations during spectral analysis is that spectral processing is limited by 1-D signal without structural information available and insufficient training samples. In this article, a dual stream neural network is proposed to reconstruct the original infrared spectroscopy, which effectively strengthens the capability to represent the feature of infrared spectrum. A novel activation function is proposed to realize the function of the dual stream network. Furthermore, a heuristic learning strategy from the perspective of balanced self-paced learning is exploited to help network train from simple to difficult, resolving the problem of high sample repeatability. Compared with other traditional methods, the experimental results show that our network can achieve state-of-the-art reconstruction result and fairly excellent performance in terms of the corresponding index within synectics and real spectrum experiments.

Traffic sign classification plays a vital role in autonomous vehicles for its powerful capability in information representation. However, the low-quality data of traffic signs captured by in-vehicle cameras often inevitably bring inherent challenges to the one-shot classification task. Apart from the problem of data degradation, learning-based classification techniques of real traffic signs also come across the challenges of intra-class and inter-class data imbalance from the training data. To overcome the aforementioned problems, we propose an end-to-end degradation robust deep model, termed PcGAN, to classify traffic signs in a manner of few-shot learning. The proposed PcGAN models the joint distribution between the degraded traffic signal data and the corresponding prototypes from both degradation removal and generation perspectives by two alternating optimized modules, which ensures the generalization of the learned embedding of latent space for novel tasks. A multi-task loss function is designed to improve the robustness of PcGAN. Numerous experiments comprehensively demonstrate that the accuracy of our proposed PcGAN is improved by 5% compared with other state-of-the-art (SOTA) approaches in few-shot classification.

Many discriminative correlation filter (DCF)-based methods have successfully leveraged the guidance for solving two problems (i.e., the boundary effect and temporal filtering degradation) as a model prior to visual tracking. The intuitive motivation of these methods is to control the degeneration of the updating loss of the objective function with a structural framework. While these methods rely mostly on various regularization items, they always ignore the loss from data fidelity term. Therefore, we propose a bilateral weighted regression ranking model termed as BWRR. Here, we resort to two procedures for solving the above problems. First, BWRR introduces a bilateral constraint into the data fidelity term to control the loss of rows and columns of the filter learning data term. The weighted matrices could impose an adaptive penalty for large data loss during the learning process to avoid the model degradation problem. Second, the data of the updated weighted matrices is not directly applied to the calculation of the filter during each iteration. Instead, a new weighted product matrix is obtained by ranking and numerical transformation for updating the filter. We show that the proposed model converts the original correlation filter regression problem into a regression-with-ranking problem, thus avoiding the problem of positive and negative sample imbalance. Overall, the BWRR model is iteratively solved by the alternating direction method of multipliers(ADMM). Qualitative and quantitative evaluations demonstrate the effectiveness and superiority of our proposed method by extensive and quantitative experiments on the OTB, VOT, and UAV datasets.

Abstract Smoke has a very bad effect on the outdoor vision system. Not only are the videos with poor visual effects obtained, but also the quality and structure of the videos are reduced. In this paper, we propose a video smoke removal method based on low‐rank tensor completion via spatial‐temporal continuity constraint. The proposed method is based on the smoke mixing model and consider the sparseness of smoke and the global and local consistency of clean video. Then, the optimal solution of the smoke removal algorithm model is quickly realized by the Alternating Direction Method of Multiplier. Finally, we evaluate the experiment results of real‐world data and simulated data from the visual effects and objective indicators. And the experiment results show that our proposed algorithm can achieve better smoke removal results.

Abstract Morphological edge detection is a principal component in pattern recognition and machine vision. Traditional edge detection operators only take pixel mutual into consideration. However, the edges are influenced not only by pixel mutual but also by the boundary characteristics. Here, the vector co‐occurrence morphological edge detection operator is proposed, which takes the pixel and boundary information both into consideration. The vector co‐occurrence algorithm is exploited to resist the influence of the noise points and detect the edges from the colour image rather than the grey image. And, we lead to define a precise definition of the manner of sorting high‐dimensional data for the colour image. The experiment results always illustrate the advancement and practicability of our methods against the baseline method. In terms of experiments, the BSDS500 dataset is introduced to compare and analyse with other algorithms. Based on the standard benchmark index evaluation in the BSDS500 dataset, the ODS and AP of various algorithms are compared and analysed.

Abstract This paper proposes an algorithm for the tensor completion problem of estimating multi‐linear data under the limitation of observation rate. Many tensor completion methods are based on nuclear norm minimization, they may fail to achieve the global solution for solving nuclear norm minimization in tensor completion problem with high missing ratio. To tackle this issue, an adaptive tensor completion method based on parallel multi‐block alternating direction method of multipliers (ADMM) algorithm is proposed, it can derive the model from the initial estimate and compute the next estimate from the current solution. The parallel multi‐block ADMM with global convergence is adopted to solve the dual problem, which greatly improves the processing power and reliability of the algorithm.

Image segmentation is a key component of image analysis, which refers to the process of partitioning the image into multiple segments. Graph cut is widely used in image segmentation by constructing a graph that the minimal cut of this graph would lead to partition the corresponding pixels of the different objects. In this paper, we reconstruct the graph cut problem as a special non-convex optimization problem instead of the traditional maximum flow problem. We extend this non-convex problem to the hypergraph method and combine it with a tensor field based on a directional bilateral filter bank to achieve segmentation in grayscale images. Accordingly, an efficient minimization algorithm is proposed to solve this non-convex problem with global convergence. Furthermore, we have selected the data of BSDS300 and BSDS500 as tests. Experimental results and evaluation index tests further demonstrate the superiority of the proposed method.

The procedure of establishing the correspondence between two sets of feature points is important in computer vision applications. In this article, an elastic net constraint-based tensor model is proposed for high-order graph matching. To control the tradeoff between the sparsity and the accuracy of the matching results, an elastic net constraint is introduced into the tensor-based graph matching model. Then, a nonmonotone spectral projected gradient (NSPG) method is derived to solve the proposed matching model. During the optimization of using NSPG, we propose an algorithm to calculate the projection on the feasible convex sets of elastic net constraint. Further, the global convergence of solving the proposed model using the NSPG method was proved. The superiority of the proposed method is verified through experiments on the synthetic data and natural images.

Infrared small target detection technology is one of the key technologies in the field of computer vision. In recent years, several methods have been proposed for detecting small infrared targets. However, the existing methods are highly sensitive to challenging heterogeneous backgrounds, which are mainly due to: 1) infrared images containing mostly heavy clouds and chaotic sea backgrounds and 2) the inefficiency of utilizing the structural prior knowledge of the target. In this article, we propose a novel approach for infrared small target detection in order to take both the structural prior knowledge of the target and the self-correlation of the background into account. First, we construct a tensor model for the high-dimensional structural characteristics of multiframe infrared images. Second, inspired by the low-rank background and morphological operator, a novel method based on low-rank tensor completion with top-hat regularization is proposed, which integrates low-rank tensor completion and a ring top-hat regularization into our model. Third, a closed solution to the optimization algorithm is given to solve the proposed tensor model. Furthermore, the experimental results from seven real infrared sequences demonstrate the superiority of the proposed small target detection method. Compared with traditional baseline methods, the proposed method can not only achieve an improvement in the signal-to-clutter ratio gain and background suppression factor but also provide a more robust detection model in situations with low false-positive rates.

In the fields of industry research, infrared spectrometers are widely used in diverse applications. However, the spectrum often suffers from band overlap and random noise due to the distortion caused by the point spread function, especially for aging instruments. The problem of reconstructing the clear spectrum from the degraded spectrum is called spectrum deconvolution. Traditional partial differential equation (PDE) methods rely on distribution assumptions in the reconstructed process. This restriction makes PDE methods sensitive to tackle complex instrumental broadening effect in the dispersive IR spectrometers. Also, we need to spend much time setting the parameters of PDE models manually. These problems intuitively degrade the performances of PDE methods. In this article, we propose an end-to-end neural network framework for spectral deconvolution problem. The novelty of this article lies in its strong robustness from dilated deconvolution and self-paced learning procedure to challenge the complicated degraded spectra. Actually, the deconvolution problem is tailored to a dense prediction problem in this article. Inspired by the extensive use and excellent effects of dilated convolutions in dense prediction, a lightweight dilated convolution module is given to detect the overlaps of degraded spectra. Experimental results demonstrate that the proposed solution has an outstanding performance against many other approaches. Such improvements have the potential to facilitate industrial applications and further exploration of an unknown chemical mixture. Our framework has a good performance on feature extracting and spectrum reconstruction, even in the case of low signal-to-noise ratio.

Recently, infrared small target detection problem has attracted substantial attention. Many works based on local low-rank model have been proven to be very successful for enhancing the discriminability during detection. However, these methods construct patches by traversing local images and ignore the correlations among different patches. Although the calculation is simplified, some texture information of the target is ignored, and targets of arbitrary forms cannot be accurately identified. In this paper, a novel target-aware method based on a non-local low-rank model and saliency filter regularization is proposed, with which the newly proposed detection framework can be tailored as a non-convex optimization problem, therein enabling joint target saliency learning in a lower dimensional discriminative manifold. More specifically, non-local patch construction is applied for the proposed target-aware low-rank model. By combining similar patches, we reconstruct them together to achieve a better generalization of non-local spatial sparsity constraints. Furthermore, to encourage target saliency learning, our proposed saliency filtering regularization term based on entropy is restricted to lie between the background and foreground. The regularization of the saliency filtering locally preserves the contexts from the target and surrounding areas and avoids the deviated approximation of the low-rank matrix. Finally, a unified optimization framework is proposed and solved with the alternative direction multiplier method (ADMM). Experimental evaluations of real infrared images demonstrate that the proposed method is more robust under different complex scenes compared with some state-of-the-art methods.

Multimodal fusion is one of the popular research directions of multimodal research, and it is also an emerging research field of artificial intelligence. Multimodal fusion is aimed at taking advantage of the complementarity of heterogeneous data and providing reliable classification for the model. Multimodal data fusion is to transform data from multiple single-mode representations to a compact multimodal representation. In previous multimodal data fusion studies, most of the research in this field used multimodal representations of tensors. As the input is converted into a tensor, the dimensions and computational complexity increase exponentially. In this paper, we propose a low-rank tensor multimodal fusion method with an attention mechanism, which improves efficiency and reduces computational complexity. We evaluate our model through three multimodal fusion tasks, which are based on a public data set: CMU-MOSI, IEMOCAP, and POM. Our model achieves a good performance while flexibly capturing the global and local connections. Compared with other multimodal fusions represented by tensors, experiments show that our model can achieve better results steadily under a series of attention mechanisms.