Transformers have been successfully applied to long-term time series forecasting (LTSF) owing to their ability to model long-term dependencies of time series by the multi-head attention mechanism. However, most existing Transformer-based forecasting models capture temporal dependency patterns, while ignoring the frequency patterns. To balance both patterns, we explore a novel approach of applying both frequency filtering and attention mechanism within the Transformer for the LTSF task. We design the frequency filtering layer that represents time series in terms of their frequency components to capture frequency features with log-linear complexity, providing deeper insights into global dependencies of the data. Then, we propose FAformer, a simple yet effective architecture built upon frequency and attention in Transformers for LTSF. In FAformer, the frequency filtering layer captures global dependencies in time series, and then the deeper attention layer further models the features. Extensive experiments demonstrate the effectiveness of our proposed method, which outperforms state-of-the-art (SOTA) baselines on nine real-world datasets. The code will be publicly available.

Detecting small objects in large-scale, high-resolution aerial images presents significant challenges. Most existing detectors focus primarily on the design of detection heads and fusion layers, often overlooking information loss in the backbone and the excessive computational resources required, which are particularly constrained in aerial image analysis. To address the aforementioned challenges, we propose the Resolution-Aware Criss-Cross Attention Detector (RACDet), which effectively leverages the contextual information embedded in an innovative backbone RACNet of aerial images. By decomposing the position information into orthogonal horizontal and vertical components, we achieve efficient modeling of spatial dependencies. For each pixel, RACNet gathers contextual information from all other pixels in the same position, establishing position relationships early, which can guide the subsequent processing in convolutional networks across different resolutions. The proposed method not only provides an adaptive representation of feature maps at multi-scale resolutions using normalized position encoding, but also enhances the detection accuracy of small objects by leveraging a regression loss function based on smooth Gaussian Wasserstein distance. We evaluate our method on two challenging aerial image datasets, including VisDrone2019 and UAVDT. Comprehensive experiments show that our approach achieves state-of-the-art performance while significantly decreasing the number of FLOPs.

Accurate and fast segmentation of 3D medical images is crucial in clinical analysis. CNNs struggle to capture long-range dependencies because of their inductive biases, whereas the Transformer can capture global features but faces a considerable computational burden. Thus, efficiently integrating global and detailed insights is key for precise segmentation. In this paper, we propose an effective and lightweight architecture named GCI-Net to address this issue. The key characteristic of GCI-Net is the global-guided feature enhancement strategy (GFES), which integrates the global context and facilitates the learning of local information; 3D convolutional attention, which captures long-range dependencies; and a progressive downsampling module, which perceives detailed information better. The GFES can capture the local range of information through global-guided feature fusion and global-local contrastive loss. All these designs collectively contribute to lower computational complexity and reliable performance improvements. The proposed model is trained and tested on four public datasets, namely MSD Brain Tumor, ACDC, BraTS2021, and MSD Lung. The experimental results show that, compared with several recent SOTA methods, our GCI-Net achieves superior computational efficiency with comparable or even better segmentation performance. The code is available athttps://github.com/qintianjian-lab/GCI-Net.

Deep multi-view subspace clustering aims to reveal a common subspace structure by exploiting rich multi-view information. Despite promising progress, current methods focus only on multi-view consistency and complementarity, often overlooking the adverse influence of entangled superfluous information in features. Moreover, most existing works lack scalability and are inefficient for large-scale scenarios. To this end, we innovatively propose a deep subspace clustering method via Multi-view Feature Decoupling (MvFD). First, MvFD incorporates well-designed multi-type auto-encoders with self-supervised learning, explicitly decoupling consistent, complementary, and superfluous features for every view. The disentangled and interpretable feature space can then better serve unified representation learning. By integrating these three types of information within a unified framework, we employ information theory to obtain a minimal and sufficient representation with high discriminability. Besides, we introduce a deep metric network to model self-expression correlation more efficiently, where network parameters remain unaffected by changes in sample numbers. Extensive experiments show that MvFD yields State-of-the-Art performance in various types of multi-view datasets.

Multivariate time series (MTS) forecasting is considered as a challenging task due to complex and nonlinear interdependencies between time steps and series. With the advance of deep learning, significant efforts have been made to model long-term and short-term temporal patterns hidden in historical information by recurrent neural networks (RNNs) with a temporal attention mechanism. Although various forecasting models have been developed, most of them are single-scale oriented, resulting in scale information loss. In this article, we seamlessly integrate multiscale analysis into deep learning frameworks to build scale-aware recurrent networks and propose two multiscale recurrent network (MRN) models for MTS forecasting. The first model called MRN-SA adopts a scale attention mechanism to dynamically select the most relevant information from different scales and simultaneously employs input attention and temporal attention to make predictions. The second one named as MRN-CSG introduces a novel cross-scale guidance mechanism to exploit the information from coarse scale to guide the decoding process at fine scale, which results in a lightweight and more easily trained model without obvious loss of accuracy. Extensive experimental results demonstrate that both MRN-SA and MRN-CSG can achieve state-of-the-art performance on five typical MTS datasets in different domains. The source codes will be publicly available at https://github.com/qguo2010/MRN.

Recently, the DEtection TRansformer (DETR) and its variants have achieved success in object detection, but they still require manual intervention in complex scenes. Object detection methods cannot entirely replace human capabilities. Leveraging human perception to assist remote sensing object detection offers a new technical approach. In the process of manual verification, the most prominent signals reflecting human identification of specific targets are electroencephalogram (EEG) and eye movement. To our best knowledge, there is currently no publicly available dataset for this task. Therefore, we collect two types of human perception information while observing remote sensing images, forming a trimodal dataset including brain signals, eye movements, and remote sensing images. Upon this, we propose a multimodal object detection method, termed as BEI-DETR (Brain-Eye-Image DETR). Specifically, we decouple the classification and localization tasks in the object detection, using EEG for classification and eye movement information for localization. Subsequently, we propose a feature matching fusion module (FMFM) to fuse multimodal features and then put them into the decoder and prediction heads to obtain the final detection results. Comparative and ablation experiments demonstrate the satisfactory performance of BEI-DETR, indicating that our model can effectively learn human perceptual information for the remote sensing object detection task. The trimodal dataset and source code are publicly available at https://github.com/Hickey-Curry/BEI-DETR and https://github.com/Hickey-Curry/BEIDataset.

Tensor robust principal component analysis (TRPCA) is a classical way for low-rank tensor recovery, which minimizes the convex surrogate of tensor rank by shrinking each tensor singular value equally. However, for real-world visual data, large singular values represent more significant information than small singular values. In this paper, we propose a nonconvex TRPCA (N-TRPCA) model based on the tensor adjustable logarithmic norm. Unlike TRPCA, our N-TRPCA can adaptively shrink small singular values more and shrink large singular values less. In addition, TRPCA assumes that the whole data tensor is of low rank. This assumption is hardly satisfied in practice for natural visual data, restricting the capability of TRPCA to recover the edges and texture details from noisy images and videos. To this end, we integrate nonlocal self-similarity into N-TRPCA, and further develop a nonconvex and nonlocal TRPCA (NN-TRPCA) model. Specifically, similar nonlocal patches are grouped as a tensor and then each group tensor is recovered by our N-TRPCA. Since the patches in one group are highly correlated, all group tensors have strong low-rank property, leading to an improvement of recovery performance. Experimental results demonstrate that the proposed NN-TRPCA outperforms existing TRPCA methods in visual data recovery. The demo code is available at https://github.com/qguo2010/NN-TRPCA.

Partially Relevant Video Retrieval (PRVR) aims to retrieve partially relevant videos from many unlabeled and untrimmed videos according to the query, which is defined as the multiple instance learning problem. The challenge of PRVR is that it utilizes untrimmed videos, which are much closer to reality. The existing methods excavate video-text semantic consistency information insufficiently and lack the capacity to highlight the semantics of key representations. To tackle these issues, we propose a transferable dual multi-granularity semantic excavating network, called T-D3N, to focus on enhancing the learning of dual-modal representations. Specifically, we first introduce a novel transferable textual semantic learning strategy by designing Adaptive Multi-scale Semantic Mining (AMSM) component to excavate significant textual semantic from multiple perspectives. Second, T-D3N distinguishes the feature differences from the frame-wise perspective to better perform contrastive learning between positive and negative samples in the video feature domain, which can further distance the positive and negative samples and improve the probability of positive samples being retrieved by query. Finally, our model constructs multi-grained video temporal dependencies and conducts cross-grained core feature perception, which enables more sufficient multimodal interactions . Extensive experiments are performed on three benchmarks, i.e., ActivityNet Captions, Charades-STA, and TVR, our T-D3N achieves state-of-the-art results. Furthermore, we also confirm that our model is transferable on a broad range of multimodal tasks such as T2VR, VMR, and MMSum.

Feature refinement and feature fusion are two key steps in convolutional neural networks–based salient object detection (SOD). In this article, we investigate how to utilize multiple guidance mechanisms to better refine and fuse extracted multi-level features and propose a novel multi-guidance SOD model dubbed as MGuid-Net. Since boundary information is beneficial for locating and sharpening salient objects, edge features are utilized in our network together with saliency features for SOD. Specifically, a self-guidance module is applied to multi-level saliency features and edge features, respectively, which aims to gradually guide the refinement of lower-level features by higher-level features. After that, a cross-guidance module is devised to mutually refine saliency features and edge features via the complementarity between them. Moreover, to better integrate refined multi-level features, we also present an accumulative guidance module, which exploits multiple high-level features to guide the fusion of different features in a hierarchical manner. Finally, a pixelwise contrast loss function is adopted as an implicit guidance to help our network retain more details in salient objects. Extensive experiments on five benchmark datasets demonstrate our model can identify salient regions of an image more effectively compared to most of state-of-the-art models.

To reproduce the latent high-quality image from its observed image, traditional image restoration approaches based on low-rank prior usually solve the low-rank matrix approximation problem by minimizing the nuclear norm. However, most of approaches work on the desired singular values individually, and ignore the underlying statistical property of singular values. In this work, we propose a probability-inducing nuclear norm minimization (PINNM) algorithm, where a probability-inducing singular value estimator is presented to estimate the desired singular values. For further filling-in the image details lost in handling the singular values, a simple yet efficient residual cascade scheme is designed to refine the image quality by using the intermediate recovered image. Then, the proposed PINNM algorithm is applied on two classic image restoration tasks: image super-resolution and denoising. Experimental results demonstrate that, the proposed PINNM algorithm outperforms many state-of-the-art approaches both quantitatively and qualitatively for the aforementioned tasks. The source code is available at https://github.com/cvzh/PINNM.

Tensor robust principal component analysis (TRPCA) is an important algorithm for color image denoising by treating the whole image as a tensor and shrinking all singular values equally. In this paper, to improve the denoising performance of TRPCA, we propose a variant of TRPCA model. Specifically, we first introduce a nonconvex TRPCA (N-TRPCA) model which can shrink large singular values more and shrink small singular values less, so that the physical meanings of different singular values can be preserved. To take advantage of the structural redundancy of an image, we further group similar patches as a tensor according to nonlocal prior, and then apply the N-TRPCA model on this tensor. The denoised image can be obtained by aggregating all processed tensors. Experimental results demonstrate the superiority of the proposed denoising method beyond state-of-the-arts.

Based on the image self‐similarity and singular value decomposition (SVD) techniques, the authors propose an iterative adaptive global denoising method. For the structural differences between image patches, they adaptively determine the size of the search window. In each window, a similar image patch matrix is constructed based on the multi‐scale similarity measure. In order to ensure the speed of the method, the adaptive step size and the number of image patches are introduced, and all image patches are denoised in different iterations. This not only ensures the speed of the method, suppresses residual noise, but also reduces the artefacts caused by the fixed step size and the number of image patches. Therefore, the problem of image denoising is converted to the estimation of low‐rank matrix. New singular values are estimated according to the noise level, and similar image patch matrices without noise are estimated using them and corresponding singular vectors. Experimental results show that compared with the state‐of‐the‐art denoising algorithms, this method has a higher PSNR and FSIM, and has a good visual effect. The new method can be applied to image and video restoration, target recognition and image classification.

Superpixel segmentation approach, as a preprocessing step in computer vision tasks, groups pixels into perceptually coherence atomic regions to replace the pixel grid in images and reduce the primitives and redundancy of subsequent works. In this study, the authors proposed a fast and robust superpixel generation method based on non‐iterative framework with the constraint of linear path. They collected neighbouring pixels as the initial superpixels one by one in the conventional order at first. To make the superpixels attach to the most object boundaries well and robust to noise, they defined a new distance measurement between pixels and superpixel seeds by considering the colour difference of pixels in the neighbourhood and along the linear path from the pixel to the seed. Meanwhile, they proposed a new way to set parameters adaptively based on the intrinsic quality of images. Then, they refined the initial superpixels by merging the smallest ones until the number of superpixel meets the expectation. The experimental results on clean and noisy images demonstrate that the proposed method is effective and presents a competitive performance in computational efficiency with the state‐of‐the‐art real‐time method.

Low‐rank approximation has shown great potential in various image tasks. It is found that there is a specific functional relationship about singular values between the original image and a series of noisy images, which can be used to construct the singular values of a noise‐free image. In this study, the authors propose a novel denoising method based on the above facts and low‐rank approximation theory. Firstly, they estimate the noise energy distribution of the group matrix in the singular value decomposition (SVD) domain using the energy characteristics of the image with different noise levels. The energy distribution of the noise is shrunk to obtain the energy distribution of the true signal. Then, based on the optimal energy compaction property of SVD, the low‐rank property of matrix is constrained in the SVD domain to obtain the low‐rank approximation of the matrix. Moreover, an iterative back projection method is adopted in this study to suppress residual noise. A new noise standard deviation estimation approach, targeted at the back projection process, is proposed to effectively optimise the denoising results during the iteration. Experimental results show that the authors’ method efficiently decreases the noise and achieves comparable denoising performance to the state‐of‐the‐art methods regarding both quantitative measurement and visual effect.

For the problem that the limited star map field angle cannot obtain the complete star map accurately, the paper study astral intrinsic and imaging features, a star map stitching algorithm based on the principle of visual perception is proposed firstly. The matching models of time and space dimensions is constructed by simulating the visual perception, then the stars and the planets points are saved by searching the matching star group dynamically, the star map is stitched and reconstructed efficiently by creating the computer sparse storage model. The experimental results show that the algorithm can achieve data compression quickly, compression ratio is 99.54%, which can reduce complexity of manual processing and can achieve star map stitching accurately.

To recover the corrupted pixels, traditional inpainting methods based on low-rank priors generally need to solve a convex optimization problem by an iterative singular value shrinkage algorithm. In this paper, we propose a simple method for image inpainting using low rank approximation, which avoids the time-consuming iterative shrinkage. Specifically, if similar patches of a corrupted image are identified and reshaped as vectors, then a patch matrix can be constructed by collecting these similar patch-vectors. Due to its columns being highly linearly correlated, this patch matrix is low-rank. Instead of using an iterative singular value shrinkage scheme, the proposed method utilizes low rank approximation with truncated singular values to derive a closed-form estimate for each patch matrix. Depending upon an observation that there exists a distinct gap in the singular spectrum of patch matrix, the rank of each patch matrix is empirically determined by a heuristic procedure. Inspired by the inpainting algorithms with component decomposition, a two-stage low rank approximation (TSLRA) scheme is designed to recover image structures and refine texture details of corrupted images. Experimental results on various inpainting tasks demonstrate that the proposed method is comparable and even superior to some state-of-the-art inpainting algorithms.

Nonlocal self-similarity of images has attracted considerable interest in the field of image processing and has led to several state-of-the-art image denoising algorithms, such as block matching and 3-D, principal component analysis with local pixel grouping, patch-based locally optimal wiener, and spatially adaptive iterative singular-value thresholding. In this paper, we propose a computationally simple denoising algorithm using the nonlocal self-similarity and the low-rank approximation (LRA). The proposed method consists of three basic steps. First, our method classifies similar image patches by the block-matching technique to form the similar patch groups, which results in the similar patch groups to be low rank. Next, each group of similar patches is factorized by singular value decomposition (SVD) and estimated by taking only a few largest singular values and corresponding singular vectors. Finally, an initial denoised image is generated by aggregating all processed patches. For low-rank matrices, SVD can provide the optimal energy compaction in the least square sense. The proposed method exploits the optimal energy compaction property of SVD to lead an LRA of similar patch groups. Unlike other SVD-based methods, the LRA in SVD domain avoids learning the local basis for representing image patches, which usually is computationally expensive. The experimental results demonstrate that the proposed method can effectively reduce noise and be competitive with the current state-of-the-art denoising algorithms in terms of both quantitative metrics and subjective visual quality.

It is challenging to consistently smooth natural images, yet smoothing results determine the quality of a broad range of applications in computer vision. To achieve consistent smoothing, we propose a novel optimization model making use of the redundancy of natural images, by defining a nonlocal concentration regularization term on the gradient. This nonlocal constraint is carefully combined with a gradient-sparsity constraint, allowing details throughout the whole image to be removed automatically in a data-driven manner. As variations in gradient between similar patches can be suppressed effectively, the new model has excellent edge preserving, detail removal, and visual consistency properties. Comparisons with state-of-the-art smoothing methods demonstrate the effectiveness of the new method. Several applications, including edge manipulation, image abstraction, detail magnification, and image resizing, show the applicability of the new method.

Region covariance descriptor that fuses multiple features compactly has proven to be very effective for visual tracking. While working effectively, the exhaustive global search strategy of covariance tracking is still inefficient, and there is much room for improvement. It may cause inconsecutive tracking trajectory and distraction. A suitable region similarity metric for covariance matching between the candidate object region and a given appearance template is of much importance. However, the computational burden of the metric, especially for large matrices under Riemannian space, may hinder its application in gradient‐based algorithms. In this study, the authors propose an algorithm which, by minimising the metric function, exploits an efficient conjugate gradient method to iteratively search the best matched candidate, and determines the search step size by non‐monotonic liner strategy. Then, an inferential reasoning in view of new efficient metric is derived for the gradient‐based algorithm. The authors test the proposed tracking method on test baseline dataset. Both quantitative and qualitative results demonstrate the effectiveness of the proposed algorithm compared with other state‐of‐the‐art methods.

In recent years, many time synchronization protocols for wireless sensor networks are presented and reducing the number of synchronous data packets in transmission has become a new research breakthrough. In this paper, the nodes with amount of power held are placed in different areas where the two nodes may exchange the synchronization information. This way does not only efficiently use of energy during the synchronization process, but also reduce the overall timing messages. Based on inherent clock drift and clock offset problem, the Kalman Filter algorithm is used to optimize the clock deviation, which can minimize the mean-square error (MSE). The performance of the proposed time synchronization algorithm is described through the simulation results, there is not present relatively large deviation after optimizing comparing to clsscical TPSN algorithm for synchronization precision.

A constraint-based sensor network nodes particle swarm search localization algorithm (CPL) is presented. First of all, a constraint domain of an unknown node must be determined; Then the positions which meet specific criteria is searched out by particle swarm optimization algorithm and the searching results within the constraint domain are recorded; Finally, the unknown node's localization can be obtained by calculating the average recording results. As is shown in the experiment results, CPL has strong robustness, and comparing with normal schemes such as least square method (LS), CPL's positioning accuracy can improve 50% when the ranging error is 35%.