Smart grids, as a critical application of the Internet of Things (IoT), integrate diverse power generation sources while serving a vast number of energy consumers. Therefore, multivariate time series (MTS) forecasting methods are essential for power forecasting. The existing deep MTS forecasting models have already achieved remarkable performance. They deploy different architectures to analyze MTS features from the temporal, frequency and variable dimensions. However, tight combinations of three types of MTS features have rarely been explored, resulting in imperfect predictions, especially for power systems, wherein the variates are correlated mostly because of the close spatial locations of power plants and clients. To address this problem, a novel MTS forecasting model, SimTFV, that is capable of simultaneously modeling temporal-frequency-variable features is proposed in this work. SimTFV mixes temporal and frequency features via an enhanced instance normalization mechanism based on Heisenberg uncertainty principle. Moreover, SimTFV possesses multiple modified attention modules for simultaneously and efficiently extracting MTS features from the temporal and variable dimensions. Extensive experiments on three energy generation benchmarks and one energy consumption benchmark demonstrate the state-of-the-art performance of SimTFV. The code is released on https://github.com/OrigamiSL/SimTFV.

Audio-visual segmentation (AVS) is a challenging multimodal task that needs to fuse the spatial-temporal audio-visual features to achieve pixel-wise segmentation of sounding objects. This work presents AV2TS, which is a novel spatial-temporal framework for AVS. In contrast to the previous AVS approaches where temporal features are secondarily concerned, we cast AVS as a multivariate time series modeling task in which each frame sequence and its corresponding audio sequence form two time series to highlight the significance of temporal features in AVS. In AV2TS, the perception of each modal sequence is expressed as an intra-series feature extraction process, and the cross-modal fusion task is described as inter-series interactions. Specifically, the intra-series features of the frame and audio sequences are extracted via temporal attention. Moreover, an inter-series cross-modal fusion module is implemented by sharing the shared temporal attention maps of the frame and audio sequences. Additionally, AV2TS contains a one-stream fusion module that unifies the learning and relation modeling processes applied to spatially compact visual features and nontrivial acoustic features. Extensive experiments conducted on three AVSBench datasets demonstrate the state-of-the-art performance of AV2TS in both segmentation and semantic segmentation scenarios. Code is released onhttps://github.com/OrigamiSL/AV2TS.

The past decade has witnessed the success of deep learning-based multivariate time series forecasting in Internet of Things (IoT) systems. However, dynamic variable correlation remains a long-standing problem. The majority of existing multivariate forecasting methods either constantly forbid the interactions of all variables or, conversely, keep extracting the correlations of all variables, which is suboptimal for real-world time series with time-varying variable correlations. In contrast, we introduce a novel variable-dynamic forecasting transformer named VDformer. By leveraging empirical mode decomposition (EMD), VDformer can sparsely identify the dominant periodic ingredients of each variable in an arbitrary multivariate sequence via Fourier spectral analysis of its intrinsic mode functions (IMFs) obtained by the EMD. Thus, a mask matrix, where only the variables with identical dominant periodic ingredients are allowed for interactions, can be generated and used in the cross-variable attention modules of VDformer to dynamically gauge and extract the variable correlations. Additionally, better decoder initialization can be obtained by reconstructing the input sequence with these dominant periodic ingredients and extending the reconstructed results to the prediction duration. Extensive experiments on 11 benchmarks, which cover five IoT-related domains, demonstrate the state-of-the-art forecasting performance of VDformer (10.02% MSE reduction relative to the current best method). Code and Appendix are released on https://github.com/OrigamiSL/VDformer.

Energy forecasting plays a vital role in smart grid technology frameworks for monitoring power systems, including energy generation and consumption systems. As a downstream task of time series forecasting, energy forecasting has been thoroughly studied on the basis of deep learning in recent years. However, the sparsity of multivariate energy data, as well as the sparsity involved in cases with multiple solutions, has received minimal attention. To fill this gap, this work analyzes the intra-series and inter-series sparsity of long-term multivariate energy data in a hierarchical manner. Specifically, hierarchical global time stamps are leveraged to represent intra-series sparsity. Moreover, wavelet theory is applied to identify inter-series sparsity according to the correlations of series at different frequency scales. Building upon the above analysis of hierarchical sparsity, this work presents a novel energy forecasting model, the hierarchically sparse transformer, which uses a novel pyramid architecture to hierarchically extract sparse intra-series and inter-series features for effective and efficient energy forecasting. Extensive experiments on four energy-related benchmarks demonstrate the state-of-the-art performance of the proposed model. The source code is released on https://github.com/OrigamiSL/HST.

The advent of deep learning and neural network has revolutionized the study of time series forecasting. Diverse forecasting networks seem to achieve more promising performances than traditional forecasting models especially when handling complicated and non-linear conditions. However, most of forecasting networks are built upon Seq2Seq model, which means that they pursue the unique forecasting result when given certain input sequence. However, due to the natural noises, tiny errors and distribution shifts in real-world time series, Seq2Seq models are vulnerable to over-fitting problem. Based on these observations, we propose Area2Area forecasting formula containing C ausal S equence-wise C ontrastive L earning ( CSCL ) and A rea L oss ( AL ) mechanisms to loosen the forecasting constraint and alleviate over-fitting problem. CSCL utilizes contrastive learning technique to transform the input sequence into an input Area while AL modifies the loss function to transform the prediction sequence into a prediction Area . We additionally propose a novel encoder network Temporal E fficient L ayer A ggregation N etwork ( Temporal ELAN ). Extensive experiments on six datasets and nine baselines demonstrate that Area2Area forecasting is literally capable of alleviating the over-fitting problem of existing Seq2Seq forecasting networks. The source code is released on https://github.com/OrigamiSL/A2A .

Recently, there has been a surge of meta-learning-based approaches for the few-shot hyperspectral image classification (FSHSIC) task. Meta-learning leverages prior knowledge to teach a base-learner how to adapt quickly to a new few-shot task, which hinges on the consistency of the prior and new tasks to guarantee validity. However, hyperspectral image classification (HSIC) is an environment-dependent task, which means that the hyperspectral features of two objects in the same category can be essentially distinctive in different environments. Consequently, whether meta-learning is a feasible solution for FSHSIC is an imperative problem to investigate, notwithstanding the promising performance shown in previous meta-learning-based approaches. To this end, this work proposes a simple multilayer perceptron (MLP)-based model named SimHSIC for FSHSIC. SimHSIC utilizes only a few labeled samples from the target HSI to train the model rapidly. Surprisingly, SimHSIC outperforms existing meta-learning-based approaches, which are built upon complex three-dimensional convolutions or transformers and need heavy training processes, in prevailing public benchmarks. On the basis of extensive experiments, we conclude that the relatively better classification performances of meta-learning-based FSHSIC solutions are due mainly to the patching of each HSI pixel with the large surroundings instead of meta-learning. The code is released on https://github.com/OrigamiSL/SimHSIC.

Visual tracking plays a vital role in modern intelligent transportation systems (ITSs) to sense traffic environments and trace targets, wherein uncrewed aerial vehicles (UAVs) are commonly used for data collection. Specifically, UAV tracking is particularly needed in road traffic surveillance (RTS) systems due to the mobility of UAV and the complexity of road traffic environments. Currently, many existing trackers leverage spatial-temporal features to increase their tracking capabilities. However, the utilization of spatial-temporal features normally involves considerable extra network modules and time-consuming recursive deduction processes, making these trackers impractical in ITSs. To address the issue of low efficiency, we propose FWTrack, a novel tracker that constructs hierarchical spatial-temporal features via three-dimensional wavelets and thus achieves efficient spatial-temporal visual tracking. FWTrack employs the spatial wavelets to reinforce its feature extraction ability in an approximately parameter-free manner and applies the temporal wavelets to adaptively separate static backgrounds from the attention modules of FWTrack, thereby speeding up the model. Moreover, the window-wise attention technique, is adopted and enhanced in FWTrack to further increase its efficiency. Thus, FWTrack can be readily deployed in UAVs for RTS purposes. Extensive experiments conducted on seven benchmarks demonstrate that FWTrack achieves state-of-the-art tracking accuracy and efficiency. Code is released on https://github.com/OrigamiSL/FWTrack

Traditional statistical time series forecasting models rely on model identification methods to identify the worthiest model variants to investigate; therefore, the model parameters change with the statistical features of rolling windows to reach optimality. Currently, although deep-learning-based methods achieve promising multivariate forecasting performance, their representations of variable correlations are consistent regardless of the observed local time series properties and dynamic cross-variable relations, rendering them prone to overfitting. To bridge this gap, we propose FPPformer-MD, a novel inconsistent time series forecasting transformer. FPPformer-MD leverages multiresolution analysis to transform each univariate series into multiple frequency scales and evaluate the local variable correlations via their variances. Thus, FPPformer-MD receives richer input features, and its inner inconsistent cross-variable attention mechanism enables the adaptive extraction of cross-variable features. To further alleviate the overfitting problem, we apply dynamic mode decomposition to perform cross-variable data augmentation, which reconstructs the sequence outliers with other correlated sequences during the model training process. Extensive experiments conducted on thirteen real-world benchmarks demonstrate the state-of-the-art performance of FPPformer-MD.

We present Exformer, a novel Siamese-like time-series forecasting Transformer with extended anomaly detection and reconstruction modules. Exformer extricates itself from addressing the non-stationarity, which is the principal bottleneck of time-series forecasting, solely from the perspective of normalization. Instead, Exformer foremost analyzes each input window with anomaly detection method before forecasting and attempts to reconstruct the anomalous parts during the forecasting period. Leveraging from this strategy, Exformer excels in mitigating the influences of pattern anomalies in input sequences, which are virtually insurmountable for existing solutions on tackling non-stationarity. To further alleviate the forecasting turbulence brought by non-stationarity, Exformer additionally employs Siamese architecture to pledge the identical feature distribution of input and forecasting windows in the latent space. Exformer is simple and direct as the anomaly detection part is devised to be non-parametric and the reconstruction part is designed to merely make modifications rather than supplementing additional components. Benefiting from exploiting these innovations, Exformer achieves state-of-the-art forecasting accuracy and robustness on eight benchmarks with considerable efficiency. The source code will be released soon.

With the development of Internet of Things (IoT) systems, precise long-term forecasting method is requisite for decision makers to evaluate current statuses and formulate future policies. Currently, Transformer and MLP are two paradigms for deep time-series forecasting and the former one is more prevailing in virtue of its exquisite attention mechanism and encoder-decoder architecture. However, data scientists seem to be more willing to dive into the research of encoder, leaving decoder unconcerned. Some researchers even adopt linear projections in lieu of the decoder to reduce the complexity. We argue that both extracting the features of input sequence and seeking the relations of input and prediction sequence, which are respective functions of encoder and decoder, are of paramount significance. Motivated from the success of FPN in CV field, we propose FPPformer to utilize bottom-up and top-down architectures respectively in encoder and decoder to build the full and rational hierarchy. The cutting-edge patch-wise attention is exploited and further developed with the combination, whose format is also different in encoder and decoder, of revamped element-wise attention in this work. Extensive experiments with six state-of-the-art baselines on twelve benchmarks verify the promising performances of FPPformer and the importance of elaborately devising decoder in time-series forecasting Transformer. The source code is released in https://github.com/OrigamiSL/FPPformer.

Forecasting-based method is one of prevalent unsupervised time series anomaly detection approaches. Currently, large portions of existing forecasting-based methods are devoted to discussing the feature extraction of input sequences and targeting at accurate predictions. In essence, their frameworks and core ideas are identical to the pure forecasting models. However, the distinctiveness of anomalies is affected by not only forecasting accuracy, but also many other factors. This paper summarizes three other dominant factors: (1) Scale disparity ; (2) Discrete variate ; (3) Input anomaly . They are common and non-negligible in real-world anomaly detection. Moreover, we propose AFMF: a time series A nomaly detection F ramework with M odified F orecasting to solve them respectively by its three key components, i.e., Local Instance Normalization, Lopsided Forecasting and Progressive Adjacent Masking . The first two are refined descendants of existing mechanisms while the third component is completely novel. Extensive experiments on ten benchmarks verify that AFMF can be combined with any forecasting or forecasting-based anomaly detection method to achieve SOTA anomaly detection performances. The source code is available at https://github.com/OrigamiSL/AFMF.

This paper presents FDNet: a Focal Decomposed Network for efficient, robust and practical time series forecasting. We break away from conventional deep time series forecasting formulas which obtain prediction results from universal feature maps of input sequences. In contrary, FDNet neglects universal correlations of input elements and only extracts fine-grained local features from input sequence. We show that: (1) Deep time series forecasting with only fine-grained local feature maps of input sequence is feasible upon theoretical basis. (2) By abandoning global coarse-grained feature maps, FDNet overcomes distribution shift problem caused by changing dynamics of time series which is common in real-world applications. (3) FDNet is not dependent on any inductive bias of time series except basic auto-regression, making it general and practical. Moreover, we propose focal input sequence decomposition method which decomposes input sequence in a focal manner for efficient and robust forecasting when facing Long Sequence Time series Input (LSTI) problem. FDNet achieves competitive forecasting performances on six real-world benchmarks and reduces prediction MSE by 38.4% on average compared with other thirteen SOTA baselines. The source code is available at https://github.com/OrigamiSL/FDNet.

This paper shows that time series forecasting Transformer (TSFT) suffers from severe over-fitting problem caused by improper initialization method of unknown decoder inputs, especially when handling non-stationary time series. Based on this observation, we propose GBT, a novel two-stage Transformer framework with Good Beginning. It decouples the prediction process of TSFT into two stages, including Auto-Regression stage and Self-Regression stage to tackle the problem of different statistical properties between input and prediction sequences. Prediction results of Auto-Regression stage serve as a 'Good Beginning', i.e., a better initialization for inputs of Self-Regression stage. We also propose the Error Score Modification module to further enhance the forecasting capability of the Self-Regression stage in GBT. Extensive experiments on seven benchmark datasets demonstrate that GBT outperforms SOTA TSFTs (FEDformer, Pyraformer, ETSformer, etc.) and many other forecasting models (SCINet, N-HiTS, etc.) with only canonical attention and convolution while owning less time and space complexity. It is also general enough to couple with these models to strengthen their forecasting capability. The source code is available at: https://github.com/OrigamiSL/GBT.

Time series forecasting is essential for a wide range of real-world applications. Recent studies have shown the superiority of Transformer in dealing with such problems, especially long sequence time series input (LSTI) and long sequence time series forecasting (LSTF) problems. To improve the efficiency and enhance the locality of Transformer, these studies combine Transformer with CNN in varying degrees. However, their combinations are loosely-coupled and do not make full use of CNN. To address this issue, we propose the concept of tightly-coupled convolutional Transformer (TCCT) and three TCCT architectures which apply transformed CNN architectures into Transformer: (1) CSPAttention: through fusing CSPNet with self-attention mechanism, the computation cost of self-attention mechanism is reduced by 30% and the memory usage is reduced by 50% while achieving equivalent or beyond prediction accuracy. (2) Dilated causal convolution: this method is to modify the distilling operation proposed by Informer through replacing canonical convolutional layers with dilated causal convolutional layers to gain exponentially receptive field growth. (3) Passthrough mechanism: the application of passthrough mechanism to stack of self-attention blocks helps Transformer-like models get more fine-grained information with negligible extra computation costs. Our experiments on real-world datasets show that our TCCT architectures could greatly improve the performance of existing state-of-the-art Transformer models on time series forecasting with much lower computation and memory costs, including canonical Transformer, LogTrans and Informer.