In the field of remote sensing self-supervised learning, the self-supervised learning paradigm based on masked image modeling (MIM) effectively promotes the learning of structural and contextual semantic information in images. However, the random masking strategies employed in past studies have not effectively utilized the distribution of semantic information within images. Furthermore, overly simple reconstruction tasks cannot effectively improve self-supervised learning performance. To address these challenges, this paper introduces a gradient-guided activation-aware autoencoder (GA-MAE). Specifically, this approach computes and captures the spatial spectral activation map (SSAM) with semantic distribution information during the process of backpropagation of spatial-spectral reconstruction loss in the pre-training phase, thereby enabling masking operations on similar visual patches with higher reconstruction loss. Additionally, we introduce an Activation-Aware Self-Attention mechanism (ASA) that adjusts the self-attention dependencies by utilizing the weight information of visual patches provided by SSAM. Extensive experiments conducted on three public datasets (Houston2013, Augsburg, and Berlin) demonstrate the effectiveness of the proposed approach. With only ten labeled training samples per class, the proposed method achieves anoverall accuracy (OA) of 91.24%, 84.69%, and 73.43% on the Houston2013, Augsburg, and Berlin datasets, respectively, indicating improvements of 1.09%, 2.05%, and 2.40% over state-of-the-art methods.

Time series analysis finds wide applications in fields such as weather forecasting, anomaly detection, and behavior recognition. Previous methods attempted to model temporal variations directly using 1D time series. However, this has been quite challenging due to the discrete nature of data points in time series and the complexity of periodic variation. In terms of periodicity, taking weather and traffic data as an example, there are multi-periodic variations such as yearly, monthly, weekly, and daily, etc. In order to break through the limitations of the previous methods, we decouple the implied complex periodic variations into inclusion and overlap relationships among different level periodic components based on the observation of the multi-periodicity therein and its inclusion relationships. This explicitly represents the naturally occurring pyramid-like properties in time series, where the top level is the original time series and lower levels consist of periodic components with gradually shorter periods, which we call the periodic pyramid. To further extract complex temporal variations, we introduce self-attention mechanism into the periodic pyramid, capturing complex periodic relationships by computing attention between periodic components based on their inclusion, overlap, and adjacency relationships. Our proposed Peri-midFormer demonstrates outstanding performance in five mainstream time series analysis tasks, including short- and long-term forecasting, imputation, classification, and anomaly detection.