Best-of-N sampling is a powerful method for improving Large Language Model (LLM) performance, but it is often limited by its dependence on massive, text-based reward models. These models are not only computationally expensive but also data-hungry, requiring extensive labeled datasets for training. This creates a significant data challenge, as they overlook a rich, readily available data source: the LLM's own internal hidden states. To address this data and efficiency gap, we introduce SWIFT (Simple Weighted Intrinsic Feedback Technique), a novel and lightweight method that learns a reward function directly from the rich information embedded in LLM hidden states. Operating at the token embedding level, SWIFT employs simple linear layers to effectively distinguish between preferred and dispreferred generations, eliminating the need for computationally intensive text-based modeling. Extensive experiments on standard benchmarks show that SWIFT outperforms existing baselines (12.7% higher accuracy than EurusRM-7B on MATH dataset) while using less than 0.005% of their parameters. Its robust scalability, compatibility with certain closed-source models via logit access, and ability to combine with traditional reward models for additional performance highlight SWIFT's practical value and contribution to more efficient data-driven LLM post-training. Our code is available at https://github.com/aster2024/SWIFT.

Dynamic graph anomaly detection (DGAD) is essential for iden- tifying anomalies in evolving graphs across domains such as fi- nance and social networks. Recently, generalist graph anomaly detection (GAD) models have shown promising results. They are pretrained on multiple source datasets and generalize across do- mains. While effective on static graphs, they struggle to capture evolving anomalies in dynamic graphs. Moreover, the continuous emergence of new domains and the lack of labeled data further challenge generalist DGAD. Effective cross-domain DGAD requires both domain-specific and domain-agnostic anomalous patterns. Importantly, these patterns evolve temporally within and across domains. Building on these insights, we propose a DGAD model with Dynamic Prototypes (DP) to capture evolving domain-specific and domain-agnostic patterns. Firstly, DP-DGAD extracts dynamic prototypes, i.e., evolving representations of normal and anomalous patterns, from temporal ego-graphs and stores them in a memory buffer. The buffer is selectively updated to retain general, domain- agnostic patterns while incorporating new domain-specific ones. Then, an anomaly scorer compares incoming data with dynamic prototypes to flag both general and domain-specific anomalies. Fi- nally, DP-DGAD employs confidence detection guided memory buffer updating for effective adaptation to target domain. Extensive experiments demonstrate state-of-the-art performance across ten real-world datasets from different domains.

Sea surface temperature (SST) prediction is a critical task in ocean science, supporting various applications, such as weather forecasting, fisheries management, and storm tracking. While existing data-driven methods have demonstrated significant success, they often neglect to leverage the rich domain knowledge accumulated over the past decades, limiting further advancements in prediction accuracy. The recent emergence of large language models (LLMs) has highlighted the potential of integrating domain knowledge for downstream tasks. However, the application of LLMs to SST prediction remains under explored, primarily due to the challenge of integrating ocean domain knowledge and numerical data. To address this issue, we propose Ocean Knowledge Graph-enhanced LLM (OKG-LLM), a novel framework for global SST prediction. To the best of our knowledge, this work presents the first systematic effort to construct an Ocean Knowledge Graph (OKG) specifically designed to represent diverse ocean knowledge for SST prediction. We then develop a graph embedding network to learn the comprehensive semantic and structural knowledge within the OKG, capturing both the unique characteristics of individual sea regions and the complex correlations between them. Finally, we align and fuse the learned knowledge with fine-grained numerical SST data and leverage a pre-trained LLM to model SST patterns for accurate prediction. Extensive experiments on the real-world dataset demonstrate that OKG-LLM consistently outperforms state-of-the-art methods, showcasing its effectiveness, robustness, and potential to advance SST prediction. The codes are available in the online repository.

Temporal causal discovery aims to uncover causal relations in time series data. Current deep learning-based methods usually analyze the parameters of some components of the trained models, which is an incomplete mapping process from the model parameters to the causality and fails to investigate the other components. To address this, this paper presents an interpretable transformer-based causal discovery model termed CausalFormer, which consists of: 1) the causality-aware transformer which learns the causal representation with the multi-kernel causal convolution under the temporal priority constraint, and 2) the decomposition-based causality detector which identifies causality by interpreting the global structure of the trained transformer with the regression relevance propagation.

Long-term prediction of Sea Surface Temperature (SST) is a pervasive issue in ocean science, particularly for understanding climate changes and improving marine disaster risk assessment. However, existing approaches typically focus either on short-term or long-term coarse-grained prediction, due to their limited ability to overcome noise interference and model complex spatio-temporal dependencies in long SST sequences. To overcome these limitations, we proposed a novel spatio-temporal decomposition Mamba model, termed STDMamba, for long-term fine-grained SST prediction. First, we introduce a Gaussian-weighted series decomposition module with smoothing mechanisms to decompose SST sequences into trend and fluctuation components, thereby mitigating noise interference. Then, we design a dual spatio-temporal representation learning module which utilizes Mamba2 to effectively capture the long-term spatio-temporal dependencies in the fluctuation component, and employs a Temporal Convolutional Network (TCN) to learn the spatio-temporal feature representation of the trend component. Finally, the dual representations are fused and passed through a prediction layer to generate the long-term fine-grained SST prediction results. Experiments on real-world datasets demonstrate that STDMamba significantly outperforms state-of-the-art prediction models. The code of STDMamba is available at https://github.com/ADMIS-TONGJI/STDMamba.

Remote sensing data are of considerable significance for monitoring global climate, detecting harmful algae bloom, and so on. However, due to sensor failures, cloud cover, and thick aerosols, the collected remote sensing data for various ocean factors, such as chlorophyll-a (Chl-a) concentration and sea surface temperature (SST), often have a high missing rate, which seriously hinders their applications. Existing data imputation models mostly ignore highly sparse spatial locations or perform badly when the data missing rate is high due to the lack of available information. Large language models (LLMs) possess powerful representation learning capabilities and can effectively capture sequential correlations even with extremely limited information, thereby presenting the promising potential for highly sparse remote sensing data imputation. Therefore, we proposed a novel LLM-based spatiotemporal model for highly sparse remote sensing data imputation, i.e., LLM4HRS. First, LLM4HRS develops an LLM-based bidirectional temporal representation learning module to learn forward and backward temporal dependencies in data sequences and fuses them together to obtain comprehensive temporal representations. Next, LLM4HRS constructs a LLM-based spatial representation learning module to learn spatial correlations with the learned temporal representation. Finally, a spatiotemporal representation fusion and data imputation module is developed to achieve data imputation. Experiments on the SST and Chl-a remote sensing datasets demonstrate that LLM4HRS significantly outperforms existing data imputation models, with its advantages becoming more pronounced as the masking rate increases. Furthermore, when extended to the remote sensing PAR data in a large region, LLM4HRS still achieves the best performance, further validating its broad applicability for remote sensing data imputation. The code of LLM4HRS is publicly available athttps://github.com/ssyuwang/LLM4HRS-master.

Accurate prediction of Sea Surface Temperature (SST) is of high importance in marine science, benefiting applications ranging from ecosystem protection to extreme weather forecasting and climate analysis. Wide-area SST usually shows diverse SST patterns in different sea areas due to the changes of temperature zones and the dynamics of ocean currents. However, existing studies on SST prediction often focus on small-area predictions and lack the consideration of diverse SST patterns. Furthermore, SST shows an annual periodicity, but the periodicity is not strictly adherent to an annual cycle. Existing SST prediction methods struggle to adapt to this non-strict periodicity. To address these two issues, we proposed the Cross-Region Graph Convolutional Network with Periodicity Shift Adaptation (RGCN-PSA) model which is equipped with the Cross-Region Graph Convolutional Network module and the Periodicity Shift Adaption module. The Cross-Region Graph Convolutional Network module enhances wide-area SST prediction by learning and incorporating diverse SST patterns. Meanwhile, the periodicity Shift Adaptation module accounts for the annual periodicity and enable the model to adapt to the possible temporal shift automatically. We conduct experiments on two real-world SST datasets, and the results demonstrate that our RGCN-PSA model obviously outperforms baseline models in terms of prediction accuracy. The code of RGCN-PSA model is available at https://github.com/ADMIS-TONGJI/RGCN-PSA/ .

With the rapid amassing of spatial-temporal (ST) ocean data, many spatial-temporal data mining (STDM) studies have been conducted to address various oceanic issues, including climate forecasting and disaster warning. Compared with typical ST data (e.g., traffic data), ST ocean data presents some unique characteristics, e.g., diverse regionality and high sparsity. These characteristics make it difficult to design and train STDM models on ST ocean data. To the best of our knowledge, a comprehensive survey of existing studies remains missing in the literature, which hinders not only computer scientists from identifying the research issues in ocean data mining but also ocean scientists to apply advanced STDM techniques. In this article, we provide a comprehensive survey of existing STDM studies for ocean science. Concretely, we first review the widely used ST ocean datasets and highlight their unique characteristics. Then, typical ST ocean data quality enhancement techniques are discussed. Next, we classify existing STDM studies for ocean science into four types of tasks, i.e., prediction, event detection, pattern mining, and anomaly detection, and elaborate the techniques for these tasks. Finally, promising research opportunities are discussed. This survey can help scientists from both computer science and ocean science better understand the fundamental concepts, key techniques, and open challenges of STDM for ocean science.

Accurate spatial-temporal (ST) traffic prediction plays an essential role in intelligent transportation systems. Existing advanced traffic prediction methods typically utilize spatial-temporal graph neural networks (STGNNs) to capture the ST correlations and achieve excellent prediction performance. However, our experimental investigation reveals that existing static and dynamic graph-based STGNNs still incur excessive noise and redundancy, and fail to discover robust and reliable ST correlations in traffic networks. Moreover, most methods cannot explain the underlying reasons behind the ST correlations. To solve these problems, we propose a novel S patial- T emporal G raph M odeling framework via A daptive contrastive learning (ST-GMA). Firstly, we design a robust augmentation learning module to generate high-level and robust data augmentations via a self-supervised task for modeling reliable correlations. Then, we develop an adaptive contrastive learning module to update correlation graphs by effectively selecting positive and negative augmentations, reducing redundant calculations, and providing insights into the correlation changes. Finally, ST-GMA integrates the generated correlation graphs with ST convolution blocks to conduct traffic prediction tasks. Experimental results on five real-world datasets demonstrate that ST-GMA not only achieves significant prediction performance compared with state-of-the-art methods but also exhibits a new perspective on the interpretability of correlation changes.

Temporal causal discovery is a crucial task aimed at uncovering the causal relations within time series data. The latest temporal causal discovery methods usually train deep learning models on prediction tasks to uncover the causality between time series. They capture causal relations by analyzing the parameters of some components of the trained models, e.g., attention weights and convolution weights. However, this is an incomplete mapping process from the model parameters to the causality and fails to investigate the other components, e.g., fully connected layers and activation functions, that are also significant for causal discovery. To facilitate the utilization of the whole deep learning models in temporal causal discovery, we proposed an interpretable transformer-based causal discovery model termed CausalFormer, which consists of the causality-aware transformer and the decomposition-based causality detector. The causality-aware transformer learns the causal representation of time series data using a prediction task with the designed multi-kernel causal convolution which aggregates each input time series along the temporal dimension under the temporal priority constraint. Then, the decomposition-based causality detector interprets the global structure of the trained causality-aware transformer with the proposed regression relevance propagation to identify potential causal relations and finally construct the causal graph. Experiments on synthetic, simulated, and real datasets demonstrate the state-of-the-art performance of CausalFormer on discovering temporal causality.

Accurate prediction of ocean factors (e.g., temperature and salinity) is crucial for plenty of applications, including weather forecasting, storm tracking, and ecosystem protection. Meanwhile, it is well-known that the ocean is a unified system and various ocean factors usually influence each other. For example, the changes in temperature would affect the distribution of salinity in ocean. However, the existing studies for ocean factor prediction mainly focus on designing individual models for predicting specific factors and ignore the correlations between different factors, thus having potentials to be further improved. Therefore, we propose a unified framework UniOcean to predict multiple ocean factors simultaneously, and capture the correlations between them to improve the prediction accuracy. First, considering that ocean factors are usually collected with different temporal scales, we develop the fine-grained multiscale data fusion module to integrate multiple ocean factors with different temporal scales, and effectively learn their hierarchical patterns at different levels. Then, since the correlations between ocean factors may vary across different time periods, the multifactor correlation learning module is constructed to adaptively learn the dynamic correlations between different factors. Finally, we utilize the factor-specific towers to predict multiple ocean factors simultaneously. Experimental results on five real-world remote sensing datasets demonstrate that UniOcean significantly improves the prediction accuracy by 11%–53% in terms of MSD for different ocean factors.

Sea surface temperature (SST) is one critical parameter of global climate change, and accurate SST prediction is important to various applications, e.g., weather forecasting, fishing directions, and disaster warnings. The global ocean system is unified and complex, and the SST patterns in different oceanic regions are highly diverse and correlated. However, existing data-driven SST prediction methods mainly consider the local patterns within a certain oceanic region, e.g., El Nino region and the Black sea. It is challenging but necessary to model the global SST correlations rather than that in a specific region to enhance the prediction accuracy of SST. In this work, we proposed a new method called Hierarchical Graph Recurrent Network (HiGRN) to address the issue. First, to learn the dynamic and diverse local SST patterns of specific locations, we design an adaptive node embedding with self-learned parameters to learn various SST patterns. Then we develop a hierarchical cluster generator to aggregate the locations with similar patterns into regional clusters and utilize a graph convolution network to learn the spatial correlations among these clusters. Finally, we introduce a multi-level attention mechanism to fuse the local patterns and regional correlations, and the output is fed into a recurrent network to achieve SST predictions. Extensive experiments on two real-world datasets show that our method largely outperforms the state-of-the-art SST prediction methods. The source code is available at https://github.com/Neoyanghc/HiGRN .