Cloud removal (CR) remains a challenging task in remote sensing image processing. Although diffusion models (DM) exhibit strong generative capabilities, their direct applications to CR are suboptimal, as they generate cloudless images from random noise, ignoring inherent information in cloudy inputs. To overcome this drawback, we develop a new CR model EMRDM based on mean-reverting diffusion models (MRDMs) to establish a direct diffusion process between cloudy and cloudless images. Compared to current MRDMs, EMRDM offers a modular framework with updatable modules and an elucidated design space, based on a reformulated forward process and a new ordinary differential equation (ODE)-based backward process. Leveraging our framework, we redesign key MRDM modules to boost CR performance, including restructuring the denoiser via a preconditioning technique, reorganizing the training process, and improving the sampling process by introducing deterministic and stochastic samplers. To achieve multi-temporal CR, we further develop a denoising network for simultaneously denoising sequential images. Experiments on mono-temporal and multi-temporal datasets demonstrate the superior performance of EMRDM. Our code is available at https://github.com/Ly403/EMRDM.

Temporal causal discovery aims to uncover causal relations in time series data. Current deep learning-based methods usually analyze the parameters of some components of the trained models, which is an incomplete mapping process from the model parameters to the causality and fails to investigate the other components. To address this, this paper presents an interpretable transformer-based causal discovery model termed CausalFormer, which consists of: 1) the causality-aware transformer which learns the causal representation with the multi-kernel causal convolution under the temporal priority constraint, and 2) the decomposition-based causality detector which identifies causality by interpreting the global structure of the trained transformer with the regression relevance propagation.

Long-term prediction of Sea Surface Temperature (SST) is a pervasive issue in ocean science, particularly for understanding climate changes and improving marine disaster risk assessment. However, existing approaches typically focus either on short-term or long-term coarse-grained prediction, due to their limited ability to overcome noise interference and model complex spatio-temporal dependencies in long SST sequences. To overcome these limitations, we proposed a novel spatio-temporal decomposition Mamba model, termed STDMamba, for long-term fine-grained SST prediction. First, we introduce a Gaussian-weighted series decomposition module with smoothing mechanisms to decompose SST sequences into trend and fluctuation components, thereby mitigating noise interference. Then, we design a dual spatio-temporal representation learning module which utilizes Mamba2 to effectively capture the long-term spatio-temporal dependencies in the fluctuation component, and employs a Temporal Convolutional Network (TCN) to learn the spatio-temporal feature representation of the trend component. Finally, the dual representations are fused and passed through a prediction layer to generate the long-term fine-grained SST prediction results. Experiments on real-world datasets demonstrate that STDMamba significantly outperforms state-of-the-art prediction models. The code of STDMamba is available at https://github.com/ADMIS-TONGJI/STDMamba.

Few-shot object detection (FSOD) has been proposed to solve the problem of insufficient data for training, and it has drawn the attention of the remote sensing community in recent years. A mainstream type of FSOD method is to generate class prototypes based on the limited samples to help the construction of classification decision boundaries. However, these constructed prototypes may be far away from the true class centroids in the few-shot scenario. Recently, the vision-language model (VLM) has shown its powerful ability to align the visual features and text features, which leads to strong zero-shot performance on various downstream computer vision tasks when given only texts. Therefore, in this work, we propose to build class prototypes from text descriptions instead of limited visual instances by leveraging a classical pretrained VLM named CLIP. Concretely, we generate prototypes by feeding the CLIP text encoder with class names and enforcing each positive proposal feature to be close to the corresponding prototype. To accelerate the alignment process, we utilize the CLIP visual encoder as another teacher to achieve visual knowledge distillation. Moreover, we adopt prompt tuning to adapt CLIP to the remote sensing scenario. Extensive experiments on two public FSOD datasets, i.e., DIOR and NWPU VHR-10.v2, demonstrate the effectiveness of our method, which yields competitive results with that of existing approaches.

Remote sensing data are of considerable significance for monitoring global climate, detecting harmful algae bloom, and so on. However, due to sensor failures, cloud cover, and thick aerosols, the collected remote sensing data for various ocean factors, such as chlorophyll-a (Chl-a) concentration and sea surface temperature (SST), often have a high missing rate, which seriously hinders their applications. Existing data imputation models mostly ignore highly sparse spatial locations or perform badly when the data missing rate is high due to the lack of available information. Large language models (LLMs) possess powerful representation learning capabilities and can effectively capture sequential correlations even with extremely limited information, thereby presenting the promising potential for highly sparse remote sensing data imputation. Therefore, we proposed a novel LLM-based spatiotemporal model for highly sparse remote sensing data imputation, i.e., LLM4HRS. First, LLM4HRS develops an LLM-based bidirectional temporal representation learning module to learn forward and backward temporal dependencies in data sequences and fuses them together to obtain comprehensive temporal representations. Next, LLM4HRS constructs a LLM-based spatial representation learning module to learn spatial correlations with the learned temporal representation. Finally, a spatiotemporal representation fusion and data imputation module is developed to achieve data imputation. Experiments on the SST and Chl-a remote sensing datasets demonstrate that LLM4HRS significantly outperforms existing data imputation models, with its advantages becoming more pronounced as the masking rate increases. Furthermore, when extended to the remote sensing PAR data in a large region, LLM4HRS still achieves the best performance, further validating its broad applicability for remote sensing data imputation. The code of LLM4HRS is publicly available athttps://github.com/ssyuwang/LLM4HRS-master.

Accurate prediction of Sea Surface Temperature (SST) is of high importance in marine science, benefiting applications ranging from ecosystem protection to extreme weather forecasting and climate analysis. Wide-area SST usually shows diverse SST patterns in different sea areas due to the changes of temperature zones and the dynamics of ocean currents. However, existing studies on SST prediction often focus on small-area predictions and lack the consideration of diverse SST patterns. Furthermore, SST shows an annual periodicity, but the periodicity is not strictly adherent to an annual cycle. Existing SST prediction methods struggle to adapt to this non-strict periodicity. To address these two issues, we proposed the Cross-Region Graph Convolutional Network with Periodicity Shift Adaptation (RGCN-PSA) model which is equipped with the Cross-Region Graph Convolutional Network module and the Periodicity Shift Adaption module. The Cross-Region Graph Convolutional Network module enhances wide-area SST prediction by learning and incorporating diverse SST patterns. Meanwhile, the periodicity Shift Adaptation module accounts for the annual periodicity and enable the model to adapt to the possible temporal shift automatically. We conduct experiments on two real-world SST datasets, and the results demonstrate that our RGCN-PSA model obviously outperforms baseline models in terms of prediction accuracy. The code of RGCN-PSA model is available at https://github.com/ADMIS-TONGJI/RGCN-PSA/ .

With the growing diversity of data formats on social media, such as text, images, and videos, there is a growing need to analyze sentiment from multiple modalities. Multimodal sentiment detection, which aims to identify users’ sentiment by jointly modeling information from different modalities, has thus attracted increasing attention. However, most existing multimodal sentiment detection methods fuse multimodal information directly after the unimodal encoding and overlook the modality consistency of multimodal vector spaces before the fusion, which may damage the accuracy of multimodal sentiment detection. To address this issue, we propose a contrastive learning-based multimodal sentiment detection model termed EPMC which can map the representations of different modalities into a unified semantic space before fusion. EPMC operates in two stages, i.e., pre-training stage and fine-tuning stage. At the pre-training stage, we designed a cross-modal transformation module to map different modalities into a unified feature space. Meanwhile, to further capture the relationship between the cross-modal transformation vectors and the unimodal encoding vectors, we propose a multimodal consistency contrastive learning task that helps the model discern and amplify the cross-modal similarity between different modalities, thereby learning more discriminative features for sentiment detection. At the fine-tuning stage, EPMC is iteratively refined using the learned multimodal representation and guided by the cross-entropy loss. Extensive experiments conducted on three public multimodal datasets validate the effectiveness of EPMC model. The official implementation of EPMC is released at https://github.com/ADMIS-TONGJI/EPMC .

Given a specific region, crime prediction aims to predict the occurrence of various crime events within a certain period of time in future, which is of high significance for guaranteeing urban safety. In practice, crime events are usually affected by a variety of factors from different views, e.g., the attributes of the region, the correlations between different regions, and the correlations between different categories of crime events. Moreover, these correlations are dynamically changing over time, which makes it difficult to learn the regularity and patterns in crime data for achieving accurate prediction. To address this issue, we proposed a new M ulti- V iew S patial- T emporal (MVST) model for fine-grained crime prediction. MVST model first builds a static region graph to capture the similarity between regions in terms of region attributes such as census records and economy statistics, and creates a time-dependent graph to capture the dynamic correlations between regions based on human mobility data. Meanwhile, both static and dynamic graphs are created to capture the correlations between different categories of crime events. After that, those graphs created from different views are fused together with a multi-view graph fusion module to achieve crime prediction with fine-grained time granularities, e.g., 4 hours and 12 hours. According to the experiments on two real crime datasets, our MVST model obviously outperforms existing crime prediction methods. The code of MVST model is available at https://github.com/weichang811/MVST .

Inductive relation prediction, an important task for knowledge graph completion, is to predict the relations between entities that are unseen at the training stage. The latest methods use Pre-Trained Language Models (PLMs) to encode the paths between the head entity and tail entity and achieve state-of-the-art prediction performance. However, these methods cannot handle no-path scenarios well and lack the capability to learn comprehensive relation representations for distinguishing different relations. To tackle this issue, we propose a novel R elation- a ware k nowledg e r easoning model entitled Raker, which introduces an adaptive reasoning information extraction method to identify relation-aware reasoning neighbors of entities in the target triple to handle no-path scenarios and enables the PLM to better distinguish different relations via the relation-specific soft prompting. Raker is evaluated on three public datasets and achieves SOTA performance in inductive relation prediction when compared with the baseline methods. Notably, the absolute improvement of Raker is even more than 5% on the FB15k-237 dataset in the inductive setting. Moreover, Raker also demonstrates the superiority in transductive, few-shot, and unseen relation settings. The code of Raker is available at https://github.com/ADMIS-TONGJI/Raker .

Accurate spatial-temporal (ST) traffic prediction plays an essential role in intelligent transportation systems. Existing advanced traffic prediction methods typically utilize spatial-temporal graph neural networks (STGNNs) to capture the ST correlations and achieve excellent prediction performance. However, our experimental investigation reveals that existing static and dynamic graph-based STGNNs still incur excessive noise and redundancy, and fail to discover robust and reliable ST correlations in traffic networks. Moreover, most methods cannot explain the underlying reasons behind the ST correlations. To solve these problems, we propose a novel S patial- T emporal G raph M odeling framework via A daptive contrastive learning (ST-GMA). Firstly, we design a robust augmentation learning module to generate high-level and robust data augmentations via a self-supervised task for modeling reliable correlations. Then, we develop an adaptive contrastive learning module to update correlation graphs by effectively selecting positive and negative augmentations, reducing redundant calculations, and providing insights into the correlation changes. Finally, ST-GMA integrates the generated correlation graphs with ST convolution blocks to conduct traffic prediction tasks. Experimental results on five real-world datasets demonstrate that ST-GMA not only achieves significant prediction performance compared with state-of-the-art methods but also exhibits a new perspective on the interpretability of correlation changes.

Temporal causal discovery is a crucial task aimed at uncovering the causal relations within time series data. The latest temporal causal discovery methods usually train deep learning models on prediction tasks to uncover the causality between time series. They capture causal relations by analyzing the parameters of some components of the trained models, e.g., attention weights and convolution weights. However, this is an incomplete mapping process from the model parameters to the causality and fails to investigate the other components, e.g., fully connected layers and activation functions, that are also significant for causal discovery. To facilitate the utilization of the whole deep learning models in temporal causal discovery, we proposed an interpretable transformer-based causal discovery model termed CausalFormer, which consists of the causality-aware transformer and the decomposition-based causality detector. The causality-aware transformer learns the causal representation of time series data using a prediction task with the designed multi-kernel causal convolution which aggregates each input time series along the temporal dimension under the temporal priority constraint. Then, the decomposition-based causality detector interprets the global structure of the trained causality-aware transformer with the proposed regression relevance propagation to identify potential causal relations and finally construct the causal graph. Experiments on synthetic, simulated, and real datasets demonstrate the state-of-the-art performance of CausalFormer on discovering temporal causality.

Existing multimodal sentiment analysis models focus more on fusing highly correlated image-text pairs, and thus achieves unsatisfactory performance on multimodal social media data which usually manifests weak correlations between different modalities. To address this issue, we first build a large multimodal social media sentiment analysis dataset RU-Senti which contains more than 100,000 image-text pairs with sentiment labels. Then, we proposed a topic-oriented model (TOM) which assumes that text is usually related to a certain portion of the image contents and significant variances exist in sentiment distribution across diverse topics. TOM learns the topic information from textual content and designs a topic-oriented feature alignment module to extract textual semantics correlated information from images, thus achieving the alignment between two modalities. Then, TOM utilizes a transformer encoder initialized with the parameters from a pre-trained vision-language model to fuse the multimodal features for sentiment prediction. According to the experiments over the public MVSA-Multiple dataset and our RU-Senti dataset, RU-Senti is of high suitability for studying weakly correlated multimodal sentiment analysis, and the proposed TOM model also largely outperforms the SOTA mulitimodal sentiment analysis methods and pre-trained vision-language models.

Monitoring Chlorophyll-a (Chl-a) concentration in ocean is of considerable significance for early warning of algae disasters, marine ecological environment protection, etc. However, due to various uncontrollable factors such as cloud cover and thick aerosols, the missing rate of observed Chl-a data is quite high, which seriously hinders its applications. The existing data imputation methods are mostly applicable to Chl-a data with a low missing rate, and perform much worse when the missing rate reaches 0.7 or above. To address this issue, we proposed a self-supervised spatiotemporal imputation (S2-STI) model for highly sparse Chl-a data imputation by fusing multisource satellite data. First, to obtain comprehensive information about the missed Chl-a data, we designed a multisource sparse data fusion (MSDF) module to fuse multisource satellite data, including Chl-a data, sea surface temperature (SST) data, and photosynthetically available radiation data. MSDF constructs a spatialtemporal graph network to learn high-quality spatiotemporal representations of SST and photosynthetically active radiation (PAR) data, and fuses the learned representations with Chl-a data to enrich the information for imputation. Then, we developed a generation-based data imputation (GDI) module to model the distribution of Chl-a data based on the outputs of MSDF. Considering the high data sparsity, we designed a self-supervised training strategy to train S2-STI in the absence of ground truth. Finally, we leverage the data generated by the GDI module in the trained S2-STI model to fill in missing values in the sparse Chl-a data. Experiments on real datasets show that S2-STI achieves much better performance than the existing data imputation methods. Specifically, for the Chl-a data with a missing rate of 0.9, S2-STI improves by at least 12% in terms of masked autoencoder (MAE) error when compared with the strong baseline methods.

Link weight prediction is an important subject in network science and machine learning. Its applications to social network analysis, network modeling, and bioinformatics are ubiquitous. Although this subject has attracted considerable attention recently, the performance and interpretability of existing prediction models have not been well balanced. This article focuses on an unsupervised mixed strategy for link weight prediction. Here, the target attribute is the link weight, which represents the correlation or strength of the interaction between a pair of nodes. The input of the model is the weighted adjacency matrix without any preprocessing, as widely adopted in the existing models. Extensive observations on a large number of networks show that the new scheme is competitive to the state-of-the-art algorithms concerning both root-mean-square error and Pearson correlation coefficient metrics. Analytic and simulation results suggest that combining the weight consistency of the network and the link weight-associated latent factors of the nodes is a very effective way to solve the link weight prediction problem.

The mining missing links and predicting upcoming links are two important topics in the link prediction. In the past decades, a variety of algorithms have been developed, the majority of which apply similarity measures to estimate the bonding probability between nodes. And for these algorithms, it is still difficult to achieve a satisfactory tradeoff among precision, computational complexity, robustness to network types, and scalability to network size. In this article, we propose a chaotic ant colony optimized (CACO) link prediction algorithm, which integrates the chaotic perturbation model and ant colony optimization. The extensive experiments on a wide variety of unweighted and weighted networks show that the proposed algorithm CACO achieves significantly higher prediction accuracy and robustness than most of the state-of-the-art algorithms. The results demonstrate that the chaotic ant colony effectively takes advantage of the fact that most real networks possess the transmission capacity and provides a new perspective for future link prediction research.

Social networks have become one of the most important research platforms in the big data era. Modelling social networks enables researchers and engineers to understand and analyze their intrinsic properties thereby implementing their real applications. A number of studies on social network modelling focus on a few characteristics, such as the number of edges (i.e., two-star motifs), scale-free degree distribution, and assortative property. This paper proposes an exponential triangle model for a typical social network, namely the Facebook network, established based on big data, and further analyzes its primary attributes of common interest on topological features. This new model has a power-law node-degree distribution with a flat top and an exponential cut-off tail, in remarkable agreement with one large-scale Facebook dataset. It can be used to predict future links of the Facebook network and help improve the friend-recommendation system. Furthermore, this work provides a useful graph-theoretic tool for Facebook network studies and enhances potential applications of social networks in general.

Summary Modern communication networks and social networks are the main tunnels of knowledge diffusion. Knowledge diffusion in complex networks is different from the epidemic‐like information spreading, because individuals are willing to learn and spread knowledge to their friends and the learning process can hardly be achieved in a few conversations. In this paper, we investigate the important issue as what topological structure is suitable for knowledge diffusion. We propose a new knowledge diffusion model, where both learning and forgetting mechanisms are considered. In this model, individuals can play imparter and learner simultaneously. Comparing knowledge diffusion on a series of complex topologies, we observe that the individuals with a large degree can quickly learn more knowledge, who are beneficial to knowledge diffusion. Our results surprisingly reveal that the networks with high degree‐heterogeneity are likely to be suitable for knowledge diffusion. Our finding suggests that enhancing the degree heterogeneity of existing social networks may help to improve the performance of knowledge diffusion. This result is well confirmed by our extensive simulation results. Our model therefore provides a theoretical framework for understanding knowledge diffusion in complex topologies. Copyright © 2016 John Wiley & Sons, Ltd.