Sea surface temperature (SST) prediction is a critical task in ocean science, supporting various applications, such as weather forecasting, fisheries management, and storm tracking. While existing data-driven methods have demonstrated significant success, they often neglect to leverage the rich domain knowledge accumulated over the past decades, limiting further advancements in prediction accuracy. The recent emergence of large language models (LLMs) has highlighted the potential of integrating domain knowledge for downstream tasks. However, the application of LLMs to SST prediction remains under explored, primarily due to the challenge of integrating ocean domain knowledge and numerical data. To address this issue, we propose Ocean Knowledge Graph-enhanced LLM (OKG-LLM), a novel framework for global SST prediction. To the best of our knowledge, this work presents the first systematic effort to construct an Ocean Knowledge Graph (OKG) specifically designed to represent diverse ocean knowledge for SST prediction. We then develop a graph embedding network to learn the comprehensive semantic and structural knowledge within the OKG, capturing both the unique characteristics of individual sea regions and the complex correlations between them. Finally, we align and fuse the learned knowledge with fine-grained numerical SST data and leverage a pre-trained LLM to model SST patterns for accurate prediction. Extensive experiments on the real-world dataset demonstrate that OKG-LLM consistently outperforms state-of-the-art methods, showcasing its effectiveness, robustness, and potential to advance SST prediction. The codes are available in the online repository.

Existing Protein Language Models (PLMs) often suffer from limited adaptability to multiple tasks and exhibit poor generalization across diverse biological contexts. In contrast, general-purpose Large Language Models (LLMs) lack the capability to interpret protein sequences and fall short in domain-specific knowledge, limiting their capacity for effective biosemantic reasoning. To combine the advantages of both, we propose BioBridge, a domain-adaptive continual pretraining framework for protein understanding. This framework employs Domain-Incremental Continual Pre-training (DICP) to infuse protein domain knowledge and general reasoning corpus into a LLM simultaneously, effectively mitigating catastrophic forgetting. Cross-modal alignment is achieved via a PLM-ProjectorLLM pipeline, which maps protein sequence embeddings into the semantic space of the language model. Ultimately, an end-toend optimization is adopted to uniformly support various tasks, including protein property prediction and knowledge questionanswering. Our proposed BioBridge demonstrates performance comparable to that of mainstream PLMs on multiple protein benchmarks, such as EC and BindingDB. It also achieves results on par with LLMs on general understanding tasks like MMLU and RACE. This showcases its innovative advantage of combining domain-specific adaptability with general-purpose language competency.

Cloud removal (CR) remains a challenging task in remote sensing image processing. Although diffusion models (DM) exhibit strong generative capabilities, their direct applications to CR are suboptimal, as they generate cloudless images from random noise, ignoring inherent information in cloudy inputs. To overcome this drawback, we develop a new CR model EMRDM based on mean-reverting diffusion models (MRDMs) to establish a direct diffusion process between cloudy and cloudless images. Compared to current MRDMs, EMRDM offers a modular framework with updatable modules and an elucidated design space, based on a reformulated forward process and a new ordinary differential equation (ODE)-based backward process. Leveraging our framework, we redesign key MRDM modules to boost CR performance, including restructuring the denoiser via a preconditioning technique, reorganizing the training process, and improving the sampling process by introducing deterministic and stochastic samplers. To achieve multi-temporal CR, we further develop a denoising network for simultaneously denoising sequential images. Experiments on mono-temporal and multi-temporal datasets demonstrate the superior performance of EMRDM. Our code is available at https://github.com/Ly403/EMRDM.

Temporal causal discovery aims to uncover causal relations in time series data. Current deep learning-based methods usually analyze the parameters of some components of the trained models, which is an incomplete mapping process from the model parameters to the causality and fails to investigate the other components. To address this, this paper presents an interpretable transformer-based causal discovery model termed CausalFormer, which consists of: 1) the causality-aware transformer which learns the causal representation with the multi-kernel causal convolution under the temporal priority constraint, and 2) the decomposition-based causality detector which identifies causality by interpreting the global structure of the trained transformer with the regression relevance propagation.

Chlorophyll-a (Chl-a) is a critical indicator of water quality, and accurate Chl-a prediction is essential for marine ecosystem protection. However, existing methods for Chl-a prediction cannot adequately uncover the correlations between Chl-a and other environmental factors, e.g., SST and PAR. In addition, it is also difficult for these methods to learn the burst distributions of Chl-a data, i.e., increasing sharply for certain short periods of time and remaining stable for the rest of time. Furthermore, as original Chl-a, SST, and PAR data are often of high sparsity, most approaches rely on complete reanalysis data, which can incur accumulated error accumulation and degrade prediction performance. To address these three issues, we proposed a Spatio-Temporal Attention Network entitled SMO-STANet for Chl-a prediction. Concretely, the multi-branch spatio-temporal embedding module and spatio-temporal attention module are developed to learn the correlations between Chl-a and the two external factors, i.e., SST and PAR, thus facilitating the learning of the underlying spatio-temporal distribution of Chl-a. In addition, we designed a scaled loss function to enable SMO-STANet to adapt to the burst distributions of Chl-a. Last but not least, we develops a sparse observation data completion module to address the issue of data sparsity. According to the experimental results on two real datasets, SMO-STANet outperforms existing methods for Chl-a prediction by a large margin. The code is available at https://github.com/ADMIS-TONGJI/SMO-STANet.

Multimodal sentiment analysis is an active research area that combines multiple data modalities, e.g., text, image and audio, to analyze human emotions, and benefits a variety of applications. Existing multimodal sentiment analysis methods can be roughly classified as modality interaction-based methods, modality transformation-based methods and modality similarity-based methods. However, most of these methods highly rely on the strong correlations between modalities, and cannot fully uncover and utilize the correlations between modalities to enhance sentiment analysis. Therefore, these methods usually achieve unsatisfactory performance for identifying the sentiment of multimodal data with weak correlations. To address this issue, we proposed a two-stage semi-supervised model termed Correlation-aware Multimodal Transformer (CorMulT) which consists of pre-training stage and prediction stage. At the pre-training stage, a modality correlation contrastive learning module is designed to efficiently learn modality correlation coefficients between different modalities. At the prediction stage, the learned correlation coefficients are fused with modality representations to make the sentiment prediction. According to the experiments on the popular multimodal dataset CMU-MOSEI, CorMulT obviously surpasses the state-of-the-art multimodal sentiment analysis methods. The code of CorMulT is available at https://github.com/YAMY1234/CorMulT

Long-term prediction of Sea Surface Temperature (SST) is a pervasive issue in ocean science, particularly for understanding climate changes and improving marine disaster risk assessment. However, existing approaches typically focus either on short-term or long-term coarse-grained prediction, due to their limited ability to overcome noise interference and model complex spatio-temporal dependencies in long SST sequences. To overcome these limitations, we proposed a novel spatio-temporal decomposition Mamba model, termed STDMamba, for long-term fine-grained SST prediction. First, we introduce a Gaussian-weighted series decomposition module with smoothing mechanisms to decompose SST sequences into trend and fluctuation components, thereby mitigating noise interference. Then, we design a dual spatio-temporal representation learning module which utilizes Mamba2 to effectively capture the long-term spatio-temporal dependencies in the fluctuation component, and employs a Temporal Convolutional Network (TCN) to learn the spatio-temporal feature representation of the trend component. Finally, the dual representations are fused and passed through a prediction layer to generate the long-term fine-grained SST prediction results. Experiments on real-world datasets demonstrate that STDMamba significantly outperforms state-of-the-art prediction models. The code of STDMamba is available at https://github.com/ADMIS-TONGJI/STDMamba.

Few-shot object detection (FSOD) has been proposed to solve the problem of insufficient data for training, and it has drawn the attention of the remote sensing community in recent years. A mainstream type of FSOD method is to generate class prototypes based on the limited samples to help the construction of classification decision boundaries. However, these constructed prototypes may be far away from the true class centroids in the few-shot scenario. Recently, the vision-language model (VLM) has shown its powerful ability to align the visual features and text features, which leads to strong zero-shot performance on various downstream computer vision tasks when given only texts. Therefore, in this work, we propose to build class prototypes from text descriptions instead of limited visual instances by leveraging a classical pretrained VLM named CLIP. Concretely, we generate prototypes by feeding the CLIP text encoder with class names and enforcing each positive proposal feature to be close to the corresponding prototype. To accelerate the alignment process, we utilize the CLIP visual encoder as another teacher to achieve visual knowledge distillation. Moreover, we adopt prompt tuning to adapt CLIP to the remote sensing scenario. Extensive experiments on two public FSOD datasets, i.e., DIOR and NWPU VHR-10.v2, demonstrate the effectiveness of our method, which yields competitive results with that of existing approaches.

Remote sensing data are of considerable significance for monitoring global climate, detecting harmful algae bloom, and so on. However, due to sensor failures, cloud cover, and thick aerosols, the collected remote sensing data for various ocean factors, such as chlorophyll-a (Chl-a) concentration and sea surface temperature (SST), often have a high missing rate, which seriously hinders their applications. Existing data imputation models mostly ignore highly sparse spatial locations or perform badly when the data missing rate is high due to the lack of available information. Large language models (LLMs) possess powerful representation learning capabilities and can effectively capture sequential correlations even with extremely limited information, thereby presenting the promising potential for highly sparse remote sensing data imputation. Therefore, we proposed a novel LLM-based spatiotemporal model for highly sparse remote sensing data imputation, i.e., LLM4HRS. First, LLM4HRS develops an LLM-based bidirectional temporal representation learning module to learn forward and backward temporal dependencies in data sequences and fuses them together to obtain comprehensive temporal representations. Next, LLM4HRS constructs a LLM-based spatial representation learning module to learn spatial correlations with the learned temporal representation. Finally, a spatiotemporal representation fusion and data imputation module is developed to achieve data imputation. Experiments on the SST and Chl-a remote sensing datasets demonstrate that LLM4HRS significantly outperforms existing data imputation models, with its advantages becoming more pronounced as the masking rate increases. Furthermore, when extended to the remote sensing PAR data in a large region, LLM4HRS still achieves the best performance, further validating its broad applicability for remote sensing data imputation. The code of LLM4HRS is publicly available athttps://github.com/ssyuwang/LLM4HRS-master.

Accurate prediction of Sea Surface Temperature (SST) is of high importance in marine science, benefiting applications ranging from ecosystem protection to extreme weather forecasting and climate analysis. Wide-area SST usually shows diverse SST patterns in different sea areas due to the changes of temperature zones and the dynamics of ocean currents. However, existing studies on SST prediction often focus on small-area predictions and lack the consideration of diverse SST patterns. Furthermore, SST shows an annual periodicity, but the periodicity is not strictly adherent to an annual cycle. Existing SST prediction methods struggle to adapt to this non-strict periodicity. To address these two issues, we proposed the Cross-Region Graph Convolutional Network with Periodicity Shift Adaptation (RGCN-PSA) model which is equipped with the Cross-Region Graph Convolutional Network module and the Periodicity Shift Adaption module. The Cross-Region Graph Convolutional Network module enhances wide-area SST prediction by learning and incorporating diverse SST patterns. Meanwhile, the periodicity Shift Adaptation module accounts for the annual periodicity and enable the model to adapt to the possible temporal shift automatically. We conduct experiments on two real-world SST datasets, and the results demonstrate that our RGCN-PSA model obviously outperforms baseline models in terms of prediction accuracy. The code of RGCN-PSA model is available at https://github.com/ADMIS-TONGJI/RGCN-PSA/ .

With the growing diversity of data formats on social media, such as text, images, and videos, there is a growing need to analyze sentiment from multiple modalities. Multimodal sentiment detection, which aims to identify users’ sentiment by jointly modeling information from different modalities, has thus attracted increasing attention. However, most existing multimodal sentiment detection methods fuse multimodal information directly after the unimodal encoding and overlook the modality consistency of multimodal vector spaces before the fusion, which may damage the accuracy of multimodal sentiment detection. To address this issue, we propose a contrastive learning-based multimodal sentiment detection model termed EPMC which can map the representations of different modalities into a unified semantic space before fusion. EPMC operates in two stages, i.e., pre-training stage and fine-tuning stage. At the pre-training stage, we designed a cross-modal transformation module to map different modalities into a unified feature space. Meanwhile, to further capture the relationship between the cross-modal transformation vectors and the unimodal encoding vectors, we propose a multimodal consistency contrastive learning task that helps the model discern and amplify the cross-modal similarity between different modalities, thereby learning more discriminative features for sentiment detection. At the fine-tuning stage, EPMC is iteratively refined using the learned multimodal representation and guided by the cross-entropy loss. Extensive experiments conducted on three public multimodal datasets validate the effectiveness of EPMC model. The official implementation of EPMC is released at https://github.com/ADMIS-TONGJI/EPMC .

Given a specific region, crime prediction aims to predict the occurrence of various crime events within a certain period of time in future, which is of high significance for guaranteeing urban safety. In practice, crime events are usually affected by a variety of factors from different views, e.g., the attributes of the region, the correlations between different regions, and the correlations between different categories of crime events. Moreover, these correlations are dynamically changing over time, which makes it difficult to learn the regularity and patterns in crime data for achieving accurate prediction. To address this issue, we proposed a new M ulti- V iew S patial- T emporal (MVST) model for fine-grained crime prediction. MVST model first builds a static region graph to capture the similarity between regions in terms of region attributes such as census records and economy statistics, and creates a time-dependent graph to capture the dynamic correlations between regions based on human mobility data. Meanwhile, both static and dynamic graphs are created to capture the correlations between different categories of crime events. After that, those graphs created from different views are fused together with a multi-view graph fusion module to achieve crime prediction with fine-grained time granularities, e.g., 4 hours and 12 hours. According to the experiments on two real crime datasets, our MVST model obviously outperforms existing crime prediction methods. The code of MVST model is available at https://github.com/weichang811/MVST .

Inductive relation prediction, an important task for knowledge graph completion, is to predict the relations between entities that are unseen at the training stage. The latest methods use Pre-Trained Language Models (PLMs) to encode the paths between the head entity and tail entity and achieve state-of-the-art prediction performance. However, these methods cannot handle no-path scenarios well and lack the capability to learn comprehensive relation representations for distinguishing different relations. To tackle this issue, we propose a novel R elation- a ware k nowledg e r easoning model entitled Raker, which introduces an adaptive reasoning information extraction method to identify relation-aware reasoning neighbors of entities in the target triple to handle no-path scenarios and enables the PLM to better distinguish different relations via the relation-specific soft prompting. Raker is evaluated on three public datasets and achieves SOTA performance in inductive relation prediction when compared with the baseline methods. Notably, the absolute improvement of Raker is even more than 5% on the FB15k-237 dataset in the inductive setting. Moreover, Raker also demonstrates the superiority in transductive, few-shot, and unseen relation settings. The code of Raker is available at https://github.com/ADMIS-TONGJI/Raker .

With the rapid amassing of spatial-temporal (ST) ocean data, many spatial-temporal data mining (STDM) studies have been conducted to address various oceanic issues, including climate forecasting and disaster warning. Compared with typical ST data (e.g., traffic data), ST ocean data presents some unique characteristics, e.g., diverse regionality and high sparsity. These characteristics make it difficult to design and train STDM models on ST ocean data. To the best of our knowledge, a comprehensive survey of existing studies remains missing in the literature, which hinders not only computer scientists from identifying the research issues in ocean data mining but also ocean scientists to apply advanced STDM techniques. In this article, we provide a comprehensive survey of existing STDM studies for ocean science. Concretely, we first review the widely used ST ocean datasets and highlight their unique characteristics. Then, typical ST ocean data quality enhancement techniques are discussed. Next, we classify existing STDM studies for ocean science into four types of tasks, i.e., prediction, event detection, pattern mining, and anomaly detection, and elaborate the techniques for these tasks. Finally, promising research opportunities are discussed. This survey can help scientists from both computer science and ocean science better understand the fundamental concepts, key techniques, and open challenges of STDM for ocean science.

Accurate spatial-temporal (ST) traffic prediction plays an essential role in intelligent transportation systems. Existing advanced traffic prediction methods typically utilize spatial-temporal graph neural networks (STGNNs) to capture the ST correlations and achieve excellent prediction performance. However, our experimental investigation reveals that existing static and dynamic graph-based STGNNs still incur excessive noise and redundancy, and fail to discover robust and reliable ST correlations in traffic networks. Moreover, most methods cannot explain the underlying reasons behind the ST correlations. To solve these problems, we propose a novel S patial- T emporal G raph M odeling framework via A daptive contrastive learning (ST-GMA). Firstly, we design a robust augmentation learning module to generate high-level and robust data augmentations via a self-supervised task for modeling reliable correlations. Then, we develop an adaptive contrastive learning module to update correlation graphs by effectively selecting positive and negative augmentations, reducing redundant calculations, and providing insights into the correlation changes. Finally, ST-GMA integrates the generated correlation graphs with ST convolution blocks to conduct traffic prediction tasks. Experimental results on five real-world datasets demonstrate that ST-GMA not only achieves significant prediction performance compared with state-of-the-art methods but also exhibits a new perspective on the interpretability of correlation changes.

Temporal causal discovery is a crucial task aimed at uncovering the causal relations within time series data. The latest temporal causal discovery methods usually train deep learning models on prediction tasks to uncover the causality between time series. They capture causal relations by analyzing the parameters of some components of the trained models, e.g., attention weights and convolution weights. However, this is an incomplete mapping process from the model parameters to the causality and fails to investigate the other components, e.g., fully connected layers and activation functions, that are also significant for causal discovery. To facilitate the utilization of the whole deep learning models in temporal causal discovery, we proposed an interpretable transformer-based causal discovery model termed CausalFormer, which consists of the causality-aware transformer and the decomposition-based causality detector. The causality-aware transformer learns the causal representation of time series data using a prediction task with the designed multi-kernel causal convolution which aggregates each input time series along the temporal dimension under the temporal priority constraint. Then, the decomposition-based causality detector interprets the global structure of the trained causality-aware transformer with the proposed regression relevance propagation to identify potential causal relations and finally construct the causal graph. Experiments on synthetic, simulated, and real datasets demonstrate the state-of-the-art performance of CausalFormer on discovering temporal causality.

Existing multimodal sentiment analysis models focus more on fusing highly correlated image-text pairs, and thus achieves unsatisfactory performance on multimodal social media data which usually manifests weak correlations between different modalities. To address this issue, we first build a large multimodal social media sentiment analysis dataset RU-Senti which contains more than 100,000 image-text pairs with sentiment labels. Then, we proposed a topic-oriented model (TOM) which assumes that text is usually related to a certain portion of the image contents and significant variances exist in sentiment distribution across diverse topics. TOM learns the topic information from textual content and designs a topic-oriented feature alignment module to extract textual semantics correlated information from images, thus achieving the alignment between two modalities. Then, TOM utilizes a transformer encoder initialized with the parameters from a pre-trained vision-language model to fuse the multimodal features for sentiment prediction. According to the experiments over the public MVSA-Multiple dataset and our RU-Senti dataset, RU-Senti is of high suitability for studying weakly correlated multimodal sentiment analysis, and the proposed TOM model also largely outperforms the SOTA mulitimodal sentiment analysis methods and pre-trained vision-language models.

Monitoring Chlorophyll-a (Chl-a) concentration in ocean is of considerable significance for early warning of algae disasters, marine ecological environment protection, etc. However, due to various uncontrollable factors such as cloud cover and thick aerosols, the missing rate of observed Chl-a data is quite high, which seriously hinders its applications. The existing data imputation methods are mostly applicable to Chl-a data with a low missing rate, and perform much worse when the missing rate reaches 0.7 or above. To address this issue, we proposed a self-supervised spatiotemporal imputation (S2-STI) model for highly sparse Chl-a data imputation by fusing multisource satellite data. First, to obtain comprehensive information about the missed Chl-a data, we designed a multisource sparse data fusion (MSDF) module to fuse multisource satellite data, including Chl-a data, sea surface temperature (SST) data, and photosynthetically available radiation data. MSDF constructs a spatialtemporal graph network to learn high-quality spatiotemporal representations of SST and photosynthetically active radiation (PAR) data, and fuses the learned representations with Chl-a data to enrich the information for imputation. Then, we developed a generation-based data imputation (GDI) module to model the distribution of Chl-a data based on the outputs of MSDF. Considering the high data sparsity, we designed a self-supervised training strategy to train S2-STI in the absence of ground truth. Finally, we leverage the data generated by the GDI module in the trained S2-STI model to fill in missing values in the sparse Chl-a data. Experiments on real datasets show that S2-STI achieves much better performance than the existing data imputation methods. Specifically, for the Chl-a data with a missing rate of 0.9, S2-STI improves by at least 12% in terms of masked autoencoder (MAE) error when compared with the strong baseline methods.

Accurate prediction of ocean factors (e.g., temperature and salinity) is crucial for plenty of applications, including weather forecasting, storm tracking, and ecosystem protection. Meanwhile, it is well-known that the ocean is a unified system and various ocean factors usually influence each other. For example, the changes in temperature would affect the distribution of salinity in ocean. However, the existing studies for ocean factor prediction mainly focus on designing individual models for predicting specific factors and ignore the correlations between different factors, thus having potentials to be further improved. Therefore, we propose a unified framework UniOcean to predict multiple ocean factors simultaneously, and capture the correlations between them to improve the prediction accuracy. First, considering that ocean factors are usually collected with different temporal scales, we develop the fine-grained multiscale data fusion module to integrate multiple ocean factors with different temporal scales, and effectively learn their hierarchical patterns at different levels. Then, since the correlations between ocean factors may vary across different time periods, the multifactor correlation learning module is constructed to adaptively learn the dynamic correlations between different factors. Finally, we utilize the factor-specific towers to predict multiple ocean factors simultaneously. Experimental results on five real-world remote sensing datasets demonstrate that UniOcean significantly improves the prediction accuracy by 11%–53% in terms of MSD for different ocean factors.

Abstract Marine plankton classification is important for monitoring marine biological populations and understanding marine ecosystems. However, it is difficult to obtain abundant annotated image data of marine plankton to train classification models of high quality. To classify marine plankton with a small number of image samples, a cross‐domain few‐shot learning model (CDFM) is proposed. First, CDFM learns knowledge from existing labelled images and then transfers the knowledge to new images. In this process, there is an issue of domain differences between the new images and the existing datasets due to differences in data acquisition time and locations. To address this issue, the authors pre‐train a model in the source domain and then use fine‐tuning to adapt it to the target domain. Second, the graph neural network is used as a meta‐learning module to learn a feature distance metric for marine plankton classification with limited samples. Extensive experiments on four marine plankton image datasets are conducted, including Kaggle Plankton, miniPPlankton, ZooScan and WHOI, and CDFM outperforms existing methods for marine plankton classification.

Sea surface temperature (SST) is one critical parameter of global climate change, and accurate SST prediction is important to various applications, e.g., weather forecasting, fishing directions, and disaster warnings. The global ocean system is unified and complex, and the SST patterns in different oceanic regions are highly diverse and correlated. However, existing data-driven SST prediction methods mainly consider the local patterns within a certain oceanic region, e.g., El Nino region and the Black sea. It is challenging but necessary to model the global SST correlations rather than that in a specific region to enhance the prediction accuracy of SST. In this work, we proposed a new method called Hierarchical Graph Recurrent Network (HiGRN) to address the issue. First, to learn the dynamic and diverse local SST patterns of specific locations, we design an adaptive node embedding with self-learned parameters to learn various SST patterns. Then we develop a hierarchical cluster generator to aggregate the locations with similar patterns into regional clusters and utilize a graph convolution network to learn the spatial correlations among these clusters. Finally, we introduce a multi-level attention mechanism to fuse the local patterns and regional correlations, and the output is fed into a recurrent network to achieve SST predictions. Extensive experiments on two real-world datasets show that our method largely outperforms the state-of-the-art SST prediction methods. The source code is available at https://github.com/Neoyanghc/HiGRN .

Cross-view geo-localization is to match scene images (e.g. ground-view images) with geo-tagged aerial images, which is crucial to a wide range of applications such as autonomous driving and street view navigation. Existing methods can neither address the perspective difference well nor effectively capture the scene information. In this work, we propose a Geometric and Scene Information Fusion (GSIF) model for more accurate cross-view geo-localization. GSIF first learns the geometric information of scene images and aerial images via log-polar transformation and spatial-attention aggregation to alleviate the perspective difference. Then, it mines the scene information of scene images via Sky View Factor (SVF) extraction. Finally, both geometric information and scene information are fused for image matching, and a balanced loss function is introduced to boost the matching accuracy. Experimental results on two real datasets show that our model can significantly outperforms the existing methods.

Network intrusion detection (NID) aims to detect various network attacks and is an important task for guaranteeing network security. However, existing NID methods usually require a large amount of labeled data for training, which is impractical in many real application scenarios due to the high cost. To address this issue, we proposed an extreme semi-supervised framework based on transformer (ESet) for NID. ESeT first developed a multi-level feature extraction module to learn both packet-level byte encoded features and flow-level frequency domain features to enrich the information for detection. Then, during the semi-supervised learning, ESeT designed the dual-encoding transformer to fuse the extracted features for intrusion detection and introduced the credibility selector to reduce the negative impacts of incorrect pseudo-labeling of unlabeled data. The experiment results show that our model achieves excellent performance (F1-score: 97.60%) with only a small proportion of labeled data (1%) on CIC-IDS2017 and CSE-CIC-IDS2018 datasets.

Nested Name Entity Recognition is to identify the entities with nested relationships from sentences, which has various applications ranging from relation extraction to semantic understanding. However, existing methods have two drawbacks, i.e., 1) error propagation when identifying entities at different nesting levels and 2) unable to uncover and utilize the complex correlations between the inner and outer entities. To address these two defects, we propose a bidirectional hierarchical(BidH) model for nested name entity recognition. BidH consists of a forward module and a backward module, where the former first extracts the inner entities and then extracts the outer ones, while the latter extracts the entities in the opposite direction. Furthermore, we design an entity masked self attention mechanism to combine the two modules by fusing their predictions and hidden states layer by layer. BidH can effectively deal with error propagation and exploit the correlations between entities at different nesting levels to improve the recognition accuracy. Experiments on the GENIA dataset show that BidH outperforms the state-of-the-art nested named entity recognition models in terms of F1 score.

Resource allocation problems often manifest as online decision-making tasks where the proper allocation strategy depends on the understanding of the allocation environment and resources workload. Most existing resource allocation methods are based on meticulously designed heuristics which ignore the patterns of incoming tasks, so the dynamics of incoming tasks cannot be properly handled. To address this problem, we mine the task patterns from the large volume of historical allocation data and propose a reinforcement learning model termed IRDA to learn the allocation strategy in an incremental way. We observe that historical allocation data is usually generated from the daily repeated operations, which is not independent and identically distributed. Training with partial of this dataset can make the allocation strategy converged already, thereby wasting a lot of remaining data. To improve the learning efficiency, we partition the whole historical allocation big dataset into multi-batch datasets, which forces the agent to continuously “explore” and learn on the distinct state spaces. IRDA reuses the strategy learned from the previous batch dataset and adapts it to the learning on the next batch dataset, so as to incrementally learn from multi-batch datasets and improve the allocation strategy. We apply the proposed method to handle baggage carousel allocation at Hong Kong International Airport (HKIA). The experimental results show that IRDA is capable of incrementally learning from multi-batch datasets, and improves the baggage carousel resource utilization by around 51.86 percent compared to the current baggage carousel allocation system at HKIA.

Recent on-demand services, such as Uber and DiDi, provide a platform for users to request services on the spot and for suppliers to meet such demand. In such platforms, demands are dispatched to suppliers round by round, and suppliers have autonomy to decide whether to accept demands or not. Existing approaches dispatch a demand to multiple suppliers in each round, while a supplier can only receive one demand. However, by using these approaches, pended demands can not be fully dispatched in a round specially when suppliers are not sufficient, and thus need to wait for many rounds to be dispatched, leading to long response time. In this paper, we propose a novel demand dispatching model, named by many-to-many model. The novelty of the model is that a supplier could receive multiple demands in a round, such that the demand has high chance to be dispatched and answered within short time. More specifically, we first learn the probability distribution function of the response time of a supplier to a given demand, by considering the features of both the demand and the supplier. Taking the learned results as input, our model generates an optimal matching between the demands and suppliers to minimize the overall response time of the demands via solving an optimization problem. Experiments on real-world datasets show that our model is better than the start-of-art models in terms of successful acceptance rate and response time.

Generic object detection has achieved significant progress in recent years. However, oriented object detection in aerial images is still a challenging task due to arbitrary orientation, complex backgrounds and large scale variation. Currently, the majority of oriented object detectors are anchor-based, achieving promising performance yet suffering from complicated anchor designs and imbalance between the positive and negative anchor boxes. In this work, we propose a new anchor-free model called RBA-CenterNet for oriented object detection. Specifically, we first detect the center point of each object after extracting feature of the input image. Then, we regress the rotated box parameters in other prediction branches. Considering that using only the center point of the object may hurt the detection performance, we introduce a refinement module called rotated border alignment (RBA) to integrate the border feature into the center point feature. Our experiments show that the proposed model RBA-CenterNet can achieve comparable detection performance to state-of-the-art methods.

Balanced rule-constrained resource allocation aims to evenly distribute tasks to different processors under allocation rule constraints. Conventional heuristic approach fails to achieve optimal solution while simple brute force method has the defect of high computational complexity. To address these limitations, we propose recursive balanced k-subset sum partition (RBkSP), in which iterative 'cut-one-out' policy is employed that in each round, only one subset whose weight of tasks sums up to 1/k of the total weight of all tasks is taken out from the set. In a single partition, we first create a dynamic programming table with its elements recursively computed, then use 'zig-zag search' method to explore the table, find out elements with optimal subset partition and assign different partitions to proper places. Next, to resolve conflicts during allocation, we use simple but effective heuristic method to adjust the allocation of tasks that is contradicted to allocation rules. Testing results show RBkSP can achieve more balanced results with lower computational complexity over classical benchmarks.

Movement patterns of intra-urban goods/things and the ways they differ from human mobility and traffic flow patterns have seldom been explored due to data access and methodological limitations, especially from systemic and long timescale perspectives. However, urban logistics big data are increasingly available, enabling unprecedented spatial and temporal resolutions to this issue. This research proposes an analytical framework for exploring intra-urban goods movement patterns by integrating spatial analysis, network analysis and spatial interaction analysis. Using daily urban logistics big data (over 10 million orders) provided by the largest online logistics company in Hong Kong (GoGoVan) from 2014 to 2016, we analyzed two spatial characteristics (displacement and direction) of urban goods movement. Results showed that the distribution of goods displaceFower law or exponential distribution of human mobility trends. The origin–destination flows of goods were used to build a spatially embedded network, revealing that Hong Kong became increasingly connected through intra-urban freight movement. Finally, spatial interaction characteristics were revealed using a fitting gravity model. Distance lacked substantial influence on the spatial interaction of goods movement. These findings have policy implications to intra-urban logistics and urban transport planning.

Emerging on-demand transport services, such as Uber and GoGoVan, usually face the dilemma of demand supply imbalance, meaning that the spatial distributions of orders and drivers are imbalanced. Due to such imbalance, much supply resource is wasted while a considerable amount of order demand cannot be met in time. To address this dilemma, knowing the unmet demand in the near future is of high importance for service providers because they can dispatch their vehicles in advance to alleviate the impending demand supply imbalance, we develop a general framework for predicting the unmet demand in future time slots. Under this framework, we first evaluate the predictability of unmet demand in on-demand transport services and find that unmet demand is highly predictable. Then, we extract both static and dynamic urban features relevant to unmet demand from data sets in multiple domains. Finally, multiple prediction models are trained to predict unmet demand by using the extracted features. As demonstrated via experiments, the proposed framework can predict unmet demand in on-demand transport services effectively and flexibly.

The widespread location-aware applications produce a vast amount of spatio-textual data that contains both spatial and textual attributes. To make use of this enriched information for users to describe their preferences for travel routes, we propose a Bounded-Cost Informative Route (BCIR) query to retrieve the routes that are the most textually relevant to the user-specified query keywords subject to a travel cost constraint. BCIR query is particularly helpful for tourists and city explorers to plan their travel routes. We will show that BCIR query is an NP-hard problem. To answer BCIR query efficiently, we propose an exact solution with effective pruning techniques and an approximate solution with performance guarantee. Extensive experiments over real data sets demonstrate that the proposed solutions achieve the expected performance.

With the proliferation of positioning technologies and GPS-enabled mobile devices, it has become very important to search for optimal paths to cover required points of interest(POIs, e.g., banks and restaurants) over road networks in many location-based services. In practice, however, traffic conditions are inherently uncertain and dynamically changing overtime, which makes it rather challenging to provide accurate results for path queries based on travelling time. Inspired by this, we considerthe practical settings of road networks and model them by uncertain road networks (URNs) on which the travelling time of each road is uncertain and captured by a set of travelling time samples. Then, we formalize the probabilistic time-constrained path (PTP) query over uncertain road networks to retrieve those paths that not only cover required POIs with constrained service time but also have the minimum travelling times in high confidence. We prove that PTPquery problem is NP-hard. In orderto answer PTPqueries efficiently, we propose an efficient PTPquery approach with effective pruning strategies regarding the time constraints on POIs and the probabilistic/rank requirements of queries. Extensive experiments on real road networks validate the efficiency and effectiveness of our PTPquery approach.

Nowadays, there is an emerging way of connecting logistics orders and van drivers, where it is crucial to predict the order response time. Accurate prediction of order response time would not only facilitate decision making on order dispatching, but also pave ways for applications such as supply-demand analysis and driver scheduling, leading to high system efficiency. In this work, we forecast order response time on current day by fusing data from order history and driver historical locations. Specifically, we propose Coupled Sparse Matrix Factorization (CSMF) to deal with the heterogeneous fusion and data sparsity challenges raised in this problem. CSMF jointly learns from multiple heterogeneous sparse data through the proposed weight setting mechanism therein. Experiments on real-world datasets demonstrate the effectiveness of our approach, compared to various baseline methods. The performances of many variants of the proposed method are also presented to show the effectiveness of each component.

The widespread location-aware applications produce a vast amount of spatio-textual data that contains both spatial and textual attributes. To make use of this enriched information for users to describe their preferences for travel routes, we propose a Bounded-Cost Informative Route (BCIR) query to retrieve the routes that are the most textually relevant to the user-specified query keywords subject to a travel cost constraint. BCIR query is particularly helpful for tourists and city explorers to plan their travel routes. We will show that BCIR query is an NP-hard problem. To answer BCIR query efficiently, we propose an exact solution with effective pruning techniques and two approximate solutions with performance guarantees. Extensive experiments over real data sets demonstrate that the proposed solutions achieve the expected performance.

Traffic congestion is a major concern in many cities around the world. Previous work mainly focuses on the prediction of congestion and analysis of traffic flows, while the congestion correlation between road segments has not been studied yet. In this paper, we propose a three-phase framework to study the congestion correlation between road segments from multiple real world data. In the first phase, we extract congestion information on each road segment from GPS trajectories of over 10,000 taxis, define congestion correlation and propose a corresponding mining algorithm to find out all the existing correlations. In the second phase, we extract various features on each pair of road segments from road network and POI data. In the last phase, the results of the first two phases are input into several classifiers to predict congestion correlation. We further analyze the important features and evaluate the results of the trained classifiers. We found some important patterns that lead to a high/low congestion correlation, and they can facilitate building various transportation applications. The proposed techniques in our framework are general, and can be applied to other pairwise correlation analysis.