Partially observable Markov decision processes (POMDPs) present significant challenges for reinforcement learning, as agents must learn optimal policies while maintaining belief states over unobserved environment states based on partial observations. We observe a compelling analogy: large language models (LLMs) autoregressively generate token probability distributions based on preceding context, mirroring how belief states are maintained and updated in POMDPs. This insight motivates leveraging the rich prior knowledge embedded in pre-trained LLMs for latent states estimation from observation-action histories. However, two critical challenges emerge: on the one hand, modality misalignment prevents LLMs from directly encoding visual observations and discrete actions; on the other hand, semantic misalignment exists between observation-action sequences and token sequences. To address these challenges, we introduce a novel framework ELSLLM that employs a Johnson-Lindenstrauss projection (JLP) module to transform input dimensions while preserving state similarity with theoretical guarantees, and utilizes modern Hopfield networks (MHN) to store all word embeddings from pre-trained LLMs as a knowledge repository. Through retrieval and querying mechanisms, ELSLLM achieves token-level knowledge alignment without requiring fine-tuning of the pre-trained LLMs. Extensive experiments on partially observable environments demonstrate that ELSLLM achieves state-of-the-art performance, significantly outperforming baseline methods with and without LSTM memory mechanisms. Our work opens new avenues for integrating pre-trained LLMs with reinforcement learning in partially observable settings.

Offline meta-reinforcement learning (OMRL) combines the strengths of learning from diverse datasets in offline RL with the adaptability to new tasks of meta-RL, promising safe and efficient knowledge acquisition by RL agents. However, OMRL still suffers extrapolation errors due to out-of-distribution (OOD) actions, compromised by broad task distributions and Markov Decision Process (MDP) ambiguity in meta-RL setups. Existing research indicates that the generalization of the Q network affects the extrapolation error in offline RL. This paper investigates this relationship by decomposing the Q value into feature and weight components, observing that while decomposition enhances adaptability and convergence in the case of high-quality data, it often leads to policy degeneration or collapse in complex tasks. We observe that decomposed Q values introduce a large estimation bias when the feature encounters OOD samples, a phenomenon we term "feature overgeneralization''. To address this issue, we propose FLORA, which identifies OOD samples by modeling feature distributions and estimating their uncertainties. FLORA integrates a return feedback mechanism to adaptively adjust feature components. Furthermore, to learn precise task representations, FLORA explicitly models the complex task distribution using a chain of invertible transformations. We theoretically and empirically demonstrate that FLORA achieves rapid adaptation and meta-policy improvement compared to baselines across various environments.

While retrieval-augmented generation systems partially address the hallucination issues in large language models, it also introduces new vulnerabilities to knowledge corruption attacks. Adversaries exploit these vulnerabilities by poisoning documents provided by RAG system to manipulate LLM outputs. To counter this threat, we propose SecureCollaRAG, a Byzantine-tolerant collaborative RAG framework leveraging Multi-source Knowledge Validation Mechanism. Our approach enables agent system to securely verify document provenance through dynamic GNN-based credibility scoring, effectively preventing stealthy knowledge corruption attacks while preserving essential domain knowledge integrity. Through extensive evaluations and formal analysis, we demonstrate that SecureCollaRAG maintains robustness against attackers under non-IID data distributions.

Recent studies have explored querying large language models (LLMs) to serve as predictors for graph mining tasks on text-attributed graphs (TAGs), establishing a promising paradigm that surpasses Graph Neural Networks (GNNs) in scalability and generalization. However, the high token costs of LLMs make this approach prohibitively expensive for large-scale node queries, and effective multi-query optimization solutions are currently lacking. By conducting information theory analysis at the single query level, we have gained insights that enabled the development of two multi-query optimization strategies: token pruning and query boosting. The token pruning strategy is designed to reduce token usage without compromising task performance by identifying saturated node queries and pruning tokens for these queries. Meanwhile, the query boosting strategy is designed to enhance task performance by enriching the context of unexecuted queries with pseudo-labels derived from previous queries through strategic scheduling, thereby maximizing the utility of these pseudo-labels. Extensive experiments applying these two strategies, either jointly or individually, to various existing methods demonstrate that the proposed approach serves our intentions well. Besides, this paper offers a fresh methodology for optimizing LLM processing of graph tasks, demonstrating great potential. For most natural graph data benchmarks in the field, it can save tokens by several orders of magnitude. For example, on the Ogbn-Products dataset, it could theoretically save up to$2\times 10^{9}$tokens.

Multi-View Reinforcement Learning (MVRL) seeks to provide agents with multi-view observations, enabling them to perceive environment with greater effectiveness and precision. Recent advancements in MVRL focus on extracting latent representations from multiview observations and leveraging them in control tasks. However, it is not straightforward to learn compact and task-relevant representations, particularly in the presence of redundancy, distracting information, or missing views. In this paper, we propose Multi-view Fusion State for Control (MFSC), firstly incorporating bisimulation metric learning into MVRL to learn task-relevant representations. Furthermore, we propose a multiview-based mask and latent reconstruction auxiliary task that exploits shared information across views and improves MFSC’s robustness in missing views by introducing a mask token. Extensive experimental results demonstrate that our method outperforms existing approaches in MVRL tasks. Even in more realistic scenarios with interference or missing views, MFSC consistently maintains high performance. The project code is available at https://github.com/zpwdev/MFSC.

Medication recommendation (MR) has undergone rapid advancement in recent years, driven by its significant practical implications in healthcare. However, such high-risk scenarios still experience two critical yet overlooked challenges: the prevalent overconfidence in raw confidence for individual medications and the lack of a robust solution for confidence quantification in medication combinations. This paper represents the first in-depth study addressing this gap. We introduce two innovative methodologies tailored to the unique challenges of MR scenarios: 1) A discernible binning-based calibration method with theoretical guarantees for the confidence of individual medication. It guarantees distinct accuracy levels between adjacent bins and maintains consistent statistical reliability across calibration and test data, enabling calibrated confidence to reflect the correctness of medication recommendations distinctively. 2) A sample-based quantification method for the set confidence of medication combination, which is applicable for various existing performance metrics in MR. Utilizing representative deep MR models as backbones and conducting extensive experiments on the widely recognized MIMIC datasets, we empirically prove the effectiveness and robustness of our proposed methods. Our approaches not only improve the reliability of MR but also pave the way for more informed decision-making in clinical settings.

Signed graph clustering is a critical technique for discovering community structures in graphs that exhibit both positive and negative relationships. We have identified two significant challenges in this domain: i) existing signed spectral methods are highly vulnerable to noise, which is prevalent in real-world scenarios; ii) the guiding principle "an enemy of my enemy is my friend", rooted in Social Balance Theory, often narrows or disrupts cluster boundaries in mainstream signed graph neural networks. Addressing these challenges, we propose the Deep Signed Graph Clustering framework (DSGC), which leverages Weak Balance Theory to enhance preprocessing and encoding for robust representation learning. First, DSGC introduces Violation Sign-Refine to denoise the signed network by correcting noisy edges with high-order neighbor information. Subsequently, Density-based Augmentation enhances semantic structures by adding positive edges within clusters and negative edges across clusters, following Weak Balance principles. The framework then utilizes Weak Balance principles to develop clustering-oriented signed neural networks to broaden cluster boundaries by emphasizing distinctions between negatively linked nodes. Finally, DSGC optimizes clustering assignments by minimizing a regularized clustering loss. Comprehensive experiments on synthetic and real-world datasets demonstrate DSGC consistently outperforms all baselines, establishing a new benchmark in signed graph clustering.

Subsequently, by executing various attacks on benchmark datasets such as MNIST, we construct Federated Learning Malicious Parameter Identification (FLMPID) dataset to enable malicious client detection. Building on this dataset, we propose FORTRESS (Federated POisoning-Resistance Defense via Dual-Domain Distance and TRust AssESSment), a framework designed to detect and mitigate malicious updates from clients. FORTRESS employs a unique encoder-decoder architecture. The encoder utilizes dual-domain distance metrics on weights and gradients to extract hidden representations, while the decoder leverages Actor-Critic (AC) reinforcement learning for trust assessment. We evaluated FORTRESS under multiple attack scenarios and demonstrated its defense effectiveness, making it a promising solution for enhancing the security of FL systems.

Web attack is a major threat to cyberspace security, so web attack detection models have become a critical task. Traditional supervised learning methods learn features of web attacks with large amounts of high-confidence labeled data, which are extremely expensive in the real world. Pre-trained models offer a novel solution with their ability to learn generic features on large unlabeled datasets. However, designing and deploying a pre-trained model for real-world web attack detection remains challenges. In this paper, we present a pre-trained model for web attack detection, including a pre-processing module, a pre-training module, and a deployment scheme. Our model significantly improves classification performance on several web attack detection datasets. Moreover, we deploy the model in real-world systems and show its potential for industrial applications.

The widespread adoption of Graph Neural Networks (GNNs) has led to an increasing focus on their reliability. To address the issue of underconfidence in GNNs, various calibration methods have been developed to gain notable reductions in calibration error. However, we observe that existing approaches generally fail to enhance consistently, and in some cases even deteriorate, GNNs' ability to discriminate between correct and incorrect predictions. In this study, we advocate the significance of discriminative ability and the inclusion of relevant evaluation metrics. Our rationale is twofold: 1) Overlooking discriminative ability can inadvertently compromise the overall quality of the model; 2) Leveraging discriminative ability can significantly inform and improve calibration outcomes. Therefore, we thoroughly explore the reasons why existing calibration methods have ineffectiveness and even degradation regarding the discriminative ability of GNNs. Building upon these insights, we conduct GNN calibration experiments across multiple datasets using a straightforward example model, denoted as DC(GNN). Its excellent performance confirms the potential of integrating discriminative ability as a key consideration in the calibration of GNNs, thereby establishing a pathway toward more effective and reliable network calibration.

Meta-Reinforcement Learning (Meta-RL) aims to reveal shared characteristics in dynamics and reward functions across diverse training tasks. This objective is achieved by meta-learning a policy that is conditioned on task representations with encoded trajectory data or context, thus allowing rapid adaptation to new tasks from a known task distribution. However, since the trajectory data generated by the policy may be biased, the task inference module tends to form spurious correlations between trajectory data and specific tasks, thereby leading to poor adaptation to new tasks. To address this issue, we propose the Meta-RL with task unCertAinty feedback through decoupled context-aware Reward and Dynamics components (MetaCARD). MetaCARD distinctly decouples the dynamics and rewards when inferring tasks and integrates task uncertainty feedback from policy evaluation into the task inference module. This design effectively reduces uncertainty in tasks with changes in dynamics or/and reward functions, thereby enabling accurate task identification and adaptation. The experiment results on both Meta-World and classical MuJoCo benchmarks show that MetaCARD significantly outperforms prevailing Meta-RL baselines, demonstrating its remarkable adaptation ability in sophisticated environments that involve changes in both reward functions and dynamics.

Graph representation of molecular data enables extracting stereoscopic features, with graph neural networks (GNNs) excelling in molecular property prediction. However, selecting optimal hyper-parameters for GNN construction is challenging due to the vast search space and high computational costs. To tackle this, we introduce a hierarchical evaluation strategy integrated with a genetic algorithm (HESGA). HESGA combines full and fast evaluations of GNNs. Full evaluation involves training a GNN with preset epochs, using root mean square error (RMSE) to measure hyperparameter quality. Fast evaluation interrupts training early, using the difference in RMSE values as a score for GNN potential. HESGA integrates these evaluations, with fast evaluation guiding candidate selection for full evaluation, maintaining elite individuals. Applying HESGA to optimise deep GNNs for molecular property prediction, experimental results on three datasets demonstrate its superiority over traditional Bayesian optimisation, Tree-structured Parzen Estimator, and CMA-ES. HESGA efficiently navigates the complex GNN hyperparameter space, offering a promising approach for molecular property prediction.

Visual Model-Based Reinforcement Learning (MBRL) promises to encapsulate agent's knowledge about the underlying dynamics of the environment, enabling learning a world model as a useful planner. However, top MBRL agents such as Dreamer often struggle with visual pixel-based inputs in the presence of exogenous or irrelevant noise in the observation space, due to failure to capture task-specific features while filtering out irrelevant spatio-temporal details. To tackle this problem, we apply a spatio-temporal masking strategy, a bisimulation principle, combined with latent reconstruction, to capture endogenous task-specific aspects of the environment for world models, effectively eliminating non-essential information. Joint training of representations, dynamics, and policy often leads to instabilities. To further address this issue, we develop a Hybrid Recurrent State-Space Model (HRSSM) structure, enhancing state representation robustness for effective policy learning. Our empirical evaluation demonstrates significant performance improvements over existing methods in a range of visually complex control tasks such as Maniskill with exogenous distractors from the Matterport environment. Our code is avaliable at https://github.com/bit1029public/HRSSM.

Covert communication is an method that plays an important role in secure data transmission. The technology embeds covert information into data and propagates it through covert channels. The communication quality depends on the choice of channel and data embedding techniques. Recently, blockchain has emerged to become the preferred channel to carry out covert communication for its decentralization and anonymity features. Existing covert transaction methods are constructed transaction-by-transaction, which makes them immune to text analysis-based detection methods. However, it is easy to expose their features on the transaction graph level. Unfortunately, there is yet no method to detect covert transactions by the features of transaction graph. In this paper, we propose a covert transaction detection method based on graph structure. By analyzing the statistical features of graph structure for addresses, we can infer whether they are the participants of covert transactions. Furthermore, we design a protection method of covert transactions based on graph generation networks. By adjusting the structural features between different addresses, our method enhances the security of multiple interrelated covert transactions. Experimental analysis on the Bitcoin Testnet verifies the security and the efficiency of the proposed methods.

Aligning users across networks using graph representation learning has been found effective where the alignment is accomplished in a low-dimensional embedding space. Yet, highly precise alignment remains challenging, especially for nodes with long-range connectivity to labeled anchors. To alleviate this limitation, we propose WL-Align which employs a regularized representation learning framework to learn distinctive node representations. It extends the Weisfeiler-Lehman Isormorphism Test and learns the alignment in alternating phases of “across-network Weisfeiler-Lehman relabeling” and “proximity-preserving representation learning”. The across-network Weisfeiler-Lehman relabeling is achieved through iterating the anchor-based label propagation and a similarity-based hashing to exploit the known anchors’ connectivity to different nodes in an efficient and robust manner. The representation learning module preserves the second-order proximity within individual networks and is regularized by the across-network Weisfeiler-Lehman hash labels. Extensive experiments on real-world and synthetic datasets have demonstrated that our proposed WL-Align outperforms the state-of-the-art methods, achieving significant performance improvements in the “exact matching” scenario.

Inspired by the impressive success of contrastive learning (CL), a variety of graph augmentation strategies have been employed to learn node representations in a self-supervised manner. Existing methods construct the contrastive samples by adding perturbations to the graph structure or node attributes. Although impressive results are achieved, it is rather blind to the wealth of prior information assumed: with the increase of the perturbation degree applied on the original graph: 1) the similarity between the original graph and the generated augmented graph gradually decreases and 2) the discrimination between all nodes within each augmented view gradually increases. In this article, we argue that both such prior information can be incorporated (differently) into the CL paradigm following our general ranking framework. In particular, we first interpret CL as a special case of learning to rank (L2R), which inspires us to leverage the ranking order among positive augmented views. Meanwhile, we introduce a self-ranking paradigm to ensure that the discriminative information among different nodes can be maintained and also be less altered to the perturbations of different degrees. Experiment results on various benchmark datasets verify the effectiveness of our algorithm compared with the supervised and unsupervised models.

Molecular structures and Drug-Drug Interactions (DDI) are recognized as important knowledge to guide medication recommendation (MR) tasks, and medical concept embedding has been applied to boost their performance. Though promising performance has been achieved by leveraging Graph Neural Network (GNN) models to encode the molecular structures of medications or/and DDI, we observe that existing models are still defective: 1) to differentiate medications with similar molecules but different functionality; or/and 2) to properly capture the unintended reactions between drugs in the embedding space. To alleviate this limitation, we propose Carmen, a cautiously designed graph embedding-based MR framework. Carmen consists of four components, including patient representation learning, context information extraction, a context-aware GNN, and DDI encoding. Carmen incorporates the visit history into the representation learning of molecular graphs to distinguish molecules with similar topology but dissimilar activity. Its DDI encoding module is specially devised for the non-transitive interaction DDI graphs. The experiments on real-world datasets demonstrate that Carmen achieves remarkable performance improvement over state-of-the-art models and can improve the safety of recommended drugs with a proper DDI graph encoding.

Multivariate time series (MTS) analysis and forecasting are crucial in many real-world applications, such as smart traffic management and weather forecasting. However, most existing work either focuses on short sequence forecasting or makes predictions predominantly with time domain features, which is not effective at removing noises with irregular frequencies in MTS. Therefore, we propose WaveForM, an end-to-end graph enhanced Wavelet learning framework for long sequence FORecasting of MTS. WaveForM first utilizes Discrete Wavelet Transform (DWT) to represent MTS in the wavelet domain, which captures both frequency and time domain features with a sound theoretical basis. To enable the effective learning in the wavelet domain, we further propose a graph constructor, which learns a global graph to represent the relationships between MTS variables, and graph-enhanced prediction modules, which utilize dilated convolution and graph convolution to capture the correlations between time series and predict the wavelet coefficients at different levels. Extensive experiments on five real-world forecasting datasets show that our model can achieve considerable performance improvement over different prediction lengths against the most competitive baseline of each dataset.

Several self-supervised representation learning methods have been proposed for reinforcement learning (RL) with rich observations. For real world applications of RL, recovering underlying latent states is crucial, particularly when sensory inputs can contain irrelevant and exogenous information. In this work, we study how information bottlenecjs can be used to construct latent states efficiently in the presence of task irrelevant information. We propose architectures that utilize variational and discrete information bottleneck, coined as RepDIB, to learn structured factorized representations. Exploiting the expressiveness bought by factorized representations, we introduce a simple, yet effective, bottleneck that can be integrated with any existing self supervised objective for RL. We demonstrate this across several online and offline RL benchmarks, along with a real robot arm task, where we find that compressed representations with RepDIB can lead to strong performance improvements, as the learnt bottlenecks can help predict only the relevant state, while ignoring irrelevant information.

Learning to control an agent from offline data collected in a rich pixel-based visual observation space is vital for real-world applications of reinforcement learning (RL). A major challenge in this setting is the presence of input information that is hard to model and irrelevant to controlling the agent. This problem has been approached by the theoretical RL community through the lens of *exogenous information*, i.e., any control-irrelevant information contained in observations. For example, a robot navigating in busy streets needs to ignore irrelevant information, such as other people walking in the background, textures of objects, or birds in the sky. In this paper, we focus on the setting with visually detailed exogenous information and introduce new offline RL benchmarks that offer the ability to study this problem. We find that contemporary representation learning techniques can fail on datasets where the noise is a complex and time-dependent process, which is prevalent in practical applications. To address these, we propose to use multi-step inverse models to learn Agent-Centric Representations for Offline-RL (ACRO). Despite being simple and reward-free, we show theoretically and empirically that the representation created by this objective greatly outperforms baselines.

While bisimulation-based approaches hold promise for learning robust state representations for Reinforcement Learning (RL) tasks, their efficacy in offline RL tasks has not been up to par. In some instances, their performance has even significantly underperformed alternative methods. We aim to understand why bisimulation methods succeed in online settings, but falter in offline tasks. Our analysis reveals that missing transitions in the dataset are particularly harmful to the bisimulation principle, leading to ineffective estimation. We also shed light on the critical role of reward scaling in bounding the scale of bisimulation measurements and of the value error they induce. Based on these findings, we propose to apply the expectile operator for representation learning to our offline RL setting, which helps to prevent overfitting to incomplete data. Meanwhile, by introducing an appropriate reward scaling strategy, we avoid the risk of feature collapse in representation space. We implement these recommendations on two state-of-the-art bisimulation-based algorithms, MICo and SimSR, and demonstrate performance gains on two benchmark suites: D4RL and Visual D4RL. Codes are provided at \url{https://github.com/zanghyu/Offline_Bisimulation}.

The interpretability of policies remains an important challenge in Deep Reinforcement Learning (DRL). This paper explores interpretable DRL via representing policy by Differentiable Inductive Logic Programming (DILP) and provides a theoretical and empirical study of DILP-based policy learning from an optimization perspective. We first identified a fundamental fact that DILP-based policy learning should be solved as a constrained policy optimization problem. We then proposed to use Mirror Descent for policy optimization (MDPO) to deal with the constraints of DILP-based policies. We derived the closed-form regret bound of MDPO with function approximation, which is helpful to the design of DRL frameworks. Moreover, we studied the convexity of DILP-based policy to further verify the benefits gained from MDPO. Empirically, we experimented MDPO, its on-policy variant, and 3 mainstream policy learning methods, and the results verified our theoretical analysis.

The alignment of multiple multi-relational networks, such as knowledge graphs, is vital for many AI applications. In comparison with existing GCNs which cannot fully utilize relational information of multiple types, we propose a relation-aware graph convolutional network (ERGCN), which is equipped with both entity convolution and relation convolution to learn the entity embeddings and relation embeddings simultaneously. The role discrimination and translation property of knowledge graphs are adopted in the entity convolutional process to incorporate the relation information. To facilitate the relation convolution, we construct quadruples to model the connection between a pair of relations thus to determine their neighborhood, which also enables the relation convolution to be conducted in an efficient way. Thereafter, AERGCN, the alignment framework based on ERGCN, is developed for multi-relational network alignment tasks. Anchors are used to supervise the objective function, which aims at minimizing the distances between anchors and to generate new cross-network triplets to build a bridge between different knowledge graphs at the level of triplet to improve the performance of alignment. Experiments on real-world datasets show that the proposed solutions outperform the competitive baselines in terms of link prediction, entity alignment, and relation alignment.

Social network alignment aims at aligning person identities across social networks. Embedding based models have been shown effective for the alignment where the structural proximity preserving objective is typically adopted for the model training. With the observation that “overly-close” user embeddings are unavoidable for such models causing alignment inaccuracy, we propose a novel learning framework which tries to enforce the resulting embeddings to be more widely apart among the users via the introduction of carefully implanted pseudo anchors. We further proposed a meta-learning algorithm to guide the updating of the pseudo anchor embeddings during the learning process. The proposed intervention via the use of pseudo anchors and meta-learning allows the learning framework to be applicable to a wide spectrum of network alignment methods. We have incorporated the proposed learning framework into several state-of-the-art models. Our experimental results demonstrate its efficacy where the methods with the pseudo anchors implanted can outperform their counterparts without pseudo anchors by a fairly large margin, especially when there only exist very few labeled anchors.

This work explores how to learn robust and generalizable state representation from image-based observations with deep reinforcement learning methods. Addressing the computational complexity, stringent assumptions and representation collapse challenges in existing work of bisimulation metric, we devise Simple State Representation (SimSR) operator. SimSR enables us to design a stochastic approximation method that can practically learn the mapping functions (encoders) from observations to latent representation space. In addition to the theoretical analysis and comparison with the existing work, we experimented and compared our work with recent state-of-the-art solutions in visual MuJoCo tasks. The results shows that our model generally achieves better performance and has better robustness and good generalization.

Goal-conditioned reinforcement learning (RL) is a promising direction for training agents that are capable of solving multiple tasks and reach a diverse set of objectives. How to \textit{specify} and \textit{ground} these goals in such a way that we can both reliably reach goals during training as well as generalize to new goals during evaluation remains an open area of research. Defining goals in the space of noisy, high-dimensional sensory inputs is one possibility, yet this poses a challenge for training goal-conditioned agents, or even for generalization to novel goals. We propose to address this by learning compositional representations of goals and processing the resulting representation via a discretization bottleneck, for coarser specification of goals, through an approach we call DGRL. We show that discretizing outputs from goal encoders through a bottleneck can work well in goal-conditioned RL setups, by experimentally evaluating this method on tasks ranging from maze environments to complex robotic navigation and manipulation tasks. Additionally, we show a theoretical result which bounds the expected return for goals not observed during training, while still allowing for specifying goals with expressive combinatorial structure.

Network alignment is a critical task in a wide variety of fields. Many existing works leverage on representation learning to accomplish this task without eliminating domain representation bias induced by domain-dependent features, which yield inferior alignment performance. This paper proposes a unified deep architecture (DANA) to obtain a domain-invariant representation for network alignment via an adversarial domain classifier. Specifically, we employ the graph convolutional networks to perform network embedding under the domain adversarial principle, given a small set of observed anchors. Then, the semi-supervised learning framework is optimized by maximizing a posterior probability distribution of observed anchors and the loss of a domain classifier simultaneously. We also develop a few variants of our model, such as, direction-aware network alignment, weight-sharing for directed networks and simplification of parameter space. Experiments on three real-world social network datasets demonstrate that our proposed approaches achieve state-of-the-art alignment results.

Next Point-of-interest (POI) recommendation is a key task in improving location-related customer experiences and business operations, but yet remains challenging due to the substantial diversity of human activities and the sparsity of the check-in records available. To address these challenges, we proposed to explore the category hierarchy knowledge graph of POIs via an attention mechanism to learn the robust representations of POIs even when there is insufficient data. We also proposed a spatial-temporal decay LSTM and a Discrete Fourier Series-based periodic attention to better facilitate the capturing of the personalized behavior pattern. Extensive experiments on two commonly adopted real-world location-based social networks (LBSNs) datasets proved that the inclusion of the aforementioned modules helps to boost the performance of next and next new POI recommendation tasks significantly. Specifically, our model in general outperforms other state-of-the-art methods by a large margin.

Medical concept embedding which aims at learning interpretable low-dimensional representations of medical codes has become one of the key technologies to enable the machine (deep) learning models to imitate the doctor’s cognitive reasoning process in a variety of clinical tasks. Most existing works focus on leveraging the medical ontology to get the representations but remains ineffective in dealing with 1) the inconsistency between the knowledge of the medical ontology and the observations in health records, and 2) the deficiency of discovering the relations among multi-types of medical concepts. To address these challenges, this paper proposes MrER(Multi-relational EHR representation learning method). It’s a heterogeneous graph convolutional network with a self-adaptive adjacency matrix, to infer the multi-relations among different types of medical concepts and align them in the same subspace for the complex knowledge inference. Moreover, an temporal convolutional network is introduced to capture the dependency patterns in the sequence of medical records. The entire framework is trained in an end-to-end fashion. The experimental results show that MrER achieves competitive performance advantages in sequential diagnosis prediction task in comparison with state-of-the-art methods and the learned embeddings have good interpretability regarding the relationship between medical codes.

The increasing popularity and diversity of social media sites, has encouraged many people to participate in different online social networks to enjoy a variety of services. Linking the same users across different social networks, also known as social network alignment, is a critical task of great research challenges. Many existing works usually focus on finding a projection function from one subspace to another for network alignment, however, the projection functions proposed in their papers are independent and updated individually, which could not effectively exploit the non-parallel data, and yield inferior alignment performance. In this paper, we propose a Shared-latent Variable Network Alignment (SVNA) architecture to effectively exploit the non-parallel data for network alignment, and jointly train projection functions and decoders in a unified framework with the shared latent variable z. Specifically, SVNA first employs the graph convolutional networks to preserve the structural information of the network. By introducing the shared latent variable z, SVNA simultaneously integrates two projection functions and two decoders for jointly training. Both projection functions and decoders share the same latent space, therefore both projection directions can learn from the non-parallel data more effectively. Thereafter, SVNA utilizes the Generative Adversarial Networks (GANs) framework to further train the projection functions, and adopts a probability-based semi-supervised method to achieve the network alignment. Experiments on three real-world datasets show that SVNA generally outperforms the state-of-the-art methods in network alignment task.

WiFi access points are sources of considerable security risks as the wireless signals have the potential to leak important private information such as passwords. This article examines the security issues posed by point-of-sale (POS) terminals which are widely used in WiFi-covered environments, such as restaurants, banks, and libraries. In particular, we envisage an attack model on passwords entered on POS terminals. We put forward the WiPOS, a password inference system based on wireless signals. Specifically, the WiPOS is a device-free system that uses two commercial off-the-shelf (COTS) devices to collect WiFi signals. Implementing a new keystroke segmentation algorithm and adopting support vector machine (SVM) classifiers with global alignment kernel (GAK), the WiPOS achieves improvement on both keystroke recognition and password prediction. The experimental results show that the WiPOS can achieve more than 73% accuracy for 6-digit password with the top 100 candidates. This article calls the community to take a closer look at the risks posed by the current ubiquitous WiFi devices.

Aligning users across different social networks has become increasingly studied as an important task to social network analysis. In this paper, we propose a novel representation learning method that mainly exploits social structures for the network alignment. In particular, the proposed network embedding framework models the follower-ship and followee-ship of each user explicitly as input and output context vectors, while preserving the proximity of users with “similar” followers and followees in the embedded space. We incorporate both known and predicted user anchors across the networks as constraints to facilitate the transfer of context information to achieve accurate user alignment. Both network embedding and user alignment are inferred under a unified optimization framework with negative sampling adopted to ensure scalability. Also, variants of the proposed framework, including the incorporation of higher-order structural features, are also explored for further boosting the alignment accuracy. Extensive experiments on large-scale social and academia network datasets demonstrate the efficacy of our proposed model compared with state-of-the-art methods.

Network embedding is capable of providing low-dimensional feature representations for various machine learning applications. Current work focuses on: 1) designing the embedding as an unsupervised learning task to explicitly preserve the structural connectivity in the network or 2) generating the embedding as a by-product during the supervised learning of a specific discriminative task in a deep neural network. In this paper, we aim to take advantage of these two lines of research in the view of multi-output learning. That is, we propose a generative adversarial network embedding (GANE) model to adapt the generative adversarial framework to achieve the network embedding learning during the specific machine learning tasks. GANE has a generator to generate link edges, and a discriminator to distinguish the generated link edges from real connections (edges) in the network. Wasserstein-1 distance is adopted to train the generator to gain better stability. GANE is further extended by utilizing the pairwise connectivity of vertices to preserve the structural information in the original network. Experiments with real-world network data sets demonstrate that our models constantly outperform state-of-the-art solutions with significant improvements for the tasks of link prediction, clustering, and network alignment.

Alignment of multiple multi-relational networks, such as knowledge graphs, is vital for AI applications. Different from the conventional alignment models, we apply the graph convolutional network (GCN) to achieve more robust network embedding for the alignment task. In comparison with existing GCNs which cannot fully utilize multi-relation information, we propose a vectorized relational graph convolutional network (VR-GCN) to learn the embeddings of both graph entities and relations simultaneously for multi-relational networks. The role discrimination and translation property of knowledge graphs are adopted in the convolutional process. Thereafter, AVR-GCN, the alignment framework based on VR-GCN, is developed for multi-relational network alignment tasks. Anchors are used to supervise the objective function which aims at minimizing the distances between anchors, and to generate new cross-network triplets to build a bridge between different knowledge graphs at the level of triplet to improve the performance of alignment. Experiments on real-world datasets show that the proposed solutions outperform the state-of-the-art methods in terms of network embedding, entity alignment, and relation alignment.

Next and next new point-of-interest (POI) recommendation are essential instruments in promoting customer experiences and business operations related to locations. However, due to the sparsity of the check-in records, they still remain insufficiently studied. In this article, we propose to utilize personalized latent behavior patterns learned from contextual features, e.g., time of day, day of week, and location category, to improve the effectiveness of the recommendations. Two variations of models are developed, including GPDM, which learns a fixed pattern distribution for all users; and PPDM, which learns personalized pattern distribution for each user. In both models, a soft-max function is applied to integrate the personalized Markov chain with the latent patterns, and a sequential Bayesian Personalized Ranking (S-BPR) is applied as the optimization criterion. Then, Expectation Maximization (EM) is in charge of finding optimized model parameters. Extensive experiments on three large-scale commonly adopted real-world LBSN data sets prove that the inclusion of location category and latent patterns helps to boost the performance of POI recommendations. Specifically, our models in general significantly outperform other state-of-the-art methods for both next and next new POI recommendation tasks. Moreover, our models are capable of making accurate recommendations regardless of the short/long duration or distance.

Most existing solutions for the alignment of multi-relational networks, such as multi-lingual knowledge bases, are ``translation''-based which facilitate the network embedding via the trans-family, such as TransE. However, they cannot address triangular or other structural properties effectively. Thus, we propose a non-translational approach, which aims to utilize a probabilistic model to offer more robust solutions to the alignment task, by exploring the structural properties as well as leveraging on anchors to project each network onto the same vector space during the process of learning the representation of individual networks. The extensive experiments on four multi-lingual knowledge graphs demonstrate the effectiveness and robustness of the proposed method over a set of state-of-the-art alignment methods.

Next Point-of-interest (POI) recommendation has become an important task for location-based social networks (LBSNs). However, previous efforts suffer from the high computational complexity and the transition pattern between POIs has not been well studied. In this paper, we propose a two-fold approach for next POI recommendation. First, the preferred next category is predicted by using a third-rank tensor optimized by a Listwise Bayesian Personalized Ranking (LBPR) approach. Specifically we introduce two functions, namely Plackett-Luce model and cross entropy, to generate the likelihood of ranking list for posterior computation. Then POI candidates filtered by the predicated category are ranked based on the spatial influence and category ranking influence. Extensive experiments on two real-world datasets demonstrate the significant improvements of our methods over several state-of-the-art methods.

Most of the existing multi-relational network embedding methods, e.g., TransE, are formulated to preserve pair-wise connectivity structures in the networks. With the observations that significant triangular connectivity structures and parallelogram connectivity structures found in many real multi-relational networks are often ignored and that a hard-constraint commonly adopted by most of the network embedding methods is inaccurate by design, we propose a novel representation learning model for multi-relational networks which can alleviate both fundamental limitations. Scalable learning algorithms are derived using the stochastic gradient descent algorithm and negative sampling. Extensive experiments on real multi-relational network datasets of WordNet and Freebase demonstrate the efficacy of the proposed model when compared with the state-of-the-art embedding methods.

Recently, location-based services (LBSs) have been increasingly popular for people to experience new possibilities, for example, personalized point-of-interest (POI) recommendations that leverage on the overlapping of user trajectories to recommend POI collaboratively. POI recommendation is yet challenging as it suffers from the problems known for the conventional recommendation tasks such as data sparsity and cold start, and to a much greater extent. In the literature, most of the related works apply collaborate filtering to POI recommendation while overlooking the personalized time-variant human behavioral tendency. In this article, we put forward a fourth-order tensor factorization-based ranking methodology to recommend users their interested locations by considering their time-varying behavioral trends while capturing their long-term preferences and short-term preferences simultaneously. We also propose to categorize the locations to alleviate data sparsity and cold-start issues, and accordingly new POIs that users have not visited can thus be bubbled up during the category ranking process. The tensor factorization is carefully studied to prune the irrelevant factors to the ranking results to achieve efficient POI recommendations. The experimental results validate the efficacy of our proposed mechanism, which outperforms the state-of-the-art approaches significantly.

In this paper, we address the problem of personalized next Point-of-interest (POI) recommendation which has become an important and very challenging task in location-based social networks (LBSNs), but not well studied yet. With the conjecture that, under different contextual scenario, human exhibits distinct mobility patterns, we attempt here to jointly model the next POI recommendation under the influence of user's latent behavior pattern. We propose to adopt a third-rank tensor to model the successive check-in behaviors. By incorporating softmax function to fuse the personalized Markov chain with latent pattern, we furnish a Bayesian Personalized Ranking (BPR) approach and derive the optimization criterion accordingly. Expectation Maximization (EM) is then used to estimate the model parameters. Extensive experiments on two large-scale LBSNs datasets demonstrate the significant improvements of our model over several state-of-the-art methods.

Location-based social networks have been increasingly used to experience users new possibilities, including personalized point-of-interest (POI) recommendation services which leverages on the overlapping of user trajectories to recommend POI collaboratively. POI recommendation is challenging as it does not just suffers from the problems known for collaborative filtering such as data sparsity and cold-start, but to a much greater extent. Most of the related works apply the conventional recommendation approaches to POI recommendation while overlooking the personalized time-variant human behavioral tendency. In this paper, we put forward a tensor factorization-based ranking methodology to recommend users their interested locations by considering their time-varying behavioral trends. We also propose to categorize the locations to address data sparsity and cold-start issues, and accordingly new locations the user have not been visited can thus be bubbled up during ranking the location candidates. The tensor factorization is carefully studied to prune the irrelevant factors to the ranking results to achieve efficient POI recommendation. The experimental results validate the effectiveness of our proposed mechanism which outperforms the state-of-the-art approaches by over 8% for precision.

Vehicular ad hoc network(VANET) is increasingly studied recently due to its promising benefits to the urban life. The roadside unit(RSU) is one of the most important components for VANET. An effective deployment of RSUs will enhance the efficiency of the data delivery in VANET. In this paper, we propose to address the RSU deployment problem by formulating it as a mobility clustering problem. We adopt affinity propagation(AP) algorithm to capture the spatial temporal mobility influence in different time period. The union set of the obtained clustering centers which have the maximum influence to its cluster members is then the solution to RSU deployment. To validate our proposed deployment scheme, we evaluate the performance of the network with the deployed RSUs in terms of delivery ratio, average delay and hop count by adapting the conventional GPSR to VANET as the routing protocol. The simulation results show our proposed approach achieves near optimal solutions with much lower complexity compared to the exhaustive method.

Vehicular networks have been increasingly used for applications like road infrastructure monitoring and traffic jam detection, etc. Data forwarding is a well-known challenging problem in vehicular networks, which suffers from delay and error due to the frequent network disruption and fast topological change. The minimizations of the delivery delay and network cost are both central to data forwarding in vehicular networks. However, previous works usually focus on only one of the two objectives and most of them do not make good use of vehicle trajectory information. In this paper, we formulate the V2V (vehicle to vehicle) data forwarding problem as a novel multi-objective Markov Decision Process (MDP). We exploit the vehicle trajectory information and traffic statistics to estimate the parameters of the MDP (i.e., transition probabilities, rewards). The optimal routing policy is then developed by solving the multi-objective MDP. We conduct extensive simulations on a taxi network in a mega-city, the experimental results validate the effectiveness of our proposed mechanism.

In Vehicular Ad Hoc Networks (VANETs), moving vehicles are considered as mobile nodes in the network and they are connected to each other via wireless links when they are within the communication radius of each other. Efficient message delivery in VANETs is still a very challenging research issue. In this paper, a Q-learning based routing protocol (i.e., QGrid) is introduced to help to improve the message delivery from mobile vehicles to a specific location. QGrid considers both macroscopic and microscopic aspects when making the routing decision, while the traditional routing methods focus on computing meeting information between different vehicles. QGrid divides the region into different grids. The macroscopic aspect determines the optimal next-hop grid and the microscopic aspect determines the specific vehicle in the optimal next-hop grid to be selected as next-hop vehicle. QGrid computes the Q-values of different movements between neighboring grids for a given destination via Q-learning. Each vehicle stores Q-value table learned offline, then selects optimal next-hop grid by querying Q-value table. Inside the selected next-hop grid, we either greedily select the nearest neighboring vehicle to the destination or select the neighboring vehicle with highest probability of moving to the optimal next-hop grid predicted by the two-order Markov chain. The performance of QGrid is evaluated by using real life trajectory GPS data of Shanghai taxies. Simulation comparison among QGrid and other existing position-based routing protocols confirms the advantages of proposed QGrid routing protocol for VANETs.

Delay Tolerant Networks (DTNs) are intermittently connected networks, such as mobile social networks formed by human-carried mobile devices. Routing in such mobile social DTNs is very challenging as it must handle network partitioning, long delays, and dynamic topology. Recently, social-based approaches, which attempt to exploit social behaviors of DTN nodes to make better routing decision, have drawn tremendous interests in DTN routing design. In this paper, we propose a novel social-based routing approach for mobile social DTNs, where a new metric social energy is introduced to quantify the ability of a node to forward packets to others, inspired by general laws in particle physics. Social energy is generated via node encounters and shared by the communities of encountering nodes. Similar to the radiation of energy in physics, the social energy of any node decays over time. Our proposed Social Energy Based Routing (SEBAR) protocol considers social energy of encountering nodes and is in favor of the Dode with a higher social energy in its or the destination's social community. Our simulations with real-life wireless traces demonstrate the efficiency and effectiveness of SEBAR method by comparing It with several existing DTN routing schemes.

This paper presents the magnetic field modeling-based energy efficient routing for WSN which models the routing problem as a current-carrying solenoid and a free to turn magnet put into a uniform magnetic field respectively. The optimal routing is then performed through the path with the maximum generated magnetic field intensity for the solenoid technique and with minimum torque for the magnetic torque technique. The proposed approaches have been incorporated into AODV. The case studies and the simulation show that the load distribution of our proposed methodology is more balanced than that of the conventional AODV.

Partially observable Markov decision process (POMDP) is a commonly adopted mathematical framework for solving planning problems in stochastic environments. However, computing the optimal policy of POMDP for large-scale problems is known to be intractable, where the high dimensionality of the underlying belief space is one of the major causes. In this paper, we propose a hybrid approach that integrates two different approaches for reducing the dimensionality of the belief space: 1) belief compression and 2) value-directed compression. In particular, a novel orthogonal nonnegative matrix factorization is derived for the belief compression, which is then integrated in a value-directed framework for computing the policy. In addition, with the conjecture that a properly partitioned belief space can have its per-cluster intrinsic dimension further reduced, we propose to apply a k-means-like clustering technique to partition the belief space to form a set of sub-POMDPs before applying the dimension reduction techniques to each of them. We have evaluated the proposed belief compression and clustering approaches based on a set of benchmark problems and demonstrated their effectiveness in reducing the cost for computing policies, with the quality of the policies being retained.

High dimensionality of POMDP's belief state space is one major cause that makes the underlying optimal policy computation intractable. Belief compression refers to the methodology that projects the belief state space to a low-dimensional one to alleviate the problem. In this paper, we propose a novel orthogonal non-negative matrix factorization (O-NMF) for the projection. The proposed O-NMF not only factors the belief state space by minimizing the reconstruction error, but also allows the compressed POMDP formulation to be efficiently computed (due to its orthogonality) in a value-directed manner so that the value function will take same values for corresponding belief states in the original and compressed state spaces. We have tested the proposed approach using a number of benchmark problems and the empirical results confirms its effectiveness in achieving substantial computational cost saving in policy computation.

Principal Component Analysis (PCA) and other SVD related approaches are commonly used in dimension reduction and reconstruction of images. However, as linear methods they may not be appropriate for some non-linear cases. Recently a new approach named as Exponential Family Principle Component Analysis (E-PCA) is proposed for non-linear compression and has been successfully used to solve the belief states' dimension reduction of Partially observable Markov Decision Process (POMDP). In this paper, we attempted to adapt E-PCA to image compression and reconstruction due to the reason that it can guarantee nonnegative reconstruction and is fit for some nonlinearly distributed data. The original E-PCA formulations are also simplified in this paper to accelerate the parameters learning process. Experiments are performed on some standard image data sets to verify the effectiveness of E-PCA on image compression. From the experimental results, we can conclude that the new adaption of E-PCA on image compression is particularly effective when the image data follows some kinds of distribution.