Feature transformation enhances downstream task performance by generating informative features through mathematical feature crossing. Despite the advancements in deep learning, feature transformation remains essential, particularly for structured data, where deep models often struggle to capture complex feature interactions effectively. Prior literature on automated feature transformation has achieved notable success but often relies on heuristics or exhaustive searches, leading to inefficient and time-consuming processes. Recent works employ reinforcement learning (RL) to enhance traditional approaches through a more effective trial-and-error way. However, two key limitations remain: 1) Dynamic feature expansion during the transformation process, which introduces instability and increases the time complexity of the learning procedure for RL agents; 2) Insufficient cooperation and communication between agents, which results in suboptimal feature crossing operations and degraded model performance. To address them, we propose a novel heterogeneous multi-agent RL framework to enable cooperative and scalable feature transformation. The framework comprises three heterogeneous agents, grouped into two types, each designed to select essential features and operations for feature crossing. To enhance communication among these agents, we implement a shared critic mechanism that facilitates information exchange during the feature transformation process. This collaboration enables the agents to learn more intelligent and effective transformation policies. To handle the dynamically expanding feature space, we tailor multi-head attention-based feature agents to select suitable features for feature crossing. This design facilitates scalable decision-making and effective candidate selection based on comprehensive global feature space information. Additionally, we introduce a state encoding technique during the optimization process to stabilize and enhance the learning dynamics of the RL agents, resulting in more robust and reliable transformation policies. Finally, we conduct extensive experiments to validate the effectiveness, efficiency, robustness, and interpretability of our model. Our code and dataset are publicly available on GitHub.

Abstract Feature generation can significantly enhance learning outcomes, particularly for tasks with limited data. An effective way to improve feature generation is to expand the current feature space using existing features and enriching the informational content. However, generating new, interpretable features usually requires domain-specific knowledge on top of the existing features. In this paper, we introduce a Retrieval-Augmented Feature Generation method, RAFG, to generate useful and explainable features specific to domain classification tasks. To increase the interpretability of the generated features, we conduct knowledge retrieval among the existing features in the domain to identify potential feature associations. These associations are expected to help generate useful features. Moreover, we develop a framework based on large language models (LLMs) for feature generation with reasoning to evaluate their semantic relevance, causal alignment, and expected utility for the downstream task. To mitigate the risk of overconfident or unsupported reasoning, we further introduce a counterfactual validation mechanism that compares reasoning-based predictions with observed performance changes. Experiments across several datasets in medical, economic, and geographic domains show that our RAFG method can produce high-quality, meaningful features and significantly improve classification performance compared with baseline methods.

Tabular data is one of the most widely used formats across industries, driving critical applications in areas such as finance, healthcare, and marketing. In the era of data-centric AI, improving data quality and representation has become essential for enhancing model performance, particularly in applications centered around tabular data. This survey examines the key aspects of tabular data-centric AI, emphasizing feature selection and feature generation as essential techniques for data space refinement. We provide a systematic review of feature selection methods, which identify and retain the most relevant data attributes, and feature generation approaches, which create new features to simplify the capture of complex data patterns. This survey offers a comprehensive overview of current methodologies through an analysis of recent advancements, practical applications, and the strengths and limitations of these techniques. Finally, we outline open challenges and suggest future perspectives to inspire continued innovation in this field.

Feature space is an environment where data points are vectorized to represent the original dataset. Reconstructing a good feature space is essential to augment the AI power of data, improve model generalization, and increase the availability of downstream ML models. Existing literature, such as feature transformation and feature selection, is labor-intensive (e.g., heavy reliance on empirical experience) and mostly designed for tabular data. Moreover, these methods regard data samples as independent, which ignores the unique topological structure when applied to graph data, thus resulting in a suboptimal reconstruction feature space. Can we consider the topological information to automatically reconstruct feature space for graph data without heavy experiential knowledge? To fill this gap, we leverage topology-aware reinforcement learning to automate and optimize feature space reconstruction for graph data. Our approach combines the extraction of core subgraphs to capture essential structural information with a graph neural network to encode topological features and reduce computing complexity. Then we introduce three reinforcement agents within a hierarchical structure to systematically generate meaningful features through an iterative process, effectively reconstructing the feature space. This framework provides a principled solution for attributed graph feature space reconstruction. The extensive experiments demonstrate the effectiveness and efficiency of including topological awareness on three widely used downstream tasks (node classification, link prediction, and graph classification). Our code and data are available at https://tinyurl.com/graphFT123 .

Iterative feature space optimization includes continuously evaluating and refining the feature space to improve downstream task performance. However, existing methods commonly suffer from three major limitations: 1) ignoring differences between samples leads to evaluation bias; 2) the feature space is overly tailored to specific models, resulting in overfitting and poor generalization; and 3) retraining the evaluator from scratch in each iteration significantly reduces overall efficiency. To bridge these gaps, we introduce EASE (gEneralized Adaptive feature Space Evaluator), a generalized framework for efficient and objective evaluation of iteratively generated feature spaces. This framework includes two key components: Feature-Sample Subspace Generator and Contextual Attention Evaluator. The first component aims to mitigate evaluation bias by decoupling the information distribution within the feature space. To achieve this, based on feedback from the subsequent evaluator, we identify the samples most challenging for evaluation and the features most relevant to prediction tasks. The second component intends to incrementally capture evolving patterns of the feature space for efficient evaluation. Specifically, we propose a weighted-sharing multi-head attention mechanism to encode the feature space into an embedding vector for evaluation, and update the evaluator incrementally to retain prior knowledge while incorporating new information. Extensive experiments on fifteen public datasets demonstrate the effectiveness of EASE. We have released our code and data to the public.

Machine learning traditionally emphasizes developing models for given datasets, but real-world data is often messy, making model improvement insufficient for enhancing performance. AI for data editing (AI4DE) is an emerging field that systematically improves datasets, leading to significant practical ML advancements. While experienced data scientists have manually refined datasets through trial-and-error and intuition, AI4DE approaches data enhancement as a systematic engineering discipline. AI4DE represents a shift from focusing on models to the underlying data used for training and evaluation. Despite the dominance of common model architectures and predictable scaling rules, building and using datasets remain labor-intensive and costly, lacking infrastructure and best practices. The AI4DE movement aims to develop efficient, high-productivity open data engineering tools for modern ML systems. This workshop seeks to foster an interdisciplinary AI4DE community to address practical data challenges, including data collection, generation, labeling, preprocessing, augmentation, quality evaluation, debt, and governance. By defining and shaping the AI4DE movement, this workshop aims to influence the future of AI and ML, inviting interested parties to contribute through paper submissions

Multivariate time series forecasting in practical deployment faces a critical challenge termed Variable Subset Forecasting (VSF), where certain variables accessible during training are entirely missing during inference. This creates a stark discrepancy between the training (source domain with full variables) and inference (target domain with partial variables) environments, disrupting cross-variable dependencies and fragmenting global temporal patterns. Existing imputation methods, limited to transferring local knowledge (e.g., temporal neighbors or pairwise correlations), fail to capture essential global dynamics, leading to severe performance degradation under distribution shifts. To address these challenges, we redefine VSF as a cross-domain knowledge transfer problem and propose VIDA, a framework that systematically transfers Variable Invariant knowledge from complete to partial observations through Domain Adaptation. Key to our approach is (1) Global time-frequency joint representation learning, which encodes temporal dynamics via dilated convolutions and captures low-frequency spectral consistency using Fourier neural operators, and (2) Sinkhorn-regularized distribution alignment to bridge non-overlapping feature supports across domains via optimal transport. Unlike imputation-first methods, VIDA enforces task-driven consistency by jointly optimizing predictions on reconstructed and original data, ensuring the transferred knowledge directly enhances forecasting robustness. Extensive experiments across four real-world datasets show that VIDA outperforms state-of-the-art imputation methods by 25% on average with partially observed variables. This work establishes a new paradigm for variable-missing scenarios by unifying imputation and forecasting through principled knowledge transfer.

Evaluation of Massive Open Online Course (MOOC) quality is crucial to enhance the educational resources, benefiting user services, and enhancing students’ learning efficiency. Despite achieving encouraging results, current efforts are hindered by complex relationships between entities and individual varies. To address the above problem, in this article, we frame the issue as a task of learning course representations and proceed to develop an U ser-Centric H ypergraph R epresentation L earning ( UHRL ) for online course quality evaluation. In particular, we initially construct a MOOC hypergraph to depict the interactions and connections between the entities and use cross-hyperedge alignment to reveal the semantics of courses. And then we incorporate an attention mechanism in the information transmission process to ensure semantic integrity. Furthermore, to tackle the bias of users’ preference, our framework exploits mutual information for preserving the fairness of representation learning. Finally, our comprehensive experiments on three real-world datasets confirm the effectiveness of our approach compared to cutting-edge methods in evaluating online course quality across various performance metrics.

In recent years, Large Language Models (LLMs) have become increasingly more powerful in their ability to complete complex tasks. One such task in which LLMs are often employed is scoring, i.e., assigning a numerical value from a certain scale to a subject. In this paper, we strive to understand how LLMs score, specifically in the context of empathy scoring. We develop a novel and comprehensive framework for investigating how effective LLMs are at measuring and scoring empathy of responses in dialogues, and what methods can be employed to deepen our understanding of LLM scoring. Our strategy is to approximate the performance of state-of-the-art and fine-tuned LLMs with explicit and explainable features. We train classifiers using various features of dialogues including embeddings, the Motivational Interviewing Treatment Integrity (MITI) Code, a set of explicit subfactors of empathy as proposed by LLMs, and a combination of the MITI Code and the explicit subfactors. Our results show that when only using embeddings, it is possible to achieve performance close to that of generic LLMs, and when utilizing the MITI Code and explicit subfactors scored by an LLM, the trained classifiers can closely match the performance of fine-tuned LLMs. We employ feature selection methods to derive the most crucial features in the process of empathy scoring. Our work provides a new perspective toward understanding LLM empathy scoring and helps the LLM community explore the potential of LLM scoring in social science studies.1

Textual information of data is of vital importance for data mining and feature engineering. However, existing methods focus on learning the data structures and overlook the textual information along with the data. Consequently, they waste this valuable resource and miss out on the deeper data relationships embedded within the texts. In this paper, we introduce Text-Informed Feature Generation (TIFG), a novel LLM-based text-informed feature generation framework. TIFG utilizes the textual information to generate features by retrieving possible relevant features within external knowledge with Retrieval Augmented Generation (RAG) technology. In this approach, the TIFG can generate new explainable features to enrich the feature space and further mine feature relationships. We design the TIFG to be an automated framework that continuously optimizes the feature generation process, adapts to new data inputs, and improves downstream task performance over iterations. A broad range of experiments in various downstream tasks showcases that our approach can generate high-quality and meaningful features, and is significantly superior to existing methods.

Recent advances in large language models (LLMs) have demonstrated their potential in handling complex reasoning tasks, which are usually achieved by constructing a thought chain to guide the model to solve the problem with multi-step thinking. However, existing methods often remain confined to previously explored solution spaces and thus overlook the critical blind spot within LLMs’ cognitive range. To address these issues, we design the Thought Space Explorer (TSE), a novel framework to expand and optimize thought structures to guide LLMs to explore their blind spots of thinking. By generating new reasoning steps and branches based on the original thought structure with various designed strategies, TSE broadens the thought space and alleviates the impact of blind spots for LLM reasoning. Experimental results on multiple levels of reasoning tasks demonstrate the efficacy of TSE. We also conduct extensive analysis to understand how structured and expansive thought can contribute to unleashing the potential of LLM reasoning capabilities.

Accurately predicting Drug-Drug Interactions (DDIs) is critical to designing effective drug combination therapies. Recently, Artificial Intelligence (AI)-powered DDI prediction approaches have emerged as a new paradigm. However, most existing methods oversimplify the complex hierarchical structure within molecules and overlook the multi-source heterogeneous information external to molecules, limiting their modeling and predictive capabilities. To address this, we propose a H ierarchical H eterogeneous graph learning framework for D DI prediction, namely H2D. H2D employs an internal-to-external, local-to-global hierarchical perspective, exploiting intra-molecular multi-granularity structures and inter-molecular biomedical interactions to mutually enhance across hierarchical levels. Extensive experimental results demonstrate H2D's effectiveness on three real-world DDI prediction tasks (binary-class, multi-class, and multi-label). In sum, H2D achieves state-of-the-art performance in DDI prediction by leveraging the multi-scale graph structures, opening up new avenues in AI-powered DDI prediction.

Machine learning traditionally emphasizes developing models for given datasets, but real-world data is often messy, making model improvement insufficient for enhancing performance. Data-Centric AI (DCAI) is an emerging field that systematically improves datasets, leading to significant practical ML advancements. While experienced data scientists have manually refined datasets through trial-and-error and intuition, DCAI approaches data enhancement as a systematic engineering discipline. DCAI represents a shift from focusing on models to the underlying data used for training and evaluation. Despite the dominance of common model architectures and predictable scaling rules, building and using datasets remain labor-intensive and costly, lacking infrastructure and best practices. The DCAI movement aims to develop efficient, high-productivity open data engineering tools for modern ML systems. This workshop seeks to foster an interdisciplinary DCAI community to address practical data challenges, including data collection, generation, labeling, preprocessing, augmentation, quality evaluation, debt, and governance. By defining and shaping the DCAI movement, this workshop aims to influence the future of AI and ML, inviting interested parties to contribute through paper submissions.

Tabular data are the most widely used data formats in almost every application domain, such as, biology, ecology, and material science. The purpose of tabular data-centric AI is to use AI to augment the predictive power of tabular data to get better AI. Tabular data-centric AI is essential because it can reconstruct distance measures, reshape discriminative patterns, and improve data AI readiness (structural, predictive, interaction, and expression levels), which is significant in industries and real-world deployments. Therefore, our tutorial is designed to capture the interest of professionals with expertise in artificial intelligence, machine learning, and data mining, as well as researchers engaged in specific application areas and interdisciplinary studies. Examples of such applications include quality control, predictive maintenance, supply chain optimization, process efficiency improvements, biomarker identification, material performance screening. In this tutorial, we will explore the emerging field of Tabular Data-Centric AI. Our discussion will provide a comprehensive overview of this domain: (1) We will demonstrate the different settings within this research domain based on distinct application scenarios. (2) We will identify and explain the significant challenges encountered in tabular data-centric AI. (3) We will highlight existing methods and benchmarks. (4) We will discuss future potential directions for this domain and examine its interconnections with other research areas. To enhance the learning experience, this tutorial will include a hands-on section designed to teach participants the fundamental aspects of developing, evaluating and visualizing techniques in tabular data-centric AI. After this tutorial, attendees will have a deep understanding of tabular data-centric AI research, including its key challenges, seminal techniques, and insights into integrating tabular data-centric AI into their own research.

Creating an effective representation space is crucial for mitigating the curse of dimensionality, enhancing model generalization, addressing data sparsity, and leveraging classical models more effectively. Recent advancements in automated feature engineering (AutoFE) have made significant progress in addressing various challenges associated with representation learning, issues such as heavy reliance on intensive labor and empirical experiences, lack of explainable explic-itness, and inflexible feature space reconstruction embedded into downstream tasks. However, these approaches are constrained by: 1) generation of potentially unintelligible and illogical reconstructed feature spaces, stemming from the neglect of expert-level cognitive processes; 2) lack of systematic exploration, which subsequently results in slower model convergence for identification of optimal feature space. To address these, we introduce an interaction-aware reinforced generation perspective. We redefine feature space reconstruction as a nested process of creating meaningful features and controlling feature set size through selection. We develop a hierarchical reinforcement learning structure with cascading Markov Decision Processes to automate feature and operation selection, as well as feature crossing. By incorporating statistical measures, we reward agents based on the interaction strength between selected features, resulting in intelligent and efficient exploration of the feature space that emulates human decision-making. Extensive experiments are conducted to validate our proposed approach.

Feature transformation aims to reconstruct an effective representation space by mathematically refining the existing features. It serves as a pivotal approach to combat the curse of dimensionality, enhance model generalization, mitigate data sparsity, and extend the applicability of classical models. Existing research predominantly focuses on domain knowledge-based feature engineering or learning latent representations. However, these methods, while insightful, lack full automation and fail to yield a traceable and optimal representation space. An indispensable question arises: Can we concurrently address these limitations when reconstructing a feature space for a machine learning task? Our initial work took a pioneering step towards this challenge by introducing a novel self-optimizing framework. This framework leverages the power of three cascading reinforced agents to automatically select candidate features and operations for generating improved feature transformation combinations. Despite the impressive strides made, there was room for enhancing its effectiveness and generalization capability. In this extended journal version, we advance our initial work from two distinct yet interconnected perspectives: 1) We propose a refinement of the original framework, which integrates a graph-based state representation method to capture the feature interactions more effectively and develop different Q-learning strategies to alleviate Q-value overestimation further. 2) We utilize a new optimization technique (actor-critic) to train the entire self-optimizing framework in order to accelerate the model convergence and improve the feature transformation performance. Finally, to validate the improved effectiveness and generalization capability of our framework, we perform extensive experiments and conduct comprehensive analyses. These provide empirical evidence of the strides made in this journal version over the initial work, solidifying our framework’s standing as a substantial contribution to the field of automated feature transformation. To improve the reproducibility, we have released the associated code and data by the Github link https://github.com/coco11563/TKDD2023_code.

Graph clustering has been a hot research topic and is widely used in many fields, such as community detection in social networks. Lots of works combining auto-encoder and graph neural networks have been applied to clustering tasks by utilizing node attributes and graph structure. These works usually assumed the inherent parameters (i.e., size and variance) of different clusters in the latent embedding space are homogeneous, and hence the assigned probability is monotonous over the Euclidean distance between node embeddings and centroids. Unfortunately, this assumption usually does not hold since the size and concentration of different clusters can be quite different, which limits the clustering accuracy. In addition, the node embeddings in deep graph clustering methods are usually L2 normalized so that it lies on the surface of a unit hyper-sphere. To solve this problem, we proposed D eep A daptive G raph C lustering via von Mises-Fisher distributions, namely DAGC. DAGC assumes the node embeddings H can be drawn from a von Mises-Fisher distribution and each cluster k is associated with cluster inherent parameters ρ k which includes cluster center μ and cluster cohesion degree κ. Then we adopt an EM-like approach (i.e., 𝒫( H | ρ ) and 𝒫( ρ | H ), respectively) to learn the embedding and cluster inherent parameters alternately. Specifically, with the node embeddings, we proposed to update the cluster centers in an attraction-repulsion manner to make the cluster centers more separable. And given the cluster inherent parameters, a likelihood-based loss is proposed to make node embeddings more concentrated around cluster centers. Thus, DAGC can simultaneously improve the intra-cluster compactness and inter-cluster heterogeneity. Finally, extensive experiments conducted on four benchmark datasets have demonstrated that the proposed DAGC consistently outperforms the state-of-the-art methods, especially on imbalanced datasets.

In many real-world scenarios, distribution shifts exist in the streaming data across time steps. Many complex sequential data can be effectively divided into distinct regimes that exhibit persistent dynamics. Discovering the shifted behaviors and the evolving patterns underlying the streaming data are important to understand the dynamic system. Existing methods typically train one robust model to work for the evolving data of distinct distributions or sequentially adapt the model utilizing explicitly given regime boundaries. However, there are two challenges: (1) shifts in data streams could happen drastically and abruptly without precursors. Boundaries of distribution shifts are usually unavailable, and (2) training a shared model for all domains could fail to capture varying patterns. This paper aims to solve the problem of sequential data modeling in the presence of sudden distribution shifts that occur without any precursors. Specifically, we design a Bayesian framework, dubbed as T-SaS, with a discrete distribution-modeling variable to capture abrupt shifts of data. Then, we design a model that enable adaptation with dynamic network selection conditioned on that discrete variable. The proposed method learns specific model parameters for each distribution by learning which neurons should be activated in the full network. A dynamic masking strategy is adopted here to support inter-distribution transfer through the overlapping of a set of sparse networks. Extensive experiments show that our proposed method is superior in both accurately detecting shift boundaries to get segments of varying distributions and effectively adapting to downstream forecast or classification tasks.

Automatic disease diagnosis from symptoms has attracted much attention in medical practices. It can assist doctors and medical practitioners in narrowing down disease candidates, reducing testing costs, improving diagnosis efficiency, and more importantly, saving human lives. Existing research has made significant progress in diagnosing disease but was limited by the gap between interpretability and accuracy. To fill this gap, in this paper, we propose a method called Reinforced Disease Diagnosis on Knowlege Graph (RDKG). Specifically, we first construct a knowledge graph containing all information from electronic medical records. To capture informative embeddings, we propose an enhanced knowledge graph embedding method that can embed information outside the knowledge graph into entity embedding. Then we transform the automatic disease diagnosis task into a Markov decision process on the knowledge graph. After that, we design a reinforcement learning method with a soft reward mechanism and a pruning strategy to solve the Markov decision process. We accomplish automated disease diagnosis by finding a path from symptoms to disease. The experimental results show that our model can effectively utilize heterogeneous information in the knowledge graph to complete the automatic disease diagnosis. Besides, our model demonstrates supreme performance in both accuracy and interpretability.

Feature generation aims to generate new and meaningful features to create a discriminative representation space. A generated feature is meaningful when the generated feature is from a feature pair with inherent feature interaction. In the real world, experienced data scientists can identify potentially useful feature-feature interactions, and generate meaningful dimensions from an exponentially large search space in an optimal crossing form over an optimal generation path. But, machines have limited human-like abilities. We generalize such learning tasks as self-optimizing feature generation. Self-optimizing feature generation imposes several under-addressed challenges on existing systems: meaningful, robust, and efficient generation. To tackle these challenges, we propose a principled and generic representation-crossing framework to solve self-optimizing feature generation. To achieve hashing representation, we propose a three-step approach: feature discretization, feature hashing, and descriptive summarization. To achieve reinforcement crossing, we develop a hierarchical reinforcement feature crossing approach. We present extensive experimental results to demonstrate the effectiveness and efficiency of the proposed method. The code is available at https://github.com/yingwangyang/HRC_feature_cross.git.

Feature transformation for AI is an essential task to boost the effectiveness and interpretability of machine learning (ML). Feature transformation aims to transform original data to identify an optimal feature space that enhances the performances of a downstream ML model. Existing studies either combines preprocessing, feature selection, and generation skills to empirically transform data, or automate feature transformation by machine intelligence, such as reinforcement learning. However, existing studies suffer from: 1) high-dimensional non-discriminative feature space; 2) inability to represent complex situational states; 3) inefficiency in integrating local and global feature information. To fill the research gap, we propose a novel group-wise cascading actor-critic perspective to develop the AI construct of automated feature transformation. Specifically, we formulate the feature transformation task as an iterative, nested process of feature generation and selection, where feature generation is to generate and add new features based on original features, and feature selection is to remove redundant features to control the size of feature space. Our proposed framework has three technical aims: 1) efficient generation; 2) effective policy learning; 3) accurate state perception. For an efficient generation, we develop a tailored feature clustering algorithm and accelerate generation by feature group-group crossing based generation. For effective policy learning, we propose a cascading actor-critic learning strategy to learn state-passing agents to select candidate feature groups and operations for fast feature generation. Such a strategy can effectively learn policies when the original feature size is large, along with exponentially growing feature generation action space, in which classic Q-value estimation methods fail. For accurate state perception of feature space, we develop a state comprehension method considering not only pointwise feature information but also pairwise feature-feature correlations. Finally, we present extensive experiments and case studies to illustrate 24.7% improvements in F1 scores compared with SOTAs and robustness in high-dimensional data.

With the explosive accumulation of urban geographic, mobile, and IoT service data, AI-assisted automated urban planning, with a goal of configuring land-uses, has become an emerging interdisciplinary topic for smart cities. Existing literature mostly views urban planning as a generative task from the perspective of generating land-use configuration images. Such perspective is limited by two issues: 1) hierarchical planning dependency across multi scales : there are hierarchical dependencies between region-level urban function configurations and block-level building configurations. 2) sequential planning dependency within a scale : when planning the buildings of a place, planning a shopping mall can impose constraints on planning subsequent Points of Interest (POIs). In response, we propose a new perspective of formulating urban planning as a hierarchical decision process. That is, given a target region with many geographic blocks, a machine planner firstly selects the optimized urban function portfolios, thereafter, sequentially selects the most appropriate POI for each block based on its urban functions and previously-placed POIs over planning steps. We reformulate this decision process into a hierarchical reinforcement learning task and develop a novel hierarchical reinforced urban planning framework. This framework includes two components: 1) In region-level configuration, we present an actor- critic based method to overcome the challenge of weak reward feedback in planning the urban functions of regions. 2) In block-level configuration, we propose a single-agent iterative POI allocation strategy to model dependencies between POIs and urban functions, and between current and previous POIs. Finally, we present extensive experimental results on real-world urban data to demonstrate the enhanced performances of the “planning as hierarchical decision process” perspective and the reinforced planning model.

Most session-based recommender systems (SBRSs) focus on extracting information from the observed items in the current session of a user to predict a next item, ignoring the causes outside the session (called outer-session causes, OSCs) that influence the user’s selection of items. However, these causes widely exist in the real world, and few studies have investigated their role in SBRSs. In this work, we analyze the causalities and correlations of the OSCs in SBRSs from the perspective of causal inference. We find that the OSCs are essentially the confounders in SBRSs, which leads to spurious correlations in the data used to train SBRS models. To address this problem, we propose a novel SBRS framework named COCO-SBRS (COunterfactual COllaborative Session-Based Recommender Systems) to learn the causality between OSCs and user-item interactions in SBRSs. COCO-SBRS first adopts a self-supervised approach to pre-train a recommendation model by designing pseudo-labels of causes for each user’s selection of the item in data to guide the training process. Next, COCO-SBRS adopts counterfactual inference to recommend items based on the outputs of the pre-trained recommendation model considering the causalities to alleviate the data sparsity problem. As a result, COCO-SBRS can learn the causalities in data, preventing the model from learning spurious correlations. The experimental results of our extensive experiments conducted on three real-world datasets demonstrate the superiority of our proposed framework over ten representative SBRSs.

In this article, we study knowledge concept recommendation in Massive Open Online Courses (MOOCs) in an explainable manner. Knowledge concepts, composing course units (e.g., videos) in MOOCs, refer to topics and skills that students are expected to master. Compared to traditional course recommendation in MOOCs, knowledge concepts recommendation has drawn more attention because students’ interests over knowledge concepts can better revealstudents’ real intention in a more refined granularity. However, there are three unique challenges in knowledge concept recommendation: (1) How to design an appropriate data structure to capture complex relationships between knowledge concepts, course units, and other participants (e.g., students, teachers)? (2) How to model interactions between students and knowledge concepts? (3) How to make explainable recommendation results to students? To tackle these challenges, we formulate the knowledge concept recommendation as a reinforcement learning task integrated with MOOC knowledge graph (KG). Specifically, we first construct MOOC KG as the environment to capture all the relationships and behavioral histories by considering all the entities (e.g., students, teachers, videos, courses, and knowledge concepts) on the MOOC provider. Then, to model the interactions between students and knowledge concepts, we train an agent to mimic students’ learning behavioral patterns facing the complex environment. Moreover, to provide explainable recommendation results, we generate recommended knowledge concepts in the format of a path from MOOC KG to indicate semantic reasons. Finally, we conduct extensive experiments on a real-world MOOC dataset to demonstrate the effectiveness of our proposed method.

We study the problem of balancing effectiveness and efficiency in automated feature selection. Feature selection is to find an optimal feature subset from large feature space. After exploring many feature selection methods, we observe a computational dilemma: 1) traditional feature selection (e.g., mRMR) is mostly efficient, but difficult to identify the best subset; 2) the emerging reinforced feature selection automatically navigates feature space to search the best subset, but is usually inefficient. Are automation and efficiency always apart from each other? Can we bridge the gap between effectiveness and efficiency under automation? Motivated by this dilemma, we aim to develop a novel feature space navigation method. In our preliminary work, we leveraged interactive reinforcement learning to accelerate feature selection by external trainer-agent interaction. Our preliminary work can be significantly improved by modeling the structured knowledge of its downstream task (e.g., decision tree) as learning feedback. In this journal version, we propose a novel interactive and closed-loop architecture to simultaneously model interactive reinforcement learning (IRL) and decision tree feedback (DTF). Specifically, IRL is to create an interactive feature selection loop and DTF is to feed structured feature knowledge back to the loop. The DTF improves IRL from two aspects. First, the tree-structured feature hierarchy generated by decision tree is leveraged to improve state representation. In particular, we represent the selected feature subset as an undirected graph of feature-feature correlations and a directed tree of decision features. We propose a new embedding method capable of empowering Graph Convolutional Network (GCN) to jointly learn state representation from both the graph and the tree. Second, the tree-structured feature hierarchy is exploited to develop a new reward scheme. In particular, we personalize reward assignment of agents based on decision tree feature importance. In addition, observing agents’ actions can also be a feedback, we devise another new reward scheme, to weigh and assign reward based on the selected frequency ratio of each agent in historical action records. Finally, we present extensive experiments with real-world datasets to demonstrate the improved performances of our method.

Feature selection is a critical step in machine learning that selects the most important features for a subsequent prediction task. Effective feature selection can help to reduce dimensionality, improve prediction accuracy, and increase result comprehensibility. It is traditionally challenging to find the optimal feature subset from the feature subset space as the space could be very large. While much effort has been made on feature selection, reinforcement learning can provide a new perspective towards a more globally-optimal searching strategy. In the preliminary work, we propose a multi-agent reinforcement learning framework for the feature selection problem. Specifically, we first reformulate feature selection with a reinforcement learning framework by regarding each feature as an agent. Besides, we obtain the state of the environment in three ways, i.e., statistic description, autoencoder, and graph convolutional network (GCN), in order to derive a fixed-length state representation as the input of reinforcement learning. In addition, we study how the coordination among feature agents can be improved by a more effective reward scheme. Also, we provide a GMM-based generative rectified sampling strategy to accelerate the convergence of multi-agent reinforcement learning. Our method searches the feature subset space more globally and can be easily adapted to real-time scenarios due to the nature of reinforcement learning. In the extended version, we further accelerate the framework from two aspects. From the sampling aspect, we show the indirect acceleration by proposing a rank-based softmax sampling strategy. From the exploration aspect, we show the direct acceleration by proposing an interactive reinforcement learning (IRL)-based exploration strategy. Extensive experimental results show the significant improvement of the proposed method over conventional approaches.

Mobile user profiling refers to the efforts of extracting users’ characteristics from mobile activities. In order to capture the dynamic varying of user characteristics for generating effective user profiling, we propose an imitation-based mobile user profiling framework. Considering the objective of teaching an autonomous agent to imitate user mobility based on the user's profile, the user profile is the most accurate when the agent can perfectly mimic the user behavior patterns. The profiling framework is formulated into a reinforcement learning task, where an agent is a next-visit planner, an action is a POI that a user will visit next, and the state of the environment is a fused representation of a user and spatial entities. An event in which a user visits a POI will construct a new state, which helps the agent predict users’ mobility more accurately. In the framework, we introduce a spatial Knowledge Graph (KG) to characterize the semantics of user visits over connected spatial entities. Additionally, we develop a mutual-updating strategy to quantify the state that evolves over time. Along these lines, we develop a reinforcement imitative graph learning framework for mobile user profiling. Finally, we conduct extensive experiments to demonstrate the superiority of our approach.

Recommender systems learn from historical user-item interactions to identify preferred items for target users. These observed interactions are usually unbalanced following a long-tailed distribution. Such long-tailed data lead to popularity bias to recommend popular but not personalized items to users. We present a gradient perspective to understand two negative impacts of popularity bias in recommendation model optimization: (i) the gradient direction of popular item embeddings is closer to that of positive interactions, and (ii) the magnitude of positive gradient for popular items are much greater than that of unpopular items. To address these issues, we propose a simple yet efficient framework to mitigate popularity bias from a gradient perspective. Specifically, we first normalize each user embedding and record accumulated gradients of users and items via popularity bias measures in model training. To address the popularity bias issues, we develop a gradient-based embedding adjustment approach used in model testing. This strategy is generic, model-agnostic, and can be seamlessly integrated into most existing recommender systems. Our extensive experiments on two classic recommendation models and four real-world datasets demonstrate the effectiveness of our method over state-of-the-art debiasing baselines.

Representation (feature) space is an environment where data points are vectorized, distances are computed, patterns are characterized, and geometric structures are embedded. Extracting a good representation space is critical to address the curse of dimensionality, improve model generalization, overcome data sparsity, and increase the availability of classic models. Existing literature, such as feature engineering and representation learning, is limited in achieving full automation (e.g., over heavy reliance on intensive labor and empirical experiences), explainable explicitness (e.g., traceable reconstruction process and explainable new features), and flexible optimal (e.g., optimal feature space reconstruction is not embedded into downstream tasks). Can we simultaneously address the automation, explicitness, and optimal challenges in representation space reconstruction for a machine learning task? To answer this question, we propose a group-wise reinforcement generation perspective. We reformulate representation space reconstruction into an interactive process of nested feature generation and selection, where feature generation is to generate new meaningful and explicit features, and feature selection is to eliminate redundant features to control feature sizes. We develop a cascading reinforcement learning method that leverages three cascading Markov Decision Processes to learn optimal generation policies to automate the selection of features and operations and the feature crossing. We design a group-wise generation strategy to cross a feature group, an operation, and another feature group to generate new features and find the strategy that can enhance exploration efficiency and augment reward signals of cascading agents. Finally, we present extensive experiments to demonstrate the effectiveness, efficiency, traceability, and explicitness of our system.

Human mobility forecasting can help us understand human movements, improve urban planning, and, ultimately, promote the development of livable, sustainable, and viable communities. While some efforts have been made for forecasting traffic or annotating trajectories, existing methods can be improved via simultaneously conducting human mobility forecasting and trajectory semantics augmentation. Along this line, in this paper, we provide a joint perspective of point processes and sequential embedding, in order to unify mobility arrival forecasting and trajectory semantics augmentation in a Hawkes-based long short-term memory (LSTM) method. Specifically, we first regard the traffic trajectories of a region as an arrival sequence according to the arrival time. Besides, we develop a method that exploits the mutual information of Hawkes processes and LSTM to model the arrival sequences of each region. Particularly, Hawkes processes predict the time and intensities of upcoming mobility arrivals; LSTM learns the embedding of arrivals, and annotates the arrival destinations and trip purposes; the mobility arrival intensities in Hawkes processes are influenced by the hidden states of LSTM. As applications, we exploit the proposed method to predict 3W (when, where, what) and discover functional regions. Finally, extensive experimental results with real-world traffic trajectory data demonstrate the enhanced performances of our method.

Knowledge graphs (KGs) have been widely used to improve recommendation accuracy. The multi-hop paths on KGs also enable recommendation reasoning, which is considered a crystal type of explainability. In this paper, we propose a reinforcement learning framework for multi-level recommendation reasoning over KGs, which leverages both ontology-view and instance-view KGs to model multi-level user interests. This framework ensures convergence to a more satisfying solution by effectively transferring high-level knowledge to lower levels. Based on the framework, we propose a multi-level reasoning path extraction method, which automatically selects between high-level concepts and low-level ones to form reasoning paths that better reveal user interests. Experiments on three datasets demonstrate the effectiveness of our method.

Automated characterization of spatial data is a kind of critical geographical intelligence. As an emerging technique for characterization, Spatial Representation Learning (SRL) uses deep neural networks (DNNs) to learn non-linear embedded features of spatial data for characterization. However, SRL extracts features by internal layers of DNNs, and thus suffers from lacking semantic labels. Texts of spatial entities, on the other hand, provide semantic understanding of latent feature labels, but is insensible to deep SRL models. How can we teach a SRL model to discover appropriate topic labels in texts and pair learned features with the labels? This paper formulates a new problem: feature-topic pairing, and proposes a novel Particle Swarm Optimization (PSO) based deep learning framework. Specifically, we formulate the feature-topic pairing problem into an automated alignment task between 1) a latent embedding feature space and 2) a textual semantic topic space. We decompose the alignment of the two spaces into: 1) point-wise alignment, denoting the correlation between a topic distribution and an embedding vector; 2) pair-wise alignment, denoting the consistency between a feature-feature similarity matrix and a topic-topic similarity matrix. We design a PSO based solver to simultaneously select an optimal set of topics and learn corresponding features based on the selected topics. We develop a closed loop algorithm to iterate between 1) minimizing losses of representation reconstruction and feature-topic alignment and 2) searching the best topics. Finally, we present extensive experiments to demonstrate the enhanced performance of our method.

In this paper, we propose a single-agent Monte Carlo based reinforced feature selection (MCRFS) method, as well as two efficiency improvement strategies, i.e., early stopping (ES) strategy and reward-level interactive (RI) strategy. Feature selection is one of the most important technologies in data prepossessing, aiming to find the optimal feature subset for a given downstream machine learning task. Enormous research has been done to improve its effectiveness and efficiency. Recently, the multi-agent reinforced feature selection (MARFS) has achieved great success in improving the performance of feature selection. However, MARFS suffers from the heavy burden of computational cost, which greatly limits its application in real-world scenarios. In this paper, we propose an efficient reinforcement feature selection method, which uses one agent to traverse the whole feature set, and decides to select or not select each feature one by one. Specifically, we first develop one behavior policy and use it to traverse the feature set and generate training data. And then, we evaluate the target policy based on the training data and improve the target policy by Bellman equation. Besides, we conduct the importance sampling in an incremental way, and propose an early stopping strategy to improve the training efficiency by the removal of skew data. In the early stopping strategy, the behavior policy stops traversing with a probability inversely proportional to the importance sampling weight. In addition, we propose a reward-level interactive strategy to improve the training efficiency via reward-level external advice. Finally, we design extensive experiments on real-world data to demonstrate the superiority of the proposed method.

The fairness issue is very important in deploying machine learning models as algorithms widely used in human society can be easily in discrimination. Researchers have studied disparity on tabular data a lot and proposed many methods to relieve bias. However, studies towards unfairness in graph are still at early stage while graph data that often represent connections among people in real-world applications can easily give rise to fairness issues and thus should be attached to great importance. Fair representation learning is one of the most effective methods to relieve bias, which aims to generate hidden representations of input data while obfuscating sensitive information. In graph setting, learning fair representations of graph (also called fair graph embeddings) is effective to solve graph unfairness problems. However, most existing works of fair graph embeddings only study fairness in a coarse granularity (i.e., group fairness), but overlook individual fairness. In this paper, we study fair graph representations from different levels. Specifically, we consider both group fairness and individual fairness on graph. To debias graph embeddings, we propose FairGAE, a fair graph auto-encoder model, to derive unbiased graph embeddings based on the tailor-designed fair Graph Convolution Network (GCN) layers. Then, to achieve multi-level fairness, we design a Wasserstein distance based regularizer to learn the optimal transport for fairer embeddings. To overcome the efficiency concern, we further bring up Sinkhorn divergence as the approximations of Wasserstein cost for computation. Finally, we apply the learned unbiased embeddings into the node classification task and conduct extensive experiments on two real-world graph datasets to demonstrate the improved performances of our approach.

Urban planning designs land-use configurations and can benefit building livable, sustainable, safe communities. Inspired by image generation, deep urban planning aims to leverage deep learning to generate land-use configurations. However, urban planning is a complex process. Existing studies usually ignore the need of personalized human guidance in planning, and spatial hierarchical structure in planning generation. Moreover, the lack of large-scale land-use configuration samples poses a data sparsity challenge. This paper studies a novel deep human guided urban planning method to jointly solve the above challenges. Specifically, we formulate the problem into a deep conditional variational autoencoder based framework. In this framework, we exploit the deep encoder-decoder design to generate land-use configurations. To capture the spatial hierarchy structure of land uses, we enforce the decoder to generate both the coarse-grained layer of functional zones, and the fine-grained layer of POI distributions. To integrate human guidance, we allow humans to describe what they need as texts and use these texts as a model condition input. To mitigate training data sparsity and improve model robustness, we introduce a variational Gaussian embedding mechanism. It not just allows us to better approximate the embedding space distribution of training data and sample a larger population to overcome sparsity, but also adds more probabilistic randomness into the urban planning generation to improve embedding diversity so as to improve robustness. Finally, we present extensive experiments to validate the enhanced performances of our method.

Effective feature selection can help reduce dimensionality, improve prediction accuracy, and increase result comprehensibility. Classic feature selection methods typically select and test feature subset in multiple iterations, and thus can be regarded as an exploratory process. In recent literature, the multi-armed bandit has become an emerging method to automate exploration for searching optimal solutions in large spaces. In this paper, our research question is: Can the multi-armed bandit formulation help us to automate feature selection? Along this line, we reformulate the feature selection problem with the combinatorial multi-armed bandit (CMAB) framework by regarding each feature as an arm. We propose two novel oracles and investigate how the super arm is formed under different oracles, and how the coordination between various features can be improved by a novel reward scheme. We present extensive experimental results to demonstrate the improved performance of the proposed methods over conventional feature selection approaches.

The COVID-19 pandemic forces schools to move teaching online and stimulates the development of online tutoring platforms.Although online tutoring platforms provide students the access to learning materials and tools anytime and anywhere, the quality of studies is impeded by the fact that students learn by watching videos, which lacks interactions between teachers and students.Such dilemma prevents us from respectively understanding and improving the online learning patterns and efficiency of students.To achieve this goal, we need to solve three challenges: (1) How can we quantify the study quality of online learning?(2) How can we design an appropriate data structure to describe online study behaviors?(3) How can we model the online study behaviors to better mine online study patterns?To address the challenges, we first propose a new measurement to quantify the online study quality from the perspective of study engagement.We then define a study behavior sequence to describe online study behaviors.The study behavior at each timestamp is an event of a video lecture watching behavior type, such as, watching, dragging forward and dragging backward.Moreover, we develop a neural hawkes process framework (namely EduHawkes ) for online study behavior modeling.The EduHawkes is a novel hierarchical encode-decode architecture with simultaneously optimizing the study behavior prediction task (event-level) and the study quality prediction task (course-level).In the experiments, we apply EduHawkes to the applications of study quality prediction and flippant student identification in order to demonstrate the improved performances of our proposed method on modeling online study behaviors.

Feature selection is a fundamental component of data mining, aiming to select optimal feature subsets for downstream task.Recently, an emerging feature selection method called reinforced feature selection applies reinforcement learning into feature selection.Reinforced Feature Selection (RFS) automates feature selection process and can effectively find the optimal subset.Generally, RFS can be categorized into single-agent RFS and multi-agent RFS.Single-agent RFS uses one reinforcement learning agent to select features, but its action space is exponentially-increasing with feature number and can merely obtain local optima.Multi-agent RFS uses multiple agents to select features; this method can achieve global optima, but it needs to optimize as many policy networks as feature number which costs huge computational resources and thus becomes computationally inefficient.This dilemma naturally leads to a research question: How can we synthesize the advantages of single-agent RFS and multi-agent RFS while avoiding their disadvantages?To answer this question, we propose a Group-based Interactive Reinforced Feature Selection (GIRFS) framework.This framework balances single-agent RFS and multi-agent RFS for better feature selection.Specifically, we formulate the feature selection problem into a group-based RFS problem.In this formulation, we first assign the given features into several groups based on feature similarity measurement.Then, we create agents for each group, where each agent decides to select/deselect features in its corresponding group.This design balances the size of action space and number of policy networks and thus makes RFS more effective and efficient.Moreover, to further improve learning efficiency, we propose a hierarchical teacher-like trainer to provide external action advice for agents.This trainer provides advice by intra-group selection and inter-group selection and fuses knowledge from mRMR and decision tree to help agents explore and learn.Finally, we present extensive experiments on real-world datasets to demonstrate the improved performances of our method.

In this paper, we solve the ambulance dispatch problem with a reinforcement learning oriented strategy. The ambulance dispatch problem is defined as deciding which ambulance to pick up which patient. Traditional studies on ambulance dispatch mainly focus on predefined protocols and are verified on simple simulation data, which are not flexible enough when facing the dynamically changing real-world cases. In this paper, we propose an efficient ambulance dispatch method based on the reinforcement learning framework, i.e., Multi-Agent Q-Network with Experience Replay(MAQR). Specifically, we firstly reformulate the ambulance dispatch problem with a multi-agent reinforcement learning framework, and then design the state, action, and reward function correspondingly for the framework. Thirdly, we design a simulator that controls ambulance status, generates patient requests and interacts with ambulances. Finally, we design extensive experiments to demonstrate the superiority of the proposed method.

In this paper, we study the problem of balancing effectiveness and efficiency in automated feature selection. Feature selection is to find the optimal feature subset from large-scale feature space, and is a fundamental intelligence for machine learning and predictive analysis. After exploring many feature selection methods, we observe a computational dilemma: 1) traditional feature selection methods (e.g., K-Best, decision tree based ranking, mRMR) are mostly efficient, but difficult to identify the best subset; 2) the emerging reinforced feature selection methods automatically navigate feature space to explore the best subset, but are usually inefficient. Are automation and efficiency always apart from each other? Can we bridge the gap between effectiveness and efficiency under automation? Motivated by such a computational dilemma, this study is to develop a novel feature space navigation method. To that end, we propose an Interactive Reinforced Feature Selection (IRFS) framework that guides agents by not just self-exploration experience, but also diverse external skilled trainers to accelerate learning for feature exploration. Specifically, we formulate the feature selection problem into an interactive reinforcement learning framework. In this framework, we first model two trainers skilled at different searching strategies: (1) KBest based trainer; (2) Decision Tree based trainer. We then develop two strategies: (1) to identify assertive and hesitant agents to diversify agent training, and (2) to enable the two trainers to take the teaching role in different stages to fuse the experience of the trainers and diversify teaching process. Such a hybrid teaching strategy can help agents to learn broader knowledge, and thereafter be more effective. Finally, we present extensive experiments on real-world datasets to demonstrate the improved performances of our method: more efficient than reinforced selection and more effective than classic feature selection.

Feature selection aims to select a subset of features to optimize the performances of downstream predictive tasks. Recently, multi-agent reinforced feature selection (MARFS) has been introduced to automate feature selection, by creating agents for each feature to select or deselect corresponding features. Although MARFS enjoys the automation of the selection process, MARFS suffers from not just the data complexity in terms of contents and dimensionality, but also the exponentially-increasing computational costs with regard to the number of agents. The raised concern leads to a new research question: Can we simplify the selection process of agents under reinforcement learning context so as to improve the efficiency and costs of feature selection? To address the question, we develop a single-agent reinforced feature selection approach integrated with restructured choice strategy. Specifically, the restructured choice strategy includes: 1) we exploit only one single agent to handle the selection task of multiple features, instead of using multiple agents. 2) we develop a scanning method to empower the single agent to make multiple selection/deselection decisions in each round of scanning. 3) we exploit the relevance to predictive labels of features to prioritize the scanning orders of the agent for multiple features. 4) we propose a convolutional auto-encoder algorithm, integrated with the encoded index information of features, to improve state representation. 5) we design a reward scheme that take into account both prediction accuracy and feature redundancy to facilitate the exploration process. Finally, we present extensive experimental results to demonstrate the efficiency and effectiveness of the proposed method.

We study the integration of reinforcement learning and spatial knowledge graph for incremental mobile user profiling, which aims to map mobile users to dynamically-updated profile vectors by incremental learning from a mixed-user event stream. After exploring many profiling methods, we identify a new imitation based criteria to better evaluate and optimize profiling accuracy. Considering the objective of teaching an autonomous agent to imitate a mobile user to plan next-visit based on the user's profile, the user profile is the most accurate when the agent can perfectly mimic the activity patterns of the user. We propose to formulate the problem into a reinforcement learning task, where an agent is a next-visit planner, an action is a POI that a user will visit next, and the state of environment is a fused representation of a user and spatial entities (e.g., POIs, activity types, functional zones). An event that a user takes an action to visit a POI, will change the environment, resulting into a new state of user profiles and spatial entities, which helps the agent to predict next visit more accurately. After analyzing such interactions among events, users, and spatial entities, we identify (1)semantic connectivity among spatial entities, and, thus, introduce a spatial Knowledge Graph (KG) to characterize the semantics of user visits over connected locations, activities, and zones. Besides, we identify (2) mutual influence between users and the spatial KG, and, thus, develop a mutual-updating strategy between users and the spatial KG, mixed with temporal context, to quantify the state representation that evolves over time. Along these lines, we develop a reinforcement learning framework integrated with spatial KG. The proposed framework can achieve incremental learning in multi-user profiling given a mixed-user event stream. Finally, we apply our approach to human mobility activity prediction and present extensive experiments to demonstrate improved performances.

Feature selection is the preprocessing step in machine learning which tries to select the most relevant features for the subsequent prediction task. Effective feature selection could help reduce dimensionality, improve prediction accuracy and increase result comprehensibility. It is very challenging to find the optimal feature subset from the subset space as the space could be very large. While much effort has been made by existing studies, reinforcement learning can provide a new perspective for the searching strategy in a more global way. In this paper, we propose a multi-agent reinforcement learning framework for the feature selection problem. Specifically, we first reformulate feature selection with a reinforcement learning framework by regarding each feature as an agent. Then, we obtain the state of environment in three ways, i.e., statistic description, autoencoder and graph convolutional network (GCN), in order to make the algorithm better understand the learning progress. We show how to learn the state representation in a graph-based way, which could tackle the case when not only the edges, but also the nodes are changing step by step. In addition, we study how the coordination between different features would be improved by more reasonable reward scheme. The proposed method could search the feature subset space globally and could be easily adapted to the real-time case (real-time feature selection) due to the nature of reinforcement learning. Also, we provide an efficient strategy to accelerate the convergence of multi-agent reinforcement learning. Finally, extensive experimental results show the significant improvement of the proposed method over conventional approaches.

Multi-task learning (MTL), improving learning performance by transferring information between related tasks, has drawn more and more attention in the data mining field. To tackle tasks whose data are stored at different locations (or nodes), distributed MTL was proposed. It not only enhances the learning performance but also improves the computing efficiency since it transforms the original centralized computing framework into a distributed computing framework under which computations can be done in parallel. The major drawback of the distributed MTL is a potential violation of confidentiality when the data stored at each node contain sensitive information (e.g., medical records). Some distributed MTL algorithms were designed to protect the original by only transferring aggregate information (e.g., supports or gradients) from each node to a server who combines the received information to produce the desired models. However, since aggregate data may still leak sensitive information, the security guarantee of the existing solutions cannot be formally proved or verified. Thus, the goal of this paper is to develop a provable privacy-preserving multi-task learning (PP-MTL) protocol that incorporates the state of the art cryptographic techniques to achieve the best security guarantee. We also conducted experiments to demonstrate the efficiency of our proposed method.

Bike-Sharing Systems (BSSs) are being introduced to more and more cities recently, and therefore they have generated huge amounts of data. Mobike is a station-less BSS which is suffering from the chaotic parking problem. To solve this problem, it is necessary to predict where the bikes are going. Traditional works dealing with destination prediction mainly focus on station-based BSSs, and they merely leverages context-aware information technically. Thus it is naturally promising to investigate how to improve the destination prediction of station-less bikes by context information. To that end, in this paper, we develop a multi-view machine (MVM) method, by incorporating the context information from Point of Interest (POI) data and human mobility data into destination prediction. Specifically, we first describe three different views, namely start position, start time and destination by features extracted from POI data and human mobility data. Then, we capture the relationship between these three views' interactions and the trip's possibility by a multi-view machine. Finally, since multi-view machine contains too many parameters to be optimized, we leverage tensor factorization (TF) to reduce the computation costs. The experimental results show that the model can effectively capture the potential relationship of three views with trip's possibility and the approach is thus much more effective than traditional prediction methods for destination.