Multimodal Document Question Answering (MDQA) systems commonly follow a retrieve-then-answer paradigm; however, the retrieval stage often introduces substantial noise, making an effective reranking component indispensable. Existing reranker training frameworks in MDQA rely predominantly on proxy supervision derived from human annotations or large language model (LLM) outputs, which are frequently noisy and, more critically, misaligned with downstream answer quality. To overcome this limitation, we propose AD-Reranker, a novel framework that shifts reranker training from proxy imitation to answer-driven utility optimization. Specifically, we reformulate the reranker as an environment-grounded agent that interacts with a downstream reader, modeled as a deterministic environment. We further design a composite reward function that integrates answer correctness, thereby explicitly incentivizing ranking strategies aligned with downstream task performance. To optimize the agent, we adopt Group Relative Policy Optimization (GRPO), enabling stable and effective group-wise policy learning. Empirical results demonstrate that AD-Reranker achieves superior reranking quality and an optimal accuracy-efficiency trade-off. When integrated into standard MDQA pipelines, AD-Reranker improves accuracy by 1.9%–5.0% while reducing the reader's context token consumption by 15%–52%, providing strong evidence for the effectiveness of answer-driven reranker training.

Learning transferable representations from unlabeled time series is crucial for improving performance in data-scarce classification. Existing self-supervised methods often operate at the point level and rely on unidirectional encoding, leading to low semantic density and a mismatch between pre-training and downstream optimization. In this paper, we propose TimeMAE, a self-supervised framework that reformulates masked modeling for time series via semantic unit elevation and decoupled representation learning. Instead of modeling individual time steps, TimeMAE segments time series into non-overlapping sub-series to form semantically enriched units, enabling more informative masked reconstruction while reducing computational cost. To address the representation discrepancy introduced by masking, we design a decoupled masked autoencoder that separately encodes visible and masked regions, avoiding artificial masked tokens in the main encoder. To guide pre-training, we introduce two complementary objectives: masked codeword classification, which discretizes sub-series semantics via a learned tokenizer and masked representation regression, which aligns continuous representations through a momentum-updated target encoder. Extensive experiments on five datasets demonstrate that TimeMAE outperforms competitive baselines, particularly in label-scarce scenarios and transfer learning scenarios. Our codes are publicly available at https://github.com/Mingyue-Cheng/TimeMAE.

Time series forecasting (TSF) traditionally relies on fast-thinking paradigms that map historical observations directly to future sequences of continuous values. While effective, such approaches often frame forecasting as a pattern-matching problem and tend to overlook explicit reasoning over temporal dynamics and contextual factors, which are critical for modeling long-range dependencies and non-stationary behaviors in real-world scenarios. Recent slow-thinking large language models (LLMs), such as OpenAI o1 and DeepSeek-R1, demonstrate strong inference-time multi-step reasoning abilities. This raises a fundamental question: can slow-thinking LLMs reason over temporal dynamics to support accurate TSF, even without task-specific training? To investigate this question, we present TimeReasoner, a systematic empirical study that reformulates TSF as a conditional reasoning process performed entirely at inference time. TimeReasoner integrates hybrid instructions consisting of task directives, timestamps, sequential values, and optional contextual features, and induces multi-step temporal reasoning in pretrained slow-thinking LLMs through chain-of-thought prompting and rollout-based reasoning strategies. Extensive experiments across diverse TSF benchmarks show that slow-thinking LLMs consistently outperform prior baselines or achieve competitive training-free forecasting performance. Beyond accuracy, we analyze how different inference-time reasoning strategies influence forecasting behaviors, highlighting both the potential and limitations of slow-thinking paradigms for TSF.

Table reasoning requires models to jointly perform comprehensive semantic understanding and precise numerical operations. Although recent large language model (LLM)-based methods have achieved promising results, most of them still rely on a single-turn reasoning paradigm that processes flattened tables in a single forward pass. This paradigm suffers from inherent limitations, including context overflow on large tables, weak sensitivity to continuous numerical values, and the absence of explicit tool-use and reflection. In this paper, we propose TableMind, a tuning-based autonomous programmatic table agent that simulates the human-like cognitive schema of multi-turn interaction within a lightweight LLM. Instead of adopting a training-free workflow design, TableMind learns to internalize planning, action, and reflection through a principled two-stage training strategy. To bootstrap structured table reasoning capabilities, we construct and filter high-quality reasoning data for the supervised fine-tuning (SFT) stage. To enable precise code generation, we introduce a designed multi-perspective reward scheme and a novel optimization objective in the reinforcement learning (RL) stage. Extensive experiments on diverse benchmarks demonstrate that TableMind consistently outperforms previous baselines, validating the effectiveness of training autonomous agents to improve overall performance.

Logical reasoning with large language models (LLMs) has received growing attention. One mainstream approach translates natural language into formal logic and then applies symbolic solvers for deduction. While effective in many tasks, these LLM-based translators often fail to generate consistent symbolic representations when the same concept appears in different linguistic forms. Such inconsistencies break logical coherence and lead to solver errors. However, most existing benchmarks lack this type of linguistic variation, which frequently occurs in real-world text, leaving the problem underexplored. To address this gap, we present SoLT, a benchmark that systematically rewrites reasoning datasets into diverse yet logically equivalent forms across multiple levels. Beyond evaluation, SoLT also provides a general method to enrich any dataset with linguistic diversity while preserving both meaning and logic. To further enhance the stability of LLM-based reasoning, we propose MenTaL, which explicitly guides models to build a concept–symbol mapping table during translation. By linking equivalent expressions to shared symbols, MenTaL maintains consistency and mitigates symbol drift. Experiments on SoLT demonstrate that LLMs indeed suffer from inconsistent symbol mapping under linguistic variation, leading to significant drops in reasoning accuracy. Meanwhile, applying MenTaL brings clear and stable performance improvements across diverse inputs. Overall, our findings reveal that overlooking linguistic diversity hides key weaknesses in LLM-based translators, and our work offers a step toward more reliable logical reasoning in varied real-world scenarios. Our code is available at https://github.com/wufeiwuwoshihua/LinguDiver.

The primary form of user-internet engagement is shifting from leveraging implicit feedback signals, such as browsing and clicks, to harnessing the rich explicit feedback provided by textual interactive behaviors. This shift unlocks a rich source of user textual history, presenting a profound opportunity for a deeper form of personalization. However, prevailing approaches offer only a shallow form of personalization, as they treat user history as a flat list of texts for retrieval and fail to model the rich temporal and semantic structures reflecting dynamic nature of user interests. In this work, we propose MemWeaver, a framework that weaves the user's entire textual history into a hierarchical memory to power deeply personalized generation. The core innovation of our memory lies in its ability to capture both the temporal evolution of interests and the semantic relationships between different activities. To achieve this, MemWeaver builds two complementary memory components that both integrate temporal and semantic information, but at different levels of abstraction: behavioral memory, which captures specific user actions, and cognitive memory, which represents long-term preferences. This dual-component memory serves as a comprehensive representation of the user, allowing large language models (LLMs) to reason over both concrete behaviors and abstracted cognitive traits. This leads to content generation that is deeply aligned with their latent preferences. Experiments on the six datasets of the Language Model Personalization (LaMP) benchmark validate the efficacy of MemWeaver. Our code is available. https://github.com/fishsure/MemWeaver.

Science Question Answering (SQA) is an important task for evaluating models' capability to reason with scientific knowledge. However, the extensive availability of scientific information (e.g., basic concepts in biology, physics, and chemistry) in pre-trained corpora may cause large language models (LLMs) to rely more on memorized information rather than actual reasoning when answering questions. This reliance persists even with techniques like Chain-of-Thought prompting, resulting in shallow understanding and limited reasoning based on scientific knowledge. Therefore, to enhance LLMs' capacity to comprehend and apply scientific knowledge, we propose a framework calledMulti-AgentCooperation-basedKnowledgeExemplification (MCKE). Specifically, MCKE leverages knowledge alongside questions to create exemplified knowledge, promoting deeper understanding through innovative knowledge representation. To better evaluate the model's ability to reason and apply knowledge, we introduceNovSciQA, a multiple-choice question answering dataset based on newly created scientific knowledge. This dataset covers multi-subject scientific knowledge and questions that do not exist in reality, making it impossible for the model to rely on memorized answer-related information to answer questions. Experimental results show that the MCKE framework outperforms baselines, and the NovSciQA dataset effectively assesses models' knowledge understanding and application. Our code and dataset are available inhttps://anonymous.4open.science/r/MCKE-NovSciQA.

In recent years, table reasoning has garnered substantial research interest, particularly regarding its integration with Large Language Models (LLMs), which have revolutionized natural language applications. Existing LLM-based studies typically achieve step-by-step thinking for table reasoning guided by task semantics. While these approaches emphasize autonomous exploration and enhance fine-grained table understanding, they often overlook systematic thinking in the reasoning process. This oversight can lead to omitted steps, disorganized logic and misleading results, especially in complex scenarios. In this paper, we proposePoTable, a novel stage-oriented plan-then-execute approach that incorporates systematic thinking into table reasoning. Specifically,PoTableinvolves several distinct analytical stages with clear objectives to provide adequate guidance. To accomplish stage-specific goals,PoTableemploys a plan-then-execute mechanism: it first plans the operation chain based on the stage objective, and then executes operations sequentially through code generation, real-time running and feedback processing. Consequently,PoTableproduces reliable table reasoning results with highly accurate, step-wise commented and completely executable programs. It mirrors the workflow of a professional data analyst, offering advantages in both accuracy and explainability. Finally, we conduct extensive experiments on four datasets from the WikiTQ and TabFact benchmarks, where the results demonstrate the effectiveness, efficiency and explainability ofPoTable. Our code is available at:https://github.com/Double680/PoTable.

Retrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs) by retrieving relevant document from external knowledge sources. By referencing this external knowledge, RAG effectively reduces the generation of factually incorrect content and addresses hallucination issues within LLMs. Recently, there has been growing attention to improving the performance and efficiency of RAG systems from various perspectives. While these advancements have yielded significant results, the application of RAG in domains with considerable societal implications raises a critical question about fairness: What impact does the introduction of the RAG paradigm have on the fairness of LLMs? To address this question, we conduct extensive experiments by varying the LLMs, retrievers, and retrieval sources. Our experimental analysis reveals that the scale of the LLMs plays a significant role in influencing fairness outcomes within the RAG framework. When the model scale is smaller than 8B, the integration of retrieval mechanisms often exacerbates unfairness in small-scale LLMs (e.g., LLaMA3.2-1B, Mistral-7B, and LLaMA3-8B). To mitigate the fairness issues introduced by RAG for small-scale LLMs, we propose two approaches, FairFT and FairFilter. Specifically, in FairFT, we align the retriever with the LLM in terms of fairness, enabling it to retrieve documents that facilitate fairer model outputs. In FairFilter, we propose a fairness filtering mechanism to filter out biased content after retrieval. Finally, we validate our proposed approaches on real-world datasets, demonstrating their effectiveness in improving fairness while maintaining performance.

Graph Neural Networks (GNNs) have been widely used across various fields under the homophily assumption that connected nodes are similar. However, in heterophilic graphs, where connected nodes tend to have dissimilar features, existing GNNs still face some limitations. From the perspective of structure, shallow GNNs could not capture the high-order node information, whereas deep GNNs may suffer from the over-smoothing problem. From the perspective of feature, the useful information of high-order similar nodes is often weakened by low-order dissimilar nodes in the feature update phase. To address the above problems, we propose a Global Structure-aware and Feature-augmented Graph Neural Network (GSF-GNN) to alleviate the limitations from the perspectives of structure and feature. Specifically, from the structure perspective, we design a Structure-based Global Propagation (SGP) module to establish global connections among nodes and adaptively adjust edge weights for message propagation. From the feature perspective, we introduce a Feature-augmented Compensatory Update (FCU) module, which employs a multi-view feature updating mechanism to enhance node features from different perspectives. Our theoretical analysis formally demonstrates the effectiveness of GSF-GNN in heterophilic graphs. Experiments on heterophilic and homophilic benchmark datasets validate the effectiveness of GSF-GNN across various graph structures. Moreover, GSF-GNN achieves stable performance across multiple layers and effectively alleviates the over-smoothing problem. Our codes are available on https://github.com/huijieliu2023/GSF-GNN .

Knowledge Tracing (KT), a pivotal component of intelligent tutoring systems, models the evolution of student knowledge states to predict future performance. While KT fundamentally relies on the premise that performance on similar questions is highly correlated, existing approaches often depend on generalized question representations, neglecting the rich, multi-faceted nature of question attributes. To address this limitation, we propose the Hierarchical Question Attribute-Fused KT (HQAF-KT) model, a novel architecture that deconstructs question similarity through three hierarchical dimensions: inherent, dynamic, and statistical. HQAF-KT first enriches foundational representations by integrating inherent question attributes. It then deploys a Dynamic Computing module that leverages student-specific dynamic attributes to personalize similarity assessments based on individual cognitive contexts. Furthermore, a Statistic Correction module refines generalized statistical attributes to account for unique student abilities. This hierarchical fusion enables a nuanced, individualized modeling of question relationships. Extensive experiments on three large-scale, real-world datasets demonstrate that HQAF-KT significantly outperforms state-of-the-art baselines by effectively capturing multi-level question similarity.

User modeling serves as a crucial foundation for researchers to capture useful potential characteristics, playing a pivotal role in various applications such as recommender systems. One common challenge in user modeling is the cold-start problem, where interactions are notably limited for new users. To tackle this issue, the paradigm of meta-learning has been introduced to user modeling, yielding promising results. Similar to a guidebook for a new traveler, meta-learning significantly influences decision-making for new users in critical scenarios, such as career recommendations. Consequently, the issue of fairness in meta-learning has become paramount. Several methods have been proposed to mitigate unfairness in meta-learning and have shown promising results. However, a fundamental question remains unexplored: What is the critical factor leading to unfairness in meta-learned user modeling? Through theoretical analysis that integrates the meta-learning paradigm with group fairness metrics, we identify group proportion imbalance as a critical factor. Subsequently, another question arises: How can we mitigate the influence of this factor to enhance fairness while ensuring accuracy? To this end, we introduce a novel F airness-aware A daptive S ampling framework for me T a-learning, abbreviated as FAST. Its core concept involves adaptively adjusting the sampling distribution for different user groups during the interleaved training process of meta-learning. Moreover, we provide theoretical guarantees demonstrating the convergence of FAST, showcasing its potential to effectively eliminate unfairness. Furthermore, to ensure model accuracy, we enhance FAST with FAST+ by introducing a hybrid sampling strategy at an individual level. This strategy prioritizes fairness and thoroughly explores important users during the sampling process, allowing for a better accuracy-fairness tradeoff. Finally, we conduct extensive experiments on real-world datasets, which demonstrate the effectiveness of both FAST and FAST+ frameworks.

Cross-Domain Recommendation (CDR) can effectively alleviate the data sparsity issue in the recommendation system by transferring the source domain knowledge to the target domain. Many CDR methods try to find a mapping of latent embeddings of the overlapping nodes (e.g., the overlapping users) from source domains to target domains. However, due to the randomness of parameter initialization and model training, the embeddings from different domains may differ a lot and the inter-domain mapping can be difficult to learn. To this end, in this work, we propose a novel Similarity-Preserving Cross-Domain Recommendation ( SimCDR ) framework. Instead of trying to learn the inter-domain mapping, SimCDR aims to preserve the intra-domain similarities of users, such that the users that are close in the source domain are still close in the target domain. Specifically, we first verify that the intra-domain similarities of user embeddings can represent domain knowledge well. Then, based on this finding, we propose the SimCDR framework, which adds the auxiliary task of predicting similar users to preserve the intra-domain similarities of the source domain user embeddings, i.e., the source domain knowledge. Besides being distinct in knowledge transfer, SimCDR also supports multiple source domains and embedding compression, making it more suitable for real-world applications. We conduct thorough experiments that verify the effectiveness of the proposed method.

Retrieval-augmented generation (RAG) is increasingly recognized as an effective approach to mitigating the hallucination of large language models (LLMs) through the integration of external knowledge. While numerous efforts, most studies focus on a single type of external knowledge source. However, in real-world applications, most situations involve diverse knowledge from various sources, yet this area has been less explored. The main dilemma is the lack of a suitable dataset containing multiple knowledge sources and pre-exploration of the associated issues. To address these challenges, we standardize a benchmark dataset that combines structured and unstructured knowledge across diverse and complementary domains. Based on this dataset, we further develop a plug-and-play RAG framework, PruningRAG, whose main characteristic is the use of multi-granularity pruning strategies to optimize the integration of relevant information while minimizing misleading context. It consistently improves performance across various existing RAG variants, demonstrating its robustness and broad applicability. Building upon the standardized dataset and PruningRAG, we also report a series of experimental results, as well as insightful findings. Our dataset and code are publicly available. https://github.com/USTCAGI/PruningRAG, with the aim of advancing future research in the RAG community.

Large language models (LLMs) have shown promise in multivariate time series classification (MTSC). To effectively adapt LLMs for MTSC, it is crucial to generate comprehensive and informative data representations. Most methods utilizing LLMs encode numerical time series into the model's latent space, aiming to align with the semantic space of LLMs for more effective learning. Despite effectiveness, we highlight three limitations that these methods overlook: (1) they struggle to incorporate temporal and channel-specific information, both of which are essential components of multivariate time series; (2) aligning the learned representation space with the semantic space of the LLMs proves to be a significant challenge; (3) they often require task-specific retraining, preventing training-free inference despite the generalization capabilities of LLMs. To bridge these gaps, we propose TableTime, which reformulates MTSC as a table understanding task. Specifically, TableTime introduces the following strategies: (1) utilizing tabular form to unify the format of time series, facilitating the transition from the model-centric approach to the data-centric approach; (2) representing time series in text format to facilitate seamless alignment with the semantic space of LLMs; (3) designing a knowledge-task dual-driven reasoning framework, TableTime, integrating contextual information and expert-level reasoning guidance to enhance LLMs' reasoning capabilities and enable training-free classification. Extensive experiments conducted on 10 publicly available benchmark datasets from the UEA archive validate the substantial potential of TableTime to be a new paradigm for MTSC. The code is publicly available. https://github.com/realwangjiahao/TableTime.

Graph In-Context Learning, with the ability to adapt pre-trained graph models to novel and diverse downstream graphs without updating any parameters, has gained much attention in the community. The key to graph in-context learning is to perform downstream graphs conditioned on chosen prompt examples. Existing methods randomly select subgraphs or edges as prompts, leading to noisy graph prompts and inferior model performance. Additionally, due to the gap between pre-training and testing graphs, when the number of classes in the testing graphs is much greater than that in the training, the in-context learning ability will also significantly deteriorate. To tackle the aforementioned challenges, we develop a multi-stage adaptive prompt optimization method GraphPrompter, which optimizes the entire process of generating, selecting, and using graph prompts for better in-context learning capabilities. Firstly, Prompt Generator introduces a reconstruction layer to highlight the most informative edges and reduce irrelevant noise for graph prompt construction. Furthermore, in the selection stage, Prompt Selector employs the k-nearest neighbors algorithm and pre-trained selection layers to dynamically choose appropriate sam-ples and minimize the influence of irrelevant prompts. Finally, we leverage a Prompt Augmenter with a cache replacement strategy to enhance the generalization capability of the pre-trained model on new datasets. Extensive experiments show that GraphPrompter effectively enhances the in-context learning ability of graph models. On average across all the settings, our approach surpasses the state-of-the-art baselines by over 8 %. Our code is released at https://ithub.com/karin0018/GraphPrompter.

Programming Knowledge Tracking (PKT) aims to dynamically diagnose learners' mastery levels of programming knowledge based on their coding activities, facilitating more effective and personalized programming education. However, current PKT studies primarily focus on the implicit relationship between code content and knowledge assessment, often overlooking two types of noise signals in long-term programming activities: unwanted signals from unrelated submissions and weak signals from minor modifications. This practical challenge significantly limits model performance and application. To address this issue, we propose Coda, a Code graph-based tuning adaptor designed to enhance existing PKT models by identifying and mitigating the impact of noise. Specifically, Coda first transforms the loose code sequences submitted by each learner into a compact code graph. By leveraging this code graph, unwanted signals can be identified from a semantic similarity perspective. We then apply a cluster-aware GCN to the code graph, which improves the discrimination of weak signals and enables their clustering for identification. Finally, a lightweight yet effective adaptor is incorporated into the PKT task through optimization with two noise feature-based constraints and a navigational regularization term, to correct knowledge states affected by noise. It is worth mentioning that the Coda framework is model-agnostic and can be adapted to most existing PKT solutions. Extensive experimental results on four real-world datasets demonstrate that Coda effectively performs the PKT task in the presence of noisy programming records, outperforming typical baselines.

In the pursuit of enhancing software reusability and developer productivity, code search has emerged as a key area, aimed at retrieving code snippets relevant to functionalities based on natural language queries. Despite significant progress in self-supervised code pre-training utilizing the vast amount of code data in repositories, existing methods have primarily focused on leveraging contrastive learning to align natural language with function-level code snippets. These studies have overlooked the abundance of fine-grained (such as block-level and statement-level) code snippets prevalent within the function-level code snippets, which results in suboptimal performance across all levels of granularity. To address this problem, we first construct a multi-granularity code search dataset called MGCodeSearchNet, which contains 536K+ pairs of natural language and code snippets. Subsequently, we introduce a novel Multi-Granularity Self-Supervised contrastive learning code Search framework (MGS3). First, MGS3 features a Hierarchical Multi-Granularity Representation module (HMGR), which leverages syntactic structural relationships for hierarchical representation and aggregates fine-grained information into coarser-grained representations. Then, during the contrastive learning phase, we endeavor to construct positive samples of the same granularity for fine-grained code, and introduce in-function negative samples for fine-grained code. Finally, we conduct extensive experiments on code search benchmarks across various granularities, demonstrating that the framework exhibits outstanding performance in code search tasks of multiple granularities. These experiments also showcase its model-agnostic nature and compatibility with existing pre-trained code representation models.

In Click-Through Rate (CTR) prediction, the long behavior sequence, comprising the user's long period of historical interactions with items has a vital influence on assessing the user's interest in the candidate item. Existing approaches strike efficiency and effectiveness through a two-stage paradigm: first retrieving hundreds of candidate-related items and then extracting interest intensity vector through target attention. However, we argue that the discrepancy in target attention's relevance distribution between the retrieved items and the full long behavior sequence inevitably leads to a performance decline. To alleviate the discrepancy, we propose the Deep Multiple Quantization Network (DMQN) to process long behavior sequence end-to-end through compressing the long behavior sequence. Firstly, the entire spectrum of long behavior sequence will be quantized into multiple codeword sequences based on multiple independent codebooks. Hierarchical Sequential Transduction Unit is incorporated to facilitate the interaction of reduced codeword sequences. Then, attention between the candidate and multiple codeword sequences will output the interest vector. To enable online serving, intermediate representations of the codeword sequences are cached, significantly reducing latency. Our extensive experiments on both industrial and public datasets confirm the effectiveness and efficiency of DMQN. The A/B test in our advertising system shows that DMQN improves CTR by 3.5% and RPM by 2.0%.

Pre-training universal models across multiple domains to enhance downstream tasks is a prevalent learning paradigm. However, there has been minimal progress in pre-training transferable models across domains for time series representation. This dilemma is incurred by two key factors: the limited availability of training set within each domain and the substantial differences in data characteristics between domains. To address these challenges, we present a novel framework, namely CrossTimeNet, designed to perform cross-domain self-supervised pre-training to benefit target tasks. Specifically, to address the issue of data scarcity, we utilize a pre-trained language model as the backbone network to effectively capture the sequence dependencies of the input time series. Meanwhile, we adopt the recovery of corrupted region inputs as a self-supervised optimization objective, taking into account the locality of the time series. To address discrepancies in data characteristics, we introduce a novel tokenization module that converts continuous time series inputs into discrete token sequences using vector quantization techniques. This approach facilitates the learning of transferable time series models across different domains. Extensive experimental results on diverse time series tasks, including classification and forecasting, demonstrate the effectiveness of our approach. Our codes are publicly available at https://github.com/Mingyue-Cheng/CrossTimeNet.

For the advancement of time series classification, we can summarize that most existing methods adopt a common learning-to-classify paradigm - a classifier model tries to learn the relation between sequence inputs and target label encoded by one-hot distribution. Although effective, this paradigm conceals two inherent limitations: (1) one-hot distribution fails to reflect the comparability and similarity between labels, and (2) it is difficult to learn transferable representation across domains. In this work, we propose InstructTime, a novel attempt to reshape time series classification as a learning-to-generate paradigm. Relying on the generative capacity of the pre-trained language model, the core idea is to formulate the classification of time series as a multimodal understanding task. Specifically, firstly, a time series discretization module is designed to convert continuous inputs into a sequence of discrete tokens to solve the inconsistency issue across modality data. Secondly, we introduce an alignment projected layer before feeding the transformed token of time series into language models. Thirdly, prior to fine-tuning the language model for the target domain, it is essential to emphasize the necessity of auto-regressive pre-training across various modality inputs. Finally, extensive experimentation are conducted on several prevalent public benchmark datasets, indicating the superior performance of the InstructTime. Our code is at https://github.com/Mingyue-Cheng/InstructTime.

An intelligent code search engine tries to find and suggest a code piece given a developer's query quickly from a large-scale program database, which can significantly promote software development efficiency. Existing solutions can search the relevant codes to some extent. However, most of them fail to precisely understand the search intent of developers since they only mine their natural language queries, while ignoring the valuable programming context (e.g., the code written by the developer). In this paper, we study the novel problem of context-aware code search. To promote a step forward, we first provide the CodeSearchNet-C dataset with constructing sufficient programming context from the GitHub website for each query-code instance. The dataset is supplemented on the CodeSearchNet benchmark, ensuring both generality and comparability for relevant research. Then, by analyzing the characteristics of programming context, we propose a novel two-stage Context-aware Code Retrieval (ConCR) framework. In the first stage, we propose a Context Walking algorithm, which simulates the programming habits of different developers. The generated programming context could ensure the diversity of search intent among developers. In the second stage, imitating the reading habits of developers, we introduce a novel Context Hierarchical Encoder, to understand the search intent with contextual information from local to global. Our ConCR framework is general, and we give three implementations on the basis of typical code search models as backbones. Extensive experimental results clearly prove that our ConCR significantly enhances the code search performance, effectively fulfilling developers' needs for efficient code resource searching on the web. These results also verify the necessity of introducing programming context to understand developers' intent.

Transmission Control Protocol (TCP) congestion control is a fundamental mechanism in the Internet that maintains network stability and performance by adjusting the sending rate of connections. Recently, Deep Reinforcement Learning (DRL) methods have shown superior performance over traditional expert-designed solutions. However, the DRL policies are often represented by black-box neural networks, they lack interpretability, making verification challenging and requiring excessive floating-point computation. This work introduces a novel approach, Programmatic reinforcement learning for Congestion Control (ProCC), designed to autonomously discover a program as a control policy from scratch. Programs in ProCC include branching structures (e.g., if blocks and if-else blocks), conditions and actions. However, directly optimizing such program structures is challenging due to their discrete non-differentiable nature, and the program space grows exponentially as the depth increases. To address this issue, ProCC defines a Domain-Specific Language (DSL) and program transformation rules, enabling the construction of a program search graph where similar programs are closer in proximity. Subsequently, ProCC employs Monte Carlo Tree Search (MCTS) to efficiently explore the discrete space and obtain promising programs. Extensive experiments conducted in multiple simulated environments demonstrate that ProCC is adaptive and consistently performs well under varying network conditions. The learned program's performance surpasses that of state-of-the-art DRL agents, and more importantly, the generated policies are concise, transparent, and computationally efficient.

Sequential Recommendation (SR) plays a pivotal role in recommender systems by tailoring recommendations to user preferences based on their non-stationary historical interactions. Achieving high-quality performance in SR requires attention to both item representation and diversity. However, designing an SR method that simultaneously optimizes these merits remains a long-standing challenge. In this study, we address this issue by integrating recent generative Diffusion Models (DM) into SR. DM has demonstrated utility in representation learning and diverse image generation. Nevertheless, a straightforward combination of SR and DM leads to sub-optimal performance due to discrepancies in learning objectives (recommendation vs. noise reconstruction) and the respective learning spaces (non-stationary vs. stationary). To overcome this, we propose a novel framework called DimeRec (Di ffusion with multi-interest enhanced Rec ommender). DimeRec synergistically combines a guidance extraction module (GEM) and a generative diffusion aggregation module (DAM). The GEM extracts crucial stationary guidance signals from the user's non-stationary interaction history, while the DAM employs a generative diffusion process conditioned on GEM's outputs to reconstruct and generate consistent recommendations. Our numerical experiments demonstrate that DimeRec significantly outperforms established baseline methods across three publicly available datasets. Furthermore, we have successfully deployed DimeRec on a large-scale short video recommendation platform, serving hundreds of millions of users. Live A/B testing confirms that our method improves both users' time spent and result diversification.

In digital education, Cognitive Diagnosis (CD) is essential for modeling learners' cognitive states, such as problem-solving ability and knowledge proficiency, by analyzing their response data, like answer correctness. However, traditional CD methods struggle with effectiveness and efficiency. They fail to capture the diversity and uncertainty of learners' cognitive states. Additionally, response prediction can be time-consuming. To address these issues, we propose BoxCD, a contrastive probabilistic box embedding model for cognitive diagnosis. BoxCD utilizes high-dimensional axis-aligned hyper-rectangles (boxes) to represent learners and exercises, with the volume of intersecting boxes used to predict learners' responses. This approach effectively captures semantic diversity and uncertainty while enhancing diagnostic effectiveness. To stabilize box embeddings, we integrate contrastive learning objectives with response prediction goals, optimizing the distance between positive and negative samples of learner and exercise boxes to improve uniformity. Additionally, we develop a rank-based response prediction method that leverages the geometric properties of box embeddings to assess learners' response correctness efficiently. Comprehensive experiments on two real-world datasets demonstrate that BoxCD outperforms traditional CD models in effectiveness and efficiency. This showcases its potential to enhance personalized learning in digital education platforms.

The success of Graph Neural Networks (GNNs) in graph classification has heightened interest in explainable GNNs, particularly through graph rationalization. This method aims to enhance GNNs explainability by identifying subgraph structures (i.e., rationales) that support model predictions. However, existing methods often rely on centralized datasets, posing challenges in scenarios where data privacy is crucial, such as in molecular property prediction. Federated Learning (FL) offers a solution by enabling collaborative model training without sharing raw data. In this context, Federated Graph Rationalization emerges as a promising research direction. However, in each client, the rationalization methods often rely on client-specific shortcuts to compose rationales and make task predictions. Data heterogeneity, characterized by non-IID data across clients, exacerbates this problem, leading to poor prediction performance. To address these challenges, we propose the Environment-aware Data Augmentation (EaDA) method for Federated Graph Rationalization. EaDA comprises two main components: the Environment-aware Rationale Extraction (ERE) module and the Local-Global Alignment (LGA) module. The ERE module employs prototype learning to infer and share abstract environment information across clients, which are then aggregated to form a global environment. This information is used to generate counterfactual samples for local clients, enhancing the robustness of task predictions. The LGA module uses contrastive learning methods to align local and global rationale representations, mitigating performance degradation due to data heterogeneity. Comprehensive experiments on benchmark datasets demonstrate the effectiveness of our approaches. Code is available at https://github.com/yuelinan/Codes-of-EaDA.

Knowledge discovery and utilization are two essential cognitive processes that enable humans to understand the world and extract new insights from their surroundings. These processes have motivated machine learning studies, particularly zero-shot (ZS) learning, which seeks to identify unseen concepts through the use of side information. Previous ZS studies primarily focused on utilizing existing knowledge to infer unseen events, yet they overlook the crucial process of knowledge discovery and the integrated modeling of these knowledge-aware processes. In this study, we present a comprehensive ZS learning approach that explores and evaluates the machine’s abilities of discovering and utilizing knowledge. More specifically, to emulate human-like knowledge discovery and utilization processes, we propose a novel visual-aware ZS knowledge graph completion task for evaluation, incorporating a traditional ZS image classification task. Technically, we develop a unified ZS learning paradigm named Cognitive Learner (CoLa) to foster the two knowledge-aware abilities. Including a knowledge representation learning (KRL) module and a knowledge adaptation (KA) module, CoLa adapts well to the two specified tasks with the corresponding data. Extensive experiments on large-scale datasets demonstrate CoLa models’ outstanding performance over compared methods in the two ZS tasks, illustrating their superior ability of discovering and utilizing knowledge.

Math word problem (MWP) serves as a critical milestone for assessing the text mining ability and knowledge mastery level of models. Recent advancements have witnessed large language models (LLMs) showcasing remarkable performance on MWP. However, current LLMs still frequently exhibit logical errors, which highlights their inability to fully grasp the knowledge required for genuine step-by-step mathematical reasoning. To this end, in this paper, we propose a novel Knowledge-guided Solver (KNOS) framework that empowers LLMs to simulate human mathematical reasoning, whose core idea is toInvoke-Verify-Injectnecessary knowledge to solve MWP. We draw inspiration from the dual-process theory to construct two cooperative systems: aKnowledge Systemand anInference System. Specifically, theKnowledge Systememploys LLMs as the knowledge base and develops a novelknowledge invokerthat can elicit their relevant knowledge to support the strict step-level mathematical reasoning. In theInference System, we propose aknowledge verifierand aknowledge injectorto evaluate the knowledge rationality and further guide the step-wise symbolic deduction in an interpretable manner based on human cognitive mechanism, respectively. Moreover, to tackle the potential scarcity issue of mathematics-specific knowledge in LLMs, we consider an open-book exam scenario and propose an improved version of KNOS called EKNOS. In EKNOS, we meticulously designknowledge selectorsto extract the most relevant commonsense and math formulas from external knowledge sources for each reasoning step. This knowledge is utilized to assist theknowledge invokerin better stimulating LLMs’ reasoning abilities. Both KNOS and EKNOS are flexible to empower different LLMs. Our experiments with GPT3, ChatGPT, and GPT4 not only demonstrate their reasoning accuracy improvement but also show how they bring the strict step-wise interpretability of mathematical thinking.

Knowledge Tracing (KT) predicts future performance by modeling students' historical interactions, and understanding students' affective states can enhance the effectiveness of KT, thereby improving the quality of education. Although traditional KT values students' cognition and learning behaviors, efficient evaluation of students' affective states and their application in KT still require further exploration due to the non-affect-oriented nature of the data and budget constraints. To address this issue, we propose a computation-driven approach,DynamicAffectSimulationKnowledgeTracing (DASKT), to explore the impact of various student affective states (such as frustration, concentration, boredom, and confusion) on their knowledge states. In this model, we first extract affective factors from students' non-affect-oriented behavioral data, then use clustering and spatiotemporal sequence modeling to accurately simulate students' dynamic affect changes when dealing with different problems. Subsequently, we incorporate affect with time-series analysis to improve the model's ability to infer knowledge states over time and space. Extensive experimental results on two public real-world educational datasets show that DASKT can achieve more reasonable knowledge states under the effect of students' affective states. Moreover, DASKT outperforms the most advanced KT methods in predicting student performance. Our research highlights a promising avenue for future KT studies, focusing on achieving high interpretability and accuracy.

Multimodal Entity Linking (MEL) is a crucial task that aims at linking ambiguous mentions within multimodal contexts to the referent entities in a multimodal knowledge base, such as Wikipedia. Existing methods focus heavily on using complex mechanisms and extensive model tuning methods to model the multimodal interaction on specific datasets. However, these methods overcomplicate the MEL task and overlook the visual semantic information, which makes them costly and hard to scale. Moreover, these methods cannot solve the issues like textual ambiguity, redundancy, and noisy images, which severely degrade their performance. Fortunately, the advent of Large Language Models (LLMs) with robust capabilities in text understanding and reasoning, particularly Multimodal Large Language Models (MLLMs) that can process multimodal inputs, provides new insights into addressing this challenge. However, how to design a universally applicable LLMs-based MEL approach remains a pressing challenge. To this end, we propose UniMEL, a unified framework which establishes a new paradigm to process multimodal entity linking tasks using LLMs. In this framework, we employ LLMs to augment the representation of mentions and entities individually by integrating textual and visual information and refining textual information. Subsequently, we employ the embedding-based method for retrieving and re-ranking candidate entities. Then, with only ~0.26% of the model parameters fine-tuned, LLMs can make the final selection from the candidate entities. Extensive experiments on three public benchmark datasets demonstrate that our solution achieves state-of-the-art performance, and ablation studies verify the effectiveness of all modules. Our code is available at https://github.com/Javkonline/UniMEL.

Knowledge Tracing (KT) is a crucial research task for dynamically monitoring students' knowledge states, particularly in online education systems. Recently, knowledge tracing has gained significant attention and in-depth research. Most existing methods rely on students' response data for question understanding and modeling, which helps better updating students' knowledge states. Meanwhile, question ID is utilized to indicate and represent questions. However, this presents a challenge when transitioning to new, cold-start questions that few students has answered before. Also, prior work has overlooked the semantic modeling of questions, which could better assist in modeling the transfer of students' knowledge states. In this paper, we explore leveraging the power of Large Language Models (LLMs) to help understand questions for knowledge tracing, which benefits mitigating cold-start and sparse problems and modeling the transfer of students' knowledge states in a sophisticated manner. Specifically, we first design an attribute estimation module to estimate the attribute of the questions (e.g., difficulty, ability requirements, expected response time) by prompting Large Language Models. Subsequently, we have developed a question embedding module that incorporates graph attention network to effectively utilizing these attributes. Extensive experiments on various datasets demonstrate that our model outperforms existing state-of-the-art models and effectively addresses the problems of cold-start and sparsity. In addition, due to the estimation of multiple attributes of the questions, our model exhibits superior interpretability.

Learning to answer multi-step complex questions requires machines to perform like a human to think and reason step by step, which is one of the core abilities of a question answering system. Recent advancements have revealed that large language models exhibit remarkable reasoning capabilities by generating intermediate chain-of-thought rationales. However, the completeness of their rationales lacks assurance as they are susceptible to omitting steps and making factual errors. In this paper, drawing inspiration from human-like reasoning processes in answering multi-step questions, we explicitly plan the rationales to ensure their completeness. We propose a two-stage Decomposition-Evaluation (Dec-Eval) framework including a step decomposition stage and a rationale generation stage. Specifically, in the first stage, we decompose the complex question into simpler sub-ones and simulate a human's ability to grasp logical clues to ensure the integrity of step planning. Then, in the second stage, based on the sub-questions, we generate and evaluate rationales step by step. Both stages work together organically, improving the completeness of rationales and the accuracy of the answer. To further control the question answering process, we propose a novel knowledge injection mechanism that incorporates external knowledge to guide both stages. Extensive experiments on three challenging multi-step QA datasets demonstrate that Dec-Eval can explicitly generate more logical rationales, and significantly improve the reasoning performances of different backbone models.

Sequential recommender systems offer personalized suggestions by modeling users' interactions chronologically to capture dynamic user interest. Existing approaches typically fail to adequately describe the dynamics of the entire recommender system, including shifts in both user interest and item availability. To address this, we propose a simple yet effective framework with three key perspectives, tailored to the dynamics of recommender system by fully exploiting the time information. Firstly, we propose a dynamic candidate set construction approach to prevent the model from learning future interactions. Secondly, assuming that user behaviors remain consistent over short terms but may evolve over long terms, we employ a interval-weighted optimization target to model the correlation of users' historical interactions. Finally, we introduce a specialized time-aware attention module to enhance recommendations within specific temporal contexts. Extensive experiments demonstrate the effectiveness and generalizability of our framework. We make our codes publicly available.

Learner modeling is a crucial task in online learning environments, where Cognitive Diagnosis Models (CDMs) are employed to assess learners' knowledge mastery levels based on recorded response logs. However, the prevalence of noise in recorded response data poses significant challenges, including various behaviors such as guess and slip, casual answers, and system-induced errors. The existence of noise degrades the accuracy of diagnosis results and learner performance predictions. In this work, we propose a general framework, Adaptive Response Denoising (AdaRD), designed to salvage CDMs from the influence of noisy learner-exercise responses. AdaRD extends existing CDMs, incorporating primary training for denoised CDMs and auxiliary training for additional denoising support. The primary training employs binary Generalized Cross Entropy (GCE) loss to slow down the large update of learner knowledge states caused by noisy responses. Simultaneously, we utilize the variance of diagnosed knowledge mastery levels between primary and auxiliary diagnosis modules as a criterion to downweight high-variance responses that are likely to be noisy. In this manner, the proposed framework can prune noisy response learning during training, thereby enhancing the accuracy and robustness of CDMs. Extensive experiments on both real-world and synthetic datasets validate AdaRD's effectiveness in mitigating the impact of noisy learner-exercise responses.

Student knowledge profile is the basis for adaptive learning applications in online learning resulting from modeling the student mastery of knowledge concepts. In recent years, typical works based on knowledge tracing (KT) expect to profile students and have achieved significant success for the next performance prediction. However, in practical online learning scenarios, current methods tend to suffer from the following challenges: 1) Prediction inconsistency: The accuracy of the next performance prediction is inconsistent with the accuracy of student knowledge profile prediction, which is the more required result. 2) Cold start of knowledge: In online learning scenarios, it is often necessary to profile some knowledge concepts without learning records in advance. In this paper, we propose a novel Graph-based Student Knowledge Profile Model (GSKPM), along with a new end-to-end training objective, to tackle these challenges. We first define a new training objective to ensure the model is capable of inferring consistent student knowledge profiles. Then in this model, a two-stage hyper-aggregation process is employed to make full use of the topological relations between knowledge concepts and knowledge domains to provide information during profiling, especially for cold start knowledge concepts. Finally, through extensive experiments on real-world datasets, we will show that GSKPM achieves better prediction performances on student knowledge profiles and well deals with the cold start problem.

To alleviate the long-standing data sparsity issue in recommender systems, numerous studies in cross-domain recommendation (CDR) have been conducted to facilitate information transfer processes across domains. In recent years, dual-target CDR has been introduced to gain mutual improvements between two domains through more general bidirectional transfer rather than traditional one-way transit. Existing methods in dual-target CDR focus primarily on designing powerful encoders to learn representative cross-domain information, without tackling the fundamental issue of interaction data shortage. In this paper, we present CrossAug, a novel data augmentation approach to leverage interactions more efficiently in two domains. Specifically, we propose intra-domain and inter-domain augmentations based on cross-reconstructed representations in terms of sampled records. To reduce the harm of domain shift, we project domain-shared representations in two domains into a joint space with Householder transformations and apply center alignments. All these modules boost the utilization of interactions with little influence from negative transfer. Extensive experimental results over public datasets demonstrate the effectiveness of CrossAug and its components in dual-target CDR.

With the development of legal intelligence, Criminal Court View Generation has attracted much attention as a crucial task of legal intelligence, which aims to generate concise and coherent texts that summarize case facts and provide explanations for verdicts. Existing researches explore the key information in case facts to yield the court views. Most of them employ a coarse-grained approach that partitions the facts into broad segments (e.g., verdict-related sentences) to make predictions. However, this approach fails to capture the complex details present in the case facts, such as various criminal elements and legal events. To this end, in this paper, we propose an Event Grounded Generation (EGG) method for criminal court view generation with cooperative (Large) Language Models, which introduces the fine-grained event information into the generation. Specifically, we first design a LLMs-based extraction method that can extract events in case facts without massive annotated events. Then, we incorporate the extracted events into court view generation by merging case facts and events. Besides, considering the computational burden posed by the use of LLMs in the extraction phase of EGG, we propose a LLMs-free EGG method that can eliminate the requirement for event extraction using LLMs in the inference phase. Extensive experimental results on a real-world dataset clearly validate the effectiveness of our proposed method.

Deep Neural Networks (DNNs), despite their notable progress across information retrieval tasks, encounter the issues of shortcut learning and struggle with poor generalization due to their reliance on spurious correlations between features and labels. Current research mainly mitigates shortcut learning behavior using augmentation and distillation techniques, but these methods could be laborious and introduce unwarranted biases. To tackle these, in this paper, we propose COMI, a novel method to COrrect and MItigate shortcut learning behavior. Inspired by the ways students solve shortcuts in educational scenarios, we aim to reduce model's reliance on shortcuts and enhance its ability to extract underlying information integrated with standard Empirical Risk Minimization (ERM). Specifically, we first design Correct Habit (CoHa) strategy to retrieve the top m challenging samples for priority training, which encourages model to rely less on shortcuts in the early training. Then, to extract more meaningful underlying information, the information derived from ERM is separated into task-relevant and task-irrelevant information, the former serves as the primary basis for model predictions, while the latter is considered non-essential. However, within task-relevant information, certain potential shortcuts contribute to overconfident predictions. To mitigate this, we design Deep Mitigation (DeMi) network with shortcut margin loss to adaptively control the feature weights of shortcuts and eliminate their influence. Besides, to counteract unknown shortcut tokens issue in NLP, we adopt locally interpretable module-LIME to help recognize shortcut tokens. Finally, extensive experiments conducted on NLP and CV tasks demonstrate the effectiveness of COMI, which can perform well on both IID and OOD samples.

Formula recognition endeavors to automatically identify mathematical formulas from images. Currently, the Encoder-Decoder model has significantly advanced the translation from image to corresponding formula markups. Nonetheless, previous research primarily concentrated on single-line formula recognition, ignoring the recognition of multi-line formulas, which presents additional challenges such as more stringent grammatical restrictions and two- dimensional positions. In this work, we present GAP (Grammar And Position-Aware formula recognition), a comprehensive framework designed to tackle the challenges in multi-line mathematical formula recognition. First, to overcome the limitations imposed by grammar, we design a novel Grammar Aware Contrastive Learning (GACL) module, integrating complex grammar rules into the transcription model through a contrastive learning mechanism. Furthermore, primitive contrastive learning lacks clear directions for comprehending grammar rules and can lead to unstable convergence or prolonged training cycles. To enhance training efficiency, we propose Rank-Based Sampling (RBS) specialized for multi-line formulas, which guides the learning process by the importance ranking of different grammar errors. Finally, spatial location information is critical considering the two-dimensional nature of multi-line formulas. To aid the model in keeping track of that global information, we introduced a Visual Coverage (VC) mechanism that incorporates historical attention information into the image features via a parameter-free way. To validate the effectiveness of our GAP framework, we construct a new dataset Multi-Line containing 12,002 multi-line formulas and conduct extensive experiments to show the efficacy of our GAP framework in capturing grammatical rules, enhancing recognition accuracy, and enhancing training efficiency. Codes and datasets are available at https://github.com/Sinon02/GAP.

Personalized learner modeling using cognitive diagnosis (CD), which aims to model learners' cognitive states by diagnosing learner traits from behavioral data, is a fundamental yet significant task in many web learning services. Existing cognitive diagnosis models (CDMs) follow theproficiency-response paradigm that views learner traits and question parameters as trainable embeddings and learns them through learner performance prediction. However, we notice that this paradigm leads to the inevitable non-identifiability and explainability overfitting problem, which is harmful to the quantification of learners' cognitive states and the quality of web learning services. To address these problems, we propose an identifiable cognitive diagnosis framework (ID-CDF) based on a novelresponse-proficiency-response paradigm inspired by encoder-decoder models. Specifically, we first devise the diagnostic module of ID-CDF, which leverages inductive learning to eliminate randomness in optimization to guarantee identifiability and captures the monotonicity between overall response data distribution and cognitive states to prevent explainability overfitting. Next, we propose a flexible predictive module for ID-CDF to ensure diagnosis preciseness. We further present an implementation of ID-CDF, i.e., ID-CDM, to illustrate its usability. Extensive experiments on four real-world datasets with different characteristics demonstrate that ID-CDF can effectively address the problems without loss of diagnosis preciseness. Our code is available at https://github.com/CSLiJT/ID-CDF.

Cognitive diagnosis models have been widely used in different areas, especially intelligent education, to measure users' proficiency levels on knowledge concepts, based on which users can get personalized instructions. As the measurement is not always reliable due to the weak links of the models and data, the uncertainty of measurement also offers important information for decisions. However, the research on the uncertainty estimation lags behind that on advanced model structures for cognitive diagnosis. Existing approaches have limited efficiency and leave an academic blank for sophisticated models which have interaction function parameters (e.g., deep learning-based models). To address these problems, we propose a unified uncertainty estimation approach for a wide range of cognitive diagnosis models. Specifically, based on the idea of estimating the posterior distributions of cognitive diagnosis model parameters, we first provide a unified objective function for mini-batch based optimization that can be more efficiently applied to a wide range of models and large datasets. Then, we modify the reparameterization approach in order to adapt to parameters defined on different domains. Furthermore, we decompose the uncertainty of diagnostic parameters into data aspect and model aspect, which better explains the source of uncertainty. Extensive experiments demonstrate that our method is effective and can provide useful insights into the uncertainty of cognitive diagnosis.

Graph Neural Networks (GNNs) have achieved impressive results in graph classification tasks, but they struggle to generalize effectively when faced with out-of-distribution (OOD) data. Several approaches have been proposed to address this problem. Among them, one solution is to diversify training distributions in vanilla classification by modifying the data environment, yet accessing the environment information is complex. Besides, another promising approach involves rationalization, extracting invariant rationales for predictions. However, extracting rationales is difficult due to limited learning signals, resulting in less accurate rationales and diminished predictions. To address these challenges, in this paper, we propose a Cooperative Classification and Rationalization (C2R) method, consisting of theclassification and therationalization module. Specifically, we first assume that multiple environments are available in theclassification module. Then, we introduce diverse training distributions using an environment-conditional generative network, enabling robust graph representations. Meanwhile, therationalization module employs a separator to identify relevant rationale subgraphs while the remaining non-rationale subgraphs are de-correlated with labels. Next, we align graph representations from theclassification module with rationale subgraph representations using the knowledge distillation methods, enhancing the learning signal for rationales. Finally, we infer multiple environments by gathering non-rationale representations and incorporate them into theclassification module for cooperative learning. Extensive experimental results on both benchmarks and synthetic datasets demonstrate the effectiveness of C2R. Code is available at https://github.com/yuelinan/Codes-of-C2R.

Meta-learning has been widely employed to tackle the cold-start problem in user modeling. Similar to a guidebook for a new traveler, meta-learning significantly affects decision-making for new users in crucial scenarios, such as career recommendations. Consequently, the issue of fairness in meta-learning has gained paramount importance. Several methods have been proposed to mitigate unfairness in meta-learning and have shown promising results. However, a fundamental question remains unexplored: What is the critical factor leading to unfairness in meta-learned user modeling? Through the theoretical analysis that integrates the meta-learning paradigm with group fairness metrics, we identify group proportion imbalance as a critical factor. Subsequently, in order to mitigate the impact of this factor, we introduce a novel Fairness-aware Adaptive Sampling framework for meTa-learning, abbreviated as FAST. Its core concept involves adaptively adjusting the sampling distribution for different user groups during the interleaved training process of meta-learning. Furthermore, we provide theoretical guarantees demonstrating the convergence of FAST. Finally, empirical experiments conducted on three datasets reveal that FAST effectively enhances fairness while maintaining high accuracy. The code for FAST is available at https://github.com/zhengz99/FAST.

The field of education has undergone a significant revolution with the advent of intelligent systems and technology, which aim to personalize the learning experience, catering to the unique needs and abilities of individual learners. In this pursuit, a fundamental challenge is designing proper test for assessing the students’ cognitive status on knowledge and skills accurately and efficiently. One promising approach, referred to as Computerized Adaptive Testing (CAT), is to administrate computer-automated tests that alternately select the next item for each examinee and estimate their cognitive states given their responses to the selected items. Nevertheless, existing CAT systems suffer from inflexibility in item selection and ineffectiveness in cognitive state estimation, respectively. In this article, we propose a Model-Agnostic adaptive testing framework via Meta-leaned Gradient Embeddings, MAMGE for short, improving both item selection and cognitive state estimation simultaneously. For item selection, we design a Gradient Embedding-based Item Selector (GEIS) which incorporates the concept of gradient embeddings to represent items and selects the best ones that are both informative and representative. For cognitive state estimation, we propose a Meta-learned Cognitive State Estimator (MCSE) to automatically control the estimation process by learning to learn a proper initialization and dynamically inferred updates. Both MCSE and GEIS are inherently model-agnostic, and the two modules have an ingenious connection via meta-learned gradient embeddings. Finally, extensive experiments evaluate the effectiveness and flexibility of MAMGE.

By predicting a personalized ranking on a set of items, item recommendation helps users determine the information they need. While optimizing a ranking-focused loss is more in line with the objectives of item recommendation, previous studies have indicated that current sampling-based ranking methods don't always surpass non-sampling ones. This is because it is either inefficient to sample a pool of representative negatives for better generalization or challenging to gauge their contributions to ranking-focused losses accurately. To this end, we propose a novel weighted ranking loss, which weights each negative with the softmax probability based on model's predictive score. Our theoretical analysis suggests that optimizing this loss boosts the normalized discounted cumulative gain. Furthermore, it appears that this loss acts as an approximate analytic solution for adversarial training of personalized ranking. To improve optimization efficiency, we approximate the weighted ranking loss with self-normalized importance sampling and show that the loss has good generalization properties. To improve generalization, we further develop efficient cluster-based negative samplers based on clustering over item vectors, to decrease approximation error caused by the divergence between the proposal and the target distribution. Comprehensive evaluations on real-world datasets show that our methods remarkably outperform leading item recommendation algorithms.

Recent years have witnessed increasing attention being paid to AI-based quantitative investment. Compared to traditional low-frequency data (e.g., daily, weekly), high-frequency data (e.g., minute-level) is often underutilized for low-frequency stock trend prediction, leaving the vast potential for improvement. However, valuable and noisy information coexist in high-frequency data. The learning process of high-frequency factor extractors can easily be overwhelmed by noise, leading to overfitting. Moreover, common techniques used to prevent overfitting often result in poor performance on this task since they usually roughly restrict the model’s capacity, making it challenging to model complex trading signals in high-frequency data. When designing high-frequency factor extractors, we face a tough dilemma. A high-capacity model may easily overfit to noise, while a simple but robust model may not capture complex high-frequency patterns. To address these problems, we propose maintaining model capacity while preventing overfitting by constructing two components that balance information and noise through interactions between them. Specifically, we propose a novel learning framework calledDigger-Guiderto extract informative stock representations from noisy high-frequency data. We develop a high-capacity model calledDiggerto extract local and detailed features from the high-frequency data, and we design a robust model calledGuiderto capture global tendency features and help the Digger overcome the noise. The Digger and Guider enhance each other through mutual distillation during training, serving as data-driven regularizations that work well on this task. Extensive experiments on real-world datasets demonstrate that our framework can produce powerful high-frequency stock factors that significantly improve stock trend prediction performance and our understanding of the finance market.

Massive legal documents have promoted the application of legal intelligence. Among them, Legal Judgment Prediction (LJP) has emerged as a critical task, garnering significant attention. LJP aims to predict judgment results for multiple subtasks, including charges, law articles, and terms of penalty. Existing studies primarily focus on utilizing the entire factual description to produce judgment results, overlooking the practical judicial scenario where judges consider various crime circumstances to decide verdicts and sentencing. To this end, in this paper, we propose a circumstance-aware LJP framework (i.e., NeurJudge) by exploring the circumstances of crime. Specifically, NeurJudge first separates the factual description into different circumstances with the predicted results of intermediate subtasks and then employs them to yield results of other subtasks. Besides, as confusing verdicts may degrade the performance of LJP, we further develop a variant of NeurJudge (NeurJudge+) that incorporates the semantics of labels (charges and law articles) into facts to yield more expressive and distinguishable fact representations. Finally, to provide explanations for LJP, we extend NeurJudge to an explainable LJP framework E-NeurJudge with a cooperative teacher-student system. The teacher system is NeurJudge which exploits legal particularities well but lacks explanation capability. The student system is a rationalization method that provides explainability but fails to utilize legal particularities. To combine the advantages of the above methods, we use a transferring function to transfer legal particularities from the teacher to the student, making a trade-off between yielding LJP results and rendering them explainable. Extensive experimental results on real-world datasets validate the effectiveness of our proposed frameworks.

Computerized Adaptive Testing (CAT) arises as a promising personalized test mode in online education, targeting at revealing students' latent knowledge state by selecting test items adaptively. The item selection strategy is the core component of CAT, which searches for the best suitable test item based on students' current estimated ability at each test step. However, existing selection strategies behave in a brute-force manner, which results in the time complexity being linear to the number of items (N) in the item pool, i.e., O(N). Thus, in reality, the search latency becomes the bottleneck for CAT with a large-scale item pool. To this end, we propose a Search-Efficient Computerized Adaptive Testing framework (SECAT), which aims at enhancing CAT with an efficient selection strategy. Specifically, SECAT contains two main phases: item pool indexing and item search. In the item pool indexing phase, we apply a student-aware spatial partition method on the item pool to divide the test items into many sub-spaces, considering the adaptability of test items. In the item search phase, we optimize the traditional single-round search strategy with the asymptotic theory and propose a multi-round search strategy that can further improve the time efficiency. Compared with existing strategies, the time complexity of SECAT decreases from O(N) to O(logN). Across two real-world datasets, SECAT achieves over 200x speed up with negligible accuracy degradation.

Click-Through Rate (CTR) prediction, estimating the probability of a user clicking on an item, is essential in industrial applications, such as online advertising. Many works focus on user behavior modeling to improve CTR prediction performance. However, most of those methods only model users' positive interests from users' click items while ignoring the context information, which is the display items around the clicks, resulting in inferior performance. In this paper, we highlight the importance of context information on user behavior modeling and propose a novel model named Deep Context Interest Network (DCIN), which integrally models the click and its display context to learn users' context-aware interests. DCIN consists of three key modules: 1) Position-aware Context Aggregation Module (PCAM), which performs aggregation of display items with an attention mechanism; 2) Feedback-Context Fusion Module (FCFM), which fuses the representation of clicks and display contexts through non-linear feature interaction; 3) Interest Matching Module (IMM), which activates interests related with the target item. Moreover, we provide our hands-on solution to implement DCIN on large-scale industrial systems. The significant improvements in both offline and online evaluations demonstrate the superiority of our proposed DCIN method. Notably, DCIN has been deployed on our online advertising system serving the main traffic, which brings 1.5% CTR and 1.5% RPM lift.

Neural-based multi-task learning (MTL) has gained significant improvement, and it has been successfully applied to recommendation system (RS). Recent deep MTL methods for RS (e.g. MMoE, PLE) focus on designing soft gating-based parameter-sharing networks that implicitly learn a generalized representation for each task. However, MTL methods may suffer from performance degeneration when dealing with conflicting tasks, as negative transfer effects can occur on the task-shared bottom representation. This can result in a reduced capacity for MTL methods to capture task-specific characteristics, ultimately impeding their effectiveness and hindering the ability to generalize well on all tasks. In this paper, we focus on the bottom representation learning of MTL in RS and propose the Deep Task-specific Bottom Representation Network (DTRN) to alleviate the negative transfer problem. DTRN obtains task-specific bottom representation explicitly by making each task have its own representation learning network in the bottom representation modeling stage. Specifically, it extracts the user's interests from multiple types of behavior sequences for each task through the parameter-efficient hypernetwork. To further obtain the dedicated representation for each task, DTRN refines the representation of each feature by employing a SENet-like network for each task. The two proposed modules can achieve the purpose of getting task-specific bottom representation to relieve tasks' mutual interference. Moreover, the proposed DTRN is flexible to combine with existing MTL methods. Experiments on one public dataset and one industrial dataset demonstrate the effectiveness of the proposed DTRN.

Representation learning of examination papers is significantly crucial for online education systems, as it benefits various applications such as estimating paper difficulty and examination paper retrieval. Previous works mainly explore the representation learning of individual questions in an examination paper, with limited attention given to the examination paper as a whole. In fact, the structure of examination papers is strongly correlated with paper properties such as paper difficulty, which existing paper representation methods fail to capture adequately. To this end, we propose a pre-trained model namely PaperLM to learn the representation of examination papers. Our model integrates both the text content and hierarchical structure of examination papers within a single framework by converting the path of the Examination Organization Tree (EOT) into embedding. Furthermore, we specially design three pre-training objectives for PaperLM, namely EOT Node Relationship Prediction (ENRP), Question Type Prediction (QTP) and Paper Contrastive Learning (PCL), aiming to capture features from text and structure effectively. We pre-train our model on a real-world examination paper dataset, and then evaluate the model with three down-stream tasks: paper difficulty estimation, examination paper retrieval, and paper clustering. The experimental results demonstrate the effectiveness of our method.

Researchers have successfully adapted the privacy-preserving Federated Learning (FL) to news recommendation tasks to better protect users' privacy, although typically at the cost of performance degradation due to the data heterogeneity issue. To address this issue, Personalized Federated Learning (PFL) has emerged, among which model interpolation is a promising approach that interpolates the local personalized models with the global model. However, the existing model interpolation method may not work well for news recommendation tasks for some reasons. First, it neglects the fine-grained personalization needs at both the temporal and spatial levels in news recommendation tasks. Second, due to the cold-user problem in real-world news recommendation tasks, the local personalized models may perform poorly, thus limiting the performance gain from model interpolation. To this end, we propose FINDING (Federated News Recommendation with Fine-grained Interpolation and Dynamic Clustering ), a novel personalized federated learning framework based on model interpolation. Specifically, we first propose the fine-grained model interpolation strategy which interpolates the local personalized models with the global model in a time-aware and layer-aware way. Then, to address the cold-user problem in news recommendation tasks, we adopt the group-level personalization approach where users are dynamically clustered into groups and the group-level personalized models are used for interpolation. Extensive experiments on two real-world datasets show that our method can effectively handle the above limitations of the current model interpolation method and alleviate the heterogeneity issue faced by traditional FL.

The fundamental task of intelligent educational systems is to offer adaptive learning services to students, such as exercise recommendations and computerized adaptive testing. However, optimizing required models in these systems would always encounter the collection difficulty of high-quality interaction data in practice. Therefore, establishing a student simulator is of great value since it can generate valid interactions to help optimize models. Existing advances have achieved success but generally suffer from exposure bias and overlook long-term intentions. To tackle these problems, we propose a novel Direct-Adversarial Imitation Student Simulator (DAISim) by formulating it as a Markov Decision Process (MDP), which unifies the workflow of the simulator in training and generating to alleviate the exposure bias and single-step optimization problems. To construct the intentions underlying the complex student interactions, we first propose a direct imitation strategy to mimic the interactions with a simple reward function. Then, we propose an adversarial imitation strategy to learn a rational distribution with the reward given by a parameterized discriminator. Furthermore, we optimize the discriminator in adversarial imitation in a pairwise manner, and the theoretical analysis shows that the pairwise discriminator would improve the generation quality. We conduct extensive experiments on real-world datasets, where the results demonstrate that our DAISim can simulate high-quality student interactions whose distribution is close to real distribution and can promote several downstream services.

Collaborative filtering (CF) from implicit datasets has attracted much attention in recent years. The current mainstream pairwise methods optimize the Area Under the Curve (AUC) and are empirically proven to be helpful to exploit implicit feedback, but lead to not addressing the rank-biased scenarios where positive items are supposed to be placed on the top-k positions. Although there exist listwise methods, they have low efficiency and are not particularly adequate for general implicit feedback situations. To that end, in this paper, we propose a new framework, namely Collaborative List-and-Pairwise Filtering (CLAPF), which aims to introduce pairwise thinking into listwise methods. Specifically, we first smooth a well-known rank-biased measure called Mean Average Precision (MAP) as a low-bound version to make it can be optimized. After that, we combined the objective functions of optimizing the MAP with pairwise comparisons. The CLAPF framework is a new hybrid model that provides the idea of utilizing a listwise measure in a pairwise way on implicit feedback.

Math formulas (e.g., "distance = speed X time'') serve as one of the fundamental commonsense knowledge in human cognition, where humans naturally acquire and manipulate them in logical thinking for mathematical reasoning problems. However, existing reasoning models mainly focus on learning heuristic linguistics or patterns to generate answers, but do not pay enough attention on learning with such formula knowledge. Thus, they are not transparent (thus uninterpretable) in terms of understanding and grasping basic mathematical logic. In this paper, to promote a step forward in the domain, we first construct two datasets (Math23K-F and MAWPS-F) with precise annotations of formula usage in each reasoning step for math word problems. Especially, our datasets are refined on the benchmark datasets, and thus ensure the generality and comparability for relevant research. Then, we propose a novel Formula-mastered Solver (FOMAS) with the guidance of mastering formula knowledge to solve the problems. Specifically, we establish FOMAS with two systems drawing insight from the dual process theory, including a Knowledge System and a Reasoning System, to learn and apply formula knowledge, respectively. The Knowledge System accumulates the math formulas, where we propose a novel pretraining manner to mimic how humans grasp the mathematical logic behind them. Then, in the Reasoning System, we develop elaborate formula-guided symbol prediction and goal generation methods that retrieve the necessary formula knowledge from Knowledge System to improve both reasoning accuracy and interpretability. It organically simulates how humans conduct complex reasoning under the explicit instruction of math formulas. Experimental results prove that FOMAS has a stronger reasoning ability and achieves a more interpretable reasoning process, which verifies the necessity of introducing formula knowledge transparently.

Zero-Shot Learning (ZSL), which aims at automatically recognizing unseen objects, is a promising learning paradigm to understand new real-world knowledge for machines continuously. Recently, the Knowledge Graph (KG) has been proven as an effective scheme for handling the zero-shot task with large-scale and non-attribute data. Prior studies always embed relationships of seen and unseen objects into visual information from existing knowledge graphs to promote the cognitive ability of the unseen data. Actually, real-world knowledge is naturally formed by multimodal facts. Compared with ordinary structural knowledge from a graph perspective, multimodal KG can provide cognitive systems with fine-grained knowledge. For example, the text description and visual content can depict more critical details of a fact than only depending on knowledge triplets. Unfortunately, this multimodal fine-grained knowledge is largely unexploited due to the bottleneck of feature alignment between different modalities. To that end, we propose a multimodal intensive ZSL framework that matches regions of images with corresponding semantic embeddings via a designed dense attention module and self-calibration loss. It makes the semantic transfer process of our ZSL framework learns more differentiated knowledge between entities. Our model also gets rid of the performance limitation of only using rough global features. We conduct extensive experiments and evaluate our model on large-scale real-world data. The experimental results clearly demonstrate the effectiveness of the proposed model in standard zero-shot classification tasks.

Exploring how learners' knowledge states evolve during the learning activities is a critical task in online learning systems, which can facilitate personalized services downstream, such as course recommendation. Most of existing methods have devoted great efforts to analyzing learners' knowledge states according to their responses (i.e., right or wrong) to different questions. However, the significant effect of learners' learning behaviors (e.g., answering speed, the number of attempts) is omitted, which can reflect their knowledge acquisition deeper and ensure the reliability of the response. In this paper, we propose a Learning Behavior-oriented Knowledge Tracing (LBKT) model, with the goal of explicitly exploring the learning behavior effects on learners' knowledge states. Specifically, we first analyze and summarize several dominated learning behaviors including Speed, Attempts and Hints in the learning process. As the characteristics of different learning behaviors vary greatly, we separately estimate their various effects on learners' knowledge acquisition in a quantitative manner. Then, considering that different learning behaviors are closely dependent with each other, we assess the fused effect of multiple learning behaviors by capturing their complex dependent patterns. Finally, we integrate the forgetting factor with learners' knowledge acquisition to comprehensively update their changing knowledge states in learning. Extensive experimental results on several public datasets demonstrate that our model generates better performance prediction for learners against existing methods. Moreover, LBKT shows good interpretability in tracking learners' knowledge state by incorporating the learning behavior effects. Our codes are available at https://github.com/xbh0720/LBKT.

Click-Through Rate (CTR) prediction of intelligent marketing systems is of great importance, in which feature interaction selection plays a key role. Most approaches model interactions of features by the same pre-defined operation under expert guidance, among which improper interactions may bring unnecessary noise and complicate the training process. To that end, in this paper, we aim to adaptively evolve the model to select proper operations to interact on feature pairs under task guidance. Inspired by natural evolution, we propose a general Cognitive EvoLutionary Search (CELS) framework, where cognitive ability refers to the malleability of organisms to orientate to the environment. Specifically, we conceptualize interactions as genomes, models as organisms, and tasks as natural environments. Mirroring how genetic malleability develops environmental adaptability, we thus diagnose the fitness of models to simulate the survival rates of organisms for natural selection, thereby an evolution path can be planned and visualized, offering an intuitive interpretation of the mechanisms underlying interaction modeling and selection. Based on the CELS framework, we develop four instantiations including individual-based search and population-based search. We demonstrate how individual mutation and population crossover enable CELS to evolve into diverse models suitable for various tasks and data, providing ready-to-use models. Extensive experiments on real-world datasets demonstrate that CELS significantly outperforms state-of-the-art approaches.

In recent years, pairwise methods, such as Bayesian Personalized Ranking (BPR), have gained significant attention in the field of collaborative filtering for recommendation systems. Group BPR is an extension of BPR that incorporates user groups to relax the strict assumption of independence between two users. However, the reliability of its user groups may be compromised as they only focus on a few behavioral similarities. To address this problem, this paper proposes a new entropy-weighted similarity measure for implicit feedback to quantify the relation between two users and sample like-minded user groups. We first introduce the group preference into several pairwise ranking algorithms and then utilize the entropy-weighted similarity to sample groups to further improve these algorithms. Unlike other approaches that rely solely on common item ratings, our method incorporates global information into the similarity measure, resulting in a more reliable approach to group sampling. We conducted experiments on two real-world datasets and evaluated our method using different metrics. The results show that our method can construct better user groups from sparse data and produce more accurate recommendations. Our approach can be applied to a wide range of recommendation systems, and this can significantly improve the performance of pairwise ranking algorithms, making it an effective tool for pairwise ranking.

Cognitive diagnosis (CD) aims to reveal the proficiency of students on specific knowledge concepts and traits of test exercises (e.g., difficulty). It plays a critical role in intelligent education systems by supporting personalized learning guidance. However, recent developments in CD mostly concentrate on improving the accuracy of diagnostic results and often overlook the important and practical task: domain-level zero-shot cognitive diagnosis (DZCD). The primary challenge of DZCD is the deficiency of student behavior data in the target domain due to the absence of student-exercise interactions or unavailability of exercising records for training purposes. To tackle the cold-start issue, we propose a two-stage solution named TechCD (Transferable knowledgE Concept grapH embedding framework for Cognitive Diagnosis). The fundamental notion involves utilizing a pedagogical knowledge concept graph (KCG) as a mediator to connect disparate domains, allowing the transmission of student cognitive signals from established domains to the zero-shot cold-start domain. Specifically, a naive yet effective graph convolutional network (GCN) with the bottom-layer discarding operation is initially employed over the KCG to learn transferable student cognitive states and domain-specific exercise traits. Moreover, we give three implementations of the general TechCD framework following the typical cognitive diagnosis solutions. Finally, extensive experiments on real-world datasets not only prove that Tech can effectively perform zero-shot diagnosis, but also give some popular applications such as exercise recommendation.

Automatically generating controllable and diverse mathematical word problems (MWPs) which conform to equations and topics is a crucial task in information retrieval and natural language generation. Recent deep learning models mainly focus on improving the problem readability but overlook the mathematical logic coherence, which tends to generate unsolvable problems. In this paper, we draw inspiration from the human problem-designing process and propose a Mathematical structure Planning and Knowledge enhanced Generation model (MaPKG), following the "plan-then-generate" steps. Specifically, we propose a novel dynamic planning module to make sentence-level equation plans and a dual-attention mechanism for word-level generation, incorporating equation structure representation and external commonsense knowledge. Extensive experiments on two MWP datasets show our model can guarantee more solvable, high-quality, and diverse problems. Our code is available at https://github.com/KenelmQLH/MaPKG.git

Deep learning-based algorithms, e.g., convolutional networks, have significantly facilitated multivariate time series classification (MTSC) task. Nevertheless, they suffer from the limitation in modeling long-range dependence due to the nature of convolution operations. Recent advancements have shown the potential of transformers to capture long-range dependence. However, it would incur severe issues, such as fixed scale representations, temporal-invariant and quadratic time complexity, with transformers directly applicable to the MTSC task because of the distinct properties of time series data. To tackle these issues, we propose FormerTime, an hierarchical representation model for improving the classification capacity for the MTSC task. In the proposed FormerTime, we employ a hierarchical network architecture to perform multi-scale feature maps. Besides, a novel transformer encoder is further designed, in which an efficient temporal reduction attention layer and a well-informed contextual positional encoding generating strategy are developed. To sum up, FormerTime exhibits three aspects of merits: (1) learning hierarchical multi-scale representations from time series data, (2) inheriting the strength of both transformers and convolutional networks, and (3) tacking the efficiency challenges incurred by the self-attention mechanism. Extensive experiments performed on 10 publicly available datasets from UEA archive verify the superiorities of the FormerTime compared to previous competitive baselines.

In online games, predicting massive battle outcomes is a fundamental task of many applications, such as team optimization and tactical formulation. Existing works do not pay adequate attention to the massive battle. They either seek to evaluate individuals in isolation or mine simple pair-wise interactions between individuals, neither of which effectively captures the intricate interactions between massive units (e.g., individuals). Furthermore, as the team size increases, the phenomenon of diminishing marginal utility of units emerges. Such a diminishing pattern is rarely noticed in previous work, and how to capture it from data remains a challenge. To this end, we propose a novel Massive battle outcome predictor with margiNal Effect modules, namely MassNE, which comprehensively incorporates individual effects, cooperation effects (i.e., intra-team interactions) and suppression effects (i.e., inter-team interactions) for predicting battle outcomes. Specifically, we design marginal effect modules to learn how units’ marginal utility changing respect to their number, where the monotonicity assumption is applied to ensure rationality. In addition, we evaluate the current classical models and provide mathematical proofs that MassNE is able to generalize several earlier works in massive settings. Massive battle datasets generated by StarCraft II APIs are adopted to evaluate the performances of MassNE. Extensive experiments empirically demonstrate the effectiveness of MassNE, and MassNE can reveal reasonable cooperation effects, suppression effects, and marginal utilities of combat units from the data.

Knowledge Tracing (KT) aims at tracing the evolution of the knowledge states along the learning process of a learner. It has become a crucial task for online learning systems to model the learning process of their users, and further provide their users a personalized learning guidance. However, recent developments in KT based on deep neural networks mostly focus on increasing the accuracy of predicting the next performance of students. We argue that current KT modeling, as well as training paradigm, can lead to models tracing patterns of learner’s learning activities, instead of their evolving knowledge states. In this paper, we propose a new architecture, Diagnostic Transformer (DTransformer), along with a new training paradigm, to tackle this challenge. With DTransformer, we build the architecture from question-level to knowledge-level, explicitly diagnosing learner’s knowledge proficiency from each question mastery states. We also propose a novel training algorithm based on contrastive learning that focuses on maintaining the stability of the knowledge state diagnosis. Through extensive experiments, we will show that with its understanding of knowledge state evolution, DTransformer achieves a better performance prediction accuracy and more stable knowledge state tracing results. We will also show that DTransformer is less sensitive to specific patterns with case study. We open-sourced our code and data at https://github.com/yxonic/DTransformer.

In recent years, due to the explosive growth of patent applications, patent mining has drawn extensive attention and interest. An important issue of patent mining is that of recognizing the technologies contained in patents, which serves as a fundamental preparation for deeper analysis. To this end, in this article, we make a focused study on constructing a technology portrait for each patent, i.e., to recognize technical phrases concerned in it, which can summarize and represent patents from a technical perspective. Along this line, a critical challenge is how to analyze the unique characteristics of technical phrases and illustrate them with definite descriptions. Therefore, we first generate the detailed descriptions about the technical phrases existing in extensive patents based on different criteria, including various previous works, practical experience, and statistical analyses. Then, considering the unique characteristics of technical phrases and the complex structure of patent documents, such as multi-aspect semantics and multi-level relevances, we further propose a novel unsupervised model, namely TechPat, which can not only automatically recognize technical phrases from massive patents but also avoid the need for expensive human labeling. After that, we evaluate the extraction results from various aspects. Specifically, we propose a novel evaluation metric called Information Retrieval Efficiency (IRE) to quantify the performance of extracted technical phrases from a new perspective. Extensive experiments on real-world patent data demonstrate that the TechPat model can effectively discriminate technical phrases in patents and greatly outperform existing methods. We further apply extracted technical phrases to two practical application tasks, namely patent search and patent classification, where the experimental results confirm the wide application prospects of technical phrases. Finally, we discuss the generalization ability of our proposed methods.

Graph Neural Networks (GNNs) such as Graph Convolutional Networks (GCNs) can effectively learn node representations via aggregating neighbors based on the relation graph. However, despite a few exceptions, most of the previous work in this line does not consider the topical semantics underlying the edges, making the node representations less effective and the learned relation between nodes hard to explain. For instance, the current GNNs make us usually don’t know what is the reason for the connection of network nodes, such as the specific research topics cited in this article and the concerns among friends on social platforms. Some methods have begun to explore the extraction of relation semantics in recent related literature, but existing studies generally face two bottlenecks, i.e., either being unable to explain the mined latent relations to ensure their reasonableness and independence, or demanding the textual content of edges which is unavailable in most real-world datasets. Actually, these two issues are both crucial in practical use. In our work, we propose a novel Topic-Disentangled Graph Neural Network (TDG) to address the above two issues at the same time, which explores the relation topics from the perspective of node contents. We design an optimized graph topic module to handle node features to construct independent and explainable semantic subspaces, then the reasonable relation topics that correspond to these subspaces are assigned to each graph relation via a neighborhood routing mechanism. Our proposed model can be easily combined with related graph tasks to form an end-to-end model, to avoid the risk of deviation between node representation space and task space. To evaluate the efficiency of our model, sufficient node-related tasks are conducted on three public datasets in the experimental section. The results show the obvious superiority of TDG compared with the state-of-the-art models.

Hierarchical multi-label classification (HMC) deals with the problem of assigning each entity to multiple classes with a taxonomic structure (e.g., tree). Within this structure, classes at different levels tend to have dependencies under the hierarchy constraints. However, most prior studies for HMC tasks tend to ignore the class dependencies within the hierarchy. Moreover, most existing methods generate incoherent predictions and do not satisfy the hierarchy constraint. To this end, based on previously developed HARNN, we propose a general framework, HmcNet, for introducing explicit and implicit class hierarchy constraints to generate coherent predictions. We develop an efficient Prune-based Coherent Prediction (PCP) strategy for the optimal paths selection, which produces coherent predictions in a principled way. HmcNet can be well explained from two perspectives. First, it develops the Hierarchical Attention-based Memory (HAM) unit with implicit class hierarchy constraints to capture class dependencies more intuitively; Second, it subsumes explicit class hierarchy constraints during training and inference phases and generates coherent predictions in a consistent manner. Finally, extensive experimental results on six real-world datasets demonstrate the effectiveness and interpretability of the HmcNet frameworks. To facilitate future research, our code has been made publicly available.

Reading psychology believes text comprehension to involve a complex psychological construction process, with the reader mind being a dynamic associative system that stores an abundance of schemata. For Chinese text, in particular, the unique ideographic writing system allows its lansign to trigger semantic association and schema recalling without the need of phonetics. In contrast to previous research efforts on text classification problems, in this paper we present an interdisciplinary modeling approach that draws inspirations from the cognitive principles of ideography, schema theory and deep learning to study Chinese text classification. Specifically, we first propose a Radical-guided Associative Model (RAM) for preliminary cognitive imitation, which comprises two coupled spaces, namely the Literal Space and Associative Space. Then, taking consideration of the schemata acquired from the mind of a reader which plays a important role in influencing text-dependent information revision, we extend RAM with a systematic Schema-aware Radical-guided Associative Model (SRAM) that embeds label semantics as essential text-independent human knowledge for real-world abstraction. In SRAM, the Schema Space is introduced and a Schema Attention module is proposed with a novel loss paradigm that includes the linkage and interaction between text-dependent prior concepts and text-independent label schemata. Extensive experiments on three real-world datasets demonstrate the effectiveness and rationality of our proposed method.

Cognitive diagnosis is widely applicable in the scenarios where users’ cognitive states need to be assessed, such as games and clinical measurement. Especially in intelligent education, which has become increasingly popular recent decades, cognitive diagnosis serves as a fundamental module for discovering the proficiency level of students on specific knowledge concepts. Existing approaches usually mine linear interactions of student exercising process by manually designed function (e.g., logistic function). However, the cognitive interactions between students and exercises is a complex process, and excessive simplifications would lead to under fitting and thus get inaccurate diagnostic results. Besides, the manually designed interaction functions are relatively inflexible and limits their extensibility. This consequently causes lack of consideration about useful non-numerical information in the cognitive process besides response logs. In this article, we propose a general Neural Cognitive Diagnosis (NeuralCD) framework as well as several implemented models (a basic implementation NeuralCDM and three extensions), where we project students and exercises to factor vectors and incorporates neural networks to learn the complex exercising interactions. To ensure the interpretability of diagnostic results, which is essential for cognitive diagnosis, we apply an monotonicity assumption to our NeuralCD framework. Moreover, NeuralCD is a general framework and has good extensibility. We show the generality of NeuralCD through proving how it can cover some traditional models. Then, we demonstrate the extensibility of NeuralCD, which benefits future developments. On one hand, we demonstrate content-based extensions where we provide examples of exploring the rich contents of exercise texts (CNCD-Q and CNCD-F). On the other hand, we demonstrate a knowledge-association based extension to show that NeuralCD is flexible for structural adjustments so as to solve specific problems. For instance, we improve the diagnostic results on uncovered knowledge concepts of a student by extending NeuralCD with the knowledge associations consideration (KaNCD). Extensive experimental results on real-world datasets show the effectiveness of NeuralCD framework with both accuracy and interpretability.

Network representation learning or network embedding has attracted wide attention due to its effectiveness on various network-oriented applications in recent years. Though large efforts have been made, they usually model the interactions between nodes reflected by network structure and attributes in a coupled way. To this end, in this article, we comprehensively study the problem of learning attributed network embedding, which focuses on characterizing different types of interactions among nodes and alleviating the sparse attribute problem as well. Specifically, we propose a novel DeCoupled Network Embedding (DCNE) model to learn node representations in a unified framework. We first respectively project both nodes and attributes into low-dimensional vectorial space. Then, we introduce a novel ‘`decoupled-fusion’' learning process into each graph layer to iteratively generate the node embeddings. In particular, we propose two adapted graph convolution modules to decouple the learning of network structure and attributes respectively, and a fusion module to adaptively aggregate the information. Next, we adopt a modified mini-batch algorithm to iteratively aggregate the higher-order information of both nodes and attributes within a multi-task learning framework. Extensive experiments on five public datasets demonstrate that DCNE could outperform state-of-the-art methods on multiple benchmark tasks.

Many data mining tasks rely on graphs to model relational structures among individuals (nodes). Since relational data are often sensitive, there is an urgent need to evaluate the privacy risks in graph data. One famous privacy attack against data analysis models is the model inversion attack, which aims to infer sensitive data in the training dataset and leads to great privacy concerns. Despite its success in grid-like domains, directly applying model inversion attacks on non-grid domains such as graph leads to poor attack performance. This is mainly due to the failure to consider the unique properties of graphs. To bridge this gap, we conduct a systematic study on model inversion attacks against Graph Neural Networks (GNNs), one of the state-of-the-art graph analysis tools in this paper. First, in the white-box setting where the attacker has full access to the target GNN model, we present GraphMI to infer the private training graph data. Specifically in GraphMI, a projected gradient module is proposed to tackle the discreteness of graph edges and preserve the sparsity and smoothness of graph features; a graph auto-encoder module is used to efficiently exploit graph topology, node attributes, and target model parameters for edge inference; a random sampling module can finally sample discrete edges. Furthermore, in the hard-label black-box setting where the attacker can only query the GNN API and receive the classification results, we propose two methods based on gradient estimation and reinforcement learning (RL-GraphMI). With the proposed methods, we study the connection between model inversion risk and edge influence and show that edges with greater influence are more likely to be recovered. Extensive experiments over several public datasets demonstrate the effectiveness of our methods. We also evaluate our attacks under two defenses: one is the well-designed differential private training, and the other is graph preprocessing. Our experimental results show that such defenses are not sufficiently effective and call for more advanced defenses against privacy attacks.

Aspect-level sentiment analysis is a granular emotional classification task that refers to identifying sentiment polarities towards aspects in a sentence. Although previous research has reached a great achievement, this task remains very challenging. First, previous approaches only focus on one specific domain, which lacks the capability of transferring to other domains. Moreover, the majority of prior studies ignore the direct relationship between aspects and the corresponding sentiment words. To this end, in this paper, we propose a novel model named Efficient Adaptive Transfer Network (EATN) for aspect-level sentiment analysis which emphasizes the need of incorporating the correlation among multiple domains. The proposed EATN provides a Domain Adaptation Module (DAM) to learn common features from the sufficiently labeled source domain and to guide the classification performance in the target domain. Specifically, DAM comprises two special tasks, with one sentiment classification task aiming to learn sentiment knowledge and the other domain classification task focusing on learning domain-invariant features. Moreover, we design aspect-aware multi-head attention mechanism to capture the direct associations between the aspects and the contextual sentiment words, which is beneficial to learn the aspect-aware semantic knowledge. Extensive experiments demonstrate the effectiveness and universality of our method.

In recent decades, innovation and entrepreneurship have become buzz words. In reality, traditional research with empirical results is not practical for analyzing these newly launched projects of small and micro enterprises before production and sale. Actually, the future market prospect is an important criterion for evaluating entrepreneurial projects. However, this direction has not been well explored due to the limitations of scenarios and technical challenges especially for these small and micro enterprises. In this paper, we construct an interesting study of exploiting the market prospect from the sales markets (i.e., E-commerce) to help evaluate newly-posted campaigns in crowdfunding. Specifically, we propose a novel Market-oriented Prospect Analysis with Transferring Attention (MoPa-A) model which contains two learning modules, i.e., HostTask Learning and GuestTask Learning connected and enhanced by transferring attention. The former is designed for funding performance modeling with heterogeneous features of crowdfunding campaigns, and the latter is to represent and transfer the latent semantics of market prospect for target campaigns from campaigns comments with the help of relevant products in sales market. The model design of MoPa-A brings some new insights on flexible knowledge transfer for different or cross domains.

Similar text search aims to find texts relevant to a given query from a database, which is fundamental in many information retrieval applications, such as question search and exercise search. Since millions of texts always exist behind practical search engine systems, a well-developed text search system usually consists of recall and ranking stages. Specifically, the recall stage serves as the basis in the system, where the main purpose is to find a small set of relevant candidates accurately and efficiently. Towards this goal, deep semantic hashing, which projects original texts into compact hash codes, can support good search performance. However, learning desired textual hash codes is extremely difficult due to the following problems. First, compact hash codes (with short length) can improve retrieval efficiency, but the demand for learning compact hash codes cannot guarantee accuracy due to severe information loss. Second, existing methods always learn the unevenly distributed codes in the space from a local perspective, leading to unsatisfactory code-balance results. Third, a large fraction of textual data contains various types of noise in real-world applications, which causes the deviation of semantics in hash codes. To this end, in this paper, we first propose a general unsupervised encoder-decoder semantic hashing framework, namely MASH (short for Memory-bAsed Semantic Hashing), to learn the balanced and compact hash codes for similar text search. Specifically, with a target of retaining semantic information as much as possible, the encoder introduces a novel relevance constraint among informative high-dimensional representations to guide the compact hash code learning. Then, we design an external memory where the hashing learning can be optimized in the global space to ensure the code balance of the learning results, which can promote search efficiency. Besides, to alleviate the performance degradation problem of the model caused by text noise, we propose an improved SMASH (short for denoiSing Memory-bAsed Semantic Hashing) model by incorporating a noise-aware encoder-decoder framework. This framework considers the noise degree for each text from the semantic deviation aspect, ensuring the robustness of hash codes. Finally, we conduct extensive experiments in three real-world datasets. The experimental results clearly demonstrate the effectiveness and efficiency of MASH and SMASH in generating balanced and compact hash codes, as well as the superior denoising ability of SMASH.

News recommender systems have become an effective manner to help users make decisions by suggesting the potential news that users may click and read, which has shown the proliferation nowadays. Many representative algorithms made great efforts to discover users’ preferences from the histories for triggering news recommendations. However, there exist some limitations due to the following two main issues. First, they mainly rely on the sufficient user data, which cannot well capture users’ temporal interests with very limited records. Second, always perceiving users’ histories for recommendation may ignore some important news (e.g., breaking news). In this article, we propose a novel Multi-factors Fusion model for news recommendation by integrating both user-dependent preference effect and user-independent timeliness effect together. First, to track the preference of a certain user, we decompose her reading history into two user-related factors, including the long-term habit and the short-term interest. Specifically, we extract her persistent habit by exploring the category effect of news that she focuses on from her whole records. Then, we characterize her temporary interests by proposing a recurrent neural network of analyzing the homogeneous relations between her latest clicked news and the candidate ones. Second, to describe the user-independent news timeliness effect, we propose a novel survival analysis model to estimate the instantaneous click probability of a certain news as the occurring probability of an event, where much sensational news tends to be picked out. Last, we fuse all effects to determine the probability of a user clicking on a certain news under the independent event assumption. We conduct extensive experiments on two real-world datasets. Experimental results demonstrate that our model can generate better news recommendations on both general scenario and cold-start scenario.

The generation of large amounts of personal data provides data centers with sufficient resources to mine idiosyncrasy from private records. User modeling has long been a fundamental task with the goal of capturing the latent characteristics of users from their behaviors. However, centralized user modeling on collected data has raised concerns about the risk of data misuse and privacy leakage. As a result, federated user modeling has come into favor, since it expects to provide secure multi-client collaboration for user modeling through federated learning. Unfortunately, to the best of our knowledge, existing federated learning methods that ignore the inconsistency among clients cannot be applied directly to practical user modeling scenarios, and moreover, they meet the following critical challenges: 1) Statistical heterogeneity . The distributions of user data in different clients are not always independently identically distributed (IID), which leads to unique clients with needful personalized information; 2) Privacy heterogeneity . User data contains both public and private information, which have different levels of privacy, indicating that we should balance different information shared and protected; 3) Model heterogeneity . The local user models trained with client records are heterogeneous, and thus require a flexible aggregation in the server; 4) Quality heterogeneity . Low-quality information from inconsistent clients poisons the reliability of user models and offsets the benefit from high-quality ones, meaning that we should augment the high-quality information during the process. To address the challenges, in this paper, we first propose a novel client-server architecture framework, namely Hierarchical Personalized Federated Learning (HPFL), with a primary goal of serving federated learning for user modeling in inconsistent clients. More specifically, the client train and deliver the local user model via the hierarchical components containing hierarchical information from privacy heterogeneity to join collaboration in federated learning. Moreover, the client updates the personalized user model with a fine-grained personalized update strategy for statistical heterogeneity. Correspondingly, the server flexibly aggregates hierarchical components from heterogeneous user models in the case of privacy and model heterogeneity with a differentiated component aggregation strategy. In order to augment high-quality information and generate high-quality user models, we expand HPFL to the Augmented-HPFL (AHPFL) framework by incorporating the augmented mechanisms, which filters out low-quality information such as noise, sparse information and redundant information. Specially, we construct two implementations of AHPFL, i.e., AHPFL-SVD and AHPFL-AE, where the augmented mechanisms follow SVD (singular value decomposition) and AE (autoencoder), respectively. Finally, we conduct extensive experiments on real-world datasets, which demonstrate the effectiveness of both HPFL and AHPFL frameworks.

With the rapid development of intelligent education platforms, how to enhance the performance of diagnosing students' knowledge proficiency has become an important issue, e.g., by incorporating the prerequisite relation of knowledge concepts. Unfortunately, the differentiated influence from different predecessor concepts to successor concepts is still underexplored in existing approaches. To this end, we propose a Prerequisite Attention model for Knowledge Proficiency diagnosis of students (PAKP) to learn the attentive weights of precursor concepts on successor concepts and model it for inferring the knowledge proficiency. Specifically, given the student response records and knowledge prerequisite graph, we design an embedding layer to output the representations of students, exercises, and concepts. Influence coefficient among concepts is calculated via an efficient attention mechanism in a fusion layer. Finally, the performance of each student is predicted based on the mined student and exercise factors. Extensive experiments on real-data sets demonstrate that PAKP exhibits great efficiency and interpretability advantages without accuracy loss.

Social recommender systems play a key role in solving the problem of information overload. In order to better extract latent hierarchical property in the data, they usually explore the user-user connections and user-item interactions in hyperbolic space. Existing methods resort tangent spaces to realize some operations (e.g., matrix multiplication) on hyperbolic manifolds. However, frequently projecting between the hyperbolic space and the tangent space will destroy the global structure of the manifold and reduce the accuracy of predictions. Besides, decisions made by users are often influenced by multi-aspect potential preferences, which are usually represented as a vector for each user. To this end, we design a novel hyperbolic-to-hyperbolic user representation with multi-aspect social recommender system, namely H2HMSR, which directly works in hyperbolic space. Extensive experiments on three public datasets demonstrate that our model can adequately extract social information of users with multi-aspect preferences and outperforms hyperbolic and Euclidean counterparts.

Civil Case Judgment Prediction (CCJP) is a fundamental task in the legal intelligence of the civil law system, which aims to automatically predict the judgment results on each plea of the plaintiff. Existing studies mainly focus on making judgment predictions only on a certain civil cause (e.g., the divorce dispute) by utilizing the fact descriptions and pleas of the plaintiff, which still suffer from the various causes and complicated legal essential elements in the real court. Thus, in this paper, we formalize CCJP as a multi-task learning problem and propose a CCJP method centering on the trial mode of essential elements, CPEE, which explores the practical judicial process and analyzes comprehensive legal essential elements to make judgment predictions. Specifically, we first construct three tasks (i.e., the predictions on the civil causes, law articles, and the final judgment on each plea) necessary for CCJP, that follow the judgment process and exploit the results of intermediate subtasks to make judgment predictions. Then we design a logic-enhanced network to predict the results of three tasks and conduct a comprehensive study of civil cases. Finally, owing to the interlinked and dependent relationships among each task, we adopt the cause prediction result to help predict law articles and incorporate them into final judgment prediction through a gate mechanism. Furthermore, since the existing dataset fails to provide sufficient case information, we construct a real-world CCJP dataset that contains various causes and comprehensive legal elements. Extensive experimental results on the dataset validate the effectiveness of our method.

Deep learning has brought significant breakthroughs in sequential recommendation (SR) for capturing dynamic user interests. A series of recent research revealed that models with more parameters usually achieve optimal performance for SR tasks, inevitably resulting in great challenges for deploying them in real systems. Following the simple assumption that light networks might already suffice for certain users, in this work, we propose CANet, a conceptually simple yet very scalable framework for assigning adaptive network architecture in an input-dependent manner to reduce unnecessary computation. The core idea of CANet is to route the input user behaviors with a light-weighted router module. Specifically, we first construct the routing space with various submodels parameterized in terms of multiple model dimensions such as the number of layers, hidden size and embedding size. To avoid extra storage overhead of the routing space, we employ a weight-slicing schema to maintain all the submodels in exactly one network. Furthermore, we leverage several solutions to solve the discrete optimization issues caused by the router module. Thanks to them, CANet could adaptively adjust its network architecture for each input in an end-to-end manner, in which the user preference can be effectively captured. To evaluate our work, we conduct extensive experiments on benchmark datasets. Experimental results show that CANet reduces computation by 55 ~ 65% while preserving the accuracy of the original model. Our codes are available at https://github.conicantnamemyself/CANet.

Reasoning answers to mathematical problems requires machines to think and operate like a human to learn knowledge from mathematical data, which is one of the fundamental tasks for exploring general artificial intelligence. Most solutions focus on mimicking how humans understand problems, which generate the necessary expressions for answers. However, they are still far from enough since they ignore the core ability of humans to acquire knowledge from experience. In this paper, we propose a Cognitive Solver (CogSolver) that is capable of autonomously learning knowledge from scratch to solve mathematical problems, inspired by two cognitive science theories. Specifically, we draw one insight from the dual process theory to establish an intelligent BRAIN-ARM framework, and refer to another information processing theory to summarize the knowledge learning process into Store-Apply-Update steps. In CogSolver, the BRAIN system stores three types of mathematical knowledge, including semantics knowledge, relation knowledge, and mathematic rule knowledge. Then, the ARM system applies the knowledge in BRAIN to answer the problems. Specifically, we design a knowledge-aware module and a commutative module in ARM to improve its reasoning ability, where the knowledge is organically integrated into answer reasoning process. After solving the problems, BRAIN updates the stored knowledge according to the feedback of ARM, where we develop knowledge filters to eliminate the redundant ones and further form a more reasonable knowledge base. Our CogSolver carries out the above three steps iteratively, which behaves more like a human. We conduct extensive experiments on real-world math word problem datasets. The experimental results demonstrate the improvement in answer reasoning and clearly show how CogSolver gains knowledge from the problems, leading to superior interpretability. Our codes are available at https://github.com/bigdata-ustc/CogSolver.

Cognitive diagnostic assessment is a fundamental task in intelligent education, which aims at quantifying students' cognitive level on knowledge attributes. Since there exists learning dependency among knowledge attributes, it is crucial for cognitive diagnosis models (CDMs) to incorporate attribute hierarchy when assessing students. The attribute hierarchy is only explored by a few CDMs such as Attribute Hierarchy Method, and there are still two significant limitations in these methods. First, the time complexity would be unbearable when the number of attributes is large. Second, the assumption used to model the attribute hierarchy is too strong so that it may lose some information of the hierarchy and is not flexible enough to fit all situations. To address these limitations, we propose a novel Bayesian network-based Hierarchical Cognitive Diagnosis Framework (HierCDF), which enables many traditional diagnostic models to flexibly integrate the attribute hierarchy for better diagnosis. Specifically, we first use an efficient Bayesian network to model the influence of attribute hierarchy on students' cognitive states. Then we design a CDM adaptor to bridge the gap between students' cognitive states and the input features of existing diagnostic models. Finally, we analyze the generality and complexity of HierCDF to show its effectiveness in modeling hierarchy information. The performance of HierCDF is experimentally proved on real-world large-scale datasets.

Cognitive diagnosis, aiming at providing an approach to reveal the proficiency level of learners on knowledge concepts, plays an important role in intelligent education area and has recently received more and more attention. Although a number of works have been proposed in recent years, most of contemporary works acquire the traits parameters of learners and items in a transductive way, which are only suitable for stationary data. However, in the real scenario, the data is collected online, where learners, test items and interactions usually grow continuously, which can rarely meet the stationary condition. To this end, we propose a novel framework, Incremental Cognitive Diagnosis (ICD), to tailor cognitive diagnosis into the online scenario of intelligent education. Specifically, we first design a Deep Trait Network (DTN), which acquires the trait parameters in an inductive way rather than a transductive way. Then, we propose an Incremental Update Algorithm (IUA) to balance the effectiveness and training efficiency. We carry out Turning Point (TP) analysis to reduce update frequency, where we derive the minimum update condition based on the monotonicity theory of cognitive diagnosis. Meanwhile, we use a momentum update strategy on the incremental data to decrease update time without sacrificing effectiveness. Moreover, to keep the trait parameters as stable as possible, we refine the loss function in the incremental updating stage. Last but no least, our ICD is a general framework which can be applied to most of contemporary cognitive diagnosis models. To the best of our knowledge, this is the first attempt to investigate the incremental cognitive diagnosis problem with theoretical results about the update condition and a tailored incremental learning strategy. Extensive experiments demonstrate the effectiveness and robustness of our method.

Knowledge Tracing (KT), which aims to assess students' dynamic knowledge states when practicing on various questions, is a fundamental research task for offering intelligent services in online learning systems. Researchers have devoted significant efforts to developing KT models with impressive performance. However, in existing KT methods, the related question difficulty level, which directly affects students' knowledge state in learning, has not been effectively explored and employed. In this paper, we focus on exploring the question difficulty effect on learning to improve student's knowledge state assessment and propose the DIfficulty Matching Knowledge Tracing (DIMKT) model. Specifically, we first explicitly incorporate the difficulty level into the question representation. Then, to establish the relation between students' knowledge state and the question difficulty level during the practice process, we accordingly design an adaptive sequential neural network in three stages: (1) measuring students' subjective feelings of the question difficulty before practice; (2) estimating students' personalized knowledge acquisition while answering questions of different difficulty levels; (3) updating students' knowledge state in varying degrees to match the question difficulty level after practice. Finally, we conduct extensive experiments on real-world datasets, and the results demonstrate that DIMKT outperforms state-of-the-art KT models. Moreover, DIMKT shows superior interpretability by exploring the question difficulty effect when making predictions. Our codes are available at https://github.com/shshen-closer/DIMKT.

Knowledge tracing (KT) which aims at predicting learner's knowledge mastery plays an important role in the computer-aided educational system. The goal of KT is to provide personalized learning paths for learners by diagnosing the mastery of each knowledge, thus improving the learning efficiency. In recent years, many deep learning models have been applied to tackle the KT task, which has shown promising results. However, most existing methods simplify the exercising records as knowledge sequences, which fail to explore the rich information that existed in exercises. Besides, the existing diagnosis results of knowledge tracing are not convincing enough since they neglect hierarchical relations between exercises. To solve the above problems, we propose a hierarchical graph knowledge tracing model called HGKT to explore the latent complex relations between exercises. Specifically, we introduce the concept of problem schema to construct a hierarchical exercise graph that could model the exercise learning dependencies. Moreover, we employ two attention mechanisms to highlight important historical states of learners. In the testing stage, we present a knowledge&schema diagnosis matrix that could trace the transition of mastery of knowledge and problem schema, which can be more easily applied to different applications. Extensive experiments show the effectiveness and interpretability of our proposed model.

Cross-domain sentiment classification (CDSC) aims to use the transferable semantics learned from the source domain to predict the sentiment of reviews in the unlabeled target domain. Existing studies in this task attach more attention to the sequence modeling of sentences while largely ignoring the rich domain-invariant semantics embedded in graph structures (i.e., the part-of-speech tags and dependency relations). As an important aspect of exploring characteristics of language comprehension, adaptive graph representations have played an essential role in recent years. To this end, in the paper, we aim to explore the possibility of learning invariant semantic features from graph-like structures in CDSC. Specifically, we present Graph Adaptive Semantic Transfer (GAST) model, an adaptive syntactic graph embedding method that is able to learn domain-invariant semantics from both word sequences and syntactic graphs. More specifically, we first raise a POS-Transformer module to extract sequential semantic features from the word sequences as well as the part-of-speech tags. Then, we design a Hybrid Graph Attention (HGAT) module to generate syntax-based semantic features by considering the transferable dependency relations. Finally, we devise an Integrated aDaptive Strategy (IDS) to guide the joint learning process of both modules. Extensive experiments on four public datasets indicate that GAST achieves comparable effectiveness to a range of state-of-the-art models.

Computerized Adaptive Testing (CAT) is a promising testing mode in personalized online education (e.g., GRE), which aims at measuring student's proficiency accurately and reducing test length. The "adaptive" is reflected in its selection algorithm that can retrieve best-suited questions for student based on his/her estimated proficiency at each test step. Although there are many sophisticated selection algorithms for improving CAT's effectiveness, they are restricted and perturbed by the accuracy of current proficiency estimate, thus lacking robustness. To this end, we investigate a general method to enhance the robustness of existing algorithms by leveraging student's "multi-facet" nature during tests. Specifically, we present a generic optimization criterion Robust Adaptive Testing (RAT) for proficiency estimation via fusing multiple estimates at each step, which maintains a multi-facet description of student's potential proficiency. We further provide theoretical analyses of such estimator's desirable statistical properties: asymptotic unbiasedness, efficiency, and consistency. Extensive experiments on perturbed synthetic data and three real-world datasets show that selection algorithms in our RAT framework are robust and yield substantial improvements.

Knowledge tracing is a crucial and fundamental task in online education systems, which can predict students' knowledge state for personalized learning. Unfortunately, existing methods are domain-specific, whereas there are many domains (e.g., subjects, schools) in the real education scene and some domains suffer from the problem of lacking sufficient data. Therefore, how to exploit the knowledge in other domains, to improve the model's performance for target domain remains pretty much open. We term this problem as Domain Adaptation for Knowledge Tracing (DAKT), which aims to transfer knowledge from the source domain to the target one for knowledge tracing. In this paper, we propose a novel adaptable method, namely Adaptable Knowledge Tracing (AdaptKT), which contains three phases to explore this problem. Specifically, phase I is instance selection. Given the question texts of two domains, we train an auto-encoder to select and embed similar instances from both domains. Phase II is distribution discrepancy minimizing. After obtaining the selected instances and their linguistic representations, we train a knowledge tracing model and adopt the Maximum Mean Discrepancy (MMD) to minimize the discrepancy between the distributions of the domain-specific knowledge states. Phase III is fine-tuning of the output layer. We replace the output layer of the model that trained in phase II by a new one to make the knowledge tracing model's output dimension matches the number of knowledge concepts in the target domain. The new output layer is trained while other parameters that before it are frozen. We conduct extensive experiments on two large-scale real-world datasets, where the experimental results clearly demonstrate the effectiveness of AdaptKT for solving DAKT problem. We will public the code on the Github after the acceptance of the paper.

Recent years have witnessed great success in deep learning-based sequential recommendation (SR), which can provide more timely and accurate recommendations. One of the most effective deep SR architectures is to stack high-performance residual blocks, e.g., prevalent self-attentive and convolutional operations, for capturing long- and short-range dependence of sequential behaviors. By carefully revisiting previous models, we observe: 1) simple architecture modification of gating each residual connection can help us train deeper SR models and yield significant improvements; 2) compared with self-attention mechanism, stacking of convolution layers also can cover each item of the whole sequential behaviors and achieve competitive or even superior performance.

Recommendation system has been a widely studied task both in academia and industry. Previous works mainly focus on homogeneous recommendation and little progress has been made for heterogeneous recommender systems. However, heterogeneous recommendations, e.g., recommending different types of items including products, videos, celebrity shopping notes, among many others, are dominant nowadays. State-of-the-art methods are incapable of leveraging attributes from different types of items and thus suffer from data sparsity problems. And it is indeed quite challenging to represent items with different feature spaces jointly. To tackle this problem, we propose a kernel-based neural network, namely deep unified representation (or DURation) for heterogeneous recommendation, to jointly model unified representations of heterogeneous items while preserving their original feature space topology structures. Theoretically, we prove the representation ability of the proposed model. Besides, we conduct extensive experiments on the real-world datasets. Experimental results demonstrate that with the unified representation, our model achieves remarkable improvement (e.g., 4.1% ~34.9% lift by AUC score and 3.7% lift by online CTR) over existing state-of-the-art models.

Accurate user interest modeling is important for news recommendation. Most existing methods for news recommendation rely on implicit feedbacks like click for inferring user interests and model training. However, click behaviors usually contain heavy noise, and cannot help infer complicated user interest such as dislike. Besides, the feed recommendation models trained solely on click behaviors cannot optimize other objectives such as user engagement. In this paper, we present a news feed recommendation method that can exploit various kinds of user feedbacks to enhance both user interest modeling and model training. We propose a unified user modeling framework to incorporate various explicit and implicit user feedbacks to infer both positive and negative user interests. In addition, we propose a strong-to-weak attention network that uses the representations of stronger feedbacks to distill positive and negative user interests from implicit weak feedbacks for accurate user interest modeling. Besides, we propose a multi-feedback model training framework to learn an engagement-aware feed recommendation model. Extensive experiments on a real-world dataset show that our approach can effectively improve the model performance in terms of both news clicks and user engagement.

The implicit feedback based collaborative filtering (CF) has attracted much attention in recent years, mainly because users implicitly express their preferences in many real-world scenarios. The current mainstream pairwise methods optimize the Area Under the Curve (AUC) and are empirically proved to be helpful to exploit binary relevance data, but lead to either not address the ranking problem, or not specifically focus on top-$k$recommendation. Although there exists the listwise method maximizes the Mean Reciprocal Rank (MRR), it has low efficiency and is not particularly adequate for general implicit feedback situations. To that end, in this paper, we propose a new framework, namelyCollaborative List-and-Pairwise Filtering (CLAPF), which aims to introduce pairwise thinking into listwise methods. Specifically, we smooth another well-known rank-biased measure called Mean Average Precision (MAP), and respectively combine two rank-biased metrics (MAP, MRR) with the pairwise objective function to capture the performance of top-$k$recommendation. Furthermore, the sampling scheme for CLAPF is discussed to accelerate the convergence speed. Our CLAPF framework is a new hybrid model that provides an idea of utilizing rank-biased measures in a pairwise way on implicit feedback. Empirical studies demonstrated CLAPF outperforms state-of-the-art approaches on real-world datasets.

Social recommendation has achieved great success in many domains including e-commerce and location-based social networks. Existing methods usually explore the user-item interactions or user-user connections to predict users’ preference behaviors. However, they usually learn both user and item representations in Euclidean space, which has large limitations for exploring the latent hierarchical property in the data. In this article, we study a novel problem of hyperbolic social recommendation, where we aim to learn the compact but strong representations for both users and items. Meanwhile, this work also addresses two critical domain-issues, which are under-explored. First, users often make trade-offs with multiple underlying aspect factors to make decisions during their interactions with items. Second, users generally build connections with others in terms of different aspects, which produces different influences with aspects in social network. To this end, we propose a novel graph neural network (GNN) framework with multiple aspect learning, namely, HyperSoRec. Specifically, we first embed all users, items, and aspects into hyperbolic space with superior representations to ensure their hierarchical properties. Then, we adapt a GNN with novel multi-aspect message-passing-receiving mechanism to capture different influences among users. Next, to characterize the multi-aspect interactions of users on items, we propose an adaptive hyperbolic metric learning method by introducing learnable interactive relations among different aspects. Finally, we utilize the hyperbolic translational distance to measure the plausibility in each user-item pair for recommendation. Experimental results on two public datasets clearly demonstrate that our HyperSoRec not only achieves significant improvement for recommendation performance but also shows better representation ability in hyperbolic space with strong robustness and reliability.

Recent studies in recommender systems have managed to achieve significantly improved performance. However, despite being extensively studied, these methods still suffer from two limitations. First, previous studies either encode the document or extract latent sentiment via neural networks, which are difficult to interpret the sentiment of reviewers intuitively. Second, they neglect the personalized interaction of reviews with user/item, i.e., each review has different contributions when modeling the preference of user/item

Learning effective user representations from sequential user-item interactions is a fundamental problem for recommender systems (RS). Recently, several unsupervised methods focusing on user representations pre-training have been explored. In general, these methods apply similar learning paradigms by first corrupting the behavior sequence, and then restoring the original input with some item-level prediction loss functions. Despite its effectiveness, we argue that there exist important gaps between such item-level optimization objective and user-level representations, and as a result, the learned user representations may only lead to sub-optimal generalization performance. In this paper, we propose a novel self-supervised pre-training framework, called CLUE, which stands for employing Contrastive Learning for modeling sequence-level User rEpresentation. The core idea of CLUE is to regard each user behavior sequence as a whole and then construct the self-supervision signals by transforming the original user behaviors by data augmentations (DA). Specifically, we employ two Siamese (weight-sharing) networks to learn the user-oriented representations, where the optimization goal is to maximize the similarity of learned representations of the same user by these two encoders. More importantly, we perform careful investigation of the impacts of view generating strategies for user behavior inputs from a more comprehensive perspective, including processing sequential behaviors by explicit DA strategies and employing dropout as implicit DA. To verify the effectiveness of CLUE, we perform extensive experiments on several user-related tasks with different scales and characteristics. Our experimental results show that the user representations learned by CLUE surpass existing item-level baselines under several evaluation protocols.

In intelligent education systems, question difficulty prediction (QDP) is a fundamental task of many applications, such as personalized question recommendation and test paper analysis. Previous work mainly focus on data-driven QDP methods, which are heavily relied on the large-scale labeled dataset of courses. To alleviate the labor intensity, an intuitive method is to introduce domain adaptation into QDP and consider each course as a domain. In educational psychology, there are two factors influencing difficulty common to different courses: the obstacles of comprehending the question and generating a response, namely stimulus and task difficulty. To this end, we propose a novel Stimulus and Task difficulty-based Adversarial Network (STAN) that models question difficulty from the views of stimulus and task. Then, in order to align the difficulty distribution of the source domain and the target domain, we utilize the conditional adversarial learning with readability-enhanced pseudo-labels. Meanwhile, we proposed a sampling method based on density estimation to implicit alignment. Finally, we conduct experiments on the real questions datasets to evaluate the effectiveness of our QDP model and domain adaptation method. Our method significantly improves accuracy over state-of-the-art methods on real-world question data of multiple courses.

Most cognitive diagnosis research in education has been concentrated on individual assessment, aiming at discovering the latent characteristics of students. However, in many real-world scenarios, group-level assessment is an important and meaningful task, e.g., class assessment in different regions can discover the difference of teaching level in different contexts. In this work, we consider assessing cognitive ability for a group of students, which aims to mine groups’ proficiency on specific knowledge concepts. The significant challenge in this task is the sparsity of group-exercise response data, which seriously affects the assessment performance. Existing works either do not make effective use of additional student-exercise response data or fail to reasonably model the relationship between group ability and individual ability in different learning contexts, resulting in sub-optimal diagnosis results. To this end, we propose a general Multi-Task based Group-Level Cognitive Diagnosis (MGCD) framework, which is featured with three special designs: 1) We jointly model student-exercise responses and group-exercise responses in a multi-task manner to alleviate the sparsity of group-exercise responses; 2) We design a context-aware attention network to model the relationship between student knowledge state and group knowledge state in different contexts; 3) We model an interpretable cognitive layer to obtain student ability, group ability and exercise factors (e.g., difficulty), and then we leverage neural networks to learn complex interaction functions among them. Extensive experiments on real-world datasets demonstrate the generality of MGCD and the effectiveness of our attention design and multi-task learning.

With the accelerated technology development, technological trend forecasting through patent mining has become a hot issue for high-tech companies. In this term, extensive attention has been attracted to forecasting technological knowledge flows (TKF), i.e., predicting the directional flows of knowledge from one technological field to another. However, existing studies either rely on labor intensive empirical analysis or do not consider the intrinsic characteristics inherent in TKF, including the double-faced aspects (i.e., act as both the source and target) of technology nodes, multiple complex relationships among different technologies, and dynamics of the TKF process. To this end, in this paper, we make a further study and propose a data-driven solution, i.e., a Hierarchical Interactive Graph Neural Network (HighTKF), to automatically find the potential flow trends of technologies. Specifically, HighTKF makes final predictions through two kinds of representations of each technology node (a diffusion vector and an absorption vector), which is realized by three components: High-Order Interaction Module (HOI), Hierarchical Delivery Module (HD) and Technology Flow Tracing Module (TFT). For one thing, HOI and HD aim to model high-order network relationships and hierarchical relationships among technologies. For another, TFT is designed for capturing the dynamic feature evolution of technologies with the above relations involved. Also, we design a hybrid loss function and propose a new evaluation metric for better predicting the unprecedented flows between technologies. Finally, we conduct extensive experiments on a real-world patent dataset, the results verify the effectiveness of our approach and reveal some interesting phenomenons on technological knowledge flow trends.

Assessing the proficiency of trial lawyers in different legal fields is of significant importance since a qualified lawyer or lawyer team can strive for his clients' best rights while ensuring the fairness of litigations. However, proficiency assessment for lawyers is very challenging due to many technical and domain challenges, such as the lack of unified evaluation standards, and the complex interactions between lawyers and cases in real legal systems. To this end, we propose a novel proficiency assessment network for trial lawyers (LawyerPAN) to quantify lawyer proficiency through online litigation records. Specifically, we first leverage the theories in psychological measurement for mapping the proficiency of lawyers in each field into a unified real number space. Meanwhile, the characteristics of cases (i.e., case difficulty and discrimination) are well modeled to ensure fairness when assessing lawyers in different cases and fields. Then, we model the interactions between lawyers and cases from two perspectives: the anticipatory perspective aims to measure the personal proficiency of anticipated strategy, and the adversarial perspective seeks to depict the gap of lawyers' proficiency between both sides (i.e., plaintiffs and defendants). Finally, we conduct extensive experiments on real-world data, and the results show the effectiveness and interpretability of our approaches on assessing the proficiency of trial lawyers.

Learning informative representations for educational questions is a fundamental problem in online learning systems, which can promote many applications, e.g., difficulty estimation. Most solutions integrate all information of one question together following a supervised manner, where the representation results are unsatisfactory sometimes due to the following issues. First, they cannot ensure the presentation ability due to the scarcity of labeled data. Then, the label-dependent representation results have poor feasibility to be transferred. Moreover, aggregating all information into the unified may introduce some noises in applications since it cannot distinguish the diverse characteristics of questions. In this paper, we aim to learn the disentangled representations of questions. We propose a novel unsupervised model, namely DisenQNet, to divide one question into two parts, i.e., a concept representation that captures its explicit concept meaning and an individual representation that preserves its personal characteristics. We achieve this goal via mutual information estimation by proposing three self-supervised estimators in a large unlabeled question corpus. Then, we propose another enhanced model, DisenQNet+, that transfers the representation knowledge from unlabeled questions to labeled questions in specific applications by maximizing the mutual information between both. Extensive experiments on real-world datasets demonstrate that DisenQNet can generate effective and meaningful disentangled representations for questions, and furthermore, DisenQNet+ can improve the performance of different applications.

Approximate nearest neighbor search (ANNs) plays an important role in many applications ranging from information retrieval, recommender systems to machine translation. Several ANN indexes, such as hashing and quantization, have been designed to update for the evolving database, but there exists a remarkable performance gap between them and retrained indexes on the entire database. To close the gap, we propose an online additive quantization algorithm (online AQ) to dynamically update quantization codebooks with the incoming streaming data. Then we derive the regret bound to theoretically guarantee the performance of the online AQ algorithm. Moreover, to improve the learning efficiency, we develop a randomized block beam search algorithm for assigning each data to the codewords of the codebook. Finally, we extensively evaluate the proposed online AQ algorithm on four real-world datasets, showing that it remarkably outperforms the state-of-the-art baselines.

Knowledge tracing (KT), which aims to trace students' changing knowledge state during their learning process, has improved students' learning efficiency in online learning systems. Recently, KT has attracted much research attention due to its critical significance in education. However, most of the existing KT methods pursue high accuracy of student performance prediction but neglect the consistency of students' changing knowledge state with their learning process. In this paper, we explore a new paradigm for the KT task and propose a novel model named Learning Process-consistent Knowledge Tracing (LPKT), which monitors students' knowledge state through directly modeling their learning process. Specifically, we first formalize the basic learning cell as the tuple exercise---answer time---answer. Then, we deeply measure the learning gain as well as its diversity from the difference of the present and previous learning cells, their interval time, and students' related knowledge state. We also design a learning gate to distinguish students' absorptive capacity of knowledge. Besides, we design a forgetting gate to model the decline of students' knowledge over time, which is based on their previous knowledge state, present learning gains, and the interval time. Extensive experimental results on three public datasets demonstrate that LPKT could obtain more reasonable knowledge state in line with the learning process. Moreover, LPKT also outperforms state-of-the-art KT methods on student performance prediction. Our work indicates a potential future research direction for KT, which is of both high interpretability and accuracy.

Recently, a new form of online shopping becomes more and more popular, which combines live streaming with E-Commerce activity. The streamers introduce products and interact with their audiences, and hence greatly improve the performance of selling products. Despite of the successful applications in industries, the live stream E-commerce has not been well studied in the data science community. To fill this gap, we investigate this brand-new scenario and collect a real-world Live Stream E-Commerce (LSEC) dataset. Different from conventional E-commerce activities, the streamers play a pivotal role in the LSEC events. Hence, the key is to make full use of rich interaction information among streamers, users, and products. We first conduct data analysis on the tripartite interaction data and quantify the streamer's influence on users' purchase behavior. Based on the analysis results, we model the tripartite information as a heterogeneous graph, which can be decomposed to multiple bipartite graphs in order to better capture the influence. We propose a novel Live Stream E-Commerce Graph Neural Network framework (LSEC-GNN) to learn the node representations of each bipartite graph, and further design a multi-task learning approach to improve product recommendation. Extensive experiments on two real-world datasets with different scales show that our method can significantly outperform various baseline approaches.

The contexts and cultures have a direct impact on student learning by affecting student's implicit cognitive states, such as the preference and the proficiency on specific knowledge. Motivated by the success of context-aware modeling in various fields, such as recommender systems, in this paper, we propose to study how to model context-aware features and adapt them for more precisely diagnosing student's knowledge proficiency. Specifically, by analyzing the characteristics of educational contexts, we design a two-stage framework ECD (Educational context-aware Cognitive Diagnosis), where a hierarchical attentive network is first proposed to represent the context impact on students and then an adaptive optimization is used to achieve diagnosis enhancement by aggregating the cognitive states reflected from both educational contexts and students' historical learning records. Moreover, we give three implementations of general ECD framework following the typical cognitive diagnosis solutions. Finally, we conduct extensive experiments on nearly 52 million records of the students sampled by PISA (Programme for International Student Assessment) from 73 countries and regions. The experimental results not only prove that ECD is more effective in student performance prediction since it can well capture the impact from educational contexts to students' cognitive states, but also give some interesting discoveries regarding the difference among different educational contexts in different countries and regions.

One-hot encoder accompanied by a softmax loss has become the default configuration to deal with the multiclass problem, and is also prevalent in deep learning (DL) based recommender systems (RS). The standard learning process of such methods is to fit the model outputs to a one-hot encoding of the ground truth, referred to as the hard target. However, it is known that these hard targets largely ignore the ambiguity of unobserved feedback in RS, and thus may lead to sub-optimal generalization performance. In this work, we propose SoftRec, a new RS optimization framework to enhance item recommendation. The core idea is that we add additional supervisory signals - well-designed soft targets - for each instance so as to better guide the recommender learning. Meanwhile, we carefully investigate the impacts of specific soft target distributions by instantiating the SoftRec with a series of strategies, including item-based, user-based, and model-based. To verify the effectiveness of SoftRec, we conduct extensive experiments on two public recommendation datasets by using various deep recommendation architectures. The experimental results show that our methods achieve superior performance compared with the standard optimization approaches. Moreover, SoftRec could also exhibit strong performance in cold-start scenarios where user-item interaction has higher sparsity.

Cognitive diagnosis (CD) is a fundamental issue in intelligent educational settings, which aims to discover the mastery levels of students on different knowledge concepts. In general, most previous works consider it as an inter-layer interaction modeling problem, e.g., student-exercise interactions in IRT or student-concept interactions in DINA, while the inner-layer structural relations, such as educational interdependencies among concepts, are still underexplored. Furthermore, there is a lack of comprehensive modeling for the student-exercise-concept hierarchical relations in CD systems. To this end, in this paper, we present a novel Relation map driven Cognitive Diagnosis (RCD) framework, uniformly modeling the interactive and structural relations via a multi-layer student-exercise-concept relation map. Specifically, we first represent students, exercises and concepts as individual nodes in a hierarchical layout, and construct three well-defined local relation maps to incorporate inter- and inner-layer relations, including a student-exercise interaction map, a concept-exercise correlation map and a concept dependency map. Then, we leverage a multi-level attention network to integrate node-level relation aggregation inside each local map and balance map-level aggregation across different maps. Finally, we design an extendable diagnosis function to predict students' performance and jointly train the networks. Extensive experimental results on real-world datasets clearly show the effectiveness and extendibility of our RCD in both diagnosis accuracy improvement and relation-aware representation learning.

Representation learning of examination papers is the cornerstone of the Examination Paper Analysis (EPA) in education area including Paper Difficulty Prediction (PDR) and Finding Similar Papers (FSP). Previous works mainly focus on the representation learning of each test item, but few works notice the hierarchical document structure in examination papers. To this end, in this paper, we propose a novel Examination Organization Encoder (EOE) to learn a robust representation of the examination paper with the hierarchical document structure. Specifically, we first propose a syntax parser to recover the hierarchical document structure and convert an examination paper to an Examination Organization Tree (EOT), where the test items are the leaf nodes and the internal nodes are summarization of their child nodes. Then, we applied a two-layer GRU-based module to obtain the representation of each leaf node. After that, we design a subtree encoder module to aggregate the representation of each leaf node, which is used to calculate an embedding for each layer in the EOT. Finally, we feed all the layer embedding into an output module, the process is over and we get the examination paper representation that can be used for downstream tasks. Extensive experiments on real-world data demonstrate the effectiveness and interpretability of our method.

Legal Judgment Prediction is a fundamental task in legal intelligence of the civil law system, which aims to automatically predict the judgment results of multiple subtasks, such as charge, law article, and term of penalty prediction. Existing studies mainly focus on the impact of the entire fact description on all subtasks. They ignore the practical judicial scenario, where judges adopt circumstances of crime (i.e., various parts of the fact) to decide judgment results. To this end, in this paper, we propose a circumstance-aware legal judgment prediction framework (i.e., NeurJudge) by exploring circumstances of crime. Specifically, NeurJudge utilizes the results of intermediate subtasks to separate the fact description into different circumstances and exploits them to make the predictions of other subtasks. In addition, considering the popularity of confusing verdicts (i.e., charges and law articles), we further extend NeurJudge to a more comprehensive framework which is denoted by NeurJudge+. Particularly, NeurJudge+ utilizes a label embedding method to incorporate the semantics of labels (i.e., charges and law articles) into facts to generate more expressive fact representations for confusing verdicts problems. Extensive experimental results on two real-world datasets clearly validate the effectiveness of our proposed frameworks.

Criminal Court View Generation is an essential task in legal intelligence, which aims to automatically generate sentences interpreting judgment results. The court view could be seen as the summary of crime circumstances in a case, including ADjudging Circumstance (ADC) and SEntencing Circumstance (SEC). However, different circumstances vary widely, and adopting them to generate court views directly may limit the generation performance. Therefore, it is necessary to identify the ADC and SEC related sentences in case facts and enhance them into the court view generation, respectively. To this end, in this paper, we propose a novel Circumstances enhanced Criminal Court View Generation (C3VG) method, consisting of the extraction and generation stage. Specifically, in the extraction stage, we design a Circumstances Selector to select ADC and SEC related sentences. After that, we apply them to two generators to generate the circumstances enhanced court views, respectively. After merging the two types of court views, we could obtain the final court views. We evaluate C3VG by conducting extensive experiments on a real-world dataset and experimental results clearly validate the effectiveness of our proposed model.

Knowledge tracing is a fundamental task in intelligent education for tracking the knowledge states of students on necessary concepts. In recent years, Deep Knowledge Tracing (DKT) utilizes recurrent neural networks to model student learning sequences. This approach has achieved significant success and has been widely used in many educational applications. However, in practical scenarios, it tends to suffer from the following critical problems due to data isolation: 1) Data scarcity. Educational data, which is usually distributed across different silos (e.g., schools), is difficult to gather. 2) Different data quality. Students in different silos have different learning schedules, which results in unbalanced learning records, meaning that it is necessary to evaluate the learning data quality independently for different silos. 3) Data incomparability. It is difficult to compare the knowledge states of students with different learning processes from different silos. Inspired by federated learning, in this paper, we propose a novel Federated Deep Knowledge Tracing (FDKT) framework to collectively train high-quality DKT models for multiple silos. In this framework, each client takes charge of training a distributed DKT model and evaluating data quality by leveraging its own local data, while a center server is responsible for aggregating models and updating the parameters for all the clients. In particular, in the client part, we evaluate data quality incorporating different education measurement theories, and we construct two quality-oriented implementations based on FDKT, i.e., FDKTCTT and FDKTIRT-where the means of data quality evaluation follow Classical Test Theory and Item Response Theory, respectively. Moreover, in the server part, we adopt hierarchical model interpolation to uptake local effects for model personalization. Extensive experiments on real-world datasets demonstrate the effectiveness and superiority of the FDKT framework.

In the Click-Through Rate (CTR) prediction scenario, user's sequential behaviors are well utilized to capture the user interest in the recent literature. However, despite being extensively studied, these sequential methods still suffer from three limitations. First, existing methods mostly utilize attention on the behavior of users, which is not always suitable for CTR prediction, because users often click on new products that are irrelevant to any historical behaviors. Second, in the real scenario, there are numerous users that have operations a long time ago, but turn relatively inactive in recent times. Thus, it is hard to precisely capture user's current preferences through early behaviors. Third, multiple representations of user's historical behaviors in different feature subspaces are largely ignored. To remedy these issues, we propose a Multi-Interactive Attention Network (MIAN) to comprehensively extract the latent relationship among all kinds of fine-grained features (e.g., gender, age and occupation in user-profile). Specifically, MIAN contains a Multi-Interactive Layer (MIL) that integrates three local interaction modules to capture multiple representations of user preference through sequential behaviors and simultaneously utilize the fine-grained user-specific as well as context information. In addition, we design a Global Interaction Module (GIM) to learn the high-order interactions and balance the different impacts of multiple features. Finally, Offline experiment results from three datasets, together with an Online A/B test in a large-scale recommendation system, demonstrate the effectiveness of our proposed approach.

Creative plays a great important role in e-commerce for exhibiting products. Sellers usually create multiple creatives for comprehensive demonstrations, thus it is crucial to display the most appealing design to maximize the Click-Through Rate (CTR). For this purpose, modern recommender systems dynamically rank creatives when a product is proposed for a user. However, this task suffers more cold-start problem than conventional products recommendation since the user-click data is more scarce and creatives potentially change more frequently. In this paper, we propose a hybrid bandit model with visual priors which first makes predictions with a visual evaluation, and then naturally evolves to focus on the specialities through the hybrid bandit model. Our contributions are three-fold: 1) We present a visual-aware ranking model (called VAM) that incorporates a list-wise ranking loss for ordering the creatives according to the visual appearance. 2) Regarding visual evaluation as a prior, the hybrid bandit model (called HBM) is proposed to evolve consistently to make better posteriori estimations by taking more observations into consideration for online scenarios. 3) A first large-scale creative dataset, CreativeRanking1, is constructed, which contains over 1.7M creatives of 500k products as well as their real impression and click data. Extensive experiments have also been conducted on both our dataset and public Mushroom dataset, demonstrating the effectiveness of the proposed method.

User modeling aims to capture the latent characteristics of users from their behaviors, and is widely applied in numerous applications. Usually, centralized user modeling suffers from the risk of privacy leakage. Instead, federated user modeling expects to provide a secure multi-client collaboration for user modeling through federated learning. Existing federated learning methods are mainly designed for consistent clients, which cannot be directly applied to practical scenarios, where different clients usually store inconsistent user data. Therefore, it is a crucial demand to design an appropriate federated solution that can better adapt to user modeling tasks, and however, meets following critical challenges: 1) Statistical heterogeneity. The distributions of user data in different clients are not always independently identically distributed which leads to personalized clients; 2) Privacy heterogeneity. User data contains both public and private information, which have different levels of privacy. It means we should balance different information to be shared and protected; 3) Model heterogeneity. The local user models trained with client records are heterogeneous which need flexible aggregation in the server. In this paper, we propose a novel client-server architecture framework, namely Hierarchical Personalized Federated Learning (HPFL) to serve federated learning in user modeling with inconsistent clients. In the framework, we first define hierarchical information to finely partition the data with privacy heterogeneity. On this basis, the client trains a user model which contains different components designed for hierarchical information. Moreover, client processes a fine-grained personalized update strategy to update personalized user model for statistical heterogeneity. Correspondingly, the server completes a differentiated component aggregation strategy to flexibly aggregate heterogeneous user models in the case of privacy and model heterogeneity. Finally, we conduct extensive experiments on real-world datasets, which demonstrate the effectiveness of the HPFL framework.

For offering proactive services (e.g., personalized exercise recommendation) to the students in computer supported intelligent education, one of the fundamental tasks is predicting student performance (e.g., scores) on future exercises, where it is necessary to track the change of each student's knowledge acquisition during her exercising activities. Unfortunately, to the best of our knowledge, existing approaches can only exploit the exercising records of students, and the problem of extracting rich information existed in the materials (e.g., knowledge concepts, exercise content) of exercises to achieve both more precise prediction of student performance and more interpretable analysis of knowledge acquisition remains underexplored. To this end, in this paper, we present a holistic study of student performance prediction. To directly achieve the primary goal of performance prediction, we first propose a general Exercise-Enhanced Recurrent Neural Network (EERNN) framework by exploring both student's exercising records and the text content of corresponding exercises. In EERNN, we simply summarize each student's state into an integrated vector and trace it with a recurrent neural network, where we design a bidirectional LSTM to learn the encoding of each exercise from its content. For making final predictions, we design two implementations on the basis of EERNN with different prediction strategies, i.e., EERNNM with Markov property and EERNNA with Attention mechanism. Then, to explicitly track student's knowledge acquisition on multiple knowledge concepts, we extend EERNN to an explainable Exercise-aware Knowledge Tracing (EKT) framework by incorporating the knowledge concept information, where the student's integrated state vector is now extended to a knowledge state matrix. In EKT, we further develop a memory network for quantifying how much each exercise can affect the mastery of students on multiple knowledge concepts during the exercising process. Finally, we conduct extensive experiments and evaluate both EERNN and EKT frameworks on a large-scale real-world data. The results in both general and cold-start scenarios clearly demonstrate the effectiveness of two frameworks in student performance prediction as well as the superior interpretability of EKT.

Online search engine has been widely regarded as the most convenient approach for information acquisition. Indeed, the intensive information-seeking behaviors of search engine users make it possible to exploit search engine queries as effective “crowd sensors” for event monitoring. While some researchers have investigated the feasibility of using search engine queries for coarse-grained event analysis, the capability of search engine queries for real-time event detection has been largely neglected. To this end, in this article, we introduce a large-scale and systematic study on exploiting real-time search engine queries for outbreak event detection, with a focus on earthquake rapid reporting. In particular, we propose a realistic system of real-time earthquake detection through monitoring millions of queries related to earthquakes from a dominant online search engine in China. Specifically, we first investigate a large set of queries for selecting the representative queries that are highly correlated with the outbreak of earthquakes. Then, based on the real-time streams of selected queries, we design a novel machine learning–enhanced two-stage burst detection approach for detecting earthquake events. Meanwhile, the location of an earthquake epicenter can be accurately estimated based on the spatial-temporal distribution of search engine queries. Finally, through the extensive comparison with earthquake catalogs from China Earthquake Networks Center, 2015, the detection precision of our system can achieve 87.9%, and the accuracy of location estimation (province level) is 95.7%. In particular, 50% of successfully detected results can be found within 62 s after earthquake, and 50% of successful locations can be found within 25.5 km of seismic epicenter. Our system also found more than 23.3% extra earthquakes that were felt by people but not publicly released, 12.1% earthquake-like special outbreaks, and meanwhile, revealed many interesting findings, such as the typical query patterns of earthquake rumor and regular memorial events. Based on these results, our system can timely feed back information to the search engine users according to various cases and accelerate the information release of felt earthquakes.

Behavior tracing or predicting is a key component in various application scenarios like online user modeling and ubiquitous computing, which significantly benefits the system design (e.g., resource pre-caching) and improves the user experience (e.g., personalized recommendation). Traditional behavior tracing methods like Markovian and sequential models take recent behaviors as input and infer the next move by using the most real-time information. However, these existing methods rarely comprehensively model the low-level temporal irregularity in the recent behavior sequence, i.e., the unevenly distributed time intervals between consecutive behaviors, and the high-level periodicity in the long-term activity cycle, i.e., the periodic behavior patterns of each user.

Computerized Adaptive Testing (CAT) is emerging as a promising testing application in many scenarios, such as education, game and recruitment, which targets at diagnosing the knowledge mastery levels of examinees on required concepts. It shows the advantage of tailoring a personalized testing procedure for each examinee, which selects questions step by step, depending on her performance. While there are many efforts on developing CAT systems, existing solutions generally follow an inflexible model-specific fashion. That is, they need to observe a specific cognitive model which can estimate examinee's knowledge levels and design the selection strategy according to the model estimation. In this paper, we study a novel model-agnostic CAT problem, where we aim to propose a flexible framework that can adapt to different cognitive models. Meanwhile, this work also figures out CAT solution with addressing the problem of how to generate both high-quality and diverse questions simultaneously, which can give a comprehensive knowledge diagnosis for each examinee. Inspired by Active Learning, we propose a novel framework, namely Model-Agnostic Adaptive Testing (MAAT) for CAT solution, where we design three sophisticated modules including Quality Module, Diversity Module and Importance Module. Specifically, at one CAT selection step, Quality Module first quantifies the informativeness of questions and generates candidate subset with the highest quality. Then, Diversity Module selects one question at each step that maximizes the concept coverage. Additionally, we propose Importance Module to estimate the importance of concepts that optimizes the CAT selection. Under MAAT, we prove that the goal of maximizing both quality and diversity is NP-hard, but we provide efficient algorithms by exploiting the inherent submodular property. Extensive experimental results on two real-world datasets clearly demonstrate that our MAAT can support CAT with guaranteeing both quality and diversity perspectives.

Point-of-Interest (POI) recommendation helps users find their interested places to visit based on the time and user location. Unlike traditional recommendation tasks, POI recommendation is personalized, spatial-aware, and temporally dependent. Although many previous works have tried modeling spatial and temporal characteristics, most of them suffer from the following two limitations: For the spatial aspect, existing works only consider the user-POI distance or POI-POI distance. However, we find that a user prefers different regions at different times, which is known as user-region periodic pattern. For the temporal aspect, most works treat user and time as two independent factors. However, different users may prefer the same POI in different time periods, which is known as user-POI periodic pattern. To address the limitation of existing works, we propose a novel Spatial-Temporal aware Graph Convolutional Neural Network (STGCN) for POI recommendation. Specifically, we first design a user record multigraph to fuse all the context information into a unified graph. Then, we propose a time-based neighborhood sampling algorithm and take advantage of the flexible propagation mechanism of GCNs to learn the representations of each node at a specific time. Furthermore, multiple scoring functions are proposed to exploit user-region periodic pattern and user-POI periodic pattern, respectively. We also develop a time smoothing strategy to alleviate the data sparsity problem. Extensive experiments are conducted on two real-world datasets, and the experimental results demonstrate the effectiveness of our method.

Recent years have witnessed a booming increase of patent applications, which provides an open chance for revealing the inner law of innovation, but in the meantime, puts forward higher requirements on patent mining techniques. Considering that patent mining highly relies on patent document analysis, this paper makes a focused study on constructing a technology portrait for each patent, i.e., to recognize technical phrases concerned in it, which can summarize and represent patents from a technology angle. To this end, we first give a clear and detailed description about technical phrases in patents based on various prior works and analyses. Then, combining characteristics of technical phrases and multi-level structures of patent documents, we develop an Unsupervised Multi-level Technical Phrase Extraction (UMTPE) model. Particularly, a novel evaluation metric called Information Retrieval Efficiency (IRE) is designed to evaluate the extracted phrases from a new perspective, which greatly supplements traditional metrics like Precision and Recall. Finally, extensive experiments on real-world patent data show the effectiveness of our UMTPE model.

Knowledge Tracing (KT) is a fundamental but challenging task in online education that traces learners' evolving knowledge states. Much attention has been drawn to this area and several works such as Bayesian Knowledge Tracing and Deep Knowledge Tracing are proposed. Recent works have explored the value of relations among concepts and proposed to introduce knowledge structure into KT task. However, the propagated influence among concepts, which has been shown to be a key factor in human learning by the educational theories, is still under-explored. In this paper, we propose a new framework called Structure-based Knowledge Tracing (SKT), which exploits the multiple relations in knowledge structure to model the influence propagation among concepts. In the SKT framework, we not only consider the temporal effect on the exercising sequence but also take the spatial effect on the knowledge structure into account. We take advantages of two novel formulations in modeling the influence propagation on the knowledge structure with multiple relations. For undirected relations such as similarity relations, the synchronization propagation method is adopted, where the influence propagates bidirectionally between neighbor concepts. For directed relations such as prerequisite relations, the partial propagation method is applied, where the influence can only unidirectionally propagate from a predecessor to a successor. Meanwhile, we employ the gated functions to update the states of concepts temporally and spatially. Extensive experiments demonstrate the effectiveness and interpretability of SKT.

In education systems, Finding Similar Exercises (FSE) is the key step for both exercise retrieval and duplicate detection. Recently, more and more attention has been drawn into this area and several works have been proposed, to utilize the exercise content (e.g., texts or images) or the labeled knowledge concepts. Such approaches, however, have failed to take knowledge hierarchy into account. To this end, we advance a novel knowledge-aware multimodal network, namely KnowNet, for finding similar exercises in large-scale online education systems by integrating the knowledge hierarchy into the heterogeneous exercise data and learning a relation-aware semantic representation. Specifically, we first propose a Content Representation Layer (CRL) to learn a unified semantic representation of the heterogeneous exercise content. Then, we design a Hierarchy Fusion Layer (HFL) to exploit the knowledge hierarchy. By combining the knowledge hierarchy, HFL can not only retrieve the relation-aware semantic representation but also provide an interpretable view to investigate the similarity of exercises. Finally, we adopt a Similarity Score Layer (SSL) for returning similar exercises. Extensive experiments demonstrate the effectiveness and interpretability of KnowNet.

Molecular property prediction (e.g., energy) is an essential problem in chemistry and biology. Unfortunately, many supervised learning methods usually suffer from the problem of scarce labeled molecules in the chemical space, where such property labels are generally obtained by Density Functional Theory (DFT) calculation which is extremely computational costly. An effective solution is to incorporate the unlabeled molecules in a semi-supervised fashion. However, learning semi-supervised representation for large amounts of molecules is challenging, including the joint representation issue of both molecular essence and structure, the conflict between representation and property leaning. Here we propose a novel framework called Active Semi-supervised Graph Neural Network (ASGN) by incorporating both labeled and unlabeled molecules. Specifically, ASGN adopts a teacher-student framework. In the teacher model, we propose a novel semi-supervised learning method to learn general representation that jointly exploits information from molecular structure and molecular distribution. Then in the student model, we target at property prediction task to deal with the learning loss conflict. At last, we proposed a novel active learning strategy in terms of molecular diversities to select informative data during the whole framework learning. We conduct extensive experiments on several public datasets. Experimental results show the remarkable performance of our ASGN framework.

Automatically answering mathematical problems is a challenging task since it requires not only the ability of linguistic understanding but also mathematical comprehension. Existing studies usually explore solutions on the elementary math word problems that aim to understand the questions described in natural language narratives, which are not capable of solving more general problems containing structural formulas. To this end, in this paper, we propose a novel Neural Mathematical Solver (NMS) with enhanced formula structures. Specifically, we first frame the formulas in a certain problem as a TeX dependency graph to preserve formula-enriched structures. Then, we design a formula graph network (FGN) to capture its mathematical relations. Next, we develop a novel architecture with two GRU models, connecting tokens from both word space and formula space together, to learn the linguistic semantics for the answers. Extensive experiments on a large-scale dataset demonstrate that NMS not only achieves better answer prediction but also visualizes reasonable mathematical representations of problems.

With the development of online education systems, a growing number of research works are focusing on Knowledge Tracing (KT), which aims to assess students' changing knowledge state and help them learn knowledge concepts more efficiently. However, only given student learning interactions, most of existing KT methods neglect the individualization of students, i.e., the prior knowledge and learning rates differ from student to student. To this end, in this paper, we propose a novel Convolutional Knowledge Tracing (CKT) method to model individualization in KT. Specifically, for individualized prior knowledge, we measure it from students' historical learning interactions. For individualized learning rates, we design hierarchical convolutional layers to extract them based on continuous learning interactions of students. Extensive experiments demonstrate that CKT could obtain better knowledge tracing results through modeling individualization in learning process. Moreover, CKT can learn meaningful exercise embeddings automatically.

As the task of predicting a personalized ranking on a set of items, item recommendation has become an important way to address information overload. Optimizing ranking loss aligns better with the ultimate goal of item recommendation, so many ranking-based methods were proposed for item recommendation, such as collaborative filtering with Bayesian Personalized Ranking (BPR) loss, and Weighted Approximate-Rank Pairwise (WARP) loss. However, the ranking-based methods can not consistently beat regression-based models with the gravity regularizer. The key challenge in ranking-based optimization is difficult to fully use the limited number of negative samples, particularly when they are not so informative. To this end, we propose a new ranking loss based on importance sampling so that more informative negative samples can be better used. We then design a series of negative samplers from simple to complex, whose informativeness of negative samples is from less to more. With these samplers, the loss function is easy to use and can be optimized by popular solvers. The proposed algorithms are evaluated with five real-world datasets of varying size and difficulty. The results show that they consistently outperform the state-of-the-art item recommendation algorithms, and the relative improvements with respect to [email protected] are more than 19.2% on average. Moreover, the loss function is verified to make better use of negative samples and to require fewer negative samples when they are more informative.

Music plays an important role in our daily life. With the development of deep learning and modern generation techniques, researchers have done plenty of works on automatic music generation. However, due to the special requirements of both melody and arrangement, most of these methods have limitations when applying to multi-track music generation. Some critical factors related to the quality of music are not well addressed, such as chord progression, rhythm pattern, and musical style. In order to tackle the problems and ensure the harmony of multi-track music, in this article, we propose an end-to-end melody and arrangement generation framework to generate a melody track with several accompany tracks played by some different instruments. To be specific, we first develop a novel Chord based Rhythm and Melody Cross-Generation Model to generate melody with a chord progression. Then, we propose a Multi-Instrument Co-Arrangement Model based on multi-task learning for multi-track music arrangement. Furthermore, to control the musical style of arrangement, we design a Multi-Style Multi-Instrument Co-Arrangement Model to learn the musical style with adversarial training. Therefore, we can not only maintain the harmony of the generated music but also control the musical style for better utilization. Extensive experiments on a real-world dataset demonstrate the superiority and effectiveness of our proposed models.

Online donation-based crowdfunding has brought new life to charity by soliciting small monetary contributions from crowd donors to help others in trouble or with dreams. However, a crucial issue for crowdfunding platforms as well as traditional charities is the problem of high donor attrition, i.e., many donors donate only once or very few times within a rather short lifecycle and then leave. Thus, it is an urgent task to analyze the factors of and then further predict the donors behaviors. Especially, we focus on two types of behavioral events, e.g., donation recurrence (whether one donor will make donations at some time slices in the future) and donor retention (whether she will remain on the crowdfunding platform until a future time). However, this problem has not been well explored due to many domain and technical challenges, such as the heterogeneous influence, the relevance of the two types of events, and the censoring phenomenon of retention records. In this paper, we present a focused study on donation recurrence and donor retention with the help of large-scale behavioral data collected from crowdfunding. Specifically, we propose a Joint Deep Survival model, i.e., JDS, which can integrate heterogeneous features, e.g., donor motives, projects recently donated to, social contacts, to jointly model the donation recurrence and donor retention since these two types of behavioral events are highly relevant. In addition, we model the censoring phenomenon and dependence relations of different behaviors from the survival analysis view by designing multiple innovative constraints and incorporating them into the objective functions. Finally, we conduct extensive analysis and validation experiments with large-scale data collected from Kiva.org. The experimental results clearly demonstrate the effectiveness of our proposed models for analyzing and predicting the donation recurrence and donor retention in crowdfunding.

The rapid development of the technologies for online learning provides students with extensive resources for self-learning and brings new opportunities for data-driven research on educational management. An important issue of online learning is to diagnose the knowledge proficiency (i.e., the mastery level of a certain knowledge concept) of each student. Considering that it is a common case that students inevitably learn and forget knowledge from time to time, it is necessary to track the change of their knowledge proficiency during the learning process. Existing approaches either relied on static scenarios or ignored the interpretability of diagnosis results. To address these problems, in this article, we present a focused study on diagnosing the knowledge proficiency of students, where the goal is to track and explain their evolutions simultaneously. Specifically, we first devise an explanatory probabilistic matrix factorization model, Knowledge Proficiency Tracing (KPT), by leveraging educational priors. KPT model first associates each exercise with a knowledge vector in which each element represents a specific knowledge concept with the help of Q -matrix. Correspondingly, at each time, each student can be represented as a proficiency vector in the same knowledge space. Then, our KPT model jointly applies two classical educational theories (i.e., learning curve and forgetting curve ) to capture the change of students’ proficiency level on concepts over time. Furthermore, for improving the predictive performance, we develop an improved version of KPT, named Exercise-correlated Knowledge Proficiency Tracing (EKPT), by considering the connectivity among exercises with the same knowledge concepts. Finally, we apply our KPT and EKPT models to three important diagnostic tasks, including knowledge estimation, score prediction, and diagnosis result visualization. Extensive experiments on four real-world datasets demonstrate that both of our models could track the knowledge proficiency of students effectively and interpretatively.

Cognitive diagnosis is the cornerstone of modern educational techniques. One of the most classic cognitive diagnosis methods is Item Response Theory (IRT), which provides interpretable parameters for analyzing student performance. However, traditional IRT only exploits student response results and has difficulties in fully utilizing the semantics of question texts, which significantly restricts its application. To this end, in this paper, we propose a simple yet surprisingly effective framework to enhance the semantic exploiting process, which we termed Deep Item Response Theory (DIRT). In DIRT, we first use a proficiency vector to represent student proficiency on knowledge concepts and represent question texts and knowledge concepts by dense embedding. Then, we use deep learning to enhance the process of diagnosing parameters of student and question by exploiting question texts and the relationship between question texts and knowledge concepts. Finally, with the diagnosed parameters, we adopt the item response function to predict student performance. Extensive experimental results on real-world data clearly demonstrate the effectiveness and the interpretability of DIRT framework.

Hierarchical multi-label text classification (HMTC) is a fundamental but challenging task of numerous applications (e.g., patent annotation), where documents are assigned to multiple categories stored in a hierarchical structure. Categories at different levels of a document tend to have dependencies. However, the majority of prior studies for the HMTC task employ classifiers to either deal with all categories simultaneously or decompose the original problem into a set of flat multi-label classification subproblems, ignoring the associations between texts and the hierarchical structure and the dependencies among different levels of the hierarchical structure. To that end, in this paper, we propose a novel framework called Hierarchical Attention-based Recurrent Neural Network (HARNN) for classifying documents into the most relevant categories level by level via integrating texts and the hierarchical category structure. Specifically, we first apply a documentation representing layer for obtaining the representation of texts and the hierarchical structure. Then, we develop an hierarchical attention-based recurrent layer to model the dependencies among different levels of the hierarchical structure in a top-down fashion. Here, a hierarchical attention strategy is proposed to capture the associations between texts and the hierarchical structure. Finally, we design a hybrid method which is capable of predicting the categories of each level while classifying all categories in the entire hierarchical structure precisely. Extensive experimental results on two real-world datasets demonstrate the effectiveness and explanatory power of HARNN.

Recommending suitable exercises to students in an online education system is highly useful. Existing approaches usually rely on machine learning techniques to mine large amounts of student interaction log data accumulated in the systems to select the most suitable exercises for each student. Generally, they mainly aim to optimize a single objective, i.e., recommending non-mastered exercises to address the immediate weakness of students. While this is a reasonable objective, there exist more beneficial multiple objectives in the long-term learning process that need to be addressed including Review & Explore, Smoothness of difficulty level and Engagement. In this paper, we propose a novel Deep Reinforcement learning framework, namely DRE, for adaptively recommending Exercises to students with optimization of above three objectives. In the framework, we propose two different Exercise Q-Networks for the agent, i.e., EQNM and EQNR, to generate recommendations following Markov property and Recurrent manner, respectively. We also propose novel reward functions to formally quantify those three objectives so that DRE could update and optimize its recommendation strategy by interactively receiving students' performance feedbacks (e.g., score). We conduct extensive experiments on two real-world datasets. Experimental results clearly show that the proposed DRE can effectively learn from the student interaction data to optimize multiple objectives in a single unified framework and adaptively recommend suitable exercises to students.

This paper introduces Alpha-Beta Sampling (ABS) strategy, which is particularly intended for the sampling problem of pairwise ranking in one-class collaborative filtering (PROCCF). Specifically, ABS strategy places more emphasis on such training examples, including positive item with a lower preference score and negative items with a higher preference score for each gradient step. Then, we provide the corresponding proofs for the ABS strategy from both gradient and ranking perspectives. First, we prove that sampled training examples by ABS strategy can update the model parameters with a large magnitude and analyze two instantiations by combining two specific pairwise algorithms. Second, it can be proved that ABS strategy is equivalent to optimizing for ranking-aware evaluation metrics like Normalized Discounted Cumulative Gain (NDCG). Furthermore, ABS strategy can be very general and applicable in a lot of pairwise structures of pairwise algorithms. Based on ABS strategy, we provide an effective sampling algorithm to dynamically draw items for each SGD update. Finally, we evaluate the ABS strategy by conducting sampling tasks in two representative pairwise algorithms. The experiment results show that the ABS strategy performs significantly better than the baseline strategies.

Concept map is an useful tool to help people organize and improve knowledge. Particularly in educational domain, it is beneficial for students and teachers to improve the learning and teaching quality. Traditionally, manual educational concept maps, provided by teachers, are quite time-consuming and limited to teachers' experience. Thus, it is meaningful to automatically construct high-quality concept maps. However, existing data-driven solutions only focus on either separate data source or single pedagogic relationship, which are not sufficient to satisfy actual demands. To this end, we propose a novel framework, named Extracting Multiple Relationships Concept Map (EMRCM), to construct multiple relations concept maps from Multi-source Data. Specifically, we design various targeted evidences to explore diverse information of multi-source data from different perspectives. Then, we employ three classic classifiers to bulid the predictive model for extracting key concepts and multiple concept relationships using the proposed evidences. We create a real dataset for empirically studying this problem. Extensive experiments on a real-world dataset show the effectiveness of our method.

Technological change and innovation are vitally important, especially for high-tech companies. However, factors influencing their future research and development (R&D) trends are both complicated and various, leading it a quite difficult task to make technology tracing for high-tech companies. To this end, in this paper, we develop a novel data-driven solution, i.e., Deep Technology Forecasting (DTF) framework, to automatically find the most possible technology directions customized to each high-tech company. Specially, DTF consists of three components: Potential Competitor Recognition (PCR), Collaborative Technology Recognition (CTR), and Deep Technology Tracing (DTT) neural network. For one thing, PCR and CTR aim to capture competitive relations among enterprises and collaborative relations among technologies, respectively. For another, DTT is designed for modeling dynamic interactions between companies and technologies with the above relations involved. Finally, we evaluate our DTF framework on real-world patent data, and the experimental results clearly prove that DTF can precisely help to prospect future technology emphasis of companies by exploiting hybrid factors.

Aftershocks refer to the smaller earthquakes that occur following large earthquakes, in the same area of the main shock. The task of aftershocks detection, as a crucial and challenging issue in disaster monitoring, has attracted wide research attention in relevant fields. Compared with the traditional detection methods like STA/LTA algorithms or heuristic matching, neural network techniques are regarded as an advanced choice with better pattern recognition ability. However, current neural network-based solutions mainly formulate the seismic wave as ordinary time series, where existing techniques are directly deployed without adaption, and thus fail to obtain competitive performance on the intensive and highly-noise waveforms of aftershocks. To that end, in this paper, we propose a novel framework named Multi-Scale Description based Neural Network (MSDNN) for enhancing aftershock detection. Specifically, MSDNN contains a delicately-designed network structure for capturing both short-term scale and long-term scale seismic features. Therefore, the unique characteristics of seismic waveforms can be fully-exploited for aftershock detection. Furthermore, a multi-task learning strategy is introduced to model the seismic waveforms of multiple monitoring stations simultaneously, which can not only refine the detection performance but also provide additionally quantitative clues for discovering homologous earthquakes. Finally, comprehensive experiments on the data set from aftershocks of the Wenchuan M8.0 Earthquake have clearly validated the effectiveness of our framework compared with several state-of-the-art baselines.

Adaptive learning, also known as adaptive teaching, relies on learning path recommendation, which sequentially recommends personalized learning items (e.g., lectures, exercises) to satisfy the unique needs of each learner. Although it is well known that modeling the cognitive structure including knowledge level of learners and knowledge structure (e.g., the prerequisite relations) of learning items is important for learning path recommendation, existing methods for adaptive learning often separately focus on either knowledge levels of learners or knowledge structure of learning items. To fully exploit the multifaceted cognitive structure for learning path recommendation, we propose a Cognitive Structure Enhanced framework for Adaptive Learning, named CSEAL. By viewing path recommendation as a Markov Decision Process and applying an actor-critic algorithm, CSEAL can sequentially identify the right learning items to different learners. Specifically, we first utilize a recurrent neural network to trace the evolving knowledge levels of learners at each learning step. Then, we design a navigation algorithm on the knowledge structure to ensure the logicality of learning paths, which reduces the search space in the decision process. Finally, the actor-critic algorithm is used to determine what to learn next and whose parameters are dynamically updated along the learning path. Extensive experiments on real-world data demonstrate the effectiveness and robustness of CSEAL.

Recently, the Network Representation Learning (NRL) techniques, which represent graph structure via low-dimension vectors to support social-oriented application, have attracted wide attention. Though large efforts have been made, they may fail to describe the multiple aspects of similarity between social users, as only a single vector for one unique aspect has been represented for each node. To that end, in this paper, we propose a novel end-to-end framework named MCNE to learn multiple conditional network representations, so that various preferences for multiple behaviors could be fully captured. Specifically, we first design a binary mask layer to divide the single vector as conditional embeddings for multiple behaviors. Then, we introduce the attention network to model interaction relationship among multiple preferences, and further utilize the adapted message sending and receiving operation of graph neural network, so that multi-aspect preference information from high-order neighbors will be captured. Finally, we utilize Bayesian Personalized Ranking loss function to learn the preference similarity on each behavior, and jointly learn multiple conditional node embeddings via multi-task learning framework. Extensive experiments on public datasets validate that our MCNE framework could significantly outperform several state-of-the-art baselines, and further support the visualization and transfer learning tasks with excellent interpretability and robustness.

Understanding learning materials (e.g. test questions) is a crucial issue in online learning systems, which can promote many applications in education domain. Unfortunately, many supervised approaches suffer from the problem of scarce human labeled data, whereas abundant unlabeled resources are highly underutilized. To alleviate this problem, an effective solution is to use pre-trained representations for question understanding. However, existing pre-training methods in NLP area are infeasible to learn test question representations due to several domain-specific characteristics in education. First, questions usually comprise of heterogeneous data including content text, images and side information. Second, there exists both basic linguistic information as well as domain logic and knowledge. To this end, in this paper, we propose a novel pre-training method, namely QuesNet, for comprehensively learning question representations. Specifically, we first design a unified framework to aggregate question information with its heterogeneous inputs into a comprehensive vector. Then we propose a two-level hierarchical pre-training algorithm to learn better understanding of test questions in an unsupervised way. Here, a novel holed language model objective is developed to extract low-level linguistic features, and a domain-oriented objective is proposed to learn high-level logic and knowledge. Moreover, we show that QuesNet has good capability of being fine-tuned in many question-based tasks. We conduct extensive experiments on large-scale real-world question data, where the experimental results clearly demonstrate the effectiveness of QuesNet for question understanding as well as its superior applicability.

As users implicitly express their preferences to items on many real-world applications, the implicit feedback based collaborative filtering has attracted much attention in recent years. Pairwise methods have shown state-of-the-art solutions for dealing with the implicit feedback, with the assumption that users prefer the observed items to the unobserved items. However, for each user, the huge unobserved items are not equal to represent her preference. In this paper, we propose a Multiple Pairwise Ranking (MPR) approach, which relaxes the simple pairwise preference assumption in previous works by further tapping the connections among items with multiple pairwise ranking criteria. Specifically, we exploit the preference difference among multiple pairs of items by dividing the unobserved items into different parts. Empirical studies show that our algorithms outperform the state-of-the-art methods on real-world datasets.

Recently, the Network Representation Learning (NRL) techniques, which target at learning the low-dimension vector representation of graph structures, have attracted wide attention due to the effectiveness on various social-oriented application. Though large efforts have been made on the joint analysis combining node attributes with the network structure, they may usually fail to summarize the weighted correlations within nodes and attributes, especially when the nodes suffer extremely sparse attributes. To that end, in this paper, we propose a novel Sparse Attributed Network Embedding (SANE) framework to learn the network structure and sparse attribute information simultaneously in a united approach. Specifically, we first embed the nodes and attributes into a low-dimensional vector space. Then we introduce the pairwise method to capture the interaction between nodes and sparse attributes, and aggregate the attribute information of neighbors to alleviate sparsity for obtaining a better vector representation of node embeddings, which will be used in following network representation learning task. Along this line, we maintain the network structure by maximizing the probability of predicting the center node according to surrounding context nodes. Different from previous work, we introduce an attention mechanism to adaptively weigh the strength of interactions between each context node and the center node, according to the node attribute similarity. Furthermore, we combine the attention network with CBOW model to learn the similarity of the network structure and node attributes simultaneously. Extensive experiments on public datasets have validated the effectiveness of our SANE model with significant margin compared with the state-of-the-art baselines, which demonstrates the potential of adaptively attribute analysis in network embedding.

Keyphrases have been widely used in large document collections for providing a concise summary of document content. While significant efforts have been made on the task of automatic keyphrase extraction, existing methods have challenges in training a robust supervised model when there are insufficient labeled data in the resource-poor domains. To this end, in this paper, we propose a novel Topic-based Adversarial Neural Network (TANN) method, which aims at exploiting the unlabeled data in the target domain and the data in the resource-rich source domain. Specifically, we first explicitly incorporate the global topic information into the document representation using a topic correlation layer. Then, domain-invariant features are learned to allow the efficient transfer from the source domain to the target by utilizing adversarial training on the topic-based representation. Meanwhile, to balance the adversarial training and preserve the domain-private features in the target domain, we reconstruct the target data from both forward and backward directions. Finally, based on the learned features, keyphrase are extracted using a tagging method. Experiments on two realworld cross-domain scenarios demonstrate that our method can significantly improve the performance of keyphrase extraction on unlabeled or insufficiently labeled target domain.

Natural Language Inference (NLI) task requires an agent to determine the semantic relation between a premise sentence (p) and a hypothesis sentence (h), which demands sufficient understanding about sentences from lexical knowledge to global semantic. Due to the issues such as polysemy, ambiguity, as well as fuzziness of sentences, fully understanding sentences is still challenging. To this end, we propose an Image-Enhanced Multi-Level Sentence Representation Net (IEMLRN), a novel architecture that is able to utilize the image to enhance the sentence semantic understanding at different scales. To be specific, we introduce the corresponding image of sentences as reference information, which can be helpful for sentence semantic understanding and inference relation evaluation. Since image information might be related to the sentence semantics at different scales, we design a multi-level architecture to understand sentences from different granularity and generate the sentence representation more precisely. Experimental results on the large-scale NLI corpus and real-world NLI alike corpus demonstrate that IEMLRN can simultaneously improve the performance. It is noteworthy that IEMLRN significantly outperforms the state-of-the-art sentence-encoding based models on the challenging hard subset and challenging lexical subset of SNLI corpus.

In the modern e-commerce, the behaviors of customers contain rich information, e.g., consumption habits, the dynamics of preferences. Recently, session-based recommendationsare becoming popular to explore the temporal characteristics of customers' interactive behaviors. However, existing works mainly exploit the short-term behaviors without fully taking the customers' long-term stable preferences and evolutions into account. In this paper, we propose a novel Behavior-Intensive Neural Network (BINN) for next-item recommendation by incorporating both users' historical stable preferences and present consumption motivations. Specifically, BINN contains two main components, i.e., Neural Item Embedding, and Discriminative Behaviors Learning. Firstly, a novel item embedding method based on user interactions is developed for obtaining an unified representation for each item. Then, with the embedded items and the interactive behaviors over item sequences, BINN discriminatively learns the historical preferences and present motivations of the target users. Thus, BINN could better perform recommendations of the next items for the target users. Finally, for evaluating the performances of BINN, we conduct extensive experiments on two real-world datasets, i.e., Tianchi and JD. The experimental results clearly demonstrate the effectiveness of BINN compared with several state-of-the-art methods.

In online education systems, finding similar exercises is a fundamental task of many applications, such as exercise retrieval and student modeling. Several approaches have been proposed for this task by simply using the specific textual content (e.g. the same knowledge concepts or the similar words) in exercises. However, the problem of how to systematically exploit the rich semantic information embedded in multiple heterogenous data (e.g. texts and images) to precisely retrieve similar exercises remains pretty much open. To this end, in this paper, we develop a novel Multimodal Attention-based Neural Network (MANN) framework for finding similar exercises in large-scale online education systems by learning a unified semantic representation from the heterogenous data. In MANN, given exercises with texts, images and knowledge concepts, we first apply a convolutional neural network to extract image representations and use an embedding layer for representing concepts. Then, we design an attention-based long short-term memory network to learn a unified semantic representation of each exercise in a multimodal way. Here, two attention strategies are proposed to capture the associations of texts and images, texts and knowledge concepts, respectively. Moreover, with a Similarity Attention, the similar parts in each exercise pair are also measured. Finally, we develop a pairwise training strategy for returning similar exercises. Extensive experimental results on real-world data clearly validate the effectiveness and the interpretation power of MANN.

Transcribing content from structural images, e.g., writing notes from music scores, is a challenging task as not only the content objects should be recognized, but the internal structure should also be preserved. Existing image recognition methods mainly work on images with simple content (e.g., text lines with characters), but are not capable to identify ones with more complex content (e.g., structured code), which often follow a fine-grained grammar. To this end, in this paper, we propose a hierarchical Spotlight Transcribing Network (STN) framework followed by a two-stage "where-to-what'' solution. Specifically, we first decide "where-to-look'' through a novel spotlight mechanism to focus on different areas of the original image following its structure. Then, we decide "what-to-write'' by developing a GRU based network with the spotlight areas for transcribing the content accordingly. Moreover, we propose two implementations on the basis of STN, i.e., STNM and STNR, where the spotlight movement follows the Markov property and Recurrent modeling, respectively. We also design a reinforcement method to refine our STN framework by self-improving the spotlight mechanism. We conduct extensive experiments on many structural image datasets, where the results clearly demonstrate the effectiveness of STN framework.

With the development of knowledge of music composition and the recent increase in demand, an increasing number of companies and research institutes have begun to study the automatic generation of music. However, previous models have limitations when applying to song generation, which requires both the melody and arrangement. Besides, many critical factors related to the quality of a song such as chord progression and rhythm patterns are not well addressed. In particular, the problem of how to ensure the harmony of multi-track music is still underexplored. To this end, we present a focused study on pop music generation, in which we take both chord and rhythm influence of melody generation and the harmony of music arrangement into consideration. We propose an end-to-end melody and arrangement generation framework, called XiaoIce Band, which generates a melody track with several accompany tracks played by several types of instruments. Specifically, we devise a Chord based Rhythm and Melody Cross-Generation Model (CRMCG) to generate melody with chord progressions. Then, we propose a Multi-Instrument Co-Arrangement Model (MICA) using multi-task learning for multi-track music arrangement. Finally, we conduct extensive experiments on a real-world dataset, where the results demonstrate the effectiveness of XiaoIce Band.

With the development of social networking services, social influence analyses, as well as the influence maximization tasks, have attracted wide attention in both academia and industry. Traditional studies mainly focus on simulating process of influence spread. However, two basic functions of social spread, i.e., information propagation and information adoption have not been clearly distinguished. Usually, as information adoption could be even more significant for information publishers in application scenarios, more comprehensive analysis for effect of adoption is urgently required. To that end, in this paper, we propose a novel framework to generally describe social spread, in which information adoption process is separately formulated as random events. Along this line, when we apply this framework to the information adoption maximization task, with proving that the adoption maximization problem is NP-hard and submodular, we further design a polling-based algorithm to achieve an effective approximation. Extensive experiments on four real-world data sets demonstrate the effectiveness and efficiency of proposed algorithms, which validates that our approach could better summarize the complete social spread process, and further support the necessity of distinguishing information adoption from information propagation.

In online social networks people often express attitudes towards others, which forms massive sentiment links among users. Predicting the sign of sentiment links is a fundamental task in many areas such as personal advertising and public opinion analysis. Previous works mainly focus on textual sentiment classification, however, text information can only disclose the "tip of the iceberg»» about users» true opinions, of which the most are unobserved but implied by other sources of information such as social relation and users» profile. To address this problem, in this paper we investigate how to predict possibly existing sentiment links in the presence of heterogeneous information. First, due to the lack of explicit sentiment links in mainstream social networks, we establish a labeled heterogeneous sentiment dataset which consists of users» sentiment relation, social relation and profile knowledge by entity-level sentiment extraction method. Then we propose a novel and flexible end-to-end Signed Heterogeneous Information Network Embedding (SHINE) framework to extract users» latent representations from heterogeneous networks and predict the sign of unobserved sentiment links. SHINE utilizes multiple deep autoencoders to map each user into a low-dimension feature space while preserving the network structure. We demonstrate the superiority of SHINE over state-of-the-art baselines on link prediction and node recommendation in two real-world datasets. The experimental results also prove the efficacy of SHINE in cold start scenario.

Recent decades have witnessed the rapid growth of educational data mining (EDM), which aims at automatically extracting valuable information from large repositories of data generated by or related to people’s learning activities in educational settings. One of the key EDM tasks is cognitive modelling with examination data, and cognitive modelling tries to profile examinees by discovering their latent knowledge state and cognitive level (e.g. the proficiency of specific skills). However, to the best of our knowledge, the problem of extracting information from both objective and subjective examination problems to achieve more precise and interpretable cognitive analysis remains underexplored. To this end, we propose a fuzzy cognitive diagnosis framework (FuzzyCDF) for examinees’ cognitive modelling with both objective and subjective problems. Specifically, to handle the partially correct responses on subjective problems, we first fuzzify the skill proficiency of examinees. Then we combine fuzzy set theory and educational hypotheses to model the examinees’ mastery on the problems based on their skill proficiency. Finally, we simulate the generation of examination score on each problem by considering slip and guess factors. In this way, the whole diagnosis framework is built. For further comprehensive verification, we apply our FuzzyCDF to three classical cognitive assessment tasks, i.e., predicting examinee performance, slip and guess detection, and cognitive diagnosis visualization. Extensive experiments on three real-world datasets for these assessment tasks prove that FuzzyCDF can reveal the knowledge states and cognitive level of the examinees effectively and interpretatively.

As the worlds of commerce and the Internet technology become more inextricably linked, a large number of user consumption series become available for online market intelligence analysis. A critical demand along this line is to predict the future product adoption state of each user, which enables a wide range of applications such as targeted marketing. Nevertheless, previous works only aimed at predicting if a user would adopt a particular product or not with a binary buy-or-not representation. The problem of tracking and predicting users' adoption rates, i.e., the frequency and regularity of using each product over time, is still under-explored. To this end, we present a comprehensive study of product adoption rate prediction in a competitive market. This task is nontrivial as there are three major challenges in modeling users' complex adoption states: the heterogeneous data sources around users, the unique user preference and the competitive product selection. To deal with these challenges, we first introduce a flexible factor-based decision function to capture the change of users' product adoption rate over time, where various factors that may influence users' decisions from heterogeneous data sources can be leveraged. Using this factor-based decision function, we then provide two corresponding models to learn the parameters of the decision function with both generalized and personalized assumptions of users' preferences. We further study how to leverage the competition among different products and simultaneously learn product competition and users' preferences with both generalized and personalized assumptions. Finally, extensive experiments on two real-world datasets show the superiority of our proposed models.

DanMu , an emerging type of user-generated comment, has become increasingly popular in recent years. Many online video platforms such as Tudou.com have provided the DanMu function. Unlike traditional online reviews such as reviews at Youtube.com that are outside the videos, DanMu is a scrolling marquee comment, which is overlaid directly on top of the video and synchronized to a specific playback time. Such comments are displayed as streams of moving subtitles overlaid on the video screen. Viewers could easily write DanMu s while watching videos, and the written DanMu s will be immediately overlaid onto the video and displayed to writers themselves and other viewers as well. Such DanMu systems have greatly enabled users to communicate with each other in a much more direct way, creating a real-time sharing experience. Although there are several unique features of DanMu and has had a great impact on online video systems, to the best of our knowledge, there is no work that has provided a comprehensive study on DanMu . In this article, as a pilot study, we analyze the unique characteristics of DanMu from various perspectives. Specifically, we first illustrate some unique distributions of DanMu s by comparing with traditional reviews (TReviews) that we collected from a real DanMu -enabled online video system. Second, we discover two interesting patterns in DanMu data: a herding effect and multiple-burst phenomena that are significantly different from those in TRviews and reveal important insights about the growth of DanMu s on a video. Towards exploring antecedents of both th herding effect and multiple-burst phenomena, we propose to further detect leading DanMu s within bursts, because those leading DanMu s make the most contribution to both patterns. A framework is proposed to detect leading DanMu s that effectively combines multiple factors contributing to leading DanMu s. Based on the identified characteristics of DanMu , finally we propose to predict the distribution of future DanMu s (i.e., the growth of DanMu s), which is important for many DanMu -enabled online video systems, for example, the predicted DanMu distribution could be an indicator of video popularity. This prediction task includes two aspects: One is to predict which videos future DanMu s will be posted for, and the other one is to predict which segments of a video future DanMu s will be posted on. We develop two sophisticated models to solve both problems. Finally, intensive experiments are conducted with a real-world dataset to validate all methods developed in this article.

Diagnosing students' knowledge proficiency, i.e., the mastery degrees of a particular knowledge point in exercises, is a crucial issue for numerous educational applications, e.g., targeted knowledge training and exercise recommendation. Educational theories have converged that students learn and forget knowledge from time to time. Thus, it is necessary to track their mastery of knowledge over time. However, traditional methods in this area either ignored the explanatory power of the diagnosis results on knowledge points or relied on a static assumption. To this end, in this paper, we devise an explanatory probabilistic approach to track the knowledge proficiency of students over time by leveraging educational priors. Specifically, we first associate each exercise with a knowledge vector in which each element represents an explicit knowledge point by leveraging educational priors (i.e., Q-matrix ). Correspondingly, each student is represented as a knowledge vector at each time in a same knowledge space. Second, given the student knowledge vector over time, we borrow two classical educational theories (i.e., Learning curve and Forgetting curve ) as priors to capture the change of each student's proficiency over time. After that, we design a probabilistic matrix factorization framework by combining student and exercise priors for tracking student knowledge proficiency. Extensive experiments on three real-world datasets demonstrate both the effectiveness and explanatory power of our proposed model.

In online advertising, Click-Through Rate (CTR) prediction is a crucial task, as it may benefit the ranking and pricing of online ads. To the best of our knowledge, most of the existing CTR prediction methods are shallow layer models (e.g., Logistic Regression and Factorization Machines) or deep layer models (e.g., Neural Networks). Unfortunately, the shallow layer models cannot capture or utilize high-order nonlinear features in ad data. On the other side, the deep layer models cannot satisfy the necessity of updating CTR models online efficiently due to their high computational complexity. To address the shortcomings above, in this paper, we propose a novel hybrid method based on feature learning of both Deep and Shallow Layers (DSL). In DSL, we utilize Deep Neural Network as a deep layer model trained offline to learn high-order nonlinear features and use Factorization Machines as a shallow layer model for CTR prediction. Furthermore, we also develop an online learning implementation based on DSL, i.e., onlineDSL. Extensive experiments on large-scale real-world datasets clearly validate the effectiveness of our DSL method and onlineDSL algorithm compared with several state-of-the-art baselines.

Crowdfunding is an emerging Internet fundraising mechanism by raising monetary contributions from the crowd for projects or ventures. In these platforms, the dynamics, i.e., daily funding amount on campaigns and perks (backing options with rewards), are the most concerned issue for creators, backers and platforms. However, tracking the dynamics in crowdfunding is very challenging and still under-explored. To that end, in this paper, we present a focused study on this important problem. A special goal is to forecast the funding amount for a given campaign and its perks in the future days. Specifically, we formalize the dynamics in crowdfunding as a hierarchical time series, i.e., campaign level and perk level. Specific to each level, we develop a special regression by modeling the decision making process of the crowd (visitors and backing probability) and exploring various factors that impact the decision; on this basis, an enhanced switching regression is proposed at each level to address the heterogeneity of funding sequences. Further, we employ a revision matrix to combine the two-level base forecasts for the final forecasting. We conduct extensive experiments on a real-world crowdfunding data collected from Indiegogo.com. The experimental results clearly demonstrate the effectiveness of our approaches on tracking the dynamics in crowdfunding.

P2P lending is an emerging Internet-based application where individuals can directly borrow money from each other. The past decade has witnessed the rapid development and prevalence of online P2P lending platforms, examples of which include Prosper, LendingClub, and Kiva. Meanwhile, extensive research has been done that mainly focuses on the studies of platform mechanisms and transaction data. In this article, we provide a comprehensive survey on the research about P2P lending, which, to the best of our knowledge, is the first focused effort in this field. Specifically, we first provide a systematic taxonomy for P2P lending by summarizing different types of mainstream platforms and comparing their working mechanisms in detail. Then, we review and organize the recent advances on P2P lending from various perspectives (e.g., economics and sociology perspective, and data-driven perspective). Finally, we propose our opinions on the prospects of P2P lending and suggest some future research directions in this field. Meanwhile, throughout this paper, some analysis on real-world data collected from Prosper and Kiva are also conducted.

For a long time, PageRank has been widely used for authority computation and has been adopted as a solid baseline for evaluating social influence related applications. However, when measuring the authority of network nodes, the traditional PageRank method does not take the nodes’ prior knowledge into consideration. Also, the connection between PageRank and social influence modeling methods is not clearly established. To that end, this article provides a focused study on understanding PageRank as well as the relationship between PageRank and social influence analysis. Along this line, we first propose a linear social influence model and reveal that this model generalizes the PageRank-based authority computation by introducing some constraints. Then, we show that the authority computation by PageRank can be enhanced if exploiting more reasonable constraints (e.g., from prior knowledge). Next, to deal with the computational challenge of linear model with general constraints, we provide an upper bound for identifying nodes with top authorities. Moreover, we extend the proposed linear model for better measuring the authority of the given node sets, and we also demonstrate the way to quickly identify the top authoritative node sets. Finally, extensive experimental evaluations on four real-world networks validate the effectiveness of the proposed linear model with respect to different constraint settings. The results show that the methods with more reasonable constraints can lead to better ranking and recommendation performance. Meanwhile, the upper bounds formed by PageRank values could be used to quickly locate the nodes and node sets with the highest authorities.

Sociologists have long converged that the evolution of a Social Networking Service(SNS) is driven by the interplay between users' preferences (reflected in user-item interaction behavior) and the social network structure (reflected in user-user interaction behavior). Nevertheless, traditional approaches either modeled these two kinds of behaviors in isolation or relied on a static assumption of a SNS. Thus, it is still unclear how do the roles of the dynamic social network structure and users' historical preferences affect the evolution of SNSs. Furthermore, can transforming the underlying social theories in the platform evolution modeling process benefit both behavior prediction tasks? In this paper, we incorporate the underlying social theories to explain and model the evolution of users' two kinds of behaviors in SNSs. Specifically, we present two kinds of representations for users' behaviors: a direct (latent) representation that presumes users' behaviors are represented directly (latently) by their historical behaviors. Under each representation, we associate each user's two kinds of behaviors with two vectors at each time. Then, for each representation, we propose the corresponding learning model to fuse the interplay between users' two kinds of behaviors. Finally, extensive experimental results demonstrate the effectiveness of our proposed models for both user preference prediction and social link suggestion.

Many recent studies on finance and social networks discovered that investor's attention is correlated to the financial market movement in terms of the price shocks. Following related findings, a significant and challenging problem is to forecast the direction of the market movement based on vast social media activities. Appropriately processing social networks data and developing models to capture investor's attention on stocks would effectively help financial forecasting. In this paper, we propose and then apply a price shocks forecasting framework, which simultaneously takes the influence of social network users and their opinions about stocks into consideration. Specifically, we develop a new method to estimate social attention to stocks by influence modeling and sentiment analysis. Then, we use it in price shocks forecasting, which we formalize as a classification problem. We also consider the effect of historical market information on the market movement. Finally, we evaluate our framework based on a series of tests on the Chinese stock data. Our results show that the newly proposed measurement of social attention effectively improves the forecasting power of our framework.

The prediction of academic performance is one of the most important tasks in educational data mining, and has been widely studied in MOOCs and intelligent tutoring systems. Academic performance could be affected with factors like personality, skills, social environment, the use of library books and so on. However, it is still less investigated that how could the use of library books affect academic performance of college students and even leverage book-loan history for predicting academic performance. To this end, we propose a supervised content-aware matrix factorization for mutual reinforcement of academic performance prediction and library book recommendation. This model not only addresses the sparsity challenge by explainable dimension reduction techniques, but also promotes library book recommendation by recommending "right" books for students based on their performance levels and book meta information. Finally, we evaluate the proposed model on three years of the book-loan history and cumulative grade point average of 13,047 undergraduate students in one university. The results show that the proposed model outperforms the competing baselines on both tasks, and that academic performance is not only predictable from the book-loan history but also improves the recommendation of library books for students.

With the rapid development of E-commerce, recent years have witnessed the booming of online advertising industry, which raises extensive concerns of both academic and business circles. Among all the issues, the task of Click-through rates (CTR) prediction plays a central role, as it may influence the ranking and pricing of online ads. To deal with this task, the Factorization Machines (FM) model is designed for better revealing proper combinations of basic features. However, the sparsity of ads transaction data, i.e., a large proportion of zero elements, may severely disturb the performance of FM models. To address this problem, in this paper, we propose a novel Sparse Factorization Machines (SFM) model, in which the Laplace distribution is introduced instead of traditional Gaussian distribution to model the parameters, as Laplace distribution could better fit the sparse data with higher ratio of zero elements. Along this line, it will be beneficial to select the most important features or conjunctions with the proposed SFM model. Furthermore, we develop a distributed implementation of our SFM model on Spark platform to support the prediction task on mass dataset in practice. Comprehensive experiments on two large-scale real-world datasets clearly validate both the effectiveness and efficiency of our SFM model compared with several state-of-the-art baselines, which also proves our assumption that Laplace distribution could be more suitable to describe the online ads transaction data.

An e-commerce website provides a platform for merchants to sell products to customers. While most existing research focuses on providing customers with personalized product suggestions by recommender systems, in this paper, we consider the role of merchants and introduce a parallel problem, i.e., how to select the most valuable customers for a merchant? Accurately answering this question can not only help merchants to gain more profits, but also benefit the ecosystem of e-commence platforms. To deal with this problem, we propose a general approach by taking into consideration the interest and profit of each customer to the merchant, i.e., select the customers who are not only interested in the merchant to ensure the visit of the merchant, but also capable of making good profits. Specifically, we first generate candidate customers for a given merchant by using traditional recommendation techniques. Then we select a set of the valuable customers from candidate customers, which has the balanced maximization between the interest and the profit metrics. Given the NP-hardness of the balanced maximization formulation, we further introduce efficient techniques to solve this maximization problem by exploiting the inherent submodularity property. Finally, extensive experimental results on a real-world dataset demonstrate the effectiveness of our proposed approach.

Group-oriented services such as group recommendations aim to provide services for a group of users. For these applications, how to aggregate the preferences of different group members is the toughest yet most important problem. Inspired by game theory, in this paper, we propose to explore the idea of Nash equilibrium to simulate the selections of members in a group by a game process. Along this line, we first compute the preferences (group-dependent optimal selections) of each individual member in a given group scene, i.e., an equilibrium solution of this group, with the help of two pruning approaches. Then, to get the aggregated unitary preference of each group from all group members, we design a matrix factorization-based method which aggregates the preferences in latent space and estimates the final group preference in rating space. After obtaining the group preference, group-oriented services (e.g., group recommendation) can be directly provided. Finally, we construct extensive experiments on two real-world data sets from multiple aspects. The results clearly demonstrate the effectiveness of our method.

With recent advances in mobile and sensor technologies, a large amount of efforts have been made on developing intelligent applications for taxi drivers, which provide beneficial guide and opportunity to improve the profit and work efficiency. However, limited scopes focus on the latent social interaction within cab drivers, and corresponding social propagation scheme to share driving behaviors has been largely ignored. To that end, in this paper, we propose a comprehensive study to reveal how the social propagation affects for better prediction of cab drivers' future behaviors. To be specific, we first investigate the correlation between drivers' skills and their mutual interactions in the latent vehicle-to-vehicle network, which intuitively indicates the effects of social influences. Along this line, by leveraging the classic social influence theory, we develop a two-stage framework for quantitatively revealing the latent driving pattern propagation within taxi drivers. Comprehensive experiments on a real-word data set collected from the New York City clearly validate the effectiveness of our proposed framework on predicting future taxi driving behaviors, which also support the hypothesis that social factors indeed improve the predictability of driving behaviors.

P2P lending is an emerging wealth-management service for individuals, which allows lenders to directly bid and invest on the loans created by borrowers. In these platforms, lenders often pursue multiple objectives (e.g., non-default probability, fully-funded probability and winning-bid probability) when they select loans to invest. How to automatically assess loans from these objectives and help lenders select loan portfolios is a very important but challenging problem. To that end, in this paper, we present a holistic study on portfolio selections in P2P lending. Specifically, we first propose to adapt gradient boosting decision tree, which combines both static features and dynamic features, to assess loans from multiple objectives. Then, we propose two strategies, i.e., weighted objective optimization strategy and multi-objective optimization strategy, to select portfolios for lenders. For each lender, the first strategy attempts to provide one optimal portfolio while the second strategy attempts to provide a Pareto-optimal portfolio set. Further, we design two algorithms, namely DPA and EVA, which can efficiently resolve the optimizations in these two strategies, respectively. Finally, extensive experiments on a large-scale real-world data set demonstrate the effectiveness of our solutions.

Days on Market (DOM) refers to the number of days a property is on the active market, which is an important measurement of market liquidity in real estate industry. Indeed, at the micro level, DOM is not only a special concern of house sellers, but also a useful indicator for potential buyers to evaluate the popularity of a house. At the macro level, DOM is an important indicator of real estate market status. However, it is very challenging to measure DOM, since there are a variety of factors which can impact on the DOM of a property. To this end, in this paper, we aim to measure real estate liquidity by examining multiple factors in a holistic manner. A special goal is to predict the DOM of a given property listing. Specifically, we first extract key features from multiple types of heterogeneous real estate-related data, such as house profiles and geo-social information of residential communities. Then, based on these features, we develop a multi-task learning based regression approach for predicting the DOM of real estates. This approach can effectively learn district-aware models for different property listings by considering multiple factors. Finally, we conduct extensive experiments on real-world real estate data collected in Beijing and develop a prototype system for practical use. The experimental results clearly validate the effectiveness of the proposed approach for measuring liquidity in real estate markets.

Collaborative filtering (CF) models offer users personalized recommendations by measuring the relevance between the active user and each individual candidate item. Following this idea, user-based collaborative filtering (UCF) usually selects the local popular items from the like-minded neighbor users. However, these traditional relevance-based models only consider the individuals (i.e., each neighbor user and candidate item) separately during neighbor set selection and recommendation set generation, thus usually incurring highly similar recommendations that lack diversity. While many researchers have recognized the importance of diversified recommendations, the proposed solutions either needed additional semantic information of items or decreased accuracy in this process. In this article, we describe how to generate both accurate and diversified recommendations from a new perspective. Along this line, we first introduce a simple measure of coverage that quantifies the usefulness of the whole set, that is, the neighbor userset and the recommended itemset as a complete entity. Then we propose a recommendation framework named REC that considers both traditional relevance-based scores and the new coverage measure based on UCF. Under REC, we further prove that the goals of maximizing relevance and coverage measures simultaneously in both the neighbor set selection step and the recommendation set generation step are NP-hard. Luckily, we can solve them effectively and efficiently by exploiting the inherent submodular property. Furthermore, we generalize the coverage notion and the REC framework from both a data perspective and an algorithm perspective. Finally, extensive experimental results on three real-world datasets show that the REC-based recommendation models can naturally generate more diversified recommendations without decreasing accuracy compared to some state-of-the-art models.

With the booming popularity of online social networks like Twitter and Weibo, online user footprints are accumulating rapidly on the social web. Simultaneously, the question of how to leverage the large-scale user-generated social media data for personal credit scoring comes into the sight of both researchers and practitioners. It has also become a topic of great importance and growing interest in the P2P lending industry. However, compared with traditional financial data, heterogeneous social data presents both opportunities and challenges for personal credit scoring. In this article, we seek a deep understanding of how to learn users’ credit labels from social data in a comprehensive and efficient way. Particularly, we explore the social-data-based credit scoring problem under the micro-blogging setting for its open, simple, and real-time nature. To identify credit-related evidence hidden in social data, we choose to conduct an analytical and empirical study on a large-scale dataset from Weibo, the largest and most popular tweet-style website in China. Summarizing results from existing credit scoring literature, we first propose three social-data-based credit scoring principles as guidelines for in-depth exploration. In addition, we glean six credit-related insights arising from empirical observations of the testbed dataset. Based on the proposed principles and insights, we extract prediction features mainly from three categories of users’ social data, including demographics, tweets, and networks. To harness this broad range of features, we put forward a two-tier stacking and boosting enhanced ensemble learning framework. Quantitative investigation of the extracted features shows that online social media data does have good potential in discriminating good credit users from bad. Furthermore, we perform experiments on the real-world Weibo dataset consisting of more than 7.3 million tweets and 200,000 users whose credit labels are known through our third-party partner. Experimental results show that (i) our approach achieves a roughly 0.625 AUC value with all the proposed social features as input, and (ii) our learning algorithm can outperform traditional credit scoring methods by as much as 17% for social-data-based personal credit scoring.

The problem of information source detection, whose goal is to identify the source of a piece of information from a diffusion process (e.g., computer virus, rumor, epidemic, and so on), has attracted ever-increasing attention from research community in recent years. Although various methods have been proposed, such as those based on centrality, spectral and belief propagation, the existing solutions still suffer from high time complexity and inadequate effectiveness. To this end, we revisit this problem in the paper and present a comprehensive study from the perspective of likelihood approximation. Different from many previous works, we consider both infected and uninfected nodes to estimate the likelihood for the detection. Specifically, we propose a Maximum A Posteriori (MAP) estimator to detect the information source for general graphs with rumor centrality as the prior. To further improve the efficiency, we design two approximate estimators, namely Brute Force Search Approximation (BFSA) and Greedy Search Bound Approximation (GSBA). BFSA tries to traverse the permitted permutations and directly computes the likelihood, while GSBA exploits a strategy of greedy search to find a surrogate upper bound of the probabilities of permitted permutations for a given node, and derives an approximate MAP estimator. Extensive experiments on several network data sets clearly demonstrate the effectiveness of our methods in detecting the single information source.

In the retail market, the consumers' indecisiveness refers to the inability to make quick and assertive decisions when they choose among competing product options. Indeed, indecisiveness has been investigated in a number of fields, such as economics and psychology. However, these studies are usually based on the subjective customer survey data with some manually defined questions. Instead, in this paper, we provide a focused study on automatically mining indecisiveness in massive customer behaviors in online stores. Specifically, we first give a general definition to measure the observed indecisiveness in each behavior session. From these observed indecisiveness, we can learn the latent factors/reasons by a probabilistic factor-based model. These two factors are the indecisive indexes of the customers and the product bundles, respectively. Next, we demonstrate that this indecisiveness mining process could be useful in several potential applications, such as the competitive product detection and personalized product bundles recommendation. Finally, we perform extensive experiments on a large-scale behavioral logs of online customers in a distributed environment. The results reveal that our measurement of indecisiveness agrees with the common sense assessment, and the discoveries are useful in predicting customer behaviors and providing better recommendation services for both customers and online retailers.

With the rapid growth of usage of social network, the patterns, the scales, and the rate of information exchange have brought profound impacts on research and practice in finance. One important topic is the stock market efficiency analysis. Traditional schemes in finance focus on identifying significant abnormal returns triggered by important events. However, those events are merely identified by regular financial announcements such as mergers, equity issuances, and financial reports. Related data-driven approaches mainly focus on developing trading strategies using social media data, while the results are usually lack of theoretical explanations. In this paper, we fill the gap between the usage of social media data and financial theories. We propose a Degree of Social Attention (DSA) framework for stock analysis based on influence propagation model. Specifically, we define the self-influence for users in a social network and the DSA for stocks. A recursive process is also designed for dynamic value updating. Furthermore, we provide two modified approaches to reduce the computational cost. Our testing results from the Chinese stock market suggest that the proposed framework effectively captures stock abnormal returns based on the related social media data, and DSA is verified to be a key factor to link social media activities to the stock market.

As the worlds of commerce and Internet technology become more inextricably linked, a large number of user consumption series become available for creative use. A critical demand along this line is to predict the future product adoption for the merchants, which enables a wide range of applications such as targeted marketing. However, previous works only aimed at predicting if one user will adopt this product or not; the problem of adoption rate (or percentage of use) prediction for each user is still underexplored due to the complexity of user decision-making process. To that end, in this paper we present a comprehensive study for this product adoption rate prediction problem. Specifically, we first introduce a decision function to capture the change of users' product adoption rate, where various factors that may influence the decision can be generally leveraged. Then, we propose two models to solve this function, the Generalized Adoption Model (GAM) that assumes all users are influenced equally by these factors and the Personalized Adoption Model (PAM) that argues each factor contributes differently among people. Furthermore, we extend the PAM to a totally Bayesian model (BPAM) that can automatically learn all parameters. Finally, extensive experiments on two real-world datasets not only show the improvement of our proposed three models, but also give insights to track the effects of the various factors for product adoption decisions.

The problem of Matrix Completion (MC) refers to the process of adding entries for unknown or missing values in a matrix. In this paper, we study the convex matrix completion problem in the form of trace norm bounding. Specifically, we propose a robust solution for this problem based on trace-ball optimization, which can creatively change the original trace norm constraint into the problem of low-rank matrix factorization. Therefore, by searching in a ball space defined by the new trace constraint, the rank of new matrix can be self-determined such that the local minimum for matrix factorization is the global minimum for the original matrix completion task. Meanwhile, we define a free parameter γ to control the model complexity of our approach in terms of how well it fits the training data. Particularly, we identify a value of γb, which is the minimal value of the trace norm, in a way such that the model can exactly fit the known entries in the matrix. Furthermore, we also empirically reveal an important property of our approach: that is, a variable η* generated by γ is always stable with the increase of the amount of training data. This can help to speed up the tuning of optimal parameters for large matrices. Finally, extensive experiments on several real-world datasets clearly validate the effectiveness of the proposed approach.

Frequent pattern mining is an important data mining problem with many broad applications. Most studies in this field use support (frequency) to measure the popularity of a pattern, namely the fraction of transactions or sequences that include the pattern in a data set. In this study, we introduce a new interesting measure, namely occupancy, to measure the completeness of a pattern in its supporting transactions or sequences. This is motivated by some real-world pattern recommendation applications in which an interesting pattern should not only be frequent, but also occupies a large portion of its supporting transactions or sequences. With the definition of occupancy we call a pattern dominant if its occupancy value is above a user-specified threshold. Then, our task is to identify the qualified patterns which are both dominant and frequent. Also, we formulate the problem of mining top-k qualified patterns , that is, finding k qualified patterns with maximum values on a user-defined function of support and occupancy, for example, weighted sum of support and occupancy. The challenge to these tasks is that the value of occupancy does not change monotonically when more items are appended to a given pattern. Therefore, we propose a general algorithm called DOFRA (DOminant and FRequent pattern mining Algorithm) for mining these qualified patterns, which explores the upper bound properties on occupancy to drastically reduce the search process. Finally, we show the effectiveness of DOFRA in two real-world applications and also demonstrate the efficiency of DOFRA on several real and large synthetic datasets.

For better viral marketing, there has been a lot of research on social influence maximization. However, the problem that who is influenced and how diverse the influenced population is, which is important in real-world marketing, has largely been neglected. To that end, in this paper, we propose to consider the magnitude of influence and the diversity of the influenced crowd simultaneously. Specifically, we formulate it as an optimization problem, i.e., diversified social influence maximization. First, we present a general framework for this problem, under which we construct a class of diversity measures to quantify the diversity of the influenced crowd. Meanwhile, we prove that a simple greedy algorithm guarantees to provide a near-optimal solution to the optimization problem. Furthermore, we relax the problem by focusing on the diversity of the nodes targeted for initial activation, and show how this relaxed form could be used to diversify the results of many heuristics, e.g., PageRank. Finally, we run extensive experiments on two real-world datasets, showing that our formulation is effective in generating diverse results.

Information diffusion in social networks is emerging as a promising solution to successful viral marketing, which relies on the effective and efficient identification of a set of nodes with the maximal social influence. While there are tremendous efforts on the development of social influence models and algorithms for social influence maximization, limited progress has been made in terms of designing both efficient and effective algorithms for finding a set of nodes with the maximal social influence. To this end, in this paper, we provide a bounded linear approach for influence computation and influence maximization. Specifically, we first adopt a linear and tractable approach to describe the influence propagation. Then, we develop a quantitative metric, named Group-PageRank, to quickly estimate the upper bound of the social influence based on this linear approach. More importantly, we provide two algorithms Linear and Bound, which exploit the linear approach and Group-PageRank for social influence maximization. Finally, extensive experimental results demonstrate that (a) the adopted linear approach has a close relationship with traditional models and Group-PageRank provides a good estimation of social influence; (b) Linear and Bound can quickly find a set of the most influential nodes and both of them are scalable for large-scale social networks.

The emergence of social networks has provided opportunities for both targeted marketing and viral marketing. By concentrating the efforts on a few key customers, targeted marketing could make the promotion of the items (products) much easier and more cost-effective. On the other hand, viral marketing aims at finding a set of individuals (seeds) to maximize the word-of-mouth propagation of an item. However, these two marketing strategies can only exploit some specific characteristics of the social networks, and the problem of how to combine them together to build a better, stronger business is still open. To that end, in this paper, we propose a general approach for integrated marketing. Specifically, to market a given item, we first generate the item-specific candidate users by a recommendation algorithm, and then select the typical users who have the best balanced utility scores and consumption/social entropy. Next, treating typical users as targeted customers, we study the problem of maximizing information awareness in viral marketing with these constrained targets. Along this line, we define it as a constrained coverage maximization problem, and propose three solutions: GMIC, LMIC and QMIC. Finally, extensive experimental results on real-world datasets demonstrate that our integrated marketing approach could outperform the methods that consider only targeted marketing or viral marketing.

P2P lending is an online platform to make borrowing and investment transactions. A central question on these platforms is how to align the right products with the right investors, thus helping investors to make better decisions. Along this line, tremendous efforts have been devoted to modeling the credits of products and borrowers from an economic perspective. However, these global models are only exploratory in nature and are not practical. In this paper, we focus on the personalized investment recommendation by reconstructing the two steps for investment decision making: what to buy and how much money to pay. Specifically, we first generate a candidate investment recommendation list for each investor that tackles "what to buy" problem. In this process, we consider various unique properties of investment recommendation. Furthermore, according to the portfolio theory, we optimize the shares of each recommended candidate by incorporating the investments an investor currently holds, thus solving the "how much money to pay" problem. Finally, extensive experimental results on a large-scale real world dataset show the effectiveness of our model under various evaluation metrics.

Many of today's online news websites have enabled users to specify different types of emotions (e.g., Angry and shocked) they have after reading news. Compared with traditional user feedbacks such as comments and ratings, these specific emotion annotations are more accurate for expressing users' personal emotions. In this paper, we propose to exploit these users' emotion annotations for online news in order to track the evolution of emotions, which plays an important role in various online services. A critical challenge is how to model emotions with respect to time spans. To this end, we propose a time-aware topic modeling perspective for solving this problem. Specifically, we first develop a model named emotion-Topic over Time (eToT), in which we represent the topics of news as a Beta distribution over time and a multinomial distribution over emotions. Whilee ToT can uncover the latent relationship among news, emotion and time directly, it cannot capture the dynamics of topics. Therefore, we further develop another model named emotion based Dynamic Topic Model (eDTM), where we explore the state space model for tracking the dynamics of topics. In addition, we demonstrate that both eToT and eDTM could enable several potential applications, such as emotion prediction, emotion-based news recommendations and emotion anomaly detections. Finally, we validate the proposed models with extensive experiments with a real-world data set.

Influence maximization is the problem of finding a set of seed nodes in social network for maximizing the spread of influence. Traditionally, researchers view influence propagation as a stochastic process and formulate the influence maximization problem as a discrete optimization problem. Thus, most previous works focus on finding efficient and effective heuristic algorithms within the greedy framework. In this paper, we view the influence maximization problem from the perspective of data reconstruction and propose a novel framework named \textsl{Data Reconstruction for Influence Maximization}(DRIM). In our framework, we first construct an influence matrix, each row of which is the influence of a node to other nodes. Then, we select $k$ most informative rows to reconstruct the matrix and the corresponding nodes are the seed nodes which could maximize the influence spread. Finally, we evaluate our framework on two real-world data sets, and the results show that DRIM is at least as effective as the traditional greedy algorithm.

Providing better travel services for tourists is one of the important applications in urban computing. Though many recommender systems have been developed for enhancing the quality of travel service, most of them lack a systematic and open framework to dynamically incorporate multiple types of additional context information existing in the tourism domain, such as the travel area, season, and price of travel packages. To that end, in this article, we propose an open framework, the Objected-Oriented Recommender System (ORS), for the developers performing personalized travel package recommendations to tourists. This framework has the ability to import all the available additional context information to the travel package recommendation process in a cost-effective way. Specifically, the different types of additional information are extracted and uniformly represented as feature--value pairs. Then, we define the Object, which is the collection of the feature--value pairs. We propose two models that can be used in the ORS framework for extracting the implicit relationships among Objects. The Objected-Oriented Topic Model (OTM) can extract the topics conditioned on the intrinsic feature--value pairs of the Objects. The Objected-Oriented Bayesian Network (OBN) can effectively infer the cotravel probability of two tourists by calculating the co-occurrence time of feature--value pairs belonging to different kinds of Objects. Based on the relationships mined by OTM or OBN, the recommendation list is generated by the collaborative filtering method. Finally, we evaluate these two models and the ORS framework on real-world travel package data, and the experimental results show that the ORS framework is more flexible in terms of incorporating additional context information, and thus leads to better performances for travel package recommendations. Meanwhile, for feature selection in ORS, we define the feature information entropy, and the experimental results demonstrate that using features with lower entropies usually leads to better recommendation results.

The problem of mobile context recognition targets the identification of semantic meaning of context in a mobile environment. This plays an important role in understanding mobile user behaviors and thus provides the opportunity for the development of better intelligent context-aware services. A key step of context recognition is to model the personalized contextual information of mobile users. Although many studies have been devoted to mobile context modeling, limited efforts have been made on the exploitation of the sequential and dependency characteristics of mobile contextual information. Also, the latent semantics behind mobile context are often ambiguous and poorly understood. Indeed, a promising direction is to incorporate some domain knowledge of common contexts, such as “waiting for a bus” or “having dinner,” by modeling both labeled and unlabeled context data from mobile users because there are often few labeled contexts available in practice. To this end, in this article, we propose a sequence-based semisupervised approach to modeling personalized context for mobile users. Specifically, we first exploit the Bayesian Hidden Markov Model (B-HMM) for modeling context in the form of probabilistic distributions and transitions of raw context data. Also, we propose a sequential model by extending B-HMM with the prior knowledge of contextual features to model context more accurately. Then, to efficiently learn the parameters and initial values of the proposed models, we develop a novel approach for parameter estimation by integrating the Dirichlet Process Mixture (DPM) model and the Mixture Unigram (MU) model. Furthermore, by incorporating both user-labeled and unlabeled data, we propose a semisupervised learning-based algorithm to identify and model the latent semantics of context. Finally, experimental results on real-world data clearly validate both the efficiency and effectiveness of the proposed approaches for recognizing personalized context of mobile users.

Recent years have witnessed an increased interest in recommender systems. Despite significant progress in this field, there still remain numerous avenues to explore. Indeed, this paper provides a study of exploiting online travel information for personalized travel package recommendation. A critical challenge along this line is to address the unique characteristics of travel data, which distinguish travel packages from traditional items for recommendation. To that end, in this paper, we first analyze the characteristics of the existing travel packages and develop a tourist-area-season topic (TAST) model. This TAST model can represent travel packages and tourists by different topic distributions, where the topic extraction is conditioned on both the tourists and the intrinsic features (i.e., locations, travel seasons) of the landscapes. Then, based on this topic model representation, we propose a cocktail approach to generate the lists for personalized travel package recommendation. Furthermore, we extend the TAST model to the tourist-relation-area-season topic (TRAST) model for capturing the latent relationships among the tourists in each travel group. Finally, we evaluate the TAST model, the TRAST model, and the cocktail recommendation approach on the real-world travel package data. Experimental results show that the TAST model can effectively capture the unique characteristics of the travel data and the cocktail approach is, thus, much more effective than traditional recommendation techniques for travel package recommendation. Also, by considering tourist relationships, the TRAST model can be used as an effective assessment for travel group formation.

Advances in tourism economics have enabled us to collect massive amounts of travel tour data. If properly analyzed, this data could be a source of rich intelligence for providing real-time decision making and for the provision of travel tour recommendations. However, tour recommendation is quite different from traditional recommendations, because the tourist’s choice is affected directly by the travel costs, which includes both financial and time costs. To that end, in this article, we provide a focused study of cost-aware tour recommendation. Along this line, we first propose two ways to represent user cost preference. One way is to represent user cost preference by a two-dimensional vector. Another way is to consider the uncertainty about the cost that a user can afford and introduce a Gaussian prior to model user cost preference. With these two ways of representing user cost preference, we develop different cost-aware latent factor models by incorporating the cost information into the probabilistic matrix factorization (PMF) model, the logistic probabilistic matrix factorization (LPMF) model, and the maximum margin matrix factorization (MMMF) model, respectively. When applied to real-world travel tour data, all the cost-aware recommendation models consistently outperform existing latent factor models with a significant margin.

Recent years have witnessed the increased interests in exploiting influence in social networks for many applications. To the best of our knowledge, from the computational aspect of social influence analysis, most of existing work focus on either describing the influence propagation process or identifying the set of most influential seed nodes. However, these work usually do not distinguish the "independent influence" of each single seed node after removing other seeds. Since it is important to quickly figure out the real contribution of each seed, in this paper we propose to measure the seed's independent influence by a linear social influence model. Specifically, we first describe the linear social influence model, and then define the independent influence under this model for eliminating the "mutual enrichment" between seed nodes. Meanwhile, we find that the influence of a set of nodes is actually the sum of their independent influence, and we also give upper bounds for independent influence. Moreover, these findings are evaluated by two applications, i.e., ranking the seeds by their independent influence and identifying the Top-K influential ones. Finally, the experimental results on several real-world datasets validate the effectiveness and efficiency of the proposed independent social influence measures.

In this paper, we propose Personalized Markov Embedding (PME), a next-song recommendation strategy for online karaoke users. By modeling the sequential singing behavior, we first embed songs and users into a Euclidean space in which distances between songs and users reflect the strength of their relationships. Then, given each user's last song, we can generate personalized recommendations by ranking the candidate songs according to the embedding. Moreover, PME can be trained without any requirement of content information. Finally, we perform an experimental evaluation on a real world data set provided by ihou.com which is an online karaoke website launched by iFLYTEK, and the results clearly demonstrate the effectiveness of PME.

Collaborative Filtering(CF) is a popular way to build recommender systems and has been successfully employed in many applications. Generally, two kinds of approaches to CF, the local neighborhood methods and the global matrix factorization models, have been widely studied. Though some previous researches target on combining the complementary advantages of both approaches, the performance is still limited due to the extreme sparsity of the rating data. Therefore, it is necessary to consider more information for better reflecting user preference and item content. To that end, in this paper, by leveraging the extra tagging data, we propose a novel unified two-stage recommendation framework, named Neighborhood-aware Probabilistic Matrix Factorization(NHPMF). Specifically, we first use the tagging data to select neighbors of each user and each item, then add unique Gaussian distributions on each user's(item's) latent feature vector in the matrix factorization to ensure similar users(items) will have similar latent features}. Since the proposed method can effectively explores the external data source(i.e., tagging data) in a unified probabilistic model, it leads to more accurate recommendations. Extensive experimental results on two real world datasets demonstrate that our NHPMF model outperforms the state-of-the-art methods.

In this paper, we present a systematic perspective study on choosing and evaluating the initial seed items that will be recommended to the cold start users. We first construct an item consumption correlation network to capture the existing users' general consumption behaviors. Then, we formalize initial items recommendation as the influential seed set selection problem. Along this line, we present several methods, each of which selects seed items according to different rules. Finally, the experimental results on two real-world data sets verify that with different seed items, the users' consumption numbers will be quite different. Meanwhile, the results also provide many deep insights into these selection methods and their recommended seed items.

As the worlds of commerce, entertainment, travel, and Internet technology become more inextricably linked, new types of business data become available for creative use and formal analysis. Indeed, this paper provides a study of exploiting online travel information for personalized travel package recommendation. A critical challenge along this line is to address the unique characteristics of travel data, which distinguish travel packages from traditional items for recommendation. To this end, we first analyze the characteristics of the travel packages and develop a Tourist-Area-Season Topic (TAST) model, which can extract the topics conditioned on both the tourists and the intrinsic features (i.e. locations, travel seasons) of the landscapes. Based on this TAST model, we propose a cocktail approach on personalized travel package recommendation. Finally, we evaluate the TAST model and the cocktail approach on real-world travel package data. The experimental results show that the TAST model can effectively capture the unique characteristics of the travel data and the cocktail approach is thus much more effective than traditional recommendation methods for travel package recommendation.

Advances in tourism economics have enabled us to collect massive amounts of travel tour data. If properly analyzed, this data can be a source of rich intelligence for providing real-time decision making and for the provision of travel tour recommendations. However, tour recommendation is quite different from traditional recommendations, because the tourist's choice is directly affected by the travel cost, which includes the financial cost and the time. To that end, in this paper, we provide a focused study of cost-aware tour recommendation. Along this line, we develop two cost-aware latent factor models to recommend travel packages by considering both the travel cost and the tourist's interests. Specifically, we first design a cPMF model, which models the tourist's cost with a 2-dimensional vector. Also, in this cPMF model, the tourist's interests and the travel cost are learnt by exploring travel tour data. Furthermore, in order to model the uncertainty in the travel cost, we further introduce a Gaussian prior into the cPMF model and develop the GcPMF model, where the Gaussian prior is used to express the uncertainty of the travel cost. Finally, experiments on real-world travel tour data show that the cost-aware recommendation models outperform state-of-the-art latent factor models with a significant margin. Also, the GcPMF model with the Gaussian prior can better capture the impact of the uncertainty of the travel cost, and thus performs better than the cPMF model.

Rating sparsity is a critical issue for collaborative filtering. For example, the well-known Netflix Movie rating data contain ratings of only about 1% user-item pairs. One way to address this rating sparsity problem is to develop more effective methods for training rating prediction models. To this end, in this paper, we introduce a collective training paradigm to automatically and effectively augment the training ratings. Essentially, the collective training paradigm builds multiple different Collaborative Filtering (CF) models separately, and augments the training ratings of each CF model by using the partial predictions of other CF models for unknown ratings. Along this line, we develop two algorithms, Bi-CF and Tri-CF, based on collective training. For Bi-CF and Tri-CF, we collectively and iteratively train two and three different CF models via iteratively augmenting training ratings for individual CF model. We also design different criteria to guide the selection of augmented training ratings for Bi-CF and Tri-CF. Finally, the experimental results show that Bi-CF and Tri-CF algorithms can significantly outperform baseline methods, such as neighborhood-based and SVD-based models.

In real applications, a given user buys or rates an item based on his/her interests. Learning to leverage this interest information is often critical for recommender systems. However, in existing recommender systems, the information about latent user interests are largely under-explored. To that end, in this paper, we propose an interest expansion strategy via personalized ranking based on the topic model, named iExpand, for building an interest-oriented collaborative filtering framework. The iExpand method introduces a three-layer, user-interest-item, representation scheme, which leads to more interpretable recommendation results and helps the understanding of the interactions among users, items, and user interests. Moreover, iExpand strategically deals with many issues, such as the overspecialization and the cold-start problems. Finally, we evaluate iExpand on benchmark data sets, and experimental results show that iExpand outperforms state-of-the-art methods.