Transmission Control Protocol (TCP) congestion control is a fundamental mechanism in the Internet that maintains network stability and performance by adjusting the sending rate of connections. Recently, Deep Reinforcement Learning (DRL) methods have shown superior performance over traditional expert-designed solutions. However, the DRL policies are often represented by black-box neural networks, they lack interpretability, making verification challenging and requiring excessive floating-point computation. This work introduces a novel approach, Programmatic reinforcement learning for Congestion Control (ProCC), designed to autonomously discover a program as a control policy from scratch. Programs in ProCC include branching structures (e.g., if blocks and if-else blocks), conditions and actions. However, directly optimizing such program structures is challenging due to their discrete non-differentiable nature, and the program space grows exponentially as the depth increases. To address this issue, ProCC defines a Domain-Specific Language (DSL) and program transformation rules, enabling the construction of a program search graph where similar programs are closer in proximity. Subsequently, ProCC employs Monte Carlo Tree Search (MCTS) to efficiently explore the discrete space and obtain promising programs. Extensive experiments conducted in multiple simulated environments demonstrate that ProCC is adaptive and consistently performs well under varying network conditions. The learned program's performance surpasses that of state-of-the-art DRL agents, and more importantly, the generated policies are concise, transparent, and computationally efficient.

Learning to answer multi-step complex questions requires machines to perform like a human to think and reason step by step, which is one of the core abilities of a question answering system. Recent advancements have revealed that large language models exhibit remarkable reasoning capabilities by generating intermediate chain-of-thought rationales. However, the completeness of their rationales lacks assurance as they are susceptible to omitting steps and making factual errors. In this paper, drawing inspiration from human-like reasoning processes in answering multi-step questions, we explicitly plan the rationales to ensure their completeness. We propose a two-stage Decomposition-Evaluation (Dec-Eval) framework including a step decomposition stage and a rationale generation stage. Specifically, in the first stage, we decompose the complex question into simpler sub-ones and simulate a human's ability to grasp logical clues to ensure the integrity of step planning. Then, in the second stage, based on the sub-questions, we generate and evaluate rationales step by step. Both stages work together organically, improving the completeness of rationales and the accuracy of the answer. To further control the question answering process, we propose a novel knowledge injection mechanism that incorporates external knowledge to guide both stages. Extensive experiments on three challenging multi-step QA datasets demonstrate that Dec-Eval can explicitly generate more logical rationales, and significantly improve the reasoning performances of different backbone models.

Knowledge-based question answering (KBQA) is a key task in natural language processing research, and also an approach to access the web data and knowledge, which requires exploiting knowledge graphs (KGs) for reasoning. In the literature, one promising solution for KBQA is to incorporate the pretrained language model (LM) with KGs by generating KG-centered pretraining corpus, which has shown its superiority. However, these methods often depend on specific techniques and resources to work, which may not always be available and restrict its application. Moreover, existing methods focus more on improving language understanding with KGs, while neglect the more important human-like complex reasoning. To this end, in this paper, we propose a general K nowledge-I njected C urriculum P retraining framework (KICP) to achieve comprehensive KG learning and exploitation for KBQA tasks, which is composed of knowledge injection (KI), knowledge adaptation (KA) and curriculum reasoning (CR). Specifically, the KI module first injects knowledge into the LM by generating KG-centered pretraining corpus, and generalizes the process into three key steps that could work with different implementations for flexible application. Next, the KA module learns knowledge from the generated corpus with LM equipped with an adapter as well as keeps its original natural language understanding ability to reduce the negative impacts of the difference between the generated and natural corpus. Last, to enable the LM with complex reasoning, the CR module follows human reasoning patterns to construct three corpora with increasing difficulties of reasoning, and further trains the LM from easy to hard in a curriculum manner to promote model learning. We provide an implementation of the general framework, and evaluate the proposed KICP on four real-word datasets. The results demonstrate that our framework can achieve higher performances, and have good generalization ability to other QA tasks.

Cognitive diagnosis (CD) aims to reveal the proficiency of students on specific knowledge concepts and traits of test exercises (e.g., difficulty). It plays a critical role in intelligent education systems by supporting personalized learning guidance. However, recent developments in CD mostly concentrate on improving the accuracy of diagnostic results and often overlook the important and practical task: domain-level zero-shot cognitive diagnosis (DZCD). The primary challenge of DZCD is the deficiency of student behavior data in the target domain due to the absence of student-exercise interactions or unavailability of exercising records for training purposes. To tackle the cold-start issue, we propose a two-stage solution named TechCD (Transferable knowledgE Concept grapH embedding framework for Cognitive Diagnosis). The fundamental notion involves utilizing a pedagogical knowledge concept graph (KCG) as a mediator to connect disparate domains, allowing the transmission of student cognitive signals from established domains to the zero-shot cold-start domain. Specifically, a naive yet effective graph convolutional network (GCN) with the bottom-layer discarding operation is initially employed over the KCG to learn transferable student cognitive states and domain-specific exercise traits. Moreover, we give three implementations of the general TechCD framework following the typical cognitive diagnosis solutions. Finally, extensive experiments on real-world datasets not only prove that Tech can effectively perform zero-shot diagnosis, but also give some popular applications such as exercise recommendation.

In online games, predicting massive battle outcomes is a fundamental task of many applications, such as team optimization and tactical formulation. Existing works do not pay adequate attention to the massive battle. They either seek to evaluate individuals in isolation or mine simple pair-wise interactions between individuals, neither of which effectively captures the intricate interactions between massive units (e.g., individuals). Furthermore, as the team size increases, the phenomenon of diminishing marginal utility of units emerges. Such a diminishing pattern is rarely noticed in previous work, and how to capture it from data remains a challenge. To this end, we propose a novel Massive battle outcome predictor with margiNal Effect modules, namely MassNE, which comprehensively incorporates individual effects, cooperation effects (i.e., intra-team interactions) and suppression effects (i.e., inter-team interactions) for predicting battle outcomes. Specifically, we design marginal effect modules to learn how units’ marginal utility changing respect to their number, where the monotonicity assumption is applied to ensure rationality. In addition, we evaluate the current classical models and provide mathematical proofs that MassNE is able to generalize several earlier works in massive settings. Massive battle datasets generated by StarCraft II APIs are adopted to evaluate the performances of MassNE. Extensive experiments empirically demonstrate the effectiveness of MassNE, and MassNE can reveal reasonable cooperation effects, suppression effects, and marginal utilities of combat units from the data.

Reasoning answers to mathematical problems requires machines to think and operate like a human to learn knowledge from mathematical data, which is one of the fundamental tasks for exploring general artificial intelligence. Most solutions focus on mimicking how humans understand problems, which generate the necessary expressions for answers. However, they are still far from enough since they ignore the core ability of humans to acquire knowledge from experience. In this paper, we propose a Cognitive Solver (CogSolver) that is capable of autonomously learning knowledge from scratch to solve mathematical problems, inspired by two cognitive science theories. Specifically, we draw one insight from the dual process theory to establish an intelligent BRAIN-ARM framework, and refer to another information processing theory to summarize the knowledge learning process into Store-Apply-Update steps. In CogSolver, the BRAIN system stores three types of mathematical knowledge, including semantics knowledge, relation knowledge, and mathematic rule knowledge. Then, the ARM system applies the knowledge in BRAIN to answer the problems. Specifically, we design a knowledge-aware module and a commutative module in ARM to improve its reasoning ability, where the knowledge is organically integrated into answer reasoning process. After solving the problems, BRAIN updates the stored knowledge according to the feedback of ARM, where we develop knowledge filters to eliminate the redundant ones and further form a more reasonable knowledge base. Our CogSolver carries out the above three steps iteratively, which behaves more like a human. We conduct extensive experiments on real-world math word problem datasets. The experimental results demonstrate the improvement in answer reasoning and clearly show how CogSolver gains knowledge from the problems, leading to superior interpretability. Our codes are available at https://github.com/bigdata-ustc/CogSolver.

Programming has become an important skill for individuals nowadays. For the demand to improve personal programming skill, tracking programming skill proficiency is getting more and more important. However, few researchers pay attention to measuring the programming skill of learners. Most of existing studies on learner capability portrait only made use of the exercise results, while the rich behavioral information contained in programming exercise process remains unused. Therefore, we propose a model that measures skill proficiency in programming exercise process named Programming Skill Tracing (PST). We designed Code Information Graph (CIG) to represent the feature of learners' solution code, and Code Tracing Graph (CTG) to measure the changes between the adjacent submissions. Furthermore, we divided programming skill into programming knowledge and coding ability to get more fine-grained assessment. Finally, we conducted various experiments to verify the effectiveness and interpretability of our PST model.

Learning informative representations for educational questions is a fundamental problem in online learning systems, which can promote many applications, e.g., difficulty estimation. Most solutions integrate all information of one question together following a supervised manner, where the representation results are unsatisfactory sometimes due to the following issues. First, they cannot ensure the presentation ability due to the scarcity of labeled data. Then, the label-dependent representation results have poor feasibility to be transferred. Moreover, aggregating all information into the unified may introduce some noises in applications since it cannot distinguish the diverse characteristics of questions. In this paper, we aim to learn the disentangled representations of questions. We propose a novel unsupervised model, namely DisenQNet, to divide one question into two parts, i.e., a concept representation that captures its explicit concept meaning and an individual representation that preserves its personal characteristics. We achieve this goal via mutual information estimation by proposing three self-supervised estimators in a large unlabeled question corpus. Then, we propose another enhanced model, DisenQNet+, that transfers the representation knowledge from unlabeled questions to labeled questions in specific applications by maximizing the mutual information between both. Extensive experiments on real-world datasets demonstrate that DisenQNet can generate effective and meaningful disentangled representations for questions, and furthermore, DisenQNet+ can improve the performance of different applications.

Recent years have witnessed the emergence of worldwide megalopolises and the accompanying public safety events, making urban safety a top priority in modern urban management. Among various threats, dangerous goods such as gas and hazardous chemicals transported through cities have bred repeated tragedies and become the deadly “bomb” we sleep with every day. While tremendous research efforts have been devoted to dealing with dangerous goods transportation (DGT) issues, further study is still in great need to quantify this problem and explore its intrinsic dynamics from a big data perspective. In this article, we present a novel system called DGeye , to feature a fusion between DGT trajectory data and residential population data for dangers perception and prediction. Specifically, DGeye first develops a probabilistic graphical model-based approach to mine spatio-temporally adjacent risk patterns from population-aware risk trajectories. Then, DGeye builds the novel causality network among risk patterns for risk pain-point identification, risk source attribution, and online risky state prediction. Experiments on both Beijing and Tianjin cities demonstrate the effectiveness of DGeye in real-life DGT risk management. As a case in point, our report powered by DGeye successfully drove the government to lay down gas pipelines for the famous Guijie food street in Beijing.

While having achieved great success in rich real-life applications, deep neural network (DNN) models have long been criticized for their vulnerability to adversarial attacks. Tremendous research efforts have been dedicated to mitigating the threats of adversarial attacks, but the essential trait of adversarial examples is not yet clear, and most existing methods are yet vulnerable to hybrid attacks and suffer from counterattacks. In light of this, in this paper, we first reveal a gradient-based correlation between sensitivity analysis-based DNN interpreters and the generation process of adversarial examples, which indicates the Achilles's heel of adversarial attacks and sheds light on linking together the two long-standing challenges of DNN: fragility and unexplainability. We then propose an interpreter-based ensemble framework called X-Ensemble for robust adversary defense. X-Ensemble adopts a novel detection-rectification process and features in building multiple sub-detectors and a rectifier upon various types of interpretation information toward target classifiers. Moreover, X-Ensemble employs the Random Forests (RF) model to combine sub-detectors into an ensemble detector for adversarial hybrid attacks defense. The non-differentiable property of RF further makes it a precious choice against the counterattack of adversaries. Extensive experiments under various types of state-of-the-art attacks and diverse attack scenarios demonstrate the advantages of X-Ensemble to competitive baseline methods.

Personalized Route Recommendation (PRR) aims to generate user-specific route suggestions in response to users' route queries. Early studies cast the PRR task as a pathfinding problem on graphs, and adopt adapted search algorithms by integrating heuristic strategies. Although these methods are effective to some extent, they require setting the cost functions with heuristics. In addition, it is difficult to utilize useful context information in the search procedure. To address these issues, we propose using neural networks to automatically learn the cost functions of a classic heuristic algorithm, namely A* algorithm, for the PRR task. Our model consists of two components. First, we employ attention-based Recurrent Neural Networks (RNN) to model the cost from the source to the candidate location by incorporating useful context information. Instead of learning a single cost value, the RNN component is able to learn a time-varying vectorized representation for the moving state of a user. Second, we propose to use a value network for estimating the cost from a candidate location to the destination. For capturing structural characteristics, the value network is built on top of improved graph attention networks by incorporating the moving state of a user and other context information. The two components are integrated in a principled way for deriving a more accurate cost of a candidate location. Extensive experiment results on three real-world datasets have shown the effectiveness and robustness of the proposed model.