Consensus decision-making uses crowd responses (usually from non-experts) to questions to reach a consensus answer based on human-machine collaboration. The crucial point is dynamic, which should not only enable rapid self-iteration toward the correct answer through crowd workers' responses but also adaptively suggest the next most valuable question(s) to accelerate the integration of the answer. However, existing methods reach consensus using either offline data or fixed question search structures, thereby largely sidestepping this dynamic nature. In response, we propose a bilevel optimization-based human-machine collaboration (BiO-HMC), which explores an inner & outer-level optimization to enable effective answer integration and efficient question selection. The resulting optimization problem is intractable because there is no closed-form expression in the inner-level optimization. We employ a gradient-based method and guarantee the method's theoretical convergence. Experimental results on synthetic and real-world datasets demonstrate the effectiveness and efficiency of the BiO-HMC model, i.e., achieving the highest confidence in the correct answer with the lowest labor cost.

This paper investigates the complexity of intelligent decision-making within multiagent systems operating under uncertainty, particularly emphasizing the challenges associated with modeling behaviors of other agents and optimizing decision-making for a subject agent in a common environment characterized by incomplete historical data. To address these challenges, we propose a generative learning method based on a general multiagent decision making framework, namely interactive dynamic influence diagrams, and apply contrastive learning to diversify the generation of potential behaviors, thereby enhancing the subject agent’s modeling and prediction capabilities. We conduct experiments on multiple classic domains to demonstrate the efficacy of the new learning method in improving decision-making quality. The empirical results highlight its substantial improvement in enhancing overall performance in multiagent decision-making. Our work contributes to multiagent decision making particularly when a subject agent interacts with other unknown agents, including humans, in many practical applications.

When intelligent agents act in a stochastic environment, the principle of maximizing expected rewards is used to optimize their policies. The rationality of the maximum rewards becomes a single objective when agents’ decision problems are solved in most cases. This sometimes leads to the agents’ behaviors (the optimal policies for solving the decision problems) that are not legible . In other words, it is difficult for users (or other agents and even humans) to understand the agents’ intentions when they are executing the optimal policies. Hence, it becomes pertinent to consider the legibility of agents’ decision problems. The key challenge lies in formulating a proper legibility function in the problems. Using domain experts’ inputs leans to be subjective and inconsistent in specifying legibility values, and the manual approach quickly becomes infeasible in a complex problem domain. In this article, we aim to learn such a legibility function parallel to developing a (conventional) reward function. We adopt inverse reinforcement learning techniques to automate a legibility function in agents’ decision problems. We first demonstrate the effectiveness of the inverse reinforcement learning technique when legibility is solely considered in a decision problem. Things become complicated when both the reward and legibility functions are to be found. We develop a multi-objective inverse reinforcement learning method to automate the two functions in a good balance simultaneously. We vary problem domains in the performance study and provide empirical results in support.

A multiagent sequential decision problem has been seen in many critical applications including urban transportation, autonomous driving cars, military operations, etc. Its widely known solution, namely multiagent reinforcement learning, has evolved tremendously in recent years. Among them, the solution paradigm of modeling other agents attracts our interest, which is different from traditional value decomposition or communication mechanisms. It enables agents to understand and anticipate others' behaviors and facilitates their collaboration. Inspired by recent research on the legibility that allows agents to reveal their intentions through their behavior, we propose a multiagent active legibility framework to improve their performance. The legibility-oriented framework drives agents to conduct legible actions so as to help others optimise their behaviors. In addition, we design a series of problem domains that emulate a common legibility-needed scenario and effectively characterize the legibility in multiagent reinforcement learning. The experimental results demonstrate that the new framework is more efficient and requires less training time compared to several multiagent reinforcement learning algorithms.

Knowing a prerequisite structure among skills in a subject domain effectively enables several educational applications, including intelligent tutoring systems and curriculum planning. Traditionally, educators or domain experts use intuition to determine the skills' prerequisite relationships, which is time-consuming and prone to fall into the trap of blind spots. In this paper, we focus on inferring the prerequisite structure given access to students' performance on exercises in a subject. Nevertheless, it is challenging since students' mastery of skills can not be directly observed, but can only be estimated, i.e., its latency in nature. To tackle this problem, we propose a causal-driven skill prerequisite structure discovery (CSPS) method in a two-stage learning framework. In the first stage, we learn the skills' correlation relationships presented in the covariance matrix from the student performance data while, through the predicted covariance matrix in the second stage, we consider a heuristic method based on conditional independence tests and standardized partial variance to discover the prerequisite structure. We demonstrate the performance of the new approach with both simulated and real-world data. The experimental results show the effectiveness of the proposed model for identifying the skills' prerequisite structure.

Realizing Generalized Zero-Shot Learning (GZSL) based on large models is emerging as a prevailing trend. However, most existing methods merely regard large models as black boxes, solely leveraging the features output by the final layer while disregarding potential performance enhancements from other layers. Indeed, numerous researchers have visually depicted variations in the features learned across different layers of neural networks. Motivated by this observation, we propose a Vision Transformer (ViT)-based GZSL method named Depth-Aware Multi-Modal ViT (DAM2ViT), which exploits multi-level features of ViT. DAM2ViT incorporates a multi-modal interaction block to align semantic information of categories across multiple layers, thereby augmenting the model's capacity to learn associations between visual and semantic spaces. Extensive experiments conducted on three benchmark datasets (i.e., CUB, SUN, AWA2) have showcased that DAM2ViT achieves competitive results compared to state-of-the-art methods.

The advantages of resource elasticity and proactive data backup in cloud databases have attracted a large number of users to consider deploying their IT systems in the cloud. Factors such as performance, reliability, security, and ease of use are the most important considerations for users when choosing to migrate to the cloud. This paper mainly discusses the necessity of Quality of Service (QoS) awareness for cloud applications in the current public cloud environment, as well as the technological trends and challenges associated with it.

In cloud computing, tenants tend to select predefined flavors with substantial resources that go beyond their needs, in order to avoid resource starvation. As a result, cloud service providers experience a high allocation ratio with a low utilization ratio of resources. To improve the resource utilization, cloud service providers offer lower-priced resources by building an overcommitted environment to reuse resources. However, the colocation interference of workloads in the overcommitted environment may cause performance anomalies and degrade quality of service (QoS). In this paper, we propose a real-time mechanism to pin a group of workloads to a certain subset of CPUs to increase resource utilization and minimize interference. To be specific, we leverage deep reinforcement learning (DRL) to automatically adjust the CPU pinning configuration in the overcommitted environment according to the performance of workloads.

Student cognitive modeling (SCM) is a fundamental task in intelligent education, with applications ranging from personalized learning to educational resource allocation. By exploiting students' response logs, SCM aims to predict their exercise performance as well as estimate knowledge proficiency in a subject. Data mining approaches such as matrix factorization can obtain high accuracy in predicting student performance on exercises, but the knowledge proficiency is unknown or poorly estimated. The situation is further exacerbated if only sparse interactions exist between exercises and students (or knowledge concepts). To solve this dilemma, we root monotonicity (a fundamental psychometric theory on educational assessments) in a co-factorization framework and present an autoencoder-like nonnegative matrix co-factorization (AE-NMCF), which improves the accuracy of estimating the student's knowledge proficiency via an encoder-decoder learning pipeline. The resulting estimation problem is nonconvex with nonnegative constraints. We introduce a projected gradient method based on block coordinate descent with Lipschitz constants and guarantee the method's theoretical convergence. Experiments on several real-world data sets demonstrate the efficacy of our approach in terms of both performance prediction accuracy and knowledge estimation ability, when compared with existing student cognitive models.

Student cognitive modeling plays an important role in the rapid development of educational data mining research. It aims to discover students' proficiency in knowledge concepts as well as to predict students' performance in conducting exercises. Studies in the past few years have been mainly centered around two types of techniques: cognitive diagnosis models and data mining approaches. Cognitive diagnosis models focus on students' cognitive states and assess their knowledge concept proficiency through handcrafted features. The subjective features may trigger cascading errors in the students' performance prediction. On the other hand, data mining techniques, e.g., matrix factorization methods, achieve high prediction accuracy by directly modeling the students' exercising process. It lacks measuring the students' knowledge concept proficiency. To address the dilemma of the aforementioned methods, in this paper, we propose a scalable non-negative matrix co-factorization (SNMCF) model by jointly modeling the students' knowledge states and their exercising process. SNMCF can achieve high accuracy in predicting students' exercise performance while modeling their states of knowledge concepts in a given domain. We conduct extensive experiments on several real-world datasets, including large sparse ones, and demonstrate the effectiveness of our new approach in terms of prediction accuracy, cognitive diagnostic ability, and scalability

Abstract The process of team composition in multiplayer sports such as football has been a main area of interest within the field of the science of teamwork, which is important for improving competition results and game experience. Recent algorithms for the football team composition problem take into account the skill proficiency of players but not the interactions between players that contribute to winning the championship. To automate the composition of a cohesive team, we consider the internal collaborations among football players. Specifically, we propose a Team Composition based on the Football Players’ Attributed Collaboration Network (TC-FPACN) model, aiming to identify a cohesive football team by maximizing football players’ capabilities and their collaborations via three network metrics, namely, network ability, network density and network heterogeneity&homogeneity. Solving the optimization problem is NP-hard; we develop an approximation method based on greedy algorithms and then improve the method through pruning strategies given a budget limit. We conduct experiments on two popular football simulation platforms. The experimental results show that our proposed approach can form effective teams that dominate others in the majority of simulated competitions.

Discovering common intentions of multiple agents is one of the important ways to detect the tendency of their collaborative behaviours. Existing work mainly focuses on intention recognition in a single-agent setting and uses a descriptive model, e.g. Bayesian networks, in the recognition process. In this article, we develop a new approach of identifying intentions for multiple agents through analysing their behaviours over time. We first define a prescriptive, behavioural model for a single agent that represents the agent’s behaviours where their intentions are hidden in the plan execution. We introduce landmarks into the behavioural model therefore enhancing informative features to identify common intentions for multiple agents. Subsequently, we refine the model by focusing only on action sequences in their plans and provide a light model for identifying and comparing their intentions. The new model provides a simple approach of grouping agents’ common intentions upon partial plans observed in agents’ interactions. After that, we transform the intention recognition into an un-supervised learning problem and adapt a clustering algorithm to group intentions of multiple agents through comparing their behavioural models. We conduct the clustering process through measuring similarity of probability distributions over potential landmarks in the behavioural models so as to discover agents’ common intentions. Finally, we examine the new intention recognition approaches in two problem domains. We demonstrate importance of recognising common intentions of multiple agents in achieving their goals and provide experimental results to show performance of the new approaches.

Preference learning, especially pairwise preference learning, is an efficient method for modeling implicit feedback in item recommendation. However, it is insufficient and not always valid to work for a basic pairwise preference model, which assumes that users prefer interacted (i.e., bought or viewed) items to un-interacted (i.e., not bought or not viewed) items. Recently, the state-of-the-art approaches have emerged as two separate but powerful methods, namely, pairwise preferences over item-sets and asymmetric pairwise preference models, respectively, to address limitations of the basic models. In spite of the success achieved by these methods, the assumption that the horizontal pairwise preference is with respect to two items does not always hold in asymmetric pairwise preference. Hence, it is appealing to integrate them into a uniform approach. In this article, we propose a novel symmetric pairwise preference assumption. We use a weighted average through a softmax function and define the overall preferences that can better discover users’ preference patterns. With the new assumption and the weighted average method, we propose a novel recommendation algorithm to improve the recommendation quality. Extensive empirical studies show that our new algorithms can significantly outperform several state-of-the-art and baseline methods over a number of public datasets.

Modeling other agents' behaviors plays an important role in decision models for interactions among multiple agents. To optimize its own decisions, a subject agent needs to model what other agents act simultaneously in an uncertain environment. However, modeling insufficiency occurs when the agents are competitive and the subject agent cannot get full knowledge about other agents. Even when the agents are collaborative, they may not share their true behaviors due to their privacy concerns. Most of the recent research still assumes that the agents have common knowledge about their environments and a subject agent has the true behavior of other agents in its mind. Consequently, the resulting techniques are not applicable in many practical problem domains. In this article, we investigate into diversifying behaviors of other agents in the subject agent's decision model before their interactions. The challenges lie in generating and measuring new behaviors of other agents. Starting with prior knowledge about other agents' behaviors, we use a linear reduction technique to extract representative behavioral features from the known behaviors. We subsequently generate their new behaviors by expanding the features and propose two diversity measurements to select top- K $K$ behaviors. We demonstrate the performance of the new techniques in two well-studied problem domains. The top- K $K$ behavior selection embarks the study of unknown behaviors in multiagent decision making and inspires investigation of diversifying agents' behaviors in competitive agent interactions. This study will contribute to intelligent systems dealing with unknown unknowns in an open artificial intelligence world.

Abstract With the availability of significant amount of data, data-driven decision making becomes an alternative way for solving complex multiagent decision problems. Instead of using domain knowledge to explicitly build decision models, the data-driven approach learns decisions (probably optimal ones) from available data. This removes the knowledge bottleneck in the traditional knowledge-driven decision making, which requires a strong support from domain experts. In this paper, we study data-driven decision making in the context of interactive dynamic influence diagrams (I-DIDs)—a general framework for multiagent sequential decision making under uncertainty. We propose a data-driven framework to solve the I-DIDs model and focus on learning the behavior of other agents in problem domains. The challenge is on learning a complete policy tree that will be embedded in the I-DIDs models due to limited data. We propose two new methods to develop complete policy trees for the other agents in the I-DIDs. The first method uses a simple clustering process, while the second one employs sophisticated statistical checks. We analyze the proposed algorithms in a theoretical way and experiment them over two problem domains.

We present a novel approach for identifying exact and approximate behavioral equivalence between models of agents. This is significant because both decision making and game play in multiagent settings must contend with behavioral models of other agents in order to predict their actions. One approach that reduces the complexity of the model space is to group models that are behaviorally equivalent. Identifying equivalence between models requires solving them and comparing entire policy trees. Because the trees grow exponentially with the horizon, our approach is to focus on partial policy trees for comparison and determining the distance between updated beliefs at the leaves of the trees. We propose a principled way to determine how much of the policy trees to consider, which trades off solution quality for efficiency. We investigate this approach in the context of the interactive dynamic influence diagram and evaluate its performance.