The capability of traffic-information systems to sense the movement of millions of users and offer trip plans through mobile phones has enabled a new way of optimizing city traffic dynamics, turning transportation big data into insights and actions in a closed-loop and evaluating this approach in the real world. Existing research has applied dynamic Bayesian networks and deep neural networks to make traffic predictions from floating car data, utilized dynamic programming and simulation approaches to identify how people normally travel with dynamic traffic assignment for policy research, and introduced Markov decision processes and reinforcement learning to optimally control traffic signals. However, none of these works utilized floating car data to suggest departure times and route choices in order to optimize city traffic dynamics. In this paper, we present a study showing that floating car data can lead to lower average trip time, higher on-time arrival ratio, and higher Charypar-Nagel score compared with how people normally travel. The study is based on optimizing a partially observable discrete-time decision process and is evaluated in one synthesized scenario, one partly synthesized scenario, and three real-world scenarios. This study points to the potential of a “living lab” approach where we learn, predict, and optimize behaviors in the real world.

Imitation learning refers to the problem where an agent learns to perform a task through observing and mimicking expert demonstrations, without knowledge of the cost function. State-of-the-art imitation learning algorithms reduce imitation learning to distribution-matching problems by minimizing some distance measures. However, the distance measure may not always provide informative signals for a policy update. To this end, we propose the variational adversarial kernel learned imitation learning (VAKLIL), which measures the distance using the maximum mean discrepancy with variational kernel learning. Our method optimizes over a large cost-function space and is sample efficient and robust to overfitting. We demonstrate the performance of our algorithm through benchmarking with four state-of-the-art imitation learning algorithms over five high-dimensional control tasks, and a complex transportation control task. Experimental results indicate that our algorithm significantly outperforms related algorithms in all scenarios.

Multi-agent Imitation learning (MAIL) refers to the problem that agents learn to perform a task interactively in a multi-agent system through observing and mimicking expert demonstrations, without any knowledge of a reward function from the environment. MAIL has received a lot of attention due to promising results achieved on synthesized tasks, with the potential to be applied to complex real-world multi-agent tasks. Key challenges for MAIL include sample efficiency and scalability. In this paper, we proposed Bayesian multi-type mean field multi-agent imitation learning (BM3IL). Our method improves sample efficiency through establishing a Bayesian formulation for MAIL, and enhances scalability through introducing a new multi-type mean field approximation. We demonstrate the performance of our algorithm through benchmarking with three state-of-the-art multi-agent imitation learning algorithms on several tasks, including solving a multi-agent traffic optimization problem in a real-world transportation network. Experimental results indicate that our algorithm significantly outperforms all other algorithms in all scenarios.

Le Fang, Chunyuan Li, Jianfeng Gao, Wen Dong, Changyou Chen. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.

Policy evaluation algorithms are essential to reinforcement learning due to their ability to predict the performance of a policy. However, there are two long-standing issues lying in this prediction problem that need to be tackled: off-policy stability and on-policy efficiency. The conventional temporal difference (TD) algorithm is known to perform very well in the on-policy setting, yet is not off-policy stable. On the other hand, the gradient TD and emphatic TD algorithms are off-policy stable, but are not on-policy efficient. This paper introduces novel algorithms that are both off-policy stable and on-policy efficient by using the oblique projection method. The empirical experimental results on various domains validate the effectiveness of the proposed approach.

Internet of Things and mobile crowdsensing provide an unprecedented opportunity to collect big data that continuously track the dynamics of large populations. One very important application is to leverage these massive, noisy, and partially observed traffic data to recover road network dynamics and to improve travel efficiency for everyone on the road; however, this process is complicated by high-dimensional and non-linear system dynamics, as well as incomplete information. In this paper we propose a versatile framework for integrating simulation modeling and machine learning in the transportation domain. Our generative approach is related to the stochastic prediction of traffic dynamics in real time using particle filters and simulation. To cope with incomplete information, we use particle filter to empirically estimate the posterior distribution of road network dynamics, and use the particles to improve driving strategies. This integration leads to better accuracy in tracking and predicting vehicle dynamics, and higher future expected return as driving strategies improve.

In the electronic system manufacturing process, the board-level functional test is recognized as the most significant step to prevent defective products from entering the market. In recent years, machine learning and data mining have proven to be efficient techniques in determining root cause from the problematic functional test result, especially when the integrated circuits (IC) are becoming increasingly highly-integrated. However, the test results are sometimes unavailable due to either abnormal ending of the test sequence or occasional system failures, which results in a decreased performance of data-driven diagnosis systems. In this paper, we propose a data imputation algorithm to predict the missing entries in the functional test result, by considering the correlation between test items with conditional specification. We evaluate our data imputation algorithm over the test results collected from three different stages of functional test on a line card used in the telecommunication system. The result shows that our proposed data imputation algorithm consistently outperforms other imputation techniques with various data-driven approaches in terms of diagnosing the root cause, increasing the diagnosis accuracy by an average of 28.13% compared to none data imputation, and 9.74% compared to the naive pass imputation.

Location information is playing a significant role in nowadays mobile applications. Performing indoor localization with existing WiFi infrastructure and smartphone motion sensor through statistical filtering has been proven to be a feasible solution. Many literature have resorted to particle filters to deal with the multi-modal and non-Gaussian problem associated with the filtering process. Although grid-based filters can approximate the true densities better compared with particle filters, their computational cost is extremely expensive, especially for large areas. In this paper, we develop a novel asymmetric grid-based filter to accommodate both high-resolution requirement and computational cost-efficiency. The evaluation over an indoor area of 3750m2has shown that our proposed method can achieve a median error of only 2.72m, which is 0.17m more accurate with only 24% of the computation cost compared to particle filters.

Technological breakthroughs allow us to collect data with increasing spatio-temporal resolution from complex interaction systems. The combination of high-resolution observations, expressive dynamic models, and efficient machine learning algorithms can lead to crucial insights into complex interaction dynamics and the functions of these systems. In this paper, we formulate the dynamics of a complex interacting network as a stochastic process driven by a sequence of events, and develop expectation propagation algorithms to make inferences from noisy observations. To avoid getting stuck at a local optimum, we formulate the problem of minimizing Bethe free energy as a constrained primal problem and take advantage of the concavity of dual problem in the feasible domain of dual variables guaranteed by duality theorem. Our expectation propagation algorithms demonstrate better performance in inferring the interaction dynamics in complex transportation networks than competing models such as particle filter, extended Kalman filter, and deep neural networks.

Location information has become a key component of many applications in mobile and pervasive computing, and the ability to accurately predict the mobility of clients allows these applications to provide better service. However, existing location predictors rely heavily on a significant amount of empirical knowledge to function well. In this paper, we develop a novel framework to predict unknown locations when only little location information is available. Specifically, we first extract WiFi locations from WiFi scan results, then a mobility model is built based on the resulted WiFi location graph with connectivity information, finally, we make location predictions with little known location data with Gibbs sampling over the mobility model. Using a data set containing 31 fairly complete WiFi traces collected over three months as ground truth, we compare our proposed approach with other existing state-of-the-art location predictors. The experimental results show that our framework can achieve 83% location prediction accuracy with only three location samples each day, 15% better than Markov and Bayesian predictors which heavily rely on empirical knowledge.

Social dynamics is concerned primarily with interactions among individuals and the resulting group behaviors, modeling the temporal evolution of social systems via the interactions of individuals within these systems. In particular, the availability of large-scale data from social networks and sensor networks offers an unprecedented opportunity to predict state-changing events at the individual level. Examples of such events include disease transmission, opinion transition in elections, and rumor propagation. Unlike previous research focusing on the collective effects of social systems, this study makes efficient inferences at the individual level. In order to cope with dynamic interactions among a large number of individuals, we introduce the stochastic kinetic model to capture adaptive transition probabilities and propose an efficient variational inference algorithm the complexity of which grows linearly — rather than exponentially— with the number of individuals. To validate this method, we have performed epidemic-dynamics experiments on wireless sensor network data collected from more than ten thousand people over three years. The proposed algorithm was used to track disease transmission and predict the probability of infection for each individual. Our results demonstrate that this method is more efficient than sampling while nonetheless achieving high accuracy.

Indoor location information plays a fundamental role in supporting various interesting location- aware indoor applications. Widely deployed WiFi networks make it feasible to perform indoor localization by first establishing a received signal strength (RSS) map covering the whole area based on a signal propagation model, then determining a location from an online RSS measurement given the RSS map. However, challenges remain in practical deployments, due to inaccurately estimated RSS values in the RSS map and insufficient number of access points (APs) in large indoor areas. To address these challenges, we develop a robust, cost-effective and scalable localization system (REAL). Our approach takes the error from the indoor radio signal propagation model into consideration. It also exploits information of unobserved APs at a given location and an optimal clustering method in the location searching phase. Our real-world experimental results demonstrate that REAL achieves considerable localization accuracy at a very low training cost even for a large indoor area. In addition, the results show that our scheme can also be effectively applied to Bluetooth networks with sparse signal coverage.

The paper addresses the automatic recognition of social and task-oriented functional roles in small-group meetings, focusing on several properties: a) the importance of non-linguistic behaviors, b) the relative time-consistency of the social roles played by a given person during the course of a meeting, and c) the interplays and mutual constraints among the roles enacted by the different participants in a social encounter. In particular, this paper proposes that the Influence Model framework can address these properties of functional roles, and compares the performance obtained by this framework to the performances of models that consider only property (a) (SVM), and to those that address both (a) and (b) (HMM). The results obtained confirm our expectations: the classification of social functional roles improves if models account for temporal dependencies among the roles played by the same subject, for the time properties of the roles played by each individual, and for the mutual constraints among the roles of different group members. The two versions of the Influence Model (IM and newIM), which encode all three properties together, outperform both the SVM and the HMM on most of the figures of merit used. Of particular interest is the capability of the Influence Model to obtain good or very good results on the less-populated classes-Orienteer and Seeker for the task area, and Attacker and Supporter for the socio-emotional area.

This paper demonstrates a novel approach that combines generative models of organizational dynamics and sensor network data with a stochastic method. Generative models specify how organizational performance is related to who interacts with whom and who performs what. Sensor network data track who interacts with whom and who performs what within an organization, and the stochastic methodology fits multi-agent models to data through the Monte Carlo method. The data set used in this paper documents how employees in a data service center handle tasks with different difficulty levels - tracked with sociometric badges for one month - and documents links between performance and behavior. This paper demonstrates the potential for improving organizational dynamics with body sensor network data, and therefore also shows the need to systematically benchmark differential organizational dynamics models on data sets for different types of organizations.

The co-evolution of social relationships and individual behavior in time and space has important implications, but is poorly understood because of the difficulty closely tracking the everyday life of a complete community. We offer evidence that relationships and behavior co-evolve in a student dormitory, based on monthly surveys and location tracking through resident cellular phones over a period of nine months. We demonstrate that a Markov jump process could capture the co-evolution in terms of the rates at which residents visit places and friends.

Creativity is an important ingredient in problem solving, and problem solving is an important activity for both individuals and societies. This paper discusses our novel approach of discovering the structure of problem-solving creativity with statistical methods, and mapping the interaction patterns of group processes to their performances through the discovered creativity structure. Our discussion is based on a lab study data set using the meeting mediator system through which we collected objective quantitative data. We hope our findings and quantitative approach could be applied to many other real-world problem-solving processes and to helping people.

In this paper, an influence model is used to recognize functional roles played during meetings. Previous works on the same corpus demonstrated a high recognition accuracy using SVMs with RBF kernels. In this paper, we discuss the problems of that approach, mainly over-fitting, the curse of dimensionality and the inability to generalize to different group configurations. We present results obtained with an influence modeling method that avoid these problems and ensures both greater robustness and generalization capability.

This paper outlines the design of a distributed sensor classification system with abnormality detection intended for groups of people who are participating in coordinated activities. The system comprises an implementation of a distributed Dynamic Bayesian Network (DBN) model called the Influence Model (IM) that relies heavily on an inter-process communication architecture called Enchantment to establish the pathways of information that the model requires. We use three examples to illustrate how the "need to know" system effectively recognizes the group structure by simulating the work of cooperating individuals.

Widespread adoption of interactive, peer-to-peer digital media will require a solution to the Privacy, Sharing, and Interest (PSI) problem: how can we know what the user wants to share with whom, and when, without burdening the user with constant updating of lists of approved users and sharing preferences? We argue that real-time analysis of user behavior provides an automatic PSI capability, allowing media to be automatically and proactively shared with a much lower user burden.