As robots increasingly permeate diverse domains like healthcare, education, service industries and homes, accurately understanding humans’ responses to and behaviour towards robots is crucial. While many human-robot interaction (HRI) studies focus on either quantitative or qualitative approaches, we advocate a mixed-method approach. This study investigated robot norm violations by implementing a scenario where a mobile manipulator robot and a human, in-person, carry out a physical, competitive task. Sixty-two participants were recruited and randomly assigned to either an experimental or a control condition (balanced for age/gender). The scenario was a competitive scavenger hunt game where participants took turns with a robot. We investigated the robot behaviours’ effects on trust, discomfort, competence, enjoyment, participant behaviour and physiological changes. The mixed-method approach integrated physiological measurements, behavioural observations and qualitative responses, thus offering a comprehensive account of HRI dynamics in the context of norm violations. Questionnaire results reveal significant shifts in human perceptions and attitudes when social norms are violated by robots, compared to a norm-compliant control condition. Specifically, trust and enjoyment decrease, discomfort increases and the robot’s perceived competence is compromised. These findings are extended through additional analyses of participants’ physiological changes, behaviours and responses to open-ended questions. Behavioural observations indicated increased verbal engagement and emotional responses, while physiological data showed elevated stress levels in the experimental group. Our study highlights the advantage of a mixed-methods approach combining different qualitative and quantitative data, providing a more comprehensive picture of participants’ perceptions of a robot, and how they react and respond to robot norm violations.

As robots integrate more into daily life, socially aware robots with specific social attributes and behaviors are necessary. This review aims to explore how social norms in Human–Robot Interaction (HRI) impact robot design and human perception. We searched for relevant articles in the following databases: ACM Digital Library, IEEE Digital Library, Scopus, Springer Link, and PsycINFO. After applying inclusion and exclusion criteria, a final set of 69 articles were included in the review. These articles were categorized based on whether they examined norm conformity or norm violations, and were further sorted into 12 categorical norm labels to assist in analysis and comparison. By examining the existing literature, this review uncovers how social norms impact aspects of HRIs like trust, acceptance, and comfort while highlighting the importance of aligning robot design with user expectations. It reveals design challenges such as accounting for cultural variations, context-specific norms, and evolving norms over time. Addressing these challenges has the potential to improve user experiences, promote broader acceptance of robots, and foster successful integration of robots into various domains. The findings contribute to the ongoing discussion on the role of social norms in HRI, offering valuable insights and a foundation for future research.

General Game Playing (GGP) is a field of study in which artificial agents are required to compete in games whose rules are not known until runtime. In this domain, time is at a premium, as there may be less than a minute for an agent to initialize and decide on each of its actions. As a result, Monte Carlo Tree Search (MCTS) has been favoured by researchers in this domain for its ability to run in whatever time it is given, by quickly simulating many instances of a game. Heuristics may be used to guide these simulations, but since the game is not known in advance, a typical MCTS agent cannot know which heuristics are likely to be useful, and must expend precious time in trying to discover them. However, an agent able to take advantage of knowledge gained from prior experience with other, different, games, can do better. In this paper, we present a technique for automatically transferring heuristic knowledge between distinct, but similar, games. We show that using this knowledge to improve the quality of game-independent heuristics can produce better performance in games within the GGP framework, especially when initialization time is short, and we show that negative transfer is possible to detect and avoid.

This study aimed to investigate the effects of norm-violating behaviour on human perception and attitudes towards robots. Specifically, we examined the impact of a robot performing social norm violations in the context of a competitive scavenger hunt game. During the game, the robot was programmed to engage in predefined behaviours considered as social norm violations, including both injunctive and descriptive norm violations (e.g., cheating, and making loud noises). The study used an experimental and control group, with participants either exposed to norm-violating behaviour or not, respectively. The results indicated that participants in the experimental group had a strong awareness of the norm-violating behaviour according to self-reported assessments. Additionally, post-questionnaire results revealed a significant difference in trust, overall enjoyment, and discomfort between the two groups. These findings show that in our study, participants expected robots to abide by both types of social norms (i.e., injunctive and descriptive) and that violations of them negatively impacted participants’ perceptions and attitudes towards robots. This further emphasizes the importance of considering social norms in the design and programming of robots for human-robot interactions.

In this article, we analyze the performance of an agent developed according to a well-accepted appraisal theory of human emotion with respect to how it modulates play in the context of a social dilemma. We ask if the agent will be capable of generating interactions that are considered to be more human-like than machine-like. We conducted an experiment with 117 participants and show how participants rated our agent on dimensions of human-uniqueness (separating humans from animals) and human-nature (separating humans from machines). We show that our appraisal theoretic agent is perceived to be more human-like than the baseline models, by significantly improving both human-nature and human-uniqueness aspects of the intelligent agent. We also show that perception of humanness positively affects enjoyment and cooperation in the social dilemma, and discuss consequences for the task duration recall.

This paper presents the design of a cooperative multi-player betting game, Trust-ya, as a model of some elements of status processes in human groups. The game is designed to elicit status-driven leader-follower behaviours as a means to observe and influence social hierarchy. It involves a “Bach/Stravinsky” game of deference in a group, in which people on each turn can either invest with another player or hope someone invests with them. Players who receive investment capital are able to gamble for payoffs from a central pool which then can be shared back with those who invested (but a portion of it may be kept, including all of it). The bigger gambles (people with more investors) get bigger payoffs. Thus, there is a natural tendency for players to coalesce as investors around a “leader” who gambles, but who also shares sufficiently from their winnings to keep the investors “hanging on.” The “leader” will want to keep as much as possible for themselves, however. The game is played anonymously, but a set of “status symbols” can be purchased which have no value in the game itself, but can serve as a “cheap talk” communication device with other players. This paper introduces the game, relates it to status theory in social psychology, and shows some simple simulated and human experiments that demonstrate how the game can be used to study status processes and dynamics in human groups.

In the field of General Game Playing (GGP), artificial agents (bots) may be required to play never-before-seen games with less than one minute to initialize and train. Although tabula rasa approaches, like Monte-Carlo Tree Search, are popular in this domain, they do not leverage information from the many different games that a bot has previously encountered. A major barrier to transfer learning has been the difficulty in identifying similar features in the rule descriptions of two different games. We present two methods, called MMap and LMap, for heuristically approximating a distance between two games' graphs, and producing a mapping for the symbols of one to the other, thereby enabling transfer. We evaluate the effectiveness of these methods across a variety of transfer scenarios, and find that both methods are far more accurate than a simpler baseline mapper. MMap is found to be more robust than LMap, but LMap is much faster, and so more suitable for general use in GGP.

Conversational Agents (CA) have become one of the common user interfaces in many online domains. In this paper, we ask whether users have a preference about the personality of CAs, and whether this preference changes depending on the length and type of the tasks CAs are used for. In an online study (N = 410), we investigated three different CA personalities (introvert, extrovert, and non-personified) in four different tasks with different natures and lengths (teaching, booking, todo, and weather). Most of the participants preferred to interact with a conversational agent (introvert or extrovert) as opposed to a non-personified interface, regardless of their own personality. Results suggested that this preference may be task dependent: when CA’s goal was to provide information, participants preferred an extrovert agent. We did not observe a difference between the preference for introvert and extrovert agents when the task’s goal was to complete an assignment.

Intelligent assistive robots can enhance the quality of life of people with dementia and their caregivers. They can increase the independence of older adults, reduce tensions between a person with dementia and their caregiver, and increase social engagement. This article provides a review of assistive robots designed for and evaluated by persons with dementia. Assistive robots that only increased mobility or brain-computer interfaces were excluded. Google Scholar, IEEE Digital Library, PubMed, and ACM Digital Library were searched. A final set of 53 articles covering research in 16 different countries are reviewed. Assistive robots are categorized into five different applications and evaluated for their effectiveness, as well as the robots’ social and emotional capabilities. Our findings show that robots used in the context of therapy or for increasing engagement received the most attention in the literature, whereas the robots that assist by providing health guidance or help with an activity of daily living received relatively limited attention. PARO was the most commonly used robot in dementia care studies. The effectiveness of each assistive robot and the outcome of the studies are discussed, and particularly, the social/emotional capabilities of each assistive robot are summarized. Gaps in the research literature are identified and we provide directions for future work.

In this paper, we examine the influence of personality traits of developers on the pull request evaluation process in GitHub. We first replicate Tsayet al.’s work that examined the influence of social factors (e.g., ‘social distance’) and technical factors (e.g., test file inclusion) for evaluating contributions, and then extend it with personality based factors. In particular, we extract the Big Five personality traits (Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism) of developers from their online digital footprints, such as pull request comments. We analyze the personality traits of 16,935 active developers from 1,860 projects and compare their relative importance to other non-personality factors from past research, in the pull request evaluation process. We find that pull requests from authors (requesters) who are more open and conscientious, but less extroverted, have a higher chance of approval. Furthermore, pull requests that are closed by developers (closers) who are more conscientious, extroverted, and neurotic, have a higher likelihood of acceptance. The larger the difference in personality traits between the requester and the closer, the more positive effect it has on pull request acceptance. Finally, although the effect of personality traits is significant and comparable to technical factors, we find that social factors are still more influential on the likelihood of pull request acceptance.

For conversational AI and virtual assistants to communicate with humans in a realistic way, they must exhibit human characteristics such as expression of emotion and personality. Current attempts toward constructing human-like dialogue agents have presented significant difficulties. We propose Human Level Attributes (HLAs) based on tropes as the basis of a method for learning dialogue agents that can imitate the personalities of fictional characters. Tropes are characteristics of fictional personalities that are observed recurrently and determined by viewers' impressions. By combining detailed HLA data with dialogue data for specific characters, we present a dataset, HLA-Chat, that models character profiles and gives dialogue agents the ability to learn characters' language styles through their HLAs. We then introduce a three-component system, ALOHA (which stands for Artificial Learning of Human Attributes), that combines character space mapping, character community detection, and language style retrieval to build a character (or personality) specific language model. Our preliminary experiments demonstrate that two variations of ALOHA, combined with our proposed dataset, can outperform baseline models at identifying the correct dialogue responses of chosen target characters, and are stable regardless of the character's identity, the genre of the show, and the context of the dialogue.

Steven Y. Feng, Aaron W. Li, Jesse Hoey. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.

Cooperation and competition are a fundamental part of human interaction. In each situation, different people will cooperate or compete with one another in different ways. In this paper, we study the relationship between how people feel about the person they are interacting with (the affective identity of that person) and their level of cooperation in different circumstances. Standard game-theoretic models provide solutions to what self-interested rational agents would do in various situations. However, humans don't respond rationally in many situations, and the decision to cooperate can be strongly influenced by the identities of the interactants. In this study, over 1,000 participants answered a survey about whether they would cooperate in various framings of the Prisoners Dilemma (PD). These framings are based on the shared cultural sentiments in a three dimensional emotion space (Evaluation, Potency and Activity or EPA) about identities measured in existing large scale surveys. We combined 27 such identities with a set of five different payoff matrices, and provide statistical correlates between the sentiments about identities and likelihood of cooperation. We show that the evaluative (E) dimension is a strong predictor of cooperation, and we discuss the other factors including mixing terms. Our results provide a novel alternative view of cooperation in PD as arising simply from culturally shared sentiments about identities, rather than from payoff estimates.

Affect-Sensitive Human-Computer Interaction is enjoying growing attention. Emotions are an essential part of interaction, whether it is between humans or human and machine. This paper analyses the interaction of a user with four different virtual avatars, each manifesting distinct emotional displays, based on the principles of Affect Control Theory. Facial expressions are represented as a vector in a 3D continuous space and different sets of static visual features are evaluated for facial expression recognition. A probabilistic framework is used to simulate the interaction between the user and the virtual avatar. The results demonstrate that the probabilistic framework enables the system to perceive user's and agent's feelings.

Research in automatic affect recognition has come a long way. This paper describes the fifth Emotion Recognition in the Wild (EmotiW) challenge 2017. EmotiW aims at providing a common benchmarking platform for researchers working on different aspects of affective computing. This year there are two sub-challenges: a) Audio-video emotion recognition and b) group-level emotion recognition. These challenges are based on the acted facial expressions in the wild and group affect databases, respectively. The particular focus of the challenge is to evaluate method in `in the wild' settings. `In the wild' here is used to describe the various environments represented in the images and videos, which represent real-world (not lab like) scenarios. The baseline, data, protocol of the two challenges and the challenge participation are discussed in detail in this paper.

This paper discusses the baseline for the Emotion Recognition in the Wild (EmotiW) 2016 challenge. Continuing on the theme of automatic affect recognition `in the wild', the EmotiW challenge 2016 consists of two sub-challenges: an audio-video based emotion and a new group-based emotion recognition sub-challenges. The audio-video based sub-challenge is based on the Acted Facial Expressions in the Wild (AFEW) database. The group-based emotion recognition sub-challenge is based on the Happy People Images (HAPPEI) database. We describe the data, baseline method, challenge protocols and the challenge results. A total of 22 and 7 teams participated in the audio-video based emotion and group-based emotion sub-challenges, respectively.

Notions of identity and of the self have long been studied in social psychology and sociology as key guiding elements of social interaction and coordination. In the AI of the future, these notions will also play a role in producing natural, socially appropriate artificially intelligent agents that encompass subtle and complex human social and affective skills. We propose here a Bayesian generalization of the sociological affect control theory of self as a theoretical foundation for socio-affectively skilled artificial agents. This theory posits that each human maintains an internal model of his or her deep sense of "self" that captures their emotional, psychological, and socio-cultural sense of being in the world. The "self" is then externalised as an identity within any given interpersonal and institutional situation, and this situational identity is the person's local (in space and time) representation of the self. Situational identities govern the actions of humans according to affect control theory. Humans will seek situations that allow them to enact identities consistent with their sense of self. This consistency is cumulative over time: if some parts of a person's self are not actualized regularly, the person will have a growing feeling of inauthenticity that they will seek to resolve. In our present generalisation, the self is represented as a probability distribution, allowing it to be multi-modal (a person can maintain multiple different identities), uncertain (a person can be unsure about who they really are), and learnable (agents can learn the identities and selves of other agents). We show how the Bayesian affect control theory of self can underpin artificial agents that are socially intelligent.

This paper proposes a novel approach to sentiment analysis that leverages work in sociology on symbolic interactionism.The proposed approach uses Affect Control Theory (ACT) to analyze readers' sentiment towards factual (objective) content and towards its entities (subject and object).ACT is a theory of affective reasoning that uses empirically derived equations to predict the sentiments and emotions that arise from events.This theory relies on several large lexicons of words with affective ratings in a three-dimensional space of evaluation, potency, and activity (EPA).The equations and lexicons of ACT were evaluated on a newly collected news-headlines corpus.ACT lexicon was expanded using a label propagation algorithm, resulting in 86,604 new words.The predicted emotions for each news headline was then computed using the augmented lexicon and ACT equations.The results had a precision of 82%, 79%, and 68% towards the event, the subject, and object, respectively.These results are significantly higher than those of standard sentiment analysis techniques.

Recent advances in planning techniques for partially observable Markov decision processes (POMDPs) have focused on online search techniques and offline point-based value iteration. While these techniques allow practitioners to obtain policies for fairly large problems, they assume that a nonnegligible amount of computation can be done between each decision point. In contrast, the recent proliferation of mobile and embedded devices has lead to a surge of applications that could benefit from state-of-the-art planning techniques if they can operate under severe constraints on computational resources. To that effect, we describe two techniques to compile policies into controllers that can be executed by a mere table lookup at each decision point. The first approach compiles policies induced by a set of alpha vectors (such as those obtained by point-based techniques) into approximately equivalent controllers, while the second approach performs a simulation to compile arbitrary policies into approximately equivalent controllers. We also describe an approach to compress controllers by removing redundant and dominated nodes, often yielding smaller and yet better controllers. Further compression and higher value can sometimes be obtained by considering stochastic controllers. The compilation and compression techniques are demonstrated on benchmark problems as well as a mobile application to help persons with Alzheimer's to way-find. The battery consumption of several POMDP policies is compared against finite-state controllers learned using methods introduced in this paper. Experiments performed on the Nexus 4 phone show that finite-state controllers are the least battery consuming POMDP policies.

Affect Control Theory is a mathematical representation of the interactions between two persons, in which it is posited that people behave in a way so as to minimize the amount of deflection between their cultural emotional sentiments and the transient emotional sentiments that are created by each situation. Affect Control Theory presents a maximum likelihood solution in which optimal behaviours or identities can be predicted based on past interactions. Here, we formulate a probabilistic and decision theoretic model of the same underlying principles, and show this to be a generalisation of the basic theory. The model is more expressive than the original theory, as it can maintain multiple hypotheses about behaviours and identities simultaneously as a probability distribution. This allows the model to generate affectively believable interactions with people by learning about their identity and predicting their behaviours. We demonstrate this generalisation with a set of simulations. We then show how our model can be used as an emotional "plug-in" for systems that interact with humans. We demonstrate human-interactive capability by building a simple intelligent tutoring application and pilot-testing it in an experiment with 20 participants.

Prioritisation of Bellman backups or updating only a small subset of actions represent important techniques for speeding up planning in MDPs. The recent literature showed new efficient approaches which exploit these directions. Backward value iteration and backing up only the best actions were shown to lead to a significant reduction of the planning time. This paper conducts a theoretical and empirical analysis of these techniques and shows new important proofs. In particular, (1) it identifies weaker requirements for the convergence of backups based on best actions only, (2) a new method for evaluation of the Bellman error is shown for the update that updates one best action once, (3) it presents the theoretical proof of backward value iteration and establishes required initialisation, (4) and shows that the default state ordering of backups in standard value iteration can significantly influence its performance. Additionally, (5) the existing literature did not compare these methods, either empirically or analytically, against policy iteration. The rigorous empirical and novel theoretical parts of the paper reveal important associations and allow drawing guidelines on which type of value or policy iteration is suitable for a given domain. Finally, our chief message is that standard value iteration can be made far more efficient by simple modifications shown in the paper.

Optical motion tracking has enhanced human movement analysis in medicine, biomechanics, and rehabilitation science by providing highly accurate joint angle measurements over time. However, analyzing the large amount of recorded data is challenging. The process is usually simplified by calculating descriptive measures, such as the minimum, mean, or maximum, from the time series data. We propose a novel technique for the analysis of human motion data, which considers the complete time series data and is based on the F-statistic traditionally used in medical and biomechanical studies. The time series data is modeled by a Hidden Markov Model (HMM) and the F-statistic is reformulated using the Kullback-Leibler divergence for comparing HMMs. This provides a novel technique to enhance the analysis of human movement data and includes an automatic generation of group-specific trajectories to simplify visual data analysis. It is further suitable as time-series based, univariate feature selection technique in machine learning.

Body movements communicate affective expressions and, in recent years, computational models have been developed to recognize affective expressions from body movements or to generate movements for virtual agents or robots which convey affective expressions. This survey summarizes the state of the art on automatic recognition and generation of such movements. For both automatic recognition and generation, important aspects such as the movements analyzed, the affective state representation used, and the use of notation systems is discussed. The survey concludes with an outline of open problems and directions for future work.

Multiagent Resource Allocation (MARA) distributes a set of resources among a set of intelligent agents in order to respect the preferences of the agents and to maximize some measure of global utility, which may include minimizing total costs or maximizing total return. We are interested in MARA solutions that provide optimal or close-to-optimal allocation of resources in terms of maximizing a global welfare function with low communication and computation cost, with respect to the priority of agents, and temporal dependencies between resources. We propose an MDP approach for resource planning in multiagent environments. Our approach formulates internal preference modeling and success of each individual agent as a single MDP and then to optimize global utility, we apply a market-based solution to coordinate these decentralized MDPs.

Most of the systems for recognition of activities aim to identify a set of normal human activities. Data is either recorded by computer vision or sensor based networks. These systems may not work properly if an unusual event or abnormal activity occurs, especially ones that have not been encountered in the past. By definition, unusual events are mostly rare and unexpected, and therefore very little or no data may be available for training. In this paper, we focus on the challenging problem of detecting unusual temporal events in a sensor network and present three Hidden Markov Models (HMM) based approaches to tackle this problem. The first approach models each normal activity separately as an HMM and the second approach models all the normal activities together as one common HMM. If the likelihood is lower than a threshold, an unusual event is identified. The third approach models all normal activities together in one HMM and approximates an HMM for the the unusual events. All the methods train HMM models on data of the usual events and do not require training data from the unusual events. We perform our experiments on a Locomotion Analysis dataset that contains gyroscope, force sensor, and accelerometer readings. To test the performance of our approaches, we generate five types of unusual events that represent random activity, extremely unusual events, unusual events similar to specific normal activities, no or little motion and normal activity followed by no or little motion. Our experiments suggest that for a moderately sized time frame window, these approaches can identify all the five types of unusual events with high confidence.

The ratio of healthcare professionals to care recipients is dropping at an alarming rate, particularly for the older population. It is estimated that the number of persons with Alzheimer's disease, for example, will top 100 million worldwide by the year 2050 [Alzheimer's Disease International 2009]. It will become harder and harder to provide needed health services to this population of older adults. Further, patients are becoming more aware and involved in their own healthcare decisions. This is creating a void in which technology has an increasingly important role to play as a tool to connect providers with recipients. Examples of interactive technologies range from telecare for remote regions to computer games promoting fitness in the home. Currently, such technologies are developed for specific applications and are difficult to modify to suit individual user needs. The future potential economic and social impact of technology in the healthcare field therefore lies in our ability to make intelligent devices that are customizable by healthcare professionals and their clients, that are adaptive to users over time, and that generalize across tasks and environments. A wide application area for technology in healthcare is for assistance and monitoring in the home. As the population ages, it becomes increasingly dependent on chronic healthcare, such as assistance for tasks of everyday life (washing, cooking, dressing), medication taking, nutrition, and fitness. This article will present a summary of work over the past decade on the development of intelligent systems that provide assistance to persons with cognitive disabilities. These systems are unique in that they are all built using a common framework, a decision-theoretic model for general-purpose assistance in the home. In this article, we will show how this type of general model can be applied to a range of assistance tasks, including prompting for activities of daily living, assistance for art therapists, and stroke rehabilitation. This model is a Partially Observable Markov Decision Process (POMDP) that can be customized by end-users, that can integrate complex sensor information, and that can adapt over time. These three characteristics of the POMDP model will allow for increasing uptake and long-term efficiency and robustness of technology for assistance.

Research on sensor-based activity recognition has, recently, made significant progress and is attracting growing attention in a number of disciplines and application domains. However, there is a lack of high-level overview on this topic that can inform related communities of the research state of the art. In this paper, we present a comprehensive survey to examine the development and current status of various aspects of sensor-based activity recognition. We first discuss the general rationale and distinctions of vision-based and sensor-based activity recognition. Then, we review the major approaches and methods associated with sensor-based activity monitoring, modeling, and recognition from which strengths and weaknesses of those approaches are highlighted. We make a primary distinction in this paper between data-driven and knowledge-driven approaches, and use this distinction to structure our survey. We also discuss some promising directions for future research.

In this paper we consider a (possibly continuous) space of Bernoulli experiments. We assume that the Bernoulli distributions are correlated. All evidence data comes in the form of successful or failed experiments at different points. Current state-ofthe-art methods for expressing a distribution over a continuum of Bernoulli distributions use logistic Gaussian processes or Gaussian copula processes. However, both of these require computationally expensive matrix operations (cubic in the general case). We introduce a more intuitive approach, directly correlating beta distributions by sharing evidence between them according to a kernel function, an approach which has linear time complexity. The approach can easily be extended to multiple outcomes, giving a continuous correlated Dirichlet process, and can be used for both classification and learning the actual probabilities of the Bernoulli distributions. We show results for a number of data sets, as well as a case-study where a mixture of continuous beta processes is used as part of an automated stroke rehabilitation system. 1

This paper describes the development of a tool that assists arts therapists working with older adults with dementia. Participation in creative activities is becoming accepted as a method for improving quality of life. This paper presents the design of a novel tool to increase the capacity of creative arts therapists to engage cognitively impaired older adults in creative activities. The tool is a creative arts touch-screen interface that presents a user with activities such as painting, drawing, or collage. It was developed with a user-centered design methodology in collaboration with a group of creative arts therapists. The tool is customizable by therapists, allowing them to design and build personalized therapeutic/goal-oriented creative activities for each client. In this paper, we evaluate the acceptability of the tool by arts therapists (our primary user group). We perform this evaluation qualitatively with a set of one-on-one interviews with arts therapists who work specifically with persons with dementia. We show how their responses during interviews support the idea of a customizable assistance tool. We evaluate the tool in simulation by showing a number of examples, and by demonstrating customizable components.

This paper presents a method for learning decision theoretic models of human behaviors from video data. Our system learns relationships between the movements of a person, the context in which they are acting, and a utility function. This learning makes explicit that the meaning of a behavior to an observer is contained in its relationship to actions and outcomes. An agent wishing to capitalize on these relationships must learn to distinguish the behaviors according to how they help the agent to maximize utility. The model we use is a partially observable Markov decision process, or POMDP. The video observations are integrated into the POMDP using a dynamic Bayesian network that creates spatial and temporal abstractions amenable to decision making at the high level. The parameters of the model are learned from training data using an a posteriori constrained optimization technique based on the expectation-maximization algorithm. The system automatically discovers classes of behaviors and determines which are important for choosing actions that optimize over the utility of possible outcomes. This type of learning obviates the need for labeled data from expert knowledge about which behaviors are significant and removes bias about what behaviors may be useful to recognize in a particular situation. We show results in three interactions: a single player imitation game, a gestural robotic control problem, and a card game played by two people.

This paper describes a method for tracking in the presence of distractors, changes in shape, and occlusions. An object is modeled as a flock of features describing its approximate shape. The flock’s dynamics keep it spatially localised and moving in concert, but also well distributed across the object being tracked. A recursive Bayesian estimation of the density of the object is approximated with a set of samples. The method is demonstrated on two simple examples, and is applied to an assistive system that tracks the hands and the towel during a handwashing task. 1

Reinforcement learning (RL) was originally proposed as a framework to allow agents to learn in an online fashion as they interact with their environment. Existing RL algorithms come short of achieving this goal because the amount of exploration required is often too costly and/or too time consuming for online learning. As a result, RL is mostly used for offline learning in simulated environments. We propose a new algorithm, called BEETLE, for effective online learning that is computationally efficient while minimizing the amount of exploration. We take a Bayesian model-based approach, framing RL as a partially observable Markov decision process. Our two main contributions are the analytical derivation that the optimal value function is the upper envelope of a set of multivariate polynomials, and an efficient point-based value iteration algorithm that exploits this simple parameterization.

Older adults with dementia often cannot remember how to complete activities of daily living and require a caregiver to aid them through the steps involved. The use of a computerized guidance system could potentially reduce the reliance on a caregiver. This paper examines the design and preliminary evaluation of a planning system that uses Markov decision processes (MDPs) to determine when and how to provide prompts to a user with dementia for guidance through the activity of handwashing. Results from the study suggest that MDPs can be applied effectively to this type of guidance problem. Considerations for the development of future guidance systems are presented.

We present a method for unsupervised learning of classes of motions in video. We project optical flow fields to a complete, orthogonal, a-priori set of basis functions in a probabilistic fashion, which improves the estimation of the projections by incorporating uncertainties in the flows. We then cluster the projections using a mixture of feature-weighted Gaussians over optical flow fields. The resulting model extracts a concise probabilistic description of the major classes of optical flow present. The method is demonstrated on a video of a person's facial expressions.

Jose is a visually guided autonomous robotic waiter. He circulates around a room populated by groups of people, politely serving appetizers to humans. The serving task combines elements of robotics with human computer interaction, challenging control architecture with multiple task integration. This paper describes our purely vision-based approach to this task. Methods for mapping, localization and navigation are presented and discussed, including issues of safety for both robots and humans. Our work on human-robot interaction is covered, as well as our solutions to various tasks specific to serving food. We present results of our methods from sample experiments in our laboratory. We further discuss our experiences at the 2001 AAAI mobile robot "Hors D'oeuvres Anyone?" competition, at which Jose took first prize.

The quest for a vision system capable of representing and recognizing arbitrary motions benefits from a low dimensional, non-specific representation of flow fields, to be used in high level classification tasks. We present Zernike polynomials as an ideal candidate for such a representation. The basis of Zernike polynomials is complete and orthogonal and can be used for describing many types of motion at many scales. Starting from image sequences, locally smooth image velocities are derived using a robust estimation procedure, from which are computed compact representations of the flow using the Zernike basis. Continuous density hidden Markov models are trained using the temporal sequences of vectors thus obtained, and are used for subsequent classification. We present results of our method applied to image sequences of facial expressions both with and without significant rigid head motion and to sequences of lip motion from a known database. We demonstrate that the Zernike representation yields results competitive with those obtained using principal components, while not committing to specific types of motion. It is therefore ideal as a fundamental building block for a vision system capable of classifying arbitrary motion types.

We propose a method of approximate dynamic programming for Markov decision processes (MDPs) using algebraic decision diagrams (ADDs). We produce near-optimal value functions and policies with much lower time and space requirements than exact dynamic programming. Our method reduces the sizes of the intermediate value functions generated during value iteration by replacing the values at the terminals of the ADD with ranges of values. Our method is demonstrated on a class of large MDPs (with up to 34 billion states), and we compare the results with the optimal value functions.