Trust between group members has many implications for how well a group performs. In this study, we predict perceived trustworthiness of group members when there are subversive group members. We collected multimodal verbal and nonverbal data from a group interaction experiment. During the interaction, we periodically surveyed the group members about their perceptions of trustworthiness of other group members. We used this data to model the relationship between observable behavior and perceptions of trustworthiness. We report the most predictive features and describe them in the context of existing literature on verbal and nonverbal correlates of trust. This research advances the study of behavioral measurement in groups and the role of behavior on perceived trustworthiness.

Most work on automated deception detection (ADD) in video has two restrictions: (i) it focuses on a video of one person, and (ii) it focuses on a single act of deception in a one or two minute video. In this paper, we propose a new ADD framework which captures long term deception in a group setting. We study deception in the well-known Resistance game (like Mafia and Werewolf) which consists of 5-8 players of whom 2-3 are spies. Spies are deceptive throughout the game (typically 30-65 minutes) to keep their identity hidden. We develop an ensemble predictive model to identify spies in Resistance videos. We show that features from low-level and high-level video analysis are insufficient, but when combined with a new class of features that we call LiarRank, produce the best results. We achieve AUCs of over 0.70 in a fully automated setting.

We consider the problems of predicting (i) the most dominant person in a group of people, and (ii) the more dominant of a pair of people, from videos depicting group interactions. We introduce a novel family of variables called Dominance Rank. We combine features not previously used for dominance prediction (e.g., facial action units, emotions), with a novel ensemble-based approach to solve these two problems. We test our models against four competing algorithms in the literature on two datasets and show that our results improve past performance. We show 2.4% to 16.7% improvement in AUC compared to baselines on one dataset, and a gain of 0.6% to 8.8% in accuracy on the other. Ablation testing shows that Dominance Rank features play a key role.

The INTERSPEECH 2016 Computational Paralinguistics Challenge addresses three different problems for the first time in research competition under well-defined conditions: classification of deceptive vs. non-deceptive speech, the estimation of the degree of sincerity, and the identification of the native language out of eleven L1 classes of English L2 speakers.In this paper, we describe these sub-challenges, their conditions, the baseline feature extraction and classifiers, and the resulting baselines, as provided to the participants.

Deception is pervasive, often leading to adverse consequences for individuals, organizations, and society. Information systems researchers are developing tools and evaluating sensors that can be used to augment human deception judgments. One sensor exhibiting particular promise is the eye tracker. Prior work evaluating eye trackers for deception detection has focused on the detection and interpretation of brief eye behavior variations in response to stimuli (e.g, images) or interview questions. However, research is needed to understand how eye behaviors evolve over the course of an interaction with a deception detection system. Using latent growth curve modeling, we test how pupil diameter evolves over one's interaction with a deception detection system. The results indicate that pupil diameter changes over the course of a deception detection interaction, and that these trends are indicative of deception during the interaction, regardless if incriminating target items are shown.

Detecting deception in interpersonal dialog is challenging since deceivers take advantage of the give-and-take of interaction to adapt to any sign of skepticism in an interlocutor's verbal and nonverbal feedback. Human detection accuracy is poor, often with no better than chance performance. In this investigation, we consider whether automated methods can produce better results and if emphasizing the possible disruption in interactional synchrony can signal whether an interactant is truthful or deceptive. We propose a data-driven and unobtrusive framework using visual cues that consists of face tracking, head movement detection, facial expression recognition, and interactional synchrony estimation. Analysis were conducted on 242 video samples from an experiment in which deceivers and truth-tellers interacted with professional interviewers either face-to-face or through computer mediation. Results revealed that the framework is able to automatically track head movements and expressions of both interlocutors to extract normalized meaningful synchrony features and to learn classification models for deception recognition. Further experiments show that these features reliably capture interactional synchrony and efficiently discriminate deception from truth.

Embodied conversational agents (ECA) are a type of intelligent, multimodal computer interface that allow computers to interact with humans in a face-to-face manner. It is quite feasible that ECAs will someday replace the common keyboard as a human–computer interface. However, we have much to understand about how people interact with such embodied virtual agents. In this study, we performed a laboratory experiment, in an airport screening context, to assess how people’s linguistic behavior changes with their perceptions of the ECA’s power and likeability. The results show that people tend to manifest more verbal immediacy and expressivity, as well as offer more information about themselves, with ECAs they perceive as more likeable and less powerful.

Automated human credibility screening is an emerging research area that has potential for high impact in fields as diverse as homeland security and accounting fraud detection. Systems that conduct interviews and make credibility judgments can provide objectivity, improved accuracy, and greater reliability to credibility assessment practices, need to be built. This study establishes a foundation for developing automated systems for human credibility screening.

Transportation and border security systems have a common goal: to allow law-abiding people to pass through security and detain those people who intend to harm. Understanding how intention is concealed and how it might be detected should help in attaining this goal. In this paper, we introduce a multidisciplinary theoretical model of intent concealment along with three verbal and nonverbal automated methods for detecting intent: message feature mining, speech act profiling, and kinesic analysis. This paper also reviews a program of empirical research supporting this model, including several previously published studies and the results of a proof-of-concept study. These studies support the model by showing that aspects of intent can be detected at a rate that is higher than chance. Finally, this paper discusses the implications of these findings in an airport-screening scenario.

Social engineering (as used by the military or law-enforcement) is the emerging technique for obtaining classified information by interacting and deceiving people who can access that information. Rather than using traditional techniques of attacking the technical shields such as firewalls, many sophisticated computer hackers find that social engineering is more effective and difficult to detect by humans. Why can people not effectively detect social engineering, or more specifically, the art of deception? What can be done to augment human abilities for the task? The current findings warrant several possibilities that influence human ability to detect deception. Factors include such things as truth-bias, stereotypical thinking and processing ability. Knowing that human detection ability is limited, we propose a method to automatically detect deception that potentially assists humans. Results show that a system, using discriminant analysis to classify deception performed significantty better than humans in detecting deception. The findings can also be applied to general situations to ensure information authentication scenarios other than social engineering.

Behavioral indicators of deception and behavioral state are extremely difficult for humans to analyze. This research effort attempts to leverage automated systems to augment humans in detecting deception by analyzing nonverbal behavior on video. By tracking faces and hands of an individual, it is anticipated that objective behavioral indicators of deception can be isolated, extracted and synthesized to create a more accurate means for detecting human deception. Blob analysis, a method for analyzing the movement of the head and hands based on the identification of skin color is presented. A proof-of-concept study is presented that uses Blob analysis to extract visual cues and events, throughout the examined videos. The integration of these cues is done using a hierarchical hidden Markov model to explore behavioral state identification in the detection of deception, mainly involving the detection of agitated and over-controlled behaviors

This paper describes a pilot study on the role of expectations in human-computer interaction on a decision-making task. Participants (N=70) were randomly assigned to one of 5 different computer partners or to a human partner. After completing the rankings for the Desert Survival Task, participants engaged in a dialog with their computer or human partners. Results revealed that interaction with human partners was more expected and more positively evaluated than interaction with computer agents. In addition, the addition of human-like qualities to computer interfaces did not increase expectedness or evaluations as predicted. Correlation analysis for the five computer conditions demonstrated that expectations and evaluations do effect influence and perceptions of the partner. Discussion focuses on ways to coordinate expectations, interface design, and task objectives.