Professional service robots are increasingly being deployed in public places, which thus increases user exposure. However, we lack an empirical understanding of complex encounters taking place in dynamic and often crowded environments as well as how people overcome breakdowns during unguided interaction with a robot in a real-world scenario. In this paper, we conducted a covert, digital ethnographic study analyzing 104 user-generated YouTube videos focusing on people’s unguided interactions with robots in several public places. We identified several types of interaction breakdowns pertaining to someone (person-initiated interaction breakdown, IB) or something (environmental disturbances, ED) having a direct, negative effect on an ongoing unguided interaction. Our findings have implications for the design and development of service robots facing multi-user scenarios entertaining active (primary and secondary) users, inactive (commentators and observers) ‘users’, and Incidentally Co-present Persons (InCoPs) . Furthermore, we contribute to and built on the limited prior use of YouTube videos and digital ethnography in HRI research, thereby demonstrating its effectiveness in studying unguided interactions in public places, while supplementing and adding to the existing knowledge base of service robots in public places.

Cross-modality face recognition is an emerging topic due to the wide-spread usage of different sensors in day-to-day life applications. The development of face recognition systems relies greatly on existing databases for evaluation and obtaining training examples for data-hungry machine learning algorithms. However, currently, there is no publicly available face database that includes more than two modalities for the same subject. In this work, we introduce the Tufts Face Database that includes images acquired in various modalities: photograph images, thermal images, near infrared images, a recorded video, a computerized facial sketch, and 3D images of each volunteer's face. An Institutional Research Board protocol was obtained and images were collected from students, staff, faculty, and their family members at Tufts University. The database includes over 10,000 images from 113 individuals from more than 15 different countries, various gender identities, ages, and ethnic backgrounds. The contributions of this work are: 1) Detailed description of the content and acquisition procedure for images in the Tufts Face Database; 2) The Tufts Face Database is publicly available to researchers worldwide, which will allow assessment and creation of more robust, consistent, and adaptable recognition algorithms; 3) A comprehensive, up-to-date review on face recognition systems and face datasets.

The advancement of wearable eye-tracking technology enables cognitive researchers to capture vast amounts of eye gaze information while participants are completing specific tasks without restrictions on their movement. However, while eye trackers can overlay a gaze indicator on the scene video, identifying the specific objects being looked at and analyzing the resulting dataset are accomplished mostly by manual annotation. This method is a cost-prohibitive and time-consuming approach that is prone to human error. Such analytic difficulty limits researchers' ability to data mine the information efficiently, ultimately restricting the number of scenarios that can feasibly be conducted within budget. Here, the first fully automated solution for eye-tracking data analysis is presented, which eliminates the need for manual annotation. The proposed software architecture, gaze to object classification (GoC), processes the gaze-overlaid video from commercially available wearable eye trackers, recognizes and classifies the specific object a user is focusing on and calculates the gaze duration time. GoC utilizes an image cross-correlation method to locate the gaze indicator and an image similarity measurement to support faster processing. The presented system has been successfully adopted by cognitive psychologists. GoC's exceptional performance in analyzing a case study spanning over 50 h of mobile eye-tracking is presented. The accuracy and a cost-analysis comparison between GoC and state-of-the-art manual annotation software are provided. GoC has game-changing potential for increasing the ecological validity of using eye-tracking technology in cognitive research.