This work evaluates a Continuous Wave (CW) Time-of-Flight (ToF) camera, Stereoscopic camera, and LiDAR to determine if they are potential candidates for point-rich on-robot sensing in Speed and separation monitoring (SSM) applications. These experiments characterize the static and dynamic behaviors of the sensors while mounted on-robot. From these tests, it was found that ToF and Stereo cameras exhibit better performance to their more expensive LiDAR counterpart. Specifically, it was observed that the ToF camera demonstrated better depth accuracy while the Stereo camera generated better 3D reconstruction accuracy. Overall, ToF and Stereo Cameras demonstrate that with continued innovation and integration, these sensors could become the building blocks to point rich on-robot SSM.

We introduce three new multi-modal data sets. They contain physiological and/or emotional information about human interactions with robotic arms in proximity to completing a task in an industrial setting. The data sets provide data from human subjects engaged in the assistive task of assembling a PVC joint pipe with robots. These data streams were collected to analyze and improve the comfort and safety of humans collaborating with robots in proximity in an industrial setting. These data sets can appeal to researchers studying human-robot collaboration, robot adaptation, and affective computing. Our data is stored in various formats, including images and human-readable Comma-Separated Values (CSV) or JavaScript Object Notation (JSON) files.

Many attempts have been made at estimating discrete emotions (calmness, anxiety, boredom, surprise, anger) and continuous emotional measures commonly used in psychology, namely ‘valence’ (The pleasantness of the emotion being displayed) and ‘arousal’ (The intensity of the emotion being displayed). Existing methods to estimate arousal and valence rely on learning from data sets, where an expert annotator labels every image frame. Access to an expert annotator is not always possible, and the annotation can also be tedious. Hence it is more practical to obtain self-reported arousal and valence values directly from the human in a real-time Human-Robot collaborative setting. Hence this paper provides an emotion data set (HRI-AVC) obtained while conducting a human-robot interaction (HRI) task. The self-reported pair of labels in this data set is associated with a set of image frames. This paper also proposes a spatial and temporal attention-based network to estimate arousal and valence from this set of image frames. The results show that an attention-based network can estimate valence and arousal on the HRI-AVC data set even when Arousal and Valence values are unavailable per frame.