Effective human–robot collaboration (HRC) requires robots to understand and adapt to humans' psychological states. This research presents a novel approach to quantitatively measure human comfort levels during HRC through the development of two metrics: a comfortability index (CI) and an uncomfortability index (UnCI). We conducted HRC experiments where participants performed assembly tasks while the robot's behavior was systematically varied. Participants' subjective responses (includingsurprise,anxiety,boredom,calmness, andcomfortabilityratings) were collected alongside physiological signals, including electrocardiogram, galvanic skin response, and pupillometry data. We propose two novel approaches for estimating CI/UnCI: an adaptation of the emotion circumplex model that maps comfort levels to the arousal–valence space, and a kernel density estimation model trained on physiological data. Time-domain features were extracted from the physiological signals and used to train machine learning models for real-time comfort levels estimation. Our results demonstrate that the proposed approaches can effectively estimate human comfort levels from physiological signals alone, with the circumplex model showing particular promise in detecting high discomfort states. This work enables real-time measurement of human comfort during HRC, providing a foundation for developing more adaptive and human-aware collaborative robots.

We introduce three new multi-modal data sets. They contain physiological and/or emotional information about human interactions with robotic arms in proximity to completing a task in an industrial setting. The data sets provide data from human subjects engaged in the assistive task of assembling a PVC joint pipe with robots. These data streams were collected to analyze and improve the comfort and safety of humans collaborating with robots in proximity in an industrial setting. These data sets can appeal to researchers studying human-robot collaboration, robot adaptation, and affective computing. Our data is stored in various formats, including images and human-readable Comma-Separated Values (CSV) or JavaScript Object Notation (JSON) files.

We propose a novel framework to passively monitor pulse rate during the time spent by users on their personal mobile devices. Our framework is based on passively capturing the user's pulse signal using the front-facing camera. Signal capture is performed in the background, while the user is interacting with the device as he/she normally would, e.g., watch movies, read emails, text, and play games. The framework does not require subject participation with the monitoring procedure, thereby addressing the well-known problem of low adherence with such procedures. We investigate various techniques to suppress the impact of spontaneous user motion and fluctuations in ambient light conditions expected in non-participatory environments. Techniques include traditional signal processing, machine learning classifiers, and deep learning methods. Our performance evaluation is based on a clinical study encompassing 113 patients with a history of atrial fibrillation (Afib) who are passively monitored at home using a tablet for a period of two weeks. Our results show that the proposed framework accurately monitors pulse rate, thereby providing a gateway for long-term monitoring without relying on subject participation or the use of a dedicated wearable device.

Industrial robots working in isolation in a highly automated system are valued for their high productivity. The shortcomings of these pure robotic cells become more apparent when flexibility in production is required to respond to varying production volumes and customized product demands. Complete automation is highly productive, but it is costly to set up and difficult to change. On the other hand, manual production, although flexible, is slower and prone to human errors. Hence, in industry, smarter automation methods that leverage the dexterity, flexibility, and decision-making capability of a human to speed, precision, and power of a robot are required. In industry, the need for flexibility in production has resulted in the acceptance of human-robot collaboration (HRC) as a viable alternative. The objective of this survey is to address the main challenges in HRC (safety, trust-in-automation, and productivity), safety measures, types of HRC, technical standards, and conceptual categorization of HRC: awareness, intelligence, and compliance.

In this paper, a framework for monitoring human physiological response during Human-Robot Collaborative (HRC) task is presented. The framework highlights the importance of generation of event markers related to both human and robot, and also synchronization of data collected. This framework enables continuous data collection during an HRC task when changing robot movements as a form of stimuli to invoke a human physiological response. It also presents two case studies based on this framework and a data visualization tool for representation and easy analysis of the collected data during an HRC experiment.

In this paper, a range sensing setup for performing detection and monitoring in an industrial robot workspace is presented. The setup uses a ring of Time-of-Flight laser range sensors mounted on a robot. A 3D simulation setup with the properties of the sensor mounted on a UR10 robot and a simple pick and place task with a human avatar is used to analyze its behavior and viability with industrial arm robots. Collision detection strategies based on human-robot separation distance and relative speeds are also implemented. These strategies are evaluated based on the human safety, robot performance and productivity of the task. The parameters and results of the experiments are tabulated. The results show the benefits of achieving dynamic awareness of the robot in comparison with the conventional methods used in industry. The development of the prototype sensor ring is also shown and the future work discussed.

In this paper, a system to aid the visually impaired by providing contextual information of the surroundings using 360° view camera combined with deep learning is proposed. The system uses a 360° view camera with a mobile device to capture surrounding scene information and provide contextual information to the user in the form of audio. The scene information from the spherical camera feed is classified by identifying objects that contain contextual information of the scene. That is achieved using convolutional neural networks (CNN) for classification by leveraging CNN transfer learning properties using the pre-trained VGG-19 network. There are two challenges related to this paper, a classification and a segmentation challenge. As an initial prototype, we have experimented with general classes such restaurants, coffee shops and street signs. We have achieved a 92.8% classification accuracy in this paper.

Sign Language Recognition (SLR) system is a novel method that allows hard of hearing people to communicate with society. In this study, an American Sign Language (ASL) recognition system was proposed by using the surface Electromyography (sEMG). The objective of this study is to recognize the American Sign Language alphabet letters and allow users to spell words and sentences. For this purpose, sEMG signals are acquired from subject's right forearm for 27 American Sign Language gestures, 26 English alphabet letters, and one for home position. Time domain, frequency domain (band power), power spectral density (band power), and average power features were used as the feature extraction methods. After feature extraction, Principal Component Analysis (PCA) was applied to obtain uncorrelated features. As a classification method, Support Vector Machine and Ensemble Learning algorithm were used and their performances were compared with tabulated results. In conclusion, the results of this study show that sEMG signal can be used for SLR systems.

Sign Language Recognition (SLR) system is a method which allow deaf people to communicate with society. In this study, Real-Time Sign Language recognition system was proposed by using the surface Electromyography (sEMG). To this purpose, sEMG data acquired from subject right forearm for all twenty six American Sign Language gestures. Raw sEMG data was filtered, feature extracted and fed into classification. Support Vector Machine (SVM) with one vs. all approach was used for multi class classification. The experiment result of offline system is reaching a recognition rate of 91.% accuracy and real-time system has a recognition rate of 82.3% accuracy. The results of the proposed system shows that sEMG signal can be used for Real-Time SLR systems.