The integration of human–robot teams is increasingly essential in dynamic task environments, particularly in sectors like warehouse management, assembly lines, search and rescue operations, material handling, and autonomous driving. This trend leverages the complementary strengths of humans and robots to enhance efficiency and tackle complex objectives. However, significant challenges arise due to differences in task execution, communication modes, empathy, mental model understanding, and adaptability between humans and robots. This survey article examines the complexities of human–robot collaboration (HRC), focusing on the “3Cs” of teamwork: collaboration, communication, and cognition. It introduces a novel 3Cs rating system to evaluate HRC systems, offering a comprehensive analysis of current research trends and identifying key challenges. The findings highlight a prevalent lack of robot adaptation based on human states and performance, underscoring the need for improved communication metrics and consistent definitions of collaborative frameworks. Key contributions include the development of the 3Cs rating system, an in-depth analysis of HRC research trends, and the identification of critical areas requiring further investigation to realize the full potential of human–robot teams. This article aims to guide future research and development, promoting more effective human–robot collaborations.

Alternative text (alt text) or image descriptions for scientific figures make them accessible to screen reader users.Yet, generating high-quality alt text remains a significant challenge, particularly for people lacking subject-matter expertise.Emerging AI tools, such as generative AI, can help people understand contexts where they lack subject expertise.This work explores how university Disability Services Office (DSO) professionals write alt text for scientific figures without subject expertise.We conducted a user study with 12 DSO professionals who authored alt text for scientific figures, first without and then using generative AI.We assessed the participants' processes for alt text generation and their confidence in the resulting output.We also conducted semi-structured interviews to understand DSO professionals' experiences with the alt text generation.Our findings reveal that AI assistance improved participants' confidence and efficiency, but introduced new challenges for interaction and trust.Participants expressed caution about relying on AI-generated descriptions without sufficient domain knowledge and editing.These insights highlight the need for AI-augmented tools that better scaffold the alt text writing process for accessibility professionals.

This work investigates Source-Free Domain Adaptation (SFDA), where a model adapts to a target domain without access to source data. A new augmentation technique, Shuffle PatchMix (SPM), and a novel reweighting strategy are introduced to enhance performance. SPM shuffles and blends image patches to generate diverse and challenging augmentations, while the reweighting strategy prioritizes reliable pseudo-labels to mitigate label noise. These techniques are particularly effective on smaller datasets like PACS, where overfitting and pseudo-label noise pose greater risks. State-of-the-art results are achieved on three major benchmarks: PACS, VisDA-C, and DomainNet-126. Notably, on PACS, improvements of 7.3% (79.4% to 86.7%) and 7.2% are observed in single-target and multi-target settings, respectively, while gains of 2.8% and 0.7% are attained on DomainNet-126 and VisDA-C. This combination of advanced augmentation and robust pseudo-label reweighting establishes a new benchmark for SFDA. The code is available at: https://github.com/PrasannaPulakurthi/SPM.

Effective human–robot collaboration (HRC) requires robots to understand and adapt to humans' psychological states. This research presents a novel approach to quantitatively measure human comfort levels during HRC through the development of two metrics: a comfortability index (CI) and an uncomfortability index (UnCI). We conducted HRC experiments where participants performed assembly tasks while the robot's behavior was systematically varied. Participants' subjective responses (includingsurprise,anxiety,boredom,calmness, andcomfortabilityratings) were collected alongside physiological signals, including electrocardiogram, galvanic skin response, and pupillometry data. We propose two novel approaches for estimating CI/UnCI: an adaptation of the emotion circumplex model that maps comfort levels to the arousal–valence space, and a kernel density estimation model trained on physiological data. Time-domain features were extracted from the physiological signals and used to train machine learning models for real-time comfort levels estimation. Our results demonstrate that the proposed approaches can effectively estimate human comfort levels from physiological signals alone, with the circumplex model showing particular promise in detecting high discomfort states. This work enables real-time measurement of human comfort during HRC, providing a foundation for developing more adaptive and human-aware collaborative robots.

Generative Adversarial Networks (GANs) have emerged as a powerful tool for generating high-fidelity content. This paper presents a new training procedure that leverages Neural Architecture Search (NAS) to discover the optimal architecture for image generation while employing the Maximum Mean Discrepancy (MMD) repulsive loss for adversarial training. Moreover, the generator network is compressed using tensor decomposition to reduce its computational footprint and inference time while preserving its generative performance. Experimental results show improvements of 34% and 28% in the FID score on the CIFAR-10 and STL-10 datasets, respectively, with corresponding footprint reductions of 14× and 31× compared to the best FID score method reported in the literature. The implementation code is available at: https://github.com/PrasannaPulakurthi/MMD-AdversarialNAS.

Advances in robot teleoperation have enabled groundbreaking innovations in many fields, such as space exploration, healthcare, and disaster relief. The human operator's performance plays a key role in the success of any teleoperation task, with prior evidence suggesting that operator stress and workload can impact task performance. As robot teleoperation is currently deployed in safety-critical domains, it is essential to analyze how different stress and workload levels impact the operator. We are unaware of any prior work investigating how both stress and workload impact teleoperation performance. We conducted a novel study ($n=24$) to jointly manipulate users' stress and workload and analyze the user's performance through objective and subjective measures. Our results indicate that, as stress increased, over 70% of our participants performed better up to a moderate level of stress; yet, the majority of participants performed worse as the workload increased. Importantly, our experimental design elucidated that stress and workload have related yet distinct impacts on task performance, with workload mediating the effects of distress on performance ($p< .05$).

Technologies in machine learning and artificial intelligence have come a long way in decision making and system automation, but still faces difficult challenges in semi-automation and human-in-the-loop frameworks. This work presents a probabilistic policy blending approach for shared control between a human operator and an intelligent agent. The proposed approach assumes that the agent can control a system and the human operator needs to communicate the system’s intended goal. A comparative study is presented between different arbitration functions that are used to blend the human and agent’s actions. The proposed approach can achieve a variable level of assistance to the human operator successfully within discrete action space using the Lunar Lander game environment developed by OpenAI. Furthermore, human physiological data have been analyzed while the human interacts with the system and the agent using different arbitration functions. A correlation between the physiological data, arbitration level, and task performance was observed.

Many attempts have been made at estimating discrete emotions (calmness, anxiety, boredom, surprise, anger) and continuous emotional measures commonly used in psychology, namely ‘valence’ (The pleasantness of the emotion being displayed) and ‘arousal’ (The intensity of the emotion being displayed). Existing methods to estimate arousal and valence rely on learning from data sets, where an expert annotator labels every image frame. Access to an expert annotator is not always possible, and the annotation can also be tedious. Hence it is more practical to obtain self-reported arousal and valence values directly from the human in a real-time Human-Robot collaborative setting. Hence this paper provides an emotion data set (HRI-AVC) obtained while conducting a human-robot interaction (HRI) task. The self-reported pair of labels in this data set is associated with a set of image frames. This paper also proposes a spatial and temporal attention-based network to estimate arousal and valence from this set of image frames. The results show that an attention-based network can estimate valence and arousal on the HRI-AVC data set even when Arousal and Valence values are unavailable per frame.

Mistakes in high stress and critical multitasking environments, such as piloting an airplane and the NASA control room, can lead to catastrophic failures. The human's internal state (e.g., workload) may be used to facilitate a robot teammate's adaptations, such that the robot can interact with the human without negatively impacting overall team performance. Human performance has a direct correlation with workload states; thus, the human's internal workload state may be leveraged to adapt a robot's interactions with the human in order to improve team performance. A reinforcement learning-based paradigm that incorporates human workload states to determine appropriate robot adaptations is presented. Preliminary results using the proposed approach in a supervisory-based NASA MATB-II environment are presented.

High-stress environments, such as first-response or a NASA control room, require optimal task performance, as a single mistake may cause monetary loss or even the loss of human life. Robots can partner with humans in a collaborative or supervisory paradigm to augment the human’s abilities and increase task performance. Such teaming paradigms require the robot to appropriately interact with the human without decreasing either’s task performance. Workload is related to task performance; thus, a robot may use a human’s workload state to modify its interactions with the human. Assessing the human’s workload state may also allow for dynamic task (re-)allocation, as a robot can predict whether a task may overload the human and, if so, allocate it elsewhere. A diagnostic workload assessment algorithm that accurately estimates workload using results from two evaluations, one peer based and one supervisory based, is presented. The algorithm correctly classified workload at least 90% of the time when trained on data from the same human--robot teaming paradigm. This algorithm is an initial step toward robots that can adapt their interactions and intelligently (re-)allocate tasks.

Supervisory control environments, such as the NASA control room can induce high workload levels in situations where a single error is capable of costing millions of dollars. An intelligent system can improve human supervisor performance by monitoring the human's workload levels and intelligently adapting the system capabilities, such as adapting the interaction medium or reallocating roles and responsibilities between the human and the system. Systems capable of responding promptly and accurately to the human's changes in workload require a workload assessment algorithm that can detect changes to all components of workload in real time. A review of 24 workload assessment algorithms across six task domains is provided. Each algorithm is reviewed based on four criteria: sensitivity, diagnosticity, suitability, and generalizability. The majority of the reviewed algorithms were developed for a specific task domain and are unable to generalize different tasks. Further, the majority of the algorithms do not account for individual differences, only assess one or two workload components, and do not classify underload.

Mass casualty events caused by a biological weapon require fully capable first response teams. However, human first responders are equipped with protective gear, which limits their capabilities to complete tasks. Robots can be employed to work collaboratively with the first responders in order to augment the human's reduced abilities. The robot needs to understand and adapt to the human's workload level in order for the human-machine team to effectively complete tasks. The automatic detection of human workload levels can provide valuable insight into the human's capabilities, as workload has a direct relationship with task performance. The robot can monitor the objective metrics of the human's workload level in order to accurately estimate workload via a workload assessment algorithm. The algorithm must be able to assess overall workload and the components of workload, in order for the robot to correctly adapt its interactions or reallocate tasks among the team. A novel workload assessment algorithm that provides an accurate estimate of overall workload and each workload component is presented and evaluated. The algorithm is capable of distinguishing between high and low workload conditions; however, the algorithm's workload values correlate poorly to a generated workload model. Modifications to enhance the algorithm's capabilities are discussed and will be investigated in future work.