As collaborative robots (cobots) become available in industry, it is crucial to enable non-coders to program cobots easily. For example, in sorting tasks that may require adjustments over time, workers could easily adjust the cobot’s behavior by utilizing learning from demonstrations (LfD) approaches. This paper presents an LfD approach for enabling a cobot to learn sorting tasks by integrating the Gaussian Mixture Model (GMM), validated in Mujoco for policy training. Quantitative analysis shows that the improved BC-GMM achieves the lowest mean squared error (MSE = 0.00207) and highest R-squared score (R2=0.89), outperforming other methods, such as standard BC, diffusion policy, and implicit behavioral cloning (IBC).

In this work, we study patterns in the movement times of human participants engaging in object handover during the pre-handover phase, to inform on conducting fluent human-robot handovers. Fluency of object handover between two agents is critical to ensure success of shared overall collaborative goals in the context of larger tasks involving the transferred objects. Human givers and receivers often coordinate their pre-handover movements to ensure fluent transfer, with receivers showing anticipatory behavior and proactive response. Since typical timing patterns of human and robot movements for short-range tasks demonstrate a speedup to a peak speed, and a slowdown to the end point, we analyze relationships between time differences of peak speed attainment relative to start and end (i.e., reach) times, across the giver and receiver. Our work helps inform the design of motion planning algorithms for robots that embody timing patterns during movement that are predictable to the partner.

Modern robots found in the workplace include high-resolution color cameras, depth cameras, thermal sensors, and microphones, as well as onboard computing that enables generation of human body pose and facial landmarks that can be used to infer the physiological state of the worker. Prior research reveals that privacy concerns in the workplace tend higher towards robots than towards humans, and that robots that include privacy controllers are perceived to be more trustworthy. However, prior research lacks a systematic understanding of how workers in lower-income positions in industries such as warehousing and manufacturing perceive sensing technologies, such as color cameras and microphones, and inferred human data, such as body pose, in their concerns for privacy. In this paper, we use survey responses from 530 workers across four countries and six job domains to understand how workers perceive robots that record human activity through video, audio, or body pose. Our studies reveal that privacy concern grows with the amount of personally identifiable data that is available, i.e., workers show higher concern for video rather than body pose inferred from video. We find that Millennials show more concern about being recorded than Gen Z individuals or than Gen X individuals and older. We find that privacy concerns vary by the worker’s country with lowest concerns in the United States and highest in Canada, and that worker concern is lowest in large-size cities and highest in suburban areas.

The fluency of handover between two agents is important to ensure safety and success of handover. In this work, we study the relationships between the timings of giver and receiver motions in human-human handover interactions, in order to inform human-robot handover. We use giver and receiver hand trajectories from the Human-Object-Human (HOH) handover dataset to study movement during the pre-handover phase, prior to the point of transfer. We find that human receivers adopt a largely proactive behavior, and plan and start motion early in the pre-handover phase. We also find that human receivers spend much of their motion moving in coordination with the giver, rather than after the giver has reached the transfer point. Further, we find that human receivers may predict future movement of the giver from early giver motion, and adjust their start times accordingly to ensure coordinated grasp at transfer. Our findings suggest that robot receivers should adopt a predictive giver-aware approach to plan motion early, and robot givers should recognize that human receivers may expect giver behavior to be human-like and predictable.

We present GraspPC, an approach to perform learning-based synthesis of multiple human hand grasps as point clouds from point clouds of objects. GraspPC benefits human-robot handover approaches by providing hypotheses of human grasp on objects to inform robotic manipulation algorithms on how to bias robotic grasp for safe handover. Existing learning-based approaches to conduct hand grasp prediction require datasets to contain annotated articulated hand models, making them difficult to train on datasets that lack hand model annotations. GraspPC treats the problem of hand point cloud generation from object point clouds as a set-to-set translation problem. We contribute a Transformer architecture to synthesize point clouds via GraspPC. To generate diverse hand grasps, we generate multiple object-dependent queries and train the network using a winner-takes-gradient strategy. We show results of diverse grasps by training and testing on a variety of real-world datasets. We demonstrate how human grasps generated by GraspPC can be used to filter robotic grasp candidates to inform human-robot handover. Our code is available at: https://github.com/Terascale-All-sensing-Research-Studio/GraspPC.

In recent years, computer and game-based cognitive tests have become popular with the advancement in mobile technology. However, these tests require very little body movements and do not consider the influence that physical motion has on cognitive development. Our work mainly focus on assessing cognition in children through their physical movements. Hence, an assessment test "Ball-Drop-to-the-Beat" that is both physically and cognitively demanding has been used where the child is expected to perform certain actions based on the commands. The task is specifically designed to measure attention, response inhibition, and coordination in children. A dataset has been created with 25 children performing this test. To automate the scoring, a computer vision-based assessment system has been developed. The vision system employs an attention-based fusion mechanism to combine multiple modalities such as optical flow, human poses, and objects in the scene to predict a child's action. The proposed method outperforms other state-of-the-art approaches by achieving an average accuracy of 89.8 percent on predicting the actions and an average accuracy of 88.5 percent on predicting the rhythm on the Ball-Drop-to-the-Beat dataset.

Robots are increasingly present in environments shared with humans. Robots can cooperate with their human teammates to achieve common goals and complete tasks. This paper focuses on developing a real-time framework that assesses the cognitive load of a human while cooperating with a robot to complete a collaborative assembly task. The framework uses multi-modal sensory data from Electrocardiography (ECG) and Electrodermal Activity (EDA) sensors, extracts novel features from the data, and utilizes machine learning methodologies to detect high or low cognitive load. The developed framework was evaluated on a collaborative assembly scenario with a user study. The results show that the framework is able to reliably recognize high cognitive load and it is a first step in enabling robots to understand better about their human teammates.