The effectiveness of human-multirobot collaboration depends on the automation capabilities of robots, human’s cognitive loads and multitasking efficiency, trust of humans in robots, as well as the supporting technological system with which human and robots interact. To improve human multitasking efficiency, automated decision support recommendation systems have been developed to alleviate cognitive load, maintain human’s attentional involvement, and potentially improve the overall performance of the human-robot team. However, these recommendation systems can produce unintended behavioral consequences when considering human attention allocation, which requires experiment validation.In this paper, we present an optimization-based recommendation system extending our previous theoretical work, which will guide human operators to select which robot to help in a simulated challenging multitasking environment, where a human participant monitors and manages multiple robots navigating through an obstacle-rich terrain. Through a controlled experimental study involving 40 human participants, we compared two conditions: optimal recommendations are provided continuously versus recommendations presented only during critical scenarios. Our findings indicate that while the recommendation system provided minimal or even negative effects on actual task performance, it significantly improved participants’ perceived performance and trust. Further analysis of user feedback revealed substantial correlations between increased perceived performance, increased trust, reduced cognitive demands, and decreased frustration levels during multitasking. These insights emphasize that recommendation systems can positively impact subjective user experiences, factors essential for successful technology adoption, even when objective performance gains are limited.

For robots to transform work processes and seamlessly integrate into human settings, such as warehouses and assistive roles, they must adeptly manage a wide range of repetitive and complex interactions with various humans. Despite great advances in developing robots capable of modeling and predicting human actions for smooth collaboration, a gap persists on how these prediction models can account for the impact of robot actions on human behavior. In this paper, we introduced a human-to-human, sequential transfer learning-based, data-driven model for human decision-making in co-learning scenarios, where humans and robots mutually learn and observe from each other. We model the problem as a sequence prediction task, employing one-dimensional convolutional neural network (or 1D CNN) and dual-input bidirectional gated recurrent units (or Bi-GRU) models learned on interaction-rich, closed-loop human-robot co-learning scenarios. Our model inherently captures the dynamic irrational strategies humans develop in response to their robot counterparts. Additionally, the transfer learning approach improved the generalizability of our model, enabling it to accommodate diverse human behaviors, ideologies, and personal beliefs with reasonable accuracy. Using user feedback, we presented an in-depth analysis of some of the key factors considered by participants during interactions, which can further help in refining prediction models.

Ocular mobility disorders such as strabismus af-fect millions of people. Patients' descriptions of their symptoms, such as what they see and how their vision has changed, are important for ophthalmologists to diagnose, monitor pro-gression, and evaluate treatment effectiveness. However, such verbal depiction may be vague and Subjective. A data-driven simulator that visualizes abnormal vision experienced by a strabismic patient can be helpful to objectively illustrate each individual's vision condition and thus to better understand and manage strabismus. To fulfill this technical void, this paper presents the first vision visualization robot that uses human eye movement data to simulate strabismic vision. We developed a robotic binocular eye platform, which is capable of displaying simulated visual scenes using its onboard cameras. Based on the hypothesis that a human's binocular vision fusion process can be mimicked as a homography transformation from one view to another view, we developed a pipeline to estimate the time-varying homography matrix, and generate the fused view of a human's binocular vision. The effectiveness of the proposed method is demonstrated through experiments with eye movement data from both healthy individuals and strabismic patients.

We present the Georgia Tech Miniature Autonomous Blimp (GT-MAB), which is designed to support human-robot interaction experiments in an indoor space for up to two hours. GT-MAB is safe while flying in close proximity to humans. It is able to detect the face of a human subject, follow the human, and recognize hand gestures. GT-MAB employs a deep neural network based on the single shot multibox detector to jointly detect a human user’s face and hands in a real-time video stream collected by the onboard camera. A human-robot interaction procedure is designed and tested with various human users. The learning algorithms recognize two hand waving gestures. The human user does not need to wear any additional tracking device when interacting with the flying blimp. Vision-based feedback controllers are designed to control the blimp to follow the human and fly in one of two distinguishable patterns in response to each of the two hand gestures. The blimp communicates its intentions to the human user by displaying visual symbols. The collected experimental data show that the visual feedback from the blimp in reaction to the human user significantly improves the interactive experience between blimp and human. The demonstrated success of this procedure indicates that GT-MAB could serve as a flying robot that is able to collect human data safely in an indoor environment.

We present an approach that allows the Georgia Tech Miniature Autonomous Blimp (GT-MAB) to detect and follow a human. This accomplishment is the first Human Robot Interaction (HRI) demonstration between an uninstrumented human and a robotic blimp. GT-MAB is an ideal platform for HRI missions because it is safe to humans and can support sufficient flight time for HRI experiments. However, due to complex aerodynamic influence on the blimp, the human following task for GT-MAB with a single on-board camera is a challenging problem. We integrate Haar face detector and KLT feature tracker to achieve robust human tracking. After a human face is detected in the real-time video stream, we estimated the 3D positions of the human with respect to GT-MAB. Visionbased PID controllers are designed based on estimated relative position and the motion primitives of GT-MAB such that it can achieve stable and continuous human following behavior. Experimental results are presented to demonstrate the human following capability on GT-MAB.