Communication is essential for successful interaction. In human-robot interaction, implicit communication holds the potential to enhance robots’ understanding of human needs, emotions, and intentions. This paper introduces a method to foster implicit communication in HRI without explicitly modelling human intentions or relying on pre-existing knowledge. Leveraging Transfer Entropy, we modulate influence between agents in social interactions in scenarios involving either collaboration or competition. By integrating influence into agents’ rewards within a partially observable Markov decision process, we demonstrate that boosting influence enhances collaboration and interaction, while resisting influence promotes social independence and diminishes performance in certain scenarios. Our findings are validated through simulations and real-world experiments with human participants in social navigation and autonomous driving settings.

Learning from Demonstration (LfD) empowers robots to acquire new skills through human demonstrations, making it feasible for everyday users to teach robots. However, the success of learning and generalization heavily depends on the quality of these demonstrations. Consistency is often used to indicate quality in LfD, yet the factors that define this consistency remain underexplored. In this article, we evaluate a comprehensive set of motion data characteristics to determine which consistency measures best predict learning performance. By ensuring demonstration consistency prior to training, we enhance models’ predictive accuracy and generalization to novel scenarios. We validate our approach with two user studies involving participants with diverse levels of robotics expertise. In the first study ( N = 24), users taught a PR2 robot to perform a button-pressing task in a constrained environment, while in the second study ( N = 30), participants trained an UR5 robot on a pick-and-place task. Results show that demonstration consistency significantly impacts success rates in both learning and generalization, with 70% and 89% of task success rates in the two studies predicted using our consistency metrics. Moreover, our metrics estimate generalized performance success rates with 76% and 91% accuracy. These findings suggest that our proposed measures provide an intuitive, practical way to assess demonstration data quality before training, without requiring expert data or algorithm-specific modifications. Our approach offers a systematic way to evaluate demonstration quality, addressing a critical gap in LfD by formalizing consistency metrics that enhance the reliability of robot learning from human demonstrations.

Communication, the flow of information between agents, is vital socially acceptably robot behaviours. Understanding and utilising social information is essential for achieving such behaviours. This research investigates the detection, analysis, and application of social information flows through the lens of information theory. This research comprises three stages: detecting and analysing social cues, applying transfer entropy to enhance human-robot interaction (HRI), and exploring real-world applications. We have proposed a framework for social cue detection and analysis, demonstrated across three human interaction settings: person-following, object-handover, and group-joining. This framework provides a systematic workflow that yields reliable results. In the second stage, our simulations and human studies have shown transfer entropy's effectiveness in improving social communication within a reinforcement learning context. In the third stage, we aim to validate our framework through practical user studies, enhancing its adaptability and exploring influence modulation across diverse HRI scenarios.

When humans work together to complete a joint task, each person builds an internal model of the situation and how it will evolve. Efficient collaboration depends on how these individual models overlap to form a shared mental model among team members; shared models are also important for collaborative processes in human–robot teams. The development and maintenance of an accurate shared mental model requires bidirectional communication of individual intent and the ability to interpret the intent of other team members. To enable effective human–robot collaboration, this article investigates the use of augmented reality (AR) technology and user eye gaze to enable bidirectional communication of intent in a joint action task. We tested this approach through a user study with 37 participants and found that this communication improves task efficiency, trust, as well as task fluency. We conclude that using AR and eye gaze to enable bidirectional communication and support shared mental models is a promising means for improving collaboration between humans and robots.

Learning from Demonstration (LfD) is a framework that allows lay users to easily program robots. However, the efficiency of robot learning and the robot’s ability to generalize to task variations hinge upon the quality and quantity of the provided demonstrations. Our objective is to guide human teachers to provide more effective demonstrations, thus facilitating efficient robot learning. To achieve this, we propose to use a measure of uncertainty, namely task-related information entropy , as a criterion for suggesting informative demonstration examples to human teachers to improve their teaching skills. This approach seeks to minimize the requisite number of demonstrations by enhancing their distribution throughout the workspace. In a conducted experiment \((N = 24)\) , an augmented reality (AR)-based guidance system was employed to train novice users to produce additional demonstrations from areas with the highest entropy within the workspace. These novice users were trained for a few trials to teach the robot a generalizable task using a limited number of demonstrations. Subsequently, the users’ performance after training was assessed first on the same task (retention) and then on a new task (transfer) without guidance. The results indicate a substantial improvement in robot learning efficiency from the teacher’s demonstrations, with an improvement of up to 198% observed on the novel task. Furthermore, the proposed approach was compared to a state-of-the-art heuristic rule and found to improve robot learning efficiency by 210% compared to the heuristic rule. The scripts used in this article are available on GitHub .

Robots that work close to humans need to understand and use social cues to act in a socially acceptable manner. Social cues are a form of communication (i.e., information flow) between people. In this paper, a framework is introduced to detect and analyse a class of perceptible social cues that are nonverbal and episodic, and the related information transfer using an information-theoretic measure, namely, transfer entropy. We use a group-joining setting to demonstrate the practicality of transfer entropy for analysing communications between humans. Then we demonstrate the framework in two settings involving social interactions between humans: object-handover and person-following. Our results show that transfer entropy can identify information flows between agents and when and where they occur. Potential applications of the framework include information flow or social cue analysis for interactive robot design and socially-aware robot planning.

Multi-robot supervision becomes increasingly cognitively demanding as the ratio of robots to human supervisors rises, potentially leading to situational awareness (SA) losses and robot system failures. Nonverbal cues have been employed to direct supervisor attention and prevent awareness loss in diverse human-computer interaction (HCI) settings. This paper compares the effects of uni-modal and multi-modal audiovisual nonverbal cues on supervisor SA in a multi-robot supervision task. In a simulation-based navigation scenario, 50 participants monitored a multi-robot mission and responded to supervision requests from the robots. We evaluated supervisor SA using response speed and the situational awareness global assessment technique. Results demonstrate that supervisor awareness hinges on the communication method employed by the robots, with greater significance observed at higher awareness levels and when the robot-to-human ratio is higher. Findings also indicate the utility of sonification mapping in human-multirobot interactions and the benefits of multi-modal cues for sustaining awareness during multi-robot supervision.

Navigating effectively and safely in unknown urban environments is a crucial ability for service robot applications such as last-mile package delivery. To reach the entrance of its target destination, the robot must make informed local and global path planning decisions. We present a mapless global planning strategy based on pedestrian following. Our method allows the robot to exploit natural routes taken by surrounding pedestrians to make informed and efficient path planning decisions for reaching its goal. The algorithm also includes a recovery system to assist the robot when insufficient progress is made (i.e. robot stuck in dead end). Once the robot is within the vicinity of the target building, a wall following behaviour is used to reach the entrance of the target building. Simulated experiments and a proof-of-concept demonstration on a real robot were shown to validate the approach.

Existing approaches to direct robot-to-human handovers are typically implemented on fixed-base robot arms, or on mobile manipulators that come to a full stop before performing the handover. We propose "on-the-go" handovers which permit a moving mobile manipulator to hand over an object to a human without stopping. The on-the-go handover motion is generated with a reactive controller that allows simultaneous control of the base and the arm. In a user study, human receivers subjectively assessed on-the-go handovers to be more efficient, predictable, natural, better timed and safer than handovers that implemented a "stop-and-deliver" behavior.

Safety is a basic requirement in any human-robot collaboration scenario. To ensure user safety, from both physical and psychological aspects, we propose a novel Virtual Barrier system facilitated by an augmented reality interface1. Our system provides two types of Virtual Barriers to ensure safety: 1) a Virtual Person Barrier which encapsulates and follows the user to protect them from collisions with the robot, and 2) Virtual Obstacle Barriers which users can spawn to protect objects or regions that the robot should not enter. Our system utilizes augmented reality to visually display these protective barriers to the user during operation. To enable effective human-robot collaboration, our system automatically replans the robot’s motion when potential collisions are detected as a result of a barrier intersecting the robot’s planned path. Comparing our novel system with a standard 2D display interface in a user study with a mock industrial manufacturing task showed that our system increases both physical and psychological safety, task efficiency and interaction intuitiveness.

Humans are highly skilled in communicating their intent for when and where a handover would occur. However, even the state-of-the-art robotic implementations for handovers typically lack of such communication skills. This study investigates visualization of the robot’s internal state and intent for Human-to-Robot Handovers using Augmented Reality. Specifically, we explore the use of visualized 3D models of the object and the robotic gripper to communicate the robot’s estimation of where the object is and the pose in which the robot intends to grasp the object. We tested this design via a user study with 16 participants, in which each participant handed over a cube-shaped object to the robot 12 times. Results show communicating robot intent via augmented reality substantially improves the perceived experience of the users for handovers. Results also indicate that the effectiveness of augmented reality is even more pronounced for the perceived safety and fluency of the interaction when the robot makes errors in localizing the object.

The use of hand-based gestures has been proposed as an intuitive way for people to communicate with robots. Typically the set of gestures is defined by the experimenter. However, existing works do not necessarily focus on gestures that are communicative, and it is unclear whether the selected gesture are actually intuitive to users. This paper investigates whether different people inherently use similar gestures to convey the same commands to robots, and how teaching of gestures when collecting demonstrations for training recognizers can improve resulting accuracy. We conducted this work in two stages. In Stage 1, we conducted an online user study (n=190) to investigate if people use similar gestures to communicate the same set of given commands to a robot when no guidance or training was given. Results revealed large variations in the gestures used among individuals With the absences of training. Training a gesture recognizer using this dataset resulted in an accuracy of around 20%. In response to this, Stage 2 involved proposing a common set of gestures for the commands. We taught these gestures through demonstrations and collected ~ 7500 videos of gestures from study participants to train another gesture recognition model. Initial results showed improved accuracy but a number of gestures had high confusion rates. Refining our gesture set and recognition model by removing those gestures, We achieved an final accuracy of 84.1 ± 2.4%. We integrated the gesture recognition model into the ROS framework and demonstrated a use case, where a person commands a robot to perform a pick and place task using the gesture set.

We provide an experimental evaluation of a wearable augmented reality (AR) system we have developed for human-robot teams working on tasks requiring collaboration in shared physical workspace. Recent advances in AR technology have facilitated the development of more intuitive user interfaces for many human-robot interaction applications. While it has been anticipated that AR can provide a more intuitive interface to robot assistants helping human workers in various manufacturing scenarios, existing studies in robotics have been largely limited to teleoperation and programming. Industry 5.0 envisions cooperation between human and robot working in teams. Indeed, there exist many industrial tasks that can benefit from human-robot collaboration. A prime example is high-value composite manufacturing. Working with our industry partner towards this example application, we evaluated our AR interface design for shared physical workspace collaboration in human-robot teams. We conducted a multi-dimensional analysis of our interface using established metrics. Results from our user study (n = 26) show that, subjectively, the AR interface feels more novel and a standard joystick interface feels more dependable to users. However, the AR interface was found to reduce physical demand and task completion time, while increasing robot utilization. Furthermore, user’s freedom of choice to collaborate with the robot may also affect the perceived usability of the system.

Mobile robots are increasingly being deployed in public spaces such as shopping malls, airports, and urban sidewalks. Most of these robots are designed with human-aware motion planning capabilities but are not designed to communicate with pedestrians. Pedestrians encounter these robots without prior understanding of the robots’ behaviour, which can cause discomfort, confusion, and delayed social acceptance. In this research, we explore the common human-robot interaction at a doorway or bottleneck in a structured environment. We designed and evaluated communication cues used by a robot when yielding to a pedestrian in this scenario. We conducted an online user study with 102 participants using videos of a set of robot-to-human yielding cues. Results show that a Robot Retreating cue was the most socially acceptable cue. Repeated measures and Friedman’s ANOVAs on components of social acceptability were statistically significant (p = .01) and had small and medium effect sizes (ηp2= .04, ηp2= .08). The results of this work help guide the development of mobile robots for public spaces.

We present an approach for visualizing mobile robots through an Augmented Reality headset when there is no line-of-sight visibility between the robot and the human. Three elements are visualized in Augmented Reality: 1) Robot’s 3D model to indicate its position, 2) An arrow emanating from the robot to indicate its planned movement direction, and 3) A 2D grid to represent the ground plane. We conduct a user study with 18 participants, in which each participant are asked to retrieve objects, one at a time, from stations at the two sides of a T-junction at the end of a hallway where a mobile robot is roaming. The results show that visualizations improved the perceived safety and efficiency of the task and led to participants being more comfortable with the robot within their personal spaces. Furthermore, visualizing the motion intent in addition to the robot model was found to be more effective than visualizing the robot model alone. The proposed system can improve the safety of automated warehouses by increasing the visibility and predictability of robots.

An appropriate user interface to collect human demonstration data for deformable object manipulation has been mostly overlooked in the literature. We present an inter-action design for demonstrating cloth folding to robots. Users choose pick and place points on the cloth and can preview a visualization of a simulated cloth before real-robot execution. Two interfaces are proposed: A 2D display-and-mouse interface where points are placed by clicking on an image of the cloth, and a 3D Augmented Reality interface where the chosen points are placed by hand gestures. We conduct a user study with 18 participants, in which each user completed two sequential folds to achieve a cloth goal shape. Results show that while both interfaces were acceptable, the 3D interface was more suitable for understanding the task, and the 2D interface was suitable for repetition. Results also found that fold previews improve three key metrics: task efficiency, the ability to predict the final shape of the cloth, and overall user satisfaction.

When the question of who should get access to a communal resource first is uncertain, people often negotiate via nonverbal communication to resolve the conflict. What should a robot be programmed to do when such conflicts arise in Human-Robot Interaction? The answer to this question varies depending on the context of the situation. Learning from how humans use hesitation gestures to negotiate a solution in such conflict situations, we present a human-inspired design of nonverbal hesitation gestures that can be used for Human-Robot Negotiation. We extracted characteristic features of such negotiative hesitations humans use, and subsequently designed a trajectory generator (Negotiative Hesitation Generator) that can re-create the features in robot responses to conflicts. Our human-subjects experiment demonstrates the efficacy of the designed robot behaviour against non-negotiative stopping behaviour of a robot. With positive results from our human-robot interaction experiment, we provide a validated trajectory generator with which one can explore the dynamics of human-robot nonverbal negotiation of resource conflicts.

This article surveys the literature on human–robot object handovers. A handover is a collaborative joint action, where an agent, the giver, gives an object to another agent, the receiver. The physical exchange starts when the receiver first contacts the object held by the giver and ends when the giver fully releases the object to the receiver. However, important cognitive and physical processes begin before the physical exchange, including initiating implicit agreement with respect to the location and timing of the exchange. From this perspective, we structure our review into the two main phases delimited by the aforementioned events: a prehandover phase and the physical exchange. We focus our analysis on the two actors (giver and receiver) and report the state of the art of robotic givers (robot-to-human handovers) and the robotic receivers (human-to-robot handovers). We report a comprehensive list of qualitative and quantitative metrics commonly used to assess the interaction. While focusing our review on the cognitive level (e.g., prediction, perception, motion planning, and learning) and the physical level (e.g., motion, grasping, and grip release) of the handover, we also discuss safety. We compare the behaviors displayed during human-to-human handovers to the state of the art of robotic assistants and identify the major areas of improvement for robotic assistants to reach performance comparable to human interactions. Finally, we propose a minimal set of metrics that should be used in order to enable a fair comparison among the approaches.

This paper investigate potential use of augmented reality (AR) for physical human-robot collaboration in large-scale, labour-intensive manufacturing tasks. While it has been shown that use of AR can help increase task efficiency in teleoperative and robot programming tasks involving smaller-scale robots, its use for physical human-robot collaboration in shared workspaces and large-scale manufacturing tasks have not been well-studied. With the eventual goal of applying our AR system to collaborative aircraft body manufacturing, we compare in a user study the use of an AR interface we developed with a standard joystick for human robot collaboration in an experiment task simulating industrial carbon-fibre-reinforced-polymer manufacturing procedure. Results show that use of AR yields reduced task time and physical demand, with increased robot utilization.

Programming and executing robot trajectories is a routine manufacturing procedure. However, current interfaces (i.e., teach pendants) are bulky, unintuitive, and interrupts task flow. Recently, augmented reality (AR) has been used to create alternative solutions. However, input modalities of such systems tend to be limited. By introducing the use of electromyography (EMG), we have created a novel multimodal wearable interface for online trajectory programming and execution. Through the use of EMG, our system aims to bridge the user's force activation to the robot arm force profile. Our proposed system provides two interaction methods for trajectory execution and force control using 1) arm EMG and 2) arm orientation. We compared these methods with a standard joystick in a user study to test their usability. Results show that proposed methods have increased physical demands but yield equivalent task performance, demonstrating the potential of our proposed interface to provide a wearable alternative solution.

Robot Learning from Demonstration (LfD) allows robots to implement autonomous manipulation by observing the movements executed by a demonstrator. As such, LfD has been established as a key element for useful user interactions in everyday environments. Kinesthetic teaching, a teaching technique within LfD, entails physically guiding the robot to achieve a task. When demonstrating complex actions on a multi-DoF manipulator, novice users typically encounter difficulties with trajectory continuity and joint orientation, necessitating training by an expert. A comparison between different training approaches is conducted in a study of nine novice users. These approaches are kinesthetic, observational and discovery-learning. The kinesthetic method utilizes record and playback functions implemented on a 7-DoF Barrett Technology WAM robot. A novice user passively holds the arm while an expert's trajectory is replayed. A visual demonstration by the expert is used for the observational training group. The discovery-learning group does not receive an expert demonstration; they use trial-and-error to produce the trajectory on their own. Task-space performance is evaluated pre- and post-training for each user to determine the relative and absolute performance improvements of the groups across the three training approaches. Absolute performance improvements are compared to the performance of an expert and a minimum-jerk trajectory to gauge how skillful the participant becomes with respect to the expert. The kinesthetic approach shows superior indicators of performance in trajectory similarity to the minimum-jerk trajectory with 39% and 13% improvement over the observational and discovery methods, respectively. Observational training shows greater improvement in terms of the smoothness of the velocity profile with 32.7% compared to 29.5% and 21.9% for both discovery and kinesthetic training, respectively.

In this paper, we propose a novel navigation system for mobile robots in pedestrian-rich sidewalk environments. Sidewalks are unique in that the pedestrian-shared space has characteristics of both roads and indoor spaces. Like vehicles on roads, pedestrian movement often manifests as linear flows in opposing directions. On the other hand, pedestrians also form crowds and can exhibit much more random movements than vehicles. Classical algorithms are insufficient for safe navigation around pedestrians and remaining on the sidewalk space. Thus, our approach takes advantage of natural human motion to allow a robot to adapt to sidewalk navigation in a safe and socially-compliant manner. We developed a group surfing method which aims to imitate the optimal pedestrian group for bringing the robot closer to its goal. For pedestrian-sparse environments, we propose a sidewalk edge detection and following method. Underlying these two navigation methods, the collision avoidance scheme is human-aware. The integrated navigation stack is evaluated and demonstrated in simulation. A hardware demonstration is also presented.

This work explores social perceptions of robots within the domain of human-to-robot handovers. Using the Robotic Social Attributes Scale (RoSAS), we explore how users socially judge robot receivers as three factors are varied: initial position of the robot arm prior to handover, grasp method employed by the robot when receiving a handover object trading off perceived object safety for time efficiency, and retraction speed of the arm following handover. Our results show that over multiple handover interactions with the robot, users gradually perceive the robot receiver as being less discomforting and having more emotional warmth. Additionally, we have found that by varying grasp method and retraction speed, users may hold significantly different judgments of robot competence and discomfort. With these results, we recognize empirically that users are able to develop social perceptions of robots which can change through modification of robot receiving behaviour and through repeated interaction with the robot. More widely, this work suggests that measurement of user social perceptions should play a larger role in the design and evaluation of human-robot interactions and that the RoSAS can serve as a standardized tool in this regard.

This paper presents a future-focused approach for robot programming based on augmented trajectories. Using a mixed reality head-mounted display (Microsoft Hololens) and a 7-DOF robot arm, we designed an augmented reality (AR) robotic interface with four interactive functions to ease the robot programming task: 1) Trajectory specification. 2) Virtual previews of robot motion. 3) Visualization of robot parameters. 4) Online reprogramming during simulation and execution. We validate our AR-robot teaching interface by comparing it with a kinesthetic teaching interface in two different scenarios as part of a pilot study: creation of contact surface path and free space path. Furthermore, we present an industrial case study that illustrates our AR manufacturing paradigm by interacting with a 7-DOF robot arm to reduce wrinkles during the pleating step of the carbon-fiber-reinforcement-polymer vacuum bagging process in a simulated scenario.

We propose an algorithm for handling visual occlusions that disrupt visual tracking of high-dimensional eye-in-hand systems. Our algorithm allows a robot to look behind an occluder during active visual target search and reacquire its target in an online manner. A particle filter continuously estimates the target location and an enhanced observation model updates the target belief state. Meanwhile, we build a simple but efficient map of the occluder boundaries to compute potential occlusion-clearing motions. Our mixed-initiative cost function balances the goal of gaining more information about the target and occluder boundary while minimizing the sensor action cost. A data-driven planner uses informed samples to strike a balance between target search and information gain to avoid exhaustive mapping of the three-dimensional occluder into Configuration space. We demonstrate the capabilities of our algorithm in simulation and a real-world experiment. We also show that our proposed solvers outperform a common approach in the literature. Our results indicate that our algorithm can quickly obtain clear views of the target when occlusion is persistent and significant camera motion is required.

The use of immersive Virtual Reality (VR) for studying Human-Robot Interaction (HRI) offers many benefits, including decreased cost and risk as well as increased experimental control and repeatability. Previous work has shown that people reliably underestimate distances in VR; however, the effect of this underestimation on gesture recognition has not been characterized. This work contributes to the validation of immersive VR as a platform for HRI investigation and simulation for training in industry. A matched pair of studies compared the location preferences of human participants when viewing gestures generated by a robot in both virtual and physical environments. Participants were asked to select up to three optimal locations within a bounded region at which they perceived the robot's gesture to be the clearest. We found that the use of VR did increase the preferred proxemic distance (χ2(1) = 18.046, p <; 0.001) by approximately 642 ± 96mm. The difference in viewing angle between the virtual and physical environments was not significant, with a 95% confidence interval limiting the difference within -8.6° to +7.9°. Observations relating gesture features to optimal viewing locations are also presented.

With advances in sensor and actuator design, intelligent computing techniques and personal care robotics, today's robots hold promise as fully interactive, therapeutic human companions. To achieve this ambitious goal, key interaction components must be identified and then systematically designed and evaluated. Based on successes of human-animal therapy, we propose affective touch as one such component. Delivering this adjunct in a controllable robot form allows us to examine its efficacy for therapeutic applications such as anxiety management. With an approach grounded in social cognitive theories for human-animal relations, we deployed a social robot, the Haptic Creature, in an interaction designed to be calming: participants held the robot on their laps and stroked it as it was breathing. As a result, their heart and respiration rates significantly decreased relative to stroking a non-breathing robot. They also reported themselves as calmer and happier.

To enable robots to learn handover orientations from observing natural handovers, we conduct a user study to measure and compare natural handover orientations with giver-centered and receiver-centered handover orientations for twenty common objects. We use a distance minimization approach to compute mean handover orientations. We posit that, computed means of receiver-centered orientations could be used by robot givers to achieve more efficient and socially acceptable handovers. Furthermore, we introduce the notion of affordance axes for comparing handover orientations, and offer a definition for computing them. Observable patterns were found in receiver-centered handover orientations. Comparisons show that depending on the object, natural handover orientations may not be receiver-centered; thus, robots may need to distinguish between good and bad handover orientations when learning from natural handovers.

In this work we explore the effectiveness of a three-fingered robotic gripper in accurately expressing directional instructions (move up, down, left, right) as gestures emulating human hand gestures. Such gestures can be necessary in noisy manufacturing environments where verbal communication is ineffective. Three studies are conducted. In Study 1 we explore hand configurations that human dyads use for nonverbal instruction (n = 17). In Study 2 we examine which hand-configurations from Study 1 are most accurately understood by observers (n = 140). In Study 3 we compare performance between a robot arm performing similar motions to those of human study participants using either an unposed or posed three-fingered robotic gripper (n =100) to observe the importance of the hand's pose. Recognition rates of directional gestures for both the human and the robot are examined. Results indicate that most gestures are better and more confidently recognized when displayed with the posed robot hand.

With the rise in popularity of robot-mediated teleconference (telepresence) systems, there is an increased demand for user interfaces that simplify control of the systems' mobility. This is especially true if the display/camera is to be controlled by users while remotely collaborating with another person. In this work, we compare the efficacy of a conventional keyboard and a non-contact, gesture-based, Leap interface in controlling the display/camera of a 7-DoF (degrees of freedom) telepresence platform for remote collaboration. Twenty subjects participated in our usability study where performance, ease of use, and workload were compared between the interfaces. While Leap allowed smoother and more continuous control of the platform, our results indicate that the keyboard provided superior performance in terms of task completion time, ease of use, and workload. We discuss the implications of novel interface designs for telepresence applications.

In this paper, we compare a touch-based human-to-robot command scheme with traditional button commands in a series of human-robot collaborative assembly tasks. We find a mapping between command style and task outcome that depends on task complexity and is influenced by robot feel. In our direct touch-based scheme, the user commands the robot through direct physical contact by tapping and pushing the robot. With a small, compliant desktop robot and a simple, scripted, bolt insertion task, button commands performed slightly better than direct physical commands in quantitative task performance metrics and qualitative user preference. In a second study with a human-scale, stiffer robot arm, physical commands performed better than button commands in a more complex and less scripted bolt insertion task, which greatly outperformed using buttons in a cooperative positioning task. We conclude that commanding a robot through direct force-transmitting contact can decrease task completion time, aid in teamwork, and improve user experience in appropriately chosen tasks. We achieve our haptic commands using only robot position sensors, demonstrating that direct, intuitive physical command is an option for existing position-controlled industrial robots.

In this paper we provide empirical evidence that using humanlike gaze cues during human-robot handovers can improve the timing and perceived quality of the handover event. Handovers serve as the foundation of many human-robot tasks. Fluent, legible handover interactions require appropriate nonverbal cues to signal handover intent, location and timing. Inspired by observations of human-human handovers, we implemented gaze behaviors on a PR2 humanoid robot. The robot handed over water bottles to a total of 102 naïve subjects while varying its gaze behaviour: no gaze, gaze designed to elicit shared attention at the handover location, and the shared attention gaze complemented with a turn-taking cue. We compared subject perception of and reaction time to the robot-initiated handovers across the three gaze conditions. Results indicate that subjects reach for the offered object significantly earlier when a robot provides a shared attention gaze cue during a handover. We also observed a statistical trend of subjects preferring handovers with turn-taking gaze cues over the other conditions. Our work demonstrates that gaze can play a key role in improving user experience of human-robot handovers, and help make handovers fast and fluent.

This paper presents a novel framework for solving the Constrained Manipulator Visual Servoing (CMVS) problem. Classical eye-in-hand visual servoing relies on a reference image to capture the end-effector positioning task, but non-convex workspace constraints (such as whole-arm collision and camera occlusion constraints) are not represented. An explicit CAD model of the workspace is typically required for collision avoidance and visibility planning algorithms. In our novel CMVS framework, during the reference image capture process, we leverage the user's kinesthetic and visual capabilities to obtain a set of qualitatively-diverse demonstrations that provide information about the robot's work environment. We investigate methods for identifying the topology of the feasible regions represented directly in the control space of the robot (i.e., image-space and joint-space). We use a combination of stochastic modeling and graphical methods to describe the feasible space, capturing both the inter-group and intra-group variations. Specifically, our method uses the inter-groups variations to build a map that describes the global connectivity of the space, while exploiting the intra-group variations to automatically derive the appropriate gains in the control law. For a given target object, we apply online Gaussian Mixture Regression to the relevant feasible space regions to provide an idealized trajectory for tracking in image-space and in joint-space. We illustrate the key advantages of our approach through a set of visual servoing experiments on a Barrett WAM 7-DOF manipulator with a Sony XC-HR70 camera.

New developments, innovations, and advancements in robotic technology are paving the way for intelligent robots to enable, support, and enhance the capabilities of human workers in manufacturing environments. We envision future industrial robot assistants that support workers in their tasks, advancing manufacturing quality and processes and increasing productivity. However, this requires new channels of fine-grained, fast and reliable communication. In this research we examined the communication required for human-robot collaboration in a vehicle door assembly scenario. We identified potential communicative gestures applicable to this scenario, implemented these gestures on a Barrett WAMTM1manipulator, and evaluated them in terms of human recognition rate and response time in a real-time interaction. Response time analysis reveals insights into the communicative structure of robot motions; namely, key short gesture segments include the bulk of the communicative information. These results will help us design more efficient and fluid task flow in human-robot interaction scenarios.

We propose a method for handling persistent visual occlusions that disrupt visual tracking for eye-in-hand systems. Our approach allows a robot to “look behind” an occluder and re-acquire its target. To allow efficient planning, we avoid exhaustive mapping of the 3D occluder into configuration space, and instead use informed samples to strike a balance between target search and information gain. A particle filter continuously estimates the target location when it is not visible. Meanwhile, we build a simple but effective map of the occluder's extents to compute potential occlusion-clearing motions using very few calls to efficient approximations of inverse kinematics. Our mixed-initiative cost function balances the goal of directly locating the target with the goal of gaining information through mapping the occluder. Monte-Carlo optimization with efficient data-driven proposals allows us to approximate one-step solutions efficiently. Experimental evaluation performed on a realistic simulator shows that our method can quickly obtain clear views of the target, even when occlusions are persistent and significant camera motion is required.

Vision feedback control techniques are desirable for a wide range of robotics applications due to their robustness to image noise and modeling errors. However in the case of a robot-mounted camera, they encounter difficulties when the camera traverses large displacements. This scenario necessitates continuous visual target feedback during the robot motion, while simultaneously considering the robot's self- and external-constraints. Herein, we propose to combine workspace (Cartesian space) path-planning with robot teach-by-demonstration to address the visibility constraint, joint limits and “whole arm” collision avoidance for vision-based control of a robot manipulator. User demonstration data generates safe regions for robot motion with respect to joint limits and potential “whole arm” collisions. Our algorithm uses these safe regions to generate new feasible trajectories under a visibility constraint that achieves the desired view of the target (e.g., a pre-grasping location) in new, undemonstrated locations. Experiments with a 7-DOF articulated arm validate the proposed method.

In collaborative tasks, people often communicate using nonverbal gestures to coordinate actions. When two people reach for the same object at the same time, they often respond to an imminent potential collision with jerky halting hand motions that we term hesitation gestures. Successful implementation of such communicative conflict response behaviour onto robots can be useful. In a myriad of human-robot interaction contexts involving shared spaces and objects, this behaviour can provide a fast and effective means for robots to express awareness of conflict and cede right-of-way during collaborative work with users. Our previous work suggests that when a six-degree-of-freedom (6-DOF) robot traces a simplified trajectory of recorded human hesitation gestures, these robot motions are also perceived by humans as hesitation gestures. In this work, we present a characteristic motion profile derived from the recorded human hesitation motions, called the Acceleration-based Hesitation Profile (AHP). We test its efficacy to generate communicative hesitation responses by a robot in a fast-paced human-robot interaction experiment.

In this study, we investigate and characterize haptic interaction in human-to-human handovers and identify key features that facilitate safe and efficient object transfer. Eighteen participants worked in pairs and transferred weighted objects to each other while we measured their grip forces and load forces. Our data show that during object transfer, both the giver and receiver employ a similar strategy for controlling their grip forces in response to changes in load forces. In addition, an implicit social contract appears to exist in which the giver is responsible for ensuring object safety in the handover and the receiver is responsible for maintaining the efficiency of the handover. Compared with prior studies, our analysis of experimental data show that there are important differences between the strategies used by humans for both picking up/placing objects on table and that used for handing over objects, indicating the need for specific robot handover strategies as well. The results of this study will be used to develop a controller for enabling robots to perform object handovers with humans safely, efficiently, and intuitively.

In the near future, as robots become more advanced and affordable, we can envision their use as intelligent assistants in a variety of domains. An exemplar human-robot task identified in many previous works is cooperatively carrying a physically large object. An important task objective is to keep the carried object level. In this work, we propose an admittance-based controller that maintains a level orientation of a cooperatively carried object. The controller raises or lowers its end of the object with a human-like behavior in response to perturbations in the height of the other end of the object (e.g., the end supported by the human user). We also propose a novel tuning procedure, and find that most users are in close agreement about preferring a slightly under-damped controller response, even though they vary in their preferences regarding the speed of the controller's response.

Nonverbal communication cues play an important role in human-human interaction and are expected to take a similar role in human-robot collaboration. In current industrial practice, human-robot turn-taking is explicitly human controlled, via a command channel such as switch or button. However, such a master-slave approach does not permit collaborative interaction, and requires the human to focus on both controlling the robot's behavior and on the task, thereby affecting overall performance. In this paper, implicit, nonverbal communication cues are examined as a non-explicit communication channel during a turn-taking task context. The aim of this study is to characterize the types and frequencies of nonverbal cues important to regulating turn taking during an assembly-task-type collaboration. This analysis will guide the selection of cues that can be expressed by the robot as implicit user inputs while human and robot complete a shared task.

This work explores a novel interaction paradigm driven by implicit, low-attention user control, accomplished by monitoring a user's physiological state. We have designed and prototyped this interaction for a first use case of bookmarking an audio stream, to holistically explore the implicit interaction concept. Here, a user's galvanic skin conductance (GSR) is monitored for orienting responses (ORs) to external interruptions; our prototype automatically bookmarks the media such that the user can attend to the interruption, then resume listening from the point he/she is interrupted. To test this approach's viability, we addressed questions such as: does GSR exhibit a detectable response to interruptions, and how should the interaction utilize this information? In evaluating this system in a controlled environment, we found an OR detection accuracy of 84%; users provided subjective feedback on its accuracy and utility.

This paper presents a model-free optimization framework for the visual servoing of eye-in-hand manipulators in cluttered environments. Visual feedback is used to solve for a set of feasible trajectories that bring the robot end-effector to a target object at a previously untaught location under a number of challenging constraints (i.e., whole-arm collisions, object occlusions, robot's joint limits, camera's sensing limits). A novel controller is proposed, which exploits the natural by-products of the teach-by-showing process, to help the robot navigate this non-convex space. Examining the user-demonstrated trajectories that lead up to the reference image, we use a combination of stochastic optimization techniques and classical optimization techniques to extract the relevant cost functions and constraints for servoing. We hypothesize that we can leverage the user's sensory capabilities and knowledge of the workspace to alleviate the burden of modeling system constraints explicitly. We verify this hypothesis via realistic experiments on a Barrett WAM 7-DOF manipulator equipped with a Sony XC-HR70 camera to show the comparative efficacy of this approach.

Unwanted conflicts are inevitable between collaborating agents that share spaces and resources. Motivated by the use of nonverbal communications as a conflict resolution mechanism by humans, this study investigates the communicative capabilities reflected in the trajectory characteristics of hesitation gestures during human-robot collaboration. Hesitation gestures and non-hesitation human arm motions were recorded from a series of reach-and-retract tasks and embodied on a 6-DOF robot arm. A total of 86 survey respondents watched and scored recordings of these motions according to whether they recognized hesitation gestures as exhibited by both the human and the robot. Using the survey's statistical evidence indicating that hesitation trajectories embodied in an articulated robot arm can be recognized by human observers, we identified trajectory characteristics of hesitation gestures. The contribution of our work is an empirically grounded robot trajectory specification that provides communicative cues for conflict resolution during collaborative reaching scenarios.

Intelligent robot assistants will be simpler for laypersons to use if the robots are able to cooperate with their users anthromimetically. However, designing an anthromimetic robot controller requires knowledge of human behavior in the domain of interest. Previous works have identified cooperative carrying as an ideal task for robotic assistants, but studies of human behavior in this domain are few. In this paper, we present an experimental study of nineteen non-expert human subjects cooperatively carrying a long object with a robot partner. Our results demonstrate that human cooperative carrying behavior is much more complex than previously supposed, which has significant implications on the development of intelligent, physically interactive robot assistants.

We present the system design for a novel robotic balance simulator that enables the investigation of the balance mechanisms involved in natural human standing. Our system allows for complete control of task dynamics to mimic normal standing while avoiding the pitfalls associated with applying external perturbations. The system enables subjects to balance themselves according to a programmable physical model of an inverted pendulum. Subjects were able to balance the system, and results show that the load stiffness curves approximate those of normal human standing to within 20.1 ± 9.7% (S.D.). Differences were within the range expected from control loop delay, reduced ankle motion, and approximations inherent to the inverted pendulum model.

This paper focuses on the challenges of vision-based motion planning for industrial manipulators. Our approach is aimed at planning paths that are within the sensing and actuation limits of industrial hardware and software. Building on recent advances in path planning, our planner augments probabilistic road maps with vision-based constraints. The resulting planner finds collision-free paths that simultaneously avoid occlusions of an image target and keep the target within the field of view of the camera. The planner can be applied to eye-in-hand visual-target-tracking tasks for manipulators that use point-to-point commands with interpolated joint motion.

status: Published

This paper presents a motion planning algorithm for industrial manipulators with the simultaneous constraints of avoiding collisions and avoiding the occlusion of specified pixellated regions of an eye-to-hand camera. The system uses a probabilistic roadmap to satisfy the constraints imposed by the command interface of typical industrial manipulators and uses dynamic collision checking to ensure collision-free motion. In the context of a task monitored by a camera, we enhance a probabilistic roadmap with a dynamic occlusion checking algorithm that is able to determine which pixels of the camera are occluded by the robot during each motion segment. The occlusion algorithm is formulated as collision algorithm where the field of view of the camera is represented as a quadtree of frustums. The proposed algorithm is demonstrated in industrial bin picking simulations where the gripper must not occlude the targeted object throughout the task.

In order to develop a friendly and safe interaction between humans and robots, it is essential for the robot to evaluate user's affective states and respond accordingly. However, affective states are typically assessed using offline questionnaires and user reports. In this paper we investigate the use of an online-device for collecting real-time user reports of affective state during interaction with a robot. These reports are compared to both previous survey reports taken after the interaction, and the affective states estimated by an inference system. The aim is to evaluate and characterize the physiological signal-based inference system and determine which factors significantly influence its performance. This analysis will be used in future work, to fine tune the affective estimations by identifying what kind of variations in physiological signals precede or accompany the variations in reported affective states.

This paper presents several methods of managing field of view constraints of an eye-in-hand system for vision- based pose control with limited controller input. Herein, the possible inverse kinematic solutions for a desired relative camera pose are evaluated to determine whether the interpolated trajectories satisfy field of view constraints for the target of interest. If no immediately feasible trajectory exists, additional waypoints are specified to guide the robot towards its goal while maintaining visibility. The insertion of an additional visible and feasible waypoint divides the problem into two sub-problems of the same form, but of lesser difficulty by reducing the robot's interpolation distance. Virtual image-based visual servoing (IBVS) is used to generate an ideal image trajectory to guide the selection of waypoints. A damped least- squares inverse kinematics solution is implemented to handle robot singularities. The methods are simulated for a CRS-A465 robot with a Sony XC-HR70 camera.

An important problem in position-based visual servoing (PBVS) is to guarantee that a target will remain within the field of view for the duration of the task. In this paper, we propose a dynamic visibility checking algorithm that, given a parametrized trajectory of the camera, determines if an arbitrary 3D target will remain within the field of view. We reformulate this problem as the problem of determining if the 3D coordinates of the target collide with the frustum formed by the camera field of view during the camera trajectory. To solve this problem, our algorithm computes and compares the shortest distance between the target and the frustum with the length of the trajectory described by the target in the camera's coordinate frame. Furthermore, we demonstrate that our algorithm can be combined with path planning algorithms and, in particular, probabilistic roadmaps (PRM). Results suggest that our algorithm is computationally efficient even when the target moves in the vicinity of image borders. In simulations, we use our dynamic visibility checking algorithm in conjunction with a PRM to plan collision free paths while providing the guarantee that a specific target will not leave the field of view.

We present a novel algorithm for path planning that avoids occlusions of a visual target for an ldquoeye-in-handrdquo sensor on an articulated robot arm. We compute paths using a probabilistic roadmap to avoid collisions between the robot and obstacles, while penalizing trajectories that do not maintain line-of-sight. The system determines the space from which line-of-sight is unimpeded to the target (the visible region). We assign penalties to trajectories within the roadmap proportional to the distance the camera travels while outside the visible region. Using Dijkstrapsilas algorithm, we compute paths of minimal occlusion (maximal visibility) through the roadmap. In our experiments, we compare a shortest-distance path to the minimal-occlusion path and discuss the impact of the improved visibility.

Dynamic Parameter Identification is a useful tool for developing and evaluating robot control strategies. However, a multi degree of freedom robot arm has many parameters, and the process of determining them is challenging. Much research has been done in this area and experimental methods have been applied on several robot arms. To our knowledge, there is currently no set of inertial parameters, either by modelling or by estimation, available for the CRS A460/A465 arm, a popular laboratory table top robot. In this paper we review and compare a number of methods for dynamic parameter identification and for generating trajectories suitable for estimating the identifiable dynamic parameters of a given robot. We then present a step by step process for dynamic parameter identification of a serial manipulator, and demonstrate this process by experimentally identifying the dynamic parameters of the CRS A460 robot.

In order for humans and robots to interact in an effective and intuitive manner, robots must obtain information about the human affective state in response to the robot's actions. This secondary mode of interactive communication is hypothesized to permit a more natural collaboration, similar to the “body language” interaction between two cooperating humans. This paper describes the implementation and validation of a hidden Markov model (HMM) for estimating human affective state in real time, using robot motions as the stimulus. Inputs to the system are physiological signals such as heart rate, perspiration rate, and facial muscle contraction. Affective state was estimated using a two-dimensional valence-arousal representation. A robot manipulator was used to generate motions expected during human–robot interaction, and human subjects were asked to report their response to these motions. The human physiological response was also measured. Robot motions were generated using both a nominal potential field planner and a recently reported safe motion planner that minimizes the potential collision forces along the path. The robot motions were tested with 36 subjects. This data was used to train and validate the HMM model. The results of the HMM affective estimation are also compared to a previously implemented fuzzy inference engine.

In order for humans and robots to interact in an effective and intuitive manner, robots must obtain information about the human affective state in response to the robot's actions. This secondary mode of interactive communication is hypothesized to permit a more natural collaboration, similar to the "body language" interaction between two cooperating humans. This paper describes the implementation and validation of a hidden Markov model for estimating human affective state in real-time, using robot motions as the stimulus. Inputs to the system are physiological signals such as heart rate, perspiration rate, and facial muscle contraction. Affective state was estimated using a two dimensional valence-arousal representation. A robot manipulator was used to generate motions simulating human-robot interaction, and human subjects were asked to report their response to the motions. The human physiological response was also measured. Robot motions were generated using both a nominal potential field planner and a recently reported safe motion planner that minimizes the potential collision forces along the path. The robot motions were tested with 36 subjects. This data was used to train and validate the HMM model. The results of the HMM affective estimation are also compared to a previously implemented fuzzy inference engine

In this paper, a novel reconfigurable surveillance system that incorporates multiple active-vision sensors is presented. The proposed system has been developed for visual-servoing and other similar applications, such as tracking and state estimation, which require accurate and reliable target surveillance data. In the specific implementation case discussed herein, the position and orientation of a single target are surveyed at predetermined time instants along its unknown trajectory. Dispatching is used to select an optimal subset of dynamic sensors, to be used in a data-fusion process, and maneuver them in response to the motion of the object. The goal is to provide information of increased quality for the task at hand, while ensuring adequate response to future object maneuvers. Our experimental system is composed of a static overhead camera to predict the object's gross motion and four mobile cameras to provide surveillance of a feature on the object (i.e., target). Object motion was simulated by placing it on an xy table and preprogramming a path that is unknown to the surveillance system. The selected cameras are independently and optimally positioned to estimate the target's pose (a circular marker in our case) at the desired time instant. The target data obtained from the cameras, together with their own position and bearing, are fed to a fusion algorithm, where the final assessment of the target's pose is determined. Experiments have shown that the use of dynamic sensors, together with a dispatching algorithm, tangibly improves the performance of a surveillance system

This paper presents an approach for haptic rendering of topological constraints to users operating serial virtual linkages. In the proposed approach, a haptic device controller is designed to penalize users’ departure from the configuration manifold of the virtual linkage. This manifold is locally approximated through the range space of the Jacobian of the virtual linkage computed at the user’s hand. Simulations and controlled experiments performed using a planar haptic interaction system demonstrate that the proposed approach successfully constrains the users’ motion as required by the topology of the virtual linkage that they manipulate.

This paper describes an experiment to determine the feasibility of using physiological signals to determine the human response to robot motions during direct human-robot interaction. A robot manipulator is used to generate common interaction motions, and human subjects are asked to report their response to the motions. The human physiological response is also measured. Motion paths are generated using a classic potential field planner and a safe motion planner, which minimizes the potential collision force along the path. A fuzzy inference engine is developed to estimate the human response based on the physiological measures. Results show that emotional arousal can be detected using physiological signals and the inference engine. Comparison of initial results between the two planners shows that subjects report less anxiety and surprise with the safe planner for high planner speeds.

A new simulation approach is proposed to improve the stability and the perceived rigidity of contacts during haptic interaction with multirigid body virtual environments. The approach computes impulsive forces upon contact and penalty and friction forces during contact. The impulsive forces are derived using a new multiple collision resolution method that never increases the kinetic energy of the system. When new contacts arise, the impulsive forces generate large hand accelerations without requiring increased contact stiffness and damping. Virtual objects and linkages are regarded as points in the configuration space, and no distinction is made between them in the proposed approach.

Recent research has considered robotic machining as an alternative to traditional computer numerical control machining, particularly for prototyping applications. However, unlike traditional machine tools, robots are subject to relatively larger dynamic disturbances and operate closer to their torque limits. These factors, combined with inaccurate manipulator and machining process models, can cause joint actuator saturation during operation. This paper presents a trajectory planner that will reduce torques that are near saturation by generating trajectories with a weighted pseudoinverse. Using a relative Jacobian, the tool path is resolved into joint trajectories at the acceleration level. This paper presents a new method for selecting the weighting matrix based on the proximity of the joint torques to saturation limits. This weighting reduces the joint accelerations contributing the most to the torques near saturation, thereby reducing the joint torques. The accelerations of other joints increase to satisfy the increased demand. The effectiveness of the acceleration and torque redistribution algorithm has been demonstrated via extensive simulations.

This paper outlines an approach to the coordination of multiple mobile sensors for the surveillance of a single moving target. A real-time dispatching algorithm is used to select and position groups of sensors in response to the observed object motion. The aim is to provide robust, high-quality data while ensuring that the system can react to unexpected object manoeuvres. Sensors are assigned to collect data at specific points on the object trajectory. A dispatching strategy learned via reinforcement learning is used to control the sensor poses with respect to these points. In using the learned strategy, each sensor adopts an egocentric view of the system state to determine the most appropriate action. Simulations demonstrate the performance of the RL-based dispatcher, in comparison to similar static-sensor systems.

A real-time trajectory planner is presented. A two-pronged approach provides alternative trajectories in response to disturbances encountered during a two-manipulator machining application. First, based on the current torque demand, a weighted pseudo-inverse technique is used to reduce the joint accelerations contributing the most to the saturated joint torques. Second, the null space of the relative Jacobian is exploited to minimize the compliance of the system. Influence coefficients are used to share the null space between the cost function and penalty functions.

An online method for obtaining smooth, jerk-bounded trajectories has been developed and implemented. Jerk limitation is important in industrial robot applications, since it results in improved path tracking and reduced wear on the robot. The method described herein uses a concatenation of fifth-order polynomials to provide a smooth trajectory between two way points. The trajectory approximates a linear segment with parabolic blends trajectory. A sine wave template is used to calculate the end conditions (control points) for ramps from zero acceleration to nonzero acceleration. Joining these control points with quintic polynomials results in a controlled quintic trajectory that does not oscillate, and is near time optimal for the jerk and acceleration limits specified. The method requires only the computation of the quintic control points, up to a maximum of eight points per trajectory way point. This provides hard bounds for online motion algorithm computation time. A method for blending these straight-line trajectories over a series of way points is also discussed. Simulations and experimental results on an industrial robot are presented.

An online method for obtaining smooth, jerk-bounded trajectories has been developed and implemented. Jerk limitation is important in industrial robot applications, since it results in improved path tracking and reduced wear on the robot. The method described herein uses a concatenation of fifth-order polynomials to provide a smooth trajectory between two points. The trajectory is determined based on approximating a linear segment with parabolic blends trajectory. A sine wave approximation is used to ramp from zero acceleration to non-zero acceleration. This results in a controlled quintic trajectory which does not oscillate, and is near time-optimal given the jerk and acceleration limits specified. The method requires only the computation of the quintic control points, up to a maximum of seven points per trajectory way-point. This provides hard bounds for online motion algorithm computation time. Simulations and experimental results on an industrial robot are presented.

The paper presents a methodology for determining the initial configuration of a set of sensors for a surveillance task. It serves to complement a dynamic dispatching methodology, which selects and maneuvers subsets of sensors to achieve optimal data acquisition in real-time. Specifically, given a priori information about the expected object trajectory, the initial sensor poses are determined such that the sensing-system effectiveness is maximized. This is achieved using a constrained, nonlinear, direct search method in combination with simulations of the sensing-system performance (i.e., dynamic dispatching to adjust the sensor poses in response to the object motion).

Sensory data must be collected in real time for the majority of autonomous decision making tasks, such as target tracking, surveillance and navigation. The use of multiple sensors may significantly improve the quality and robustness of the data. Given an environment containing a set of mobile sensors, capable of altering their position and orientation, this work addresses the problem of selecting and maneuvering subsets of these sensors for optimal data acquisition in realtime. A heuristic approach to the dispatching problem suitable for on-line implementation is illustrated by a computer simulated example.

In this work, a limited survey of the diverse issues related to robot control design architectures is presented. Based on this review, an initiatory taxonomy of robot control tasks, motions, architectures, and controllers is proposed. The objective for this taxonomy is to develop an expert system for industrial robot control under the emerging open architecture controller paradigm.

A number of active prediction planning and execution (APPE) systems have recently been proposed for robotic interception of moving objects. The cornerstone of such systems is the selection of a robot-object rendezvous-point on the predicted object trajectory. Unlike tracking-based systems, which minimize the state difference between the object and the robot at each control period, in this methodology the robot is sent directly to the selected rendezvous-point. A fine-motion tracking strategy would then be employed for grasping the moving object. Herein, a novel strategy for selecting the optimal (earliest) rendezvous-point is presented. For objects with predictable trajectories, this is a significant improvement over previous APPE strategies which select the rendezvous-point from a limited number of non-optimally chosen candidates.