Human-Robot Collaboration (HRC) scenarios demand computationally efficient frameworks that enable natural and safe actions and interactions in shared workspaces. To address this, we propose a novel framework that utilises interactive Gaussian Process (GP) distance fields applying Riemannian Motion Policies (RMP) for key HRC functionality. Unlike traditional Euclidean distance field methods, our framework provides continuous and differentiable distance fields resulting in smooth collision avoidance, efficient updates in dynamic scenes and readily available surface information such as normal vectors and curvature. By leveraging RMPs, our framework supports fast, reactive motion generation, utilising both the distance and gradient fields generated by the GP model. In addition, we propose a Hessian-based normal vector estimation technique that elegantly leverages the GP's second-order derivative information which we utilise for object manipulation. We demonstrate the versatility of our CPU-only system in common HRC scenarios where a collaborative robot (cobot) interacts safely and naturally with a human and performs grasping actions in a dynamic environment. Our framework offers an open-source11https://uts-ri.github.io/IDMP-RMP/, comprehensive and low-computational resource solution for HRC, making it an ideal tool for conducting a wide range of user studies. By providing a continuous and differentiable distance field and combining motion generation, obstacle avoidance, and object manipulation within a single system, we aim to broaden the scope and accessibility of HRC research in real dynamic environments.

This study examines the efficacy of gesture-based versus touch-based control interfaces in managing unanticipated scenarios in human-robot collaborative assembly (HRCA). As collaborative robots (cobots) become increasingly integrated into industrial environments, there is a need for control interfaces that enable workers to respond to unanticipated deviations from the assembly process. Such control interfaces must be intuitive to use, acceptable to the worker, and not result in an increased workload. We conducted a controlled experiment with 30 participants, comparing the performance of gesture-based against touch-based control interfaces in an assembly task, which included simulated disruptions. Workload was evaluated using the NASA Task Load Index (NASA-TLX), and acceptance via a Technology Acceptance Model (TAM), adapted for HRCA. The results demonstrate no statistically significant differences between the two measures. However, they indicate that touch control results in a lower workload and higher user acceptance, which may be attributed to users' familiarity with touch interfaces.

Human-Robot Collaboration (HRC) in dynamic assembly environments requires intuitive communication for seamless interactions. Eye gaze, a nonverbal cue, is critical for interpreting human intent and enhancing robot adaptability. This research introduces a three-tier framework categorizing eye gaze into fixation, scanning, and task-switching behaviors, aiming to improve real-time robot responsiveness. Preliminary evaluations with multiple human subjects demonstrated the potential of this classification to enhance task alignment and detect anomalies in human behavior. Future work will explore automated personalization of the algorithm for individuals and integrate multimodal data to refine adaptability in human-robot collaborative systems.

This paper presents an algorithm for real-time sensor-based motion planning under kinodynamic constraints, in unknown environments. The objective of the trajectory-generation algorithm is to optimise a cost function out to a limited time horizon. The space of control trajectories is searched by expanding a tree using randomised sampling, in a manner similar to an RRT. The algorithm is improved by seeding the tree using the best control trajectory from the previous iteration, and by pruning branches based on a bound to the cost function and the best trajectory found so far. Performance of the algorithm is analysed in simulation. In addition, the algorithm has been implemented on two kinds of vehicles: the Segway RMP and a four-wheel-drive. The algorithm has been used to drive autonomously for a combined total on the order of hundreds of hours.

Humans and robots need to exchange information if the objective is to achieve a task cooperatively. Two questions are considered in this paper: what type of information to communicate, and how to cope with the limited resources of human operators. Decision-theoretic human-robot communication can provide answers to both questions: the type of information is determined by the underlying probabilistic representation, and value-of-information theory helps decide when it is appropriate to query operators for information. A robot navigation task is used to evaluate the system by comparing it to conventional teleoperation. The results of a user study show that the developed system is superior with respect to performance, operator workload, and usability.

This paper proposes a methodology to measure the effectiveness of a human-robot team as part of an adjustable autonomy system. The effectiveness measure is aimed at determining an appropriate autonomy level prior to the system's deployment. Two competing goals need to be traded off: maximising robot performance while minimising the amount of human input. The relative importance of the two goals depend on the mission priorities and constraints which are taken into account. The proposed methodology is applied to a human-robot communication system developed for task- oriented information exchange. The robot uses a decision- theoretic framework to act autonomously and to decide when to request input from human operators. The latter is achieved by computing the value-of-information an operator is able to provide which is compared to the cost of obtaining the information. For our system, the cost parameter represents the autonomy level to be determined. We demonstrate how an appropriate autonomy level can be found experimentally using a navigation task. In our experiment, the robot navigates through a set of simulated worlds with human input being generated by a software component. The results are used to find appropriate autonomy levels for three example missions and a subsequent user study.

This paper considers the problem of building a software architecture for a human-robot team. The objective of the team is to build a multi-attribute map of the world by performing information fusion. A decentralized approach to information fusion is adopted to achieve the system properties of scalability and survivability. Decentralization imposes constraints on the design of the architecture and its implementation. We show how a component-based software engineering approach can address these constraints. The architecture is implemented using Orca - a component-based software framework for robotic systems. Experimental results from a deployed system comprised of an unmanned air vehicle, a ground vehicle, and two human operators are presented. A section on the lessons learned is included which may be applicable to other distributed systems with complex algorithms. We also compare Orca to the player software framework in the context of distributed systems.

In this paper, we combined the application of a non-linear dimensionality reduction technique, isomap, with expectation maximisation in graphical probabilistic models for learning and classification of hyperspectral image. Hyperspectral image spectroscopy gives much greater information content per pixel on the image than a normal colour image. This should greatly help with the autonomous identification of natural and man-made objects in unfamiliar terrains for robotic vehicles. However, the large information content of such data makes interpretation of hyperspectral images time-consuming and user-intensive. Isomap is used to find the underlying manifold of the training data. This low dimensional representation of the hyperspectral data facilitates the learning of a mixture of linear models representation similar to a mixture of factor analysers, the joint probability distributions of the model can be calculated offline. The learnt model is then applied to the hyperspectral image at run-time and data classification can be performed. We also show the comparison with results from standard techniques

This paper considers the problem of building environment models by fusing information gathered by robotic platforms with human perceptual information. Rich environment models are required in real applications for both autonomous operation of robots and to support human decision making. Hierarchical models are well suited to represent complex environments because they: offer multiple abstractions of the available information to support analysis and decision-making, and permit the incorporation of higher-level human observations. The contributions of this paper are two-fold: (1) development of a probabilistic three-level environment model for distributed information gathering, and (2) experimental demonstration of fully decentralized, cooperative human-robot information gathering using an outdoor sensor network comprised of an unmanned air vehicle, a ground vehicle, and two human operators. Several information exchange patterns are presented which qualitatively demonstrate human-robot information fusion

This paper gives an overview of component-based software engineering (CBSE), motivates its application to the field of mobile robotics, and proposes a particular component model. CBSE is an approach to system-building that aims to shift the emphasis from programming to composing systems from a mixture of off-the-shelf and custom-built software components. This paper argues that robotics is particularly well-suited for and in need of component-based ideas. Furthermore, now is the right time for their introduction. The paper introduces Orca - an open-source component-based software engineering framework proposed for mobile robotics with an associated repository of free, reusable components for building mobile robotic systems.

This paper presents an approach of integrating human operators into a sensor network formed by a heterogeneous team of unmanned air and ground vehicles. Several objectives of human-network interaction are identified. The main focus of this work is on human-to-network information flow, i.e. human operators are regarded as information sources. It is argued that operators should make raw observations which are converted into the sensor network's common representation by a probabilistic model. The concepts are discussed in the context of an outdoor sensor network under development. Human operators contribute geometric feature information in the form of range and bearing observations. Visual feature properties are specified via meaningful class labels. A sensor model, represented as a Bayesian network, translates label observations into the system's representation. The model is also used to classify features as observed by robotic sensors.