We present an Unmanned Aerial Vehicle (UAV) human-robot interaction simulation environment to evaluate operator situational awareness (SA) during Beyond Visual Line of Sight (BVLOS) operations. We have developed a modular simulator that functions as a testbed for the design of custom scenarios and the study of human-robot interaction. This simulator facilitates data collection and allows for the comparative analysis of various interface designs in terms of their effectiveness in enhancing users' acquisition of situational awareness. We implemented three operational scenarios and corresponding assessment questions via real-time SA probes specifically tailored for teleoperator roles with a combination of flight data and user interaction logs. We conducted initial usability tests and received feedback on the potential use of the setup in training teleoperators and designing UAV scenarios. The insights obtained from these pilot evaluations and further, planned, studies can benefit interface design processes that improve operator situational awareness.

Simulation of conflict situations for autonomous driving research is crucial for understanding and managing interactions between Automated Vehicles (AVs) and human drivers. This paper presents a set of exemplary conflict scenarios in CARLA that arise in shared autonomy settings, where both AVs and human drivers must navigate complex traffic environments. We explore various conflict situations, focusing on the impact of driver behavior and decision-making processes on overall traffic safety and efficiency. We build a simple extendable toolkit for situation awareness research, in which the implemented conflicts can be demonstrated.

We propose a novel multi-pose loss function to train a neural network for 6D pose estimation, using synthetic data and evaluating it on real images. Our loss is inspired by the VSD (Visible Surface Discrepancy) metric and relies on a differentiable renderer and CAD models. This novel multi-pose approach produces multiple weighted pose estimates to avoid getting stuck in local minima. Our method resolves pose ambiguities without using predefined symmetries. It is trained only on synthetic data. We test on real-world RGB images from the T-LESS dataset, containing highly symmetric objects common in industrial settings. We show that our solution can be used to replace the codebook in a state-of-the-art approach. So far, the codebook approach has had the shortest inference time in the field. Our approach reduces inference time further while a) avoiding discretization, b) requiring a much smaller memory footprint and c) improving pose recall.3

To make autonomous, affordable ships feasible in the real world, they must be capable of safely navigating without fully relying on GPS, high-resolution 3D maps, or high-performance navigation sensors. We suggest a method for estimating the position using affordable navigation sensors (compass and speed log or inertial navigation sensor), sensors used for perception of the environment (cameras, echo sounder, magnetometer), and publicly available maps (sea charts and magnetic intensity anomalies maps). A real-world field trial has shown that the proposed fusion mechanism provides accurate and robust navigation, applicable for affordable autonomous ships.

We compared three different Graphical User Interfaces (GUI) that we have designed and implemented to enable human supervision of an unmanned ship. Our findings indicate that a 3D GUI presented either on a screen or in a Virtual Reality (VR) setting provides several objective and subjective benefits compared to a Baseline GUI representing traditional tools.

We assume that an intuitive means for the specification, re-use, modification and transfer of synchronized motions-both regarding the two arms of a dual-arm robotic system, as well as regarding the coordination of a user and a robot-is key in interactive and collaborative settings as they are currently targeted for industrial applications. We show, how our knowledge based approach to end-user programming of synchronized motions and other generalizable, robot-agnostic skills can support such specification of coordinated actions between two robot arms and explain how that could be extended to include coordination with a human user. We describe the underlying ontologies and possibilities to populate those with an interface for intuitive programming, and show the generality of our approach through a task transfer between different kinematics (different robots), where the user is supported through underlying reasoning about the fulfillment of certain parameters or constraints for the involved skills.

This paper presents a study on iconic programming support for mainly position-based lead-through programming of an ABB YuMi collaborative robot. A prototype tool supporting a hybrid programming and execution mode was developed and evaluated with 21 non-expert users with varying programming and robotics experience. We also present a comparison of the programming times for an expert robot programmer using traditional tools versus the new tool. The expert programmed the same tasks in 1/5 of the time compared to traditional tools and the non-experts were able to program and debug a LEGO building task using the robot within 30 minutes.

The findings from a user study regarding particular observable "interaction patterns" in the interaction during a "guided tour" are summarized in this paper.

In this paper, we describe a concise method for the feature-based representation ofregionsin an indoor environment and show how it can also be applied for door-passage-independent detection of transitions betweenregionsto improve communication with a human user.

In scenarios that require a close collaboration and knowledge transfer between inexperienced users and robots, the ldquolearning by interactingrdquo paradigm goes hand in hand with appropriate representations and learning methods. In this paper we discuss a mixed initiative strategy for robotic learning by interacting with a user in a joint map acquisition process. We propose the integration of an environment representation approach into our interactive learning framework. The environment representation and mapping system supports both user driven and data driven strategies for the acquisition of spatial information, so that a mixed initiative strategy for the learning process is realised. We evaluate our system with test runs according to the scenario of a guided tour, extending the area of operation from structured laboratory environment to less predictable domestic settings.

Service robots designed for domestic settings need to navigate in an environment that they have to share with their users. Thus, they have to be able to report their current state and whereabouts in a way that is comprehensible for the user. Pure metric maps do not usually correspond to the understanding of the environment a user would provide. Thus, the robotic map needs to be integrated with the human representation. With our framework for Human Augmented Mapping we aim to deal with this issue and assume a guided tour as basis for an initial mapping process. During such a tour the robotic system needs to be able to detect significant changes in its environment representation - structural ambiguities - to be able to invoke a clarification discourse with the user. In this paper we present our approach to the detection of such ambiguities, that is independent from prior specification and training of particular spatial categories. We evaluate our method on data sets obtained during several runs in indoor environments in the context of a guided tour scenario.

An annotated data set is presented meant to help researchers in developing, evaluating, and comparing various approaches in robotics for building space representations appropriate for communicating with humans. The data consist of omnidirectional images, laser range scans, sonar readings, and robot odometry. A set of base-level human spatial concepts is used to annotate the data.

An important goal for research on service robots is the cooperation of a human and a robot as team. A service robot in a domestic environment needs to build a representation of its future workspace that corresponds to the human user's understanding of these surroundings. But it also needs to apply this model about the "where" and "what" in its current interaction to allow communication about objects and places in a human-adequate way. In this paper we present the integration of a hierarchical robotic mapping system into an interactive framework controlled by a dialog system. The goal is to use interactively acquired environment models to implement a robot with interaction aware behaviors. A major contribution of this work is a three-level hierarchy of spatial representation affecting three different communication dimensions. This hierarchy is consequently applied in the design of the grounding-based dialog, laser-based topological mapping, and an objects attention system. We demonstrate the benefits of this integration for learning and tour guiding in a human- comprehensible interaction between a robot and its user in a home-tour scenario. The enhanced interaction capabilities are crucial for developing a new generation of robots that will be accepted not only as service robots but also as robot companions.

Interacting with a domestic service robot implies the existence for a joint environment model for a user and a robot. We present a pilot study that investigates, how humans present a familiar environment to a mobile robot. Results from this study are used to evaluate a generic environment model for a service robot that can be personalised by interaction.

Co-presence and embodied interaction are two fundamental characteristics of the command and control situation for service robots. This paper presents a study of spatial distances and orientation of a robot with respect to a human user in an experimental setting. Relevant concepts of spatiality from social interaction studies are introduced and related to human-robot interaction (HRI). A Wizard-of-Oz study quantifies the observed spatial distances and spatial formations encountered. However, it is claimed that a simplistic parameterization and measurement of spatial interaction misses the dynamic character and might be counter-productive in the design of socially appropriate robots

Service robots designed for domestic settings need to navigate in an environment that they have to share with their users. Thus, they have to be able to report their current state and whereabouts in a way that is comprehensible for the user. Pure metric maps do not usually correspond to the understanding of the environment a user would provide. Thus, the robotic map needs to be integrated with the human representation. This paper describes our framework of human augmented mapping that allows us to achieve this integration. We propose further a method to specify and represent regions that relate to a user's view on the environment. We assume an interactive setup for the specification of regions and show the applicability of our method in terms of distinctiveness for space segmentation and in terms of localisation purposes

Human interaction with a service robot requires a shared representation of the environment for spoken dialogue and task specification where names used for particular locations are depending on personal preferences. A question is how such human oriented models can be tied to the geometric robotic models needed for precise localisation and navigation. We assume that this integration can be based on the information potential users give to a service robot about its working environment. We further believe that this information is best given in an interactive setting (a "guided tour") in this particular environment. This paper presents a pilot study that investigates how humans present a familiar environment to a mobile robot. The study is set up within our concept of human augmented mapping, for which we assume an initial "guided tour" scenario to teach a robot its environment. Results from this pilot study are used to validate a proposed generic environment model for a service robot

This paper presents an in-depth analysis from a human robot interaction (HRI) study on spatial positioning and interaction episode transitions. Subjects showed a living room to a robot to teach it new places and objects. This joint task was analyzed with respect to organizing strategies for interaction episodes. Noticing the importance of transitions between interaction episodes, small adaptive movements in posture were observed. This finding needs to be incorporated into HRI modules that plan and execute robots' spatial behavior in interaction, e.g., through dynamic adaptation of spatial formations and distances depending on interaction episode

This paper presents a multiple target tracking approach for following and passing persons in the context of human-robot interaction. The general purpose for the approach is the use in human augmented mapping. This concept is presented and it is described how navigation and person following are subsumed under it. Results from experiments under test conditions and from data collected during a user study are also provided.

In this paper, we present an initial design of an interactive interface for a service robot based on multisensor fusion. We show how the integration of speech, vision and laser range data can be performed using a high level of abstraction. Guided by a number of scenarios commonly used in a service robot framework, the experimental evaluation will show the benefit of sensory integration which allows the design of a robust and natural interaction system using a set of simple perceptual algorithms.