The advancement of AI is driving the integration of robots into everyday environments. The acceptance of these robots not only depends on direct users, but also on others who share these spaces, often referred to as bystanders or non-users. A frequently discussed prerequisite for acceptance is the proper handling of personal information as robots are equipped with means for environmental awareness, data inference, and human interaction. Although bystanders are not the main target of such processing, they can be affected by robot operation. Despite its significance, bystander privacy concerns have received limited attention in prior robotics research. In this paper, we address bystander privacy in the context of robots operating in everyday environments. We conduct a scoping review of bystander privacy issues associated with related technologies exhibiting agentic qualities comparable to robots. We analyze how agency may reshape conventional attributions of actor roles and transmission principles within the established privacy framework of Contextual Integrity. This allows us to derive transferable insights about privacy expectations as well as unique opportunities and open research issues for robots in public spaces.

Many human activities are performed in groups—making decisions in workplace meetings, cooperating on a sports team, or meeting with friends for dinner. All these activities involve complex conditions and interaction processes that influence their outcomes in terms of performance, personal goals, and group objectives. As robots are increasingly being positioned within groups, improving these outcomes has emerged as an important application area in social robotics, particularly through robotic facilitation. Robot facilitators aim to elicit positive changes by deliberately influencing group processes. While research in this field has demonstrated that robots can effectively influence interpersonal dynamics, there remains a notable gap in consolidating these insights into a coherent understanding that can guide the design and development of better facilitators. We present a scoping review of literature targeting changes in interactions between multiple humans that are driven by intentional actions from robotic agents. To identify key considerations for the design of robot facilitators, we take inspiration from human group research theories to organize existing approaches. Our review includes 108 publications that meet our inclusion criteria, yielding 85 distinct application targets for group facilitation using robots. Based on the identified instances, we extract categories of possible application targets and a set of design concepts that can guide future work on robotic group facilitators.

Robots can play a role in mediating human group interactions.This study examines how robot gestures affect the perception of a human group member's personality, role in the group, and contribution.In a vignette study (n=96), participants imagined being in a group discussion and watched a short video of another group member presenting an argument.In one condition (affirmative gesture), a robot nodded while the member spoke; in the other, it shook its head (negating gesture).A control condition featured no robot.The affirmative gesture enhanced perceptions of the speaker's personality and role in the group, though their contribution was not affected.The negating gesture showed no adverse effects.Additionally, participants perceived the robot as a group member when it nodded but as an onlooker when it shook its head.This suggests that positive robot gestures can improve group dynamics by fostering favorable interpersonal perceptions.

In interactions between automated vehicles (AVs) and crossing pedestrians, modeling implicit vehicle communication is crucial. In this work, we present a combined prediction and planning approach that allows to consider the influence of the planned vehicle behavior on a pedestrian and predict a pedestrian's reaction. We plan the behavior by solving two consecutive optimal control problems (OCPs) analytically, using variational calculus. We perform a validation step that assesses whether the planned vehicle behavior is adequate to trigger a certain pedestrian reaction, which accounts for the closed-loop characteristics of prediction and planning influencing each other. In this step, we model the influence of the planned vehicle behavior on the pedestrian using a probabilistic behavior acceptance model that returns an estimate for the crossing probability. The probabilistic modeling of the pedestrian reaction facilitates considering the pedestrian's costs, thereby improving cooperative behavior planning. We demonstrate the performance of the proposed approach in simulated vehicle-pedestrian interactions with varying initial settings and high-light the decision making capabilities of the planning approach.

Interaction modeling plays a huge role in understanding human behavior in traffic. This is especially relevant when it comes to interactions between vehicles and vulnerable road users such as pedestrians. Thus, pedestrian intention prediction is an ongoing field of research in order to understand the pedestrians' decision making. Most state-of-the-art prediction frameworks are trained on large-scale datasets and evaluated with respect to acknowledged benchmarks. These datasets lack the ability to account for the reciprocal nature of interactions between pedestrians and vehicles and the effects of the two agents influencing each other. In this work, we demonstrate first steps towards assessing pedestrian prediction algorithms within realistic scenarios including the interaction effects arising from its interplay with a planning component. For this, we validate an existing prediction framework trained on benchmark datasets with situations from a virtual reality (VR) pedestrian-vehicle co-simulator that allows us to include the effect of vehicle planning on pedestrian behavior. We evaluate the performance of the prediction framework comparing data from pre-recorded real-world datasets with data from our co-simulation study and conduct an ablation analysis to identify the most important features for pedestrian intention prediction. The results highlight the significance of pedestrian action and proximity to the road.

Abstract Research on multi-agent interaction involving humans is still in its infancy. Most approaches have focused on environments with collaborative human behavior or a small, defined set of situations. When deploying robots in human-inhabited environments in the future, the diversity of interactions surpasses the capabilities of pre-trained collaboration models. ”Coexistence” environments, characterized by agents with varying or partially aligned objectives, present a unique challenge for robotic collaboration. Traditional reinforcement learning methods fall short in these settings. These approaches lack the flexibility to adapt to changing agent counts or task requirements without undergoing retraining. Moreover, existing models do not adequately support scenarios where robots should exhibit helpful behavior toward others without compromising their primary goals. To tackle this issue, we introduce a novel framework that decomposes interaction and task-solving into separate learning problems and blends the resulting policies at inference time using a goal inference model for task estimation. We create impact-aware agents and linearly scale the cost of training agents with the number of agents and available tasks. To this end, a weighting function blending action distributions for individual interactions with the original task action distribution is proposed. To support our claims we demonstrate that our framework scales in task and agent count across several environments and considers collaboration opportunities when present. The new learning paradigm opens the path to more complex multi-robot, multi-human interactions.

Abstract Significant multi-agent advances addressing the challenge of learning policies for acting in ad hoc teamwork have been made. In ad hoc teamwork, a team of agents must cooperate effectively without prior coordination or communication. Many existing approaches, however, struggle to perform well in open environments where the setting can change significantly during deployment. This paper presents a new reinforcement learning approach to tackle collaboration in open environments controlling one agent with a changing number of distinct other agents, each with an individual task. The approach uses policy blending based on an online goal inference module and a collection of learned policies modeling the individual interaction impact between the agent and populations of partners with different tasks. Blending is done using the estimated goals of others and a posterior-based action blending with entropy adjustment and regularization. Our approach addresses issues of existing policy blending mechanisms, such as handling conflicting modes in action distributions leading to oscillation and instability and adapting to uncertain states dynamically. In experiments in two collaborative open environments based on Overcooked and Level-based Foraging, our approach outperforms a baseline learner, trained with the joint reward of all agents, across changes to both agents and tasks. Ablation studies further highlight the importance of our posterior-based blending mechanism to achieve high rewards as well as the provided goal weighting. The proposed approach provides an important step towards the application of reinforcement learning to AI assistance beyond strictly closed worlds and towards more realistic scenarios.

This paper explores how a gesture-based robot influences human-human interaction in group ideation. The robot was mounted on a whiteboard and responded with six different gestures (e.g., nodding, following speakers with gaze) to specific situations. We coded the participants’ interactions from videos and gathered their experience through post-session interviews. The most frequently invoked robot behavior was following the speaker with gaze. As a result, participants felt socio-emotionally supported and responded by moving the ideation ahead (individual level) and consensus-building (group level). In fact, the groups with the robot showed more consensus-building than the two reference groups without the robot. Participants had different views on the role of the robot in the group, such as active outsider, supportive group member, or assistant. The latter tried to use the robot as decision-support. All in all, to include a robot to mediate human groups seems a promising future application domain.

Interactions between humans and automated vehicles (AVs) will become increasingly common in future traffic. Cooperative behavior enables comfortable and efficient resolutions of such interactions through joint actions. Thus, AVs need the ability to communicate their intention and planned behavior to outside road users to facilitate cooperation among the interaction partners. Many recent studies suggest external human-machine-interfaces (eHMIs) as a feasible way to realize the communication between AVs and other road users. These studies often focus on the design of eHMIs and evaluate their usability based on subjective measures from the perspective of the outside road user. There is only little research on the objective benefits of explicit external communication on the resolution of interactions with AVs. The decision making behind the activation of the communication and the consideration of the actual interaction in the communication are still open research topics. In this work, we present a communication framework that facilitates cooperation between an AV and pedestrians by communicating the vehicle’s yielding intention to give right of way. We demonstrate the proposed system’s ability to improve the joint utility as well as the driving comfort by empirically evaluating simulated interactions between AVs and pedestrians. The results indicate that timely communication leads to more efficient and more predictable pedestrian-vehicle interactions.

Research on multi-agent interaction involving both multiple artificial agents and humans is still in its infancy. Most recent approaches have focused on environments with collaboration-focused human behavior, or providing only a small, defined set of situations. When deploying robots in human-inhabited environments in the future, it will be unlikely that all interactions fit a predefined model of collaboration, where collaborative behavior is still expected from the robot. Existing approaches are unlikely to effectively create such behaviors in such “coexistence” environments. To tackle this issue, we introduce a novel framework that decomposes interaction and task-solving into separate learning problems and blends the resulting policies at inference time. Policies are learned with maximum entropy reinforcement learning, allowing us to create interaction-impact-aware agents and scale the cost of training agents linearly with the number of agents and available tasks. We propose a weighting function covering the alignment of interaction distributions with the original task. We demonstrate that our framework addresses the scaling problem while solving a given task and considering collaboration opportunities in a co-existence particle environment and a new cooking environment. Our work introduces a new learning paradigm that opens the path to more complex multi-robot, multi-human interactions.

Many everyday traffic situations require cooperation among traffic participants to establish the order in which they pass a shared part of the road. Behavior planners which do not take this cooperative aspect into account properly struggle to find efficient solutions if the situation is nontrivial. Improper modelling may lead to overly aggressive or conservative behavior. In this paper, we propose an extension to state-of-the-art systems that enables behavior planners to efficiently cope with narrow passage scenarios even without car-to-car communication. The extended system is based on an asymmetry measure which takes the shared traffic space and the cooperation partners into account. This measure is then used to continuously predict which potential outcome is more likely to occur, to infer the assumed strategy of the cooperation partner, and to match the own strategy accordingly. Experiments show that the proposed system significantly reduces the cumulative passing time of the shared traffic space as compared to baseline systems. The resulting solutions are robust against variations in the behavior of both cooperation partners, and explicitly account for oblivious traffic participants which behave uncooperatively.

For human-robot cooperation, inferring a hu-man's cognitive state is very important for an efficient and natural interaction. Similar to human-human cooperation, understanding what the partner plans and knowing, if he is situation aware, is necessary to prevent collisions, offer support at the right time, correct mistakes before they happen or choose the best actions for oneself as early as possible. We propose a model-based belief filter to extract relevant aspects of a human's mental state online during cooperation. It performs inference based on human actions and its own task knowledge, modeling cognitive processes like perception and action selection. In contrast to most prior work, we explicitly estimate the human belief instead of inferring only a single mode or intention. Since this is a double inference process, we focus on representing the human estimates of environmental state and task as well as corresponding uncertainties. We designed a human-robot cooperation experiment that allowed for a variety of cognitive states of both agents and collected data to test and evaluate the proposed belief filter. The results are promising, as our system can be used to provide reasonable predictions of the human action and insights into his situation awareness. At the same time it is inferring interpretable information about the underlying cognitive states - A belief about the human's belief about the environment.

The detection of road area in the surroundings of the ego-vehicle is a key issue for modern ADAS. Camera-based direct detection systems are able to reliably accomplish this task only within a limited spatial range or in simple environments, due to hardware limitations and unfavorable situations, like shadows or occlusions. In complex environments, like inner city, traffic is a real issue, since the mere presence of other cars can significantly restrict the field of view of the ego-vehicle. In order to extend the spatial range of road detection, indirect detection systems are a viable resource. They can complement state-of-the-art direct detection systems and help motion control systems to plan smooth and stable trajectories. In this paper we propose a probabilistic grid-based approach based on the interpretation of the motion of other vehicles in the scene. The approach uses the position and velocity of those vehicles in order to infer the presence and location of occluded road area. We will show that this approach can complement an already established feature-based detection system, taking advantage of those situations that are the most challenging for the latter. Evaluations on real-world scenes show that the union between this approach and direct road detection significantly extends the spatial range of detection, thus is able to provide a motion control system a longer horizon for planning trajectories.

Detecting the road terrain ahead of the ego-vehicle is an important issue for modern driver assistance systems. In particular, vehicle motion planning in inner city environment requires the detection of road terrain up to 3 seconds in advance. State-of-the-art visual road terrain detection systems have a hard time fulfilling this task, due to their limited range and the presence of occlusions (other vehicles, buildings, etc.), which are expected to occur often in complex scenarios. However, those systems provide significant information where the conditions are favorable (proximity to the ego-vehicle, no occlusions). Therefore, a complementary approach is needed to enhance already existing and established detection systems. In this paper we propose a probabilistic grid-based approach based on the observation and interpretation of other vehicles' behavior in the scene. It exploits their movements in order to infer the presence and location of occluded road surface. We will show that this approach presents various advantages over current visual road terrain detection systems, especially in those situations that are the most challenging for them. We will illustrate how our approach is designed to work in concert also with other available resources, e.g. offline road maps. Qualitative results on real-world scenes taken from the KITTI benchmark[11] demonstrate that the fusion of this method with visual road terrain detection can potentially extend our time horizon well over the 3 seconds mentioned above. Finally, we will show how our approach is planned to develop into a semantically enriched representation of the road, including road properties such as availability, lanes and directions.

Before taking a decision, a driver anticipates the future behavior of other traffic participants. However, if a driver is inattentive or overloaded, he may fail to consider relevant information. This can lead to bad decisions and potentially result in an accident. A computational system that is designed to anticipate other traffic participants' behaviors could assist the driver in his decision making by sending him an early warning when a risk of collision is predicted. Existing research in this area usually focuses on only one of two aspects, i.e., quality or scope. Quality refers to the ability to warn a driver early before a dangerous situation happens. Scope is the diversity of scenarios in which the approach can work. In general, we see methods targeting a broad scope but showing low quality, with others having a narrow scope but high quality. Our goal is to create a system with high quality and high scope. To achieve this, we propose an architecture that combines classifiers to predict behaviors for many scenarios. In this paper, we will first introduce the generic concept of such a system applicable to highway and inner-city scenarios. We will show that a combination of general and specific classifiers is a solution to improve quality and scope based on a concrete implementation for lane-change prediction in highway scenarios.

To drive safely, a good driver observes his surroundings, anticipates the actions of other traffic participants and then decides for a maneuver. But if a driver is inattentive or overloaded he may fail to include some relevant information. This can then lead to wrong decisions and potentially result in an accident. In order to assist a driver in his decision making, Advanced Driver Assistance Systems (ADAS) are becoming more and more popular in commercial cars. The quality of these existing systems compared to an experienced driver is weak, because they rely purely on physical observation and thus react shortly before an accident. For an earlier warning of the driver behavior prediction is used. We classify existing research in this area with respect to two aspects: quality and scope. Quality means the ability to warn a driver early before a dangerous situation. Scope means the diversity of scenes in which the approach can work. In general we see two tendencies, methods targeting for broad scope but having low quality and those targeting for narrow scope but high quality. Our goal is to have a system with high quality and wide scope. To achieve this, we propose a system that combines classifiers to predict behaviors for many scenarios. To show that a combination of general and specific classifiers is a solution to improve quality and scope, this paper will introduce the generic concept of our system followed by a concrete implementation for lane change prediction for highway scenarios.

Advanced Driver Assistance Systems (ADAS) are becoming more and more popular. Many of these systems though are limited to specific scenes and often detect risky situations very late so they can only mitigate accidents. These effects are mainly caused by the use of simple physical prediction methods, e.g. to estimate the time-to-contact with another vehicle. In this paper we show an ADAS that extends the functionality of physical collision warnings by additionally estimating potential risks based on implicit predictions. As an example we demonstrate the use of vehicle orientation information for classifying situations. Through this, we can in particular assess the risk of static cars, for which physical prediction does not apply, but which can nevertheless easily cause an accident if they start moving into our driving corridor. The proposed system is evaluated online in a test car and is shown to reliably detect classical risky situations as well as those involving static cars.