The task of supporting a human operator to understand generated plans, and to explore the plan space, are important problems in automated planning. In this work, we consider the problem of plan explainability and plan space exploration in underwater autonomous vehicle missions. In this context, concepts that are useful for querying the system, such as distance and duration, will not necessarily map directly onto components of the planning model, such as actions. To overcome this difficulty, we focus on an important substructure of these problems: the multi-agent spatial-temporal (MAST) structure. Using this structure, we define a collection of model extensions, which include additional concepts relevant to the MAST structure. We then consider the problem of user-guided plan space exploration, and identify useful query types in this domain, including user queries based on numeric functions. These queries can make use of the extended model, allowing the user to directly reference the new concepts. In an empirical study, we demonstrate the use of the new structure within queries, and compare the new query types in our target domain, and in benchmark domains with the MAST structure. Finally, we report on a qualitative user study, where we investigate the use of these new structural concepts in underwater autonomous vehicle scenarios. Our study indicates that the extended concepts can be used in user queries and agent responses, enabling the user to better communicate their intent in shaping mission objectives, and supporting explanations with more relevant information.

• First beam image dataset capturing diverse optical-alignment patterns and parameters • Optical resonator alignment cast as a pairwise beam-pattern regression task • Two-stage model with both feature interaction and refinement for coarse-to-fine alignment • Trained on one device, the model generalizes to another without re-training • Achieves high accuracy with real-time inference on embedded edge hardware Accurate alignment of a laser resonator is essential for upscaling industrial laser manufacturing and precision processing. However, traditional manual or semi-automatic methods depend heavily on operator expertise, and struggle with the interdependence among multiple alignment parameters. To tackle this, we introduce the first real-world image dataset for automatic laser resonator alignment, collected on a laboratory-built resonator setup. It comprises over 6,000 beam profiler images annotated with four key alignment parameters (intracavity iris aperture diameter, output coupler pitch and yaw actuator displacements, and axial position of the output coupler), with over 500,000 paired samples for data‐driven alignment. Given a pair of beam profiler images exhibiting distinct beam patterns under different configurations, the system predicts the control-parameter changes required to realign the resonator. Leveraging this dataset, we propose a novel two-stage deep learning framework for automatic resonator alignment. In Stage 1, a multi-scale CNN augmented with cross-attention and correlation-difference modules, extracts features and outputs an initial coarse prediction of alignment parameters. In Stage 2, a feature-difference map is computed by subtracting the paired feature representations and fed into an iterative refinement module to correct residual misalignments. The final prediction combines coarse and refined estimates, integrating global context with fine-grained corrections for accurate inference. Experiments on our dataset and a different instance of the same physical system from which the CNN was trained suggest superior accuracy and practicality to manual alignment.

In autonomous vehicle mission planning, supporting human operators to understand and influence the decision-making process is crucial for building the operator’s trust and establishing effective collaboration. However, it has been observed that human and agent representations will typically not align. As a consequence, concepts that are useful for effective human-agent communication, will not necessarily feature in the agent’s representation. Focusing on specific spatial-temporal concepts, we define automatic model extensions, which can introduce these additional concepts. We report on a qualitative user study, where we investigate the use of these new structural concepts in underwater autonomous vehicle scenarios. Our study indicates that the extended concepts can be used in user queries and agent responses, enabling the user to better communicate their intent in shaping mission objectives, and supporting explanations with more relevant information.

Studying Human-Robot Interaction over time can provide insights into what really happens when a robot becomes part of people’s everyday lives. “In the Wild” studies inform the design of social robots, such as for the service industry, to enable them to remain engaging and useful beyond the novelty effect and initial adoption. This paper presents an “In the Wild” experiment where we explored the evolution of interaction between users and a Robo-Barista. We show that perceived trust and prior attitudes are both important factors associated with the usefulness, adaptability and likeability of the Robo-Barista. A combination of interaction features and user attributes are used to predict user satisfaction. Qualitative insights illuminated users’ Robo-Barista experience and contribute to a number of lessons learned for future long-term studies.

Social Robots in human environments need to be able to reason about their physical surroundings while interacting with people. Furthermore, human proxemics behaviours around robots can indicate how people perceive the robots and can inform robot personality and interaction design. Here, we introduce Charlie, a situated robot receptionist that can interact with people using verbal and non-verbal communication in a dynamic environment, where users might enter or leave the scene at any time. The robot receptionist is stationary and cannot navigate. Therefore, people have full control over their personal space as they are the ones approaching the robot. We investigated the influence of different apparent robot personalities on the proxemics behaviours of the humans. The results indicate that different types of robot personalities, specifically introversion and extroversion, can influence human proxemics behaviours. participants maintained shorter distances with the introvert robot receptionist, compared to the extrovert robot. Interestingly, we observed that human-robot proxemics were not the same as typical human-human interpersonal distances, as defined in the literature. We therefore propose new proxemics zones for human-robot interaction.

Abstract Cognitive load has been widely studied to help understand human performance. It is desirable to monitor user cognitive load in applications such as automation, robotics, and aerospace to achieve operational safety and to improve user experience. This can allow efficient workload management and can help to avoid or to reduce human error. However, tracking cognitive load in real time with high accuracy remains a challenge. Hence, we propose a framework to detect cognitive load by non-intrusively measuring physiological data from the eyes and heart. We exemplify and evaluate the framework where participants engage in a task that induces different levels of cognitive load. The framework uses a set of classifiers to accurately predict low, medium and high levels of cognitive load. The classifiers achieve high predictive accuracy. In particular, Random Forest and Naive Bayes performed best with accuracies of 91.66% and 85.83% respectively. Furthermore, we found that, while mean pupil diameter change for both right and left eye were the most prominent features, blinking rate also made a moderately important contribution to this highly accurate prediction of low, medium and high cognitive load. The existing results on accuracy considerably outperform prior approaches and demonstrate the applicability of our framework to detect cognitive load.

We present a demonstration of a Robo-Barista: a social robot that takes hot beverage orders through verbal interaction and completes them via a Bluetooth enabled coffee machine. The demonstration is highly robust and it is the intention that this could be installed as a permanent feature, enabling “In the Wild” experimentation and long term studies. In the demonstration video, we show a user interacting with a Furhat robot to order a coffee. The robot has a novel architecture that allows it to exhibit both verbal and non-verbal cues, such as shared attention and chitchat. Furthermore, it is enabled with a unique tiredness detector based on visual facial features.

As robots take on roles in our society, it is important that their appearance, behaviour and personality are appropriate for the job they are given and are perceived favourably by the people with whom they interact. Here, we provide an extensive quantitative and qualitative study exploring robot personality but, importantly, with respect to individual human traits. Firstly, we show that we can accurately portray personality in a social robot, in terms of extroversion-introversion using vocal cues and linguistic features. Secondly, through garnering preferences and trust ratings for these different robot personalities, we establish that, for a Robo-Barista, an extrovert robot is preferred and trusted more than an introvert robot, regardless of the subject’s own personality. Thirdly, we find that individual attitudes and predispositions towards robots do impact trust in the Robo-Baristas, and are therefore important considerations in addition to robot personality, roles and interaction context when designing any human-robot interaction study.

The ability to impute mental states to oneself or others, or Theory of Mind (ToM), has been intrinsically linked to trust between humans. However, less is known about how a robot mimicking ToM affects users’ trust and behaviour. We explore this through an online study, where we compare three robot personas in a cooperative maze navigation task: one neutral, one that explains its reasoning in technical terms, and one that mimics ToM. We show that ToM influences human decision-making behaviour and trust in a way that makes it more appropriate with respect to the competencies of the robot. This is key for human-robot collaboration and adoption of robotics moving forward.

Public perceptions of Robotics and Artificial Intelligence (RAI) are important in the acceptance, uptake, government regulation and research funding of this technology. Recent research has shown that the public's understanding of RAI can be negative or inaccurate. We believe effective public engagement can help ensure that public opinion is better informed. In this paper, we describe our first iteration of a high throughput in-person public engagement activity. We describe the use of a light touch quiz-format survey instrument to integrate in-the-wild research participation into the engagement, allowing us to probe both the effectiveness of our engagement strategy, and public perceptions of the future roles of robots and humans working in dangerous settings, such as in the off-shore energy sector. We critique our methods and share interesting results into generational differences within the public's view of the future of Robotics and AI in hazardous environments. These findings include that older peoples' views about the future of robots in hazardous environments were not swayed by exposure to our exhibit, while the views of younger people were affected by our exhibit, leading us to consider carefully in future how to more effectively engage with and inform older people.

Autonomous vehicles and robots are increasingly being deployed to remote, dangerous environments in the energy sector, search and rescue and the military. As a result, there is a need for humans to interact with these robots to monitor their tasks, such as inspecting and repairing offshore wind-turbines. Conversational Agents can improve situation awareness and transparency, while being a hands-free medium to communicate key information quickly and succinctly. As part of our user-centered design of such systems, we conducted an in-depth immersive qualitative study of twelve marine research scientists and engineers, interacting with a prototype Conversational Agent. Our results expose insights into the appropriate content and style for the natural language interaction and, from this study, we derive nine design recommendations to inform future Conversational Agent design for remote autonomous systems.

There are many challenges when it comes to deploying robots remotely including lack of operator situation awareness and decreased trust. Here, we present a conversational agent embodied in a Furhat robot that can help with the deployment of such remote robots by facilitating teaming with varying levels of operator control.

Automated tools are increasingly being used to generate highly engaging concept maps as an aid to strategic planning and other decision-making tasks. Unless stakeholders can understand the principles of the underlying layout process, however, we have found that they lack confidence and are therefore reluctant to use these maps. In this paper, we present a qualitative study exploring the effect on users' confidence of using data-driven explanation mechanisms, by conducting in-depth scenario-based interviews with ten participants. To provide diversity in stimulus and approach we use two explanation mechanisms based on projection and agglomerative layout methods. The themes exposed in our results indicate that the data-driven explanations improved user confidence in several ways, and that process clarity and layout density also affected users' views of the credibility of the concept maps. We discuss how these factors can increase uptake of automated tools and affect user confidence.

Autonomous systems in remote locations have a high degree of autonomy and there is a need to explain what they are doing and why , in order to increase transparency and maintain trust. This is particularly important in hazardous, high-risk scenarios. Here, we describe a multimodal interface, MIRIAM, that enables remote vehicle behaviour to be queried by the user, along with mission and vehicle status. These explanations, as part of the multimodal interface, help improve the operator's mental model of what the vehicle can and can't do, increase transparency and assist with operator training.

Autonomous systems are designed to carry out activities in remote, hazardous environments without the need for operators to micro-manage them. It is, however, essential that operators maintain situation awareness in order to monitor vehicle status and handle unforeseen circumstances that may affect their intended behaviour, such as a change in the environment. We present MIRIAM, a multimodal interface that combines visual indicators of status with a conversational agent component. This multimodal interface offers a fluid and natural way for operators to gain information on vehicle status and faults, mission progress and to set reminders. We describe the system and an evaluation study providing evidence that such an interactive multimodal interface can assist in maintaining situation awareness for operators of autonomous systems, irrespective of cognitive styles.

As unmanned vehicles become more autonomous, it is important to maintain a high level of transparency regarding their behaviour and how they operate.This is particularly important in remote locations where they cannot be directly observed.Here, we describe a method for generating explanations in natural language of autonomous system behaviour and reasoning.Our method involves deriving an interpretable model of autonomy through having an expert 'speak aloud' and providing various levels of detail based on this model.Through an online evaluation study with operators, we show it is best to generate explanations with multiple possible reasons but tersely worded.This work has implications for designing interfaces for autonomy as well as for explainable AI and operator training.

In previous work we developed a method for interior designers to receive image-based feedback about a crowd's emotions when viewing their designs. Although the designers clearly desired a service which provided the new style of feedback, we wanted to find out if an internet crowd would enjoy, and become engaged in, giving emotion feedback this way. In this paper, through a mixed methods study, we expose whether and why internet users enjoy giving emotion feedback using images compared to responding with text. We measured the participants' cognitive styles and found that they correlate with the reported utility and engagement of using images. Those more visual than they are verbal were more engaged by using images to express emotion compared to text. Enlightening qualitative insights reveal, surprisingly, that half of our participants have an appetite for expressing emotions this way, value engagement over clarity, and would use images for emotion feedback in contexts other than design feedback.

Research into creating visualisations that organise ideas into concise concept maps often focuses on implicit mathematical and statistical theories which are built around algorithmic efficacy or visual complexity. Although there are multiple techniques which attempt to mathematically optimise this multi-dimensional problem, it is still unknown how to create concept maps that are immediately understandable to people. In this paper, we present an in-depth qualitative study observing the behaviour and discussing the strategy used by non-expert participants to create, interact, update and communicate a concept map that represents a collection of research ideas. Our results show non-expert individuals create concept maps differently to visualisation algorithms. We found that our participants prioritised narrative, landmarks, abstraction, clarity, and simplicity. Finally, we derive design recommendations from our results which we hope will inspire future algorithms that automatically create more usable and compelling concept maps better suited to the natural behaviours and needs of users.

We present MIRIAM (Multimodal Intelligent inteRactIon for Autonomous systeMs), a multimodal interface to support situation awareness of autonomous vehicles through chat-based interaction. The user is able to chat about the vehicle's plan, objectives, previous activities and mission progress. The system is mixed initiative in that it pro-actively sends messages about key events, such as fault warnings. We will demonstrate MIRIAM using SeeByte's SeeTrack command and control interface and Neptune autonomy simulator.

Imagery and language are often seen as serving different aspects of cognition, with cognitive styles theories proposing that people can be visual or verbal thinkers. Most feedback systems, however, only cater to verbal thinkers. To help rectify this, we have developed a novel method of crowd communication which appeals to those more visual people. Designers can ask a crowd to feedback on their designs using specially constructed image banks to discover the perceptual and emotional theme perceived by possible future customers. A major component of the method is a summarization process in which the crowd's feedback, consisting of a mass of images, is presented to the designer as a digest of representative images. In this paper we describe an experiment showing that these image summaries are as effective as the full image selections at communicating terms. This means that designers can consume the new feedback confident that it represents a fair representation of the total image feedback from the crowd.

Cognitive styles theories suggest that we divide into visual and verbal thinkers. In this paper we describe a method designed to encourage visual communication between designers and their audiences. This new visual feedback method is based on enabling fast intuitive selections by the crowd from image banks when responding to an idea. Visual summarization reduces the massed image choices to a small number of representative images. These summaries are then consumed at a glance by designers receiving the feedback leading to thoughtful reflection on their designs. We report an evaluation using two types of imagery for feedback. Twelve designers took part, receiving visual feedback in response to their designs. In semi-structured interviews they described their interpretation of the feedback, how it inspired them to change their designs and contrasted it with text feedback. Eleven of the twelve designers revealed that they would be enthusiastic users of a service providing this new mode of feedback.