A critical challenge in modelling Heterogeneous-Agent Teams is training agents to collaborate with teammates whose policies are inaccessible or non-stationary, such as humans. Traditional approaches rely on expensive human-in-the-loop data, which limits scalability. We propose using Large Language Models (LLMs) as policy-agnostic human proxies to generate synthetic data that mimics human decision-making. To evaluate this, we conduct three experiments in a grid-world capture game inspired by Stag Hunt, a game theory paradigm that balances risk and reward. In Experiment 1, we compare decisions from 30 human participants and 2 expert judges with outputs from LLaMA 3.1 and Mixtral 8x22B models. LLMs, prompted with game-state observations and reward structures, align more closely with experts than participants, demonstrating consistency in applying underlying decision criteria. Experiment 2 modifies prompts to induce risk-sensitive strategies (e.g. “be risk averse”). LLM outputs mirror human participants’ variability, shifting between risk-averse and risk-seeking behaviours. Finally, Experiment 3 tests LLMs in a dynamic grid-world where the LLM agents generate movement actions. LLMs produce trajectories resembling human participants’ paths. While LLMs cannot yet fully replicate human adaptability, their prompt-guided diversity offers a scalable foundation for simulating policy-agnostic teammates.

Artificial General intelligence (AGI) offers enormous benefits for humanity, yet it also poses great risk. The aim of this systematic review was to summarise the peer reviewed literature on the risks associated with AGI. The review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Sixteen articles were deemed eligible for inclusion. Article types included in the review were classified as philosophical discussions, applications of modelling techniques, and assessment of current frameworks and processes in relation to AGI. The review identified a range of risks associated with AGI, including AGI removing itself from the control of human owners/managers, being given or developing unsafe goals, development of unsafe AGI, AGIs with poor ethics, morals and values; inadequate management of AGI, and existential risks. Several limitations of the AGI literature base were also identified, including a limited number of peer reviewed articles and modelling techniques focused on AGI risk, a lack of specific risk research in which domains that AGI may be implemented, a lack of specific definitions of the AGI functionality, and a lack of standardised AGI terminology. Recommendations to address the identified issues with AGI risk research are required to guide AGI design, implementation, and management.

The version of distributed situation awareness (DSA) used in this article originated in human factors/ergonomics. Typically, this has involved the study of human operators working in teams and has used various forms of concept maps to qualitatively describe the information that team members use. In this article, we apply the concept of DSA to multi-agent teams (where team members might be human or automation) and extend the concept using the formal properties of Bayesian belief networks. In particular, we show how the Bayesian belief network can define DSA and how such a network can (using expectation maximization) adapt, even on the basis of limited information. The approach was considered in terms of situation awareness levels of perception (i.e., sensor state values conversion to belief) and projection (in terms of prediction and mission missing values estimation accuracy).

This thesis focuses attention on achieving Distributed Situation Awareness (DSA) with minimal resources (energy, processing cost, etc.) using small low-capacity agents (e.g., UAVs) coordinated in a decentralised fashion while conducting searching activity. This is in contrast to the existing works involving convoluted communication and information processing.

Broadband sonar analysis is used to identify contacts of interest and develop a tactical picture when operating underwater. Sonar analysts must identify significant features from hydrophone recordings and use these to determine the type of contact presented and in which direction it is moving. This is performed with no external views available, leading to dynamic levels of information uncertainty. This work explores how sonar analysts process sonar data and how this differs from the analysis performed by experienced sound engineers. Subject Matter Experts described aural and visual features from hydrophone recordings using a repertory grid method. This was used to gain access to the ‘frames’ that the different experts were using, described in terms of constructs and concepts. The concepts are used to design a novel visual display, Visual Intelligent Narrative for Autonomous Systems (VINAS), which is designed to aid contact classification.

In this paper, we propose an architecture that uses predictions tools obtained via Bayesian learning algorithms to monitor the issues of communication, fault tolerance, and adaptation in human-agent mission. The architecture describes different level of knowledge, planning, and commands differ by their priorities. We tested the model using forest fire lookouts problem on a simulation platform (AMASE). The process uses the conjugate gradient descent algorithm to perform the Bayesian Belief Network training. The output of the training process is a well-trained BBN for agents’ prediction, estimation, and decision making during communication failure. The prediction perfection of the human and agents were compared and studied. Although results proof that human approach is prone to error but is good in terms of emergency commands execution. We suggested that the use of a well-trained prediction tool (i.e., the output BBN) could be used in monitoring mission during communication link, hardware, or software breakdown.

In this paper, we propose a priority-based publish-subscribe approach to tackle reasoning in beliefs conflicts for a heterogeneous multi-agent mission. Agents subscribe to other agents' topics and rank them based on agents' situation awareness. Bayesian Belief Network (BBN) was used in maintaining agents' belief and recorded mission information could be used for the BBN training using conjugate gradient descent or expectation-maximization algorithms. The output of the training is the learned network for agents' predictions, estimations, and conclusions. We also propose an agent's self presumption inferential reasoning where agents learned heuristics and used them for future inferences. We test the system by using a team of heterogeneous Unmanned Aerial Vehicles (UAVs) with different sensor profiles and capacities tasked together to perform forest fire searching. To verify belief and settle conflicts, agents follow these steps: sequentially assess the prioritized publish-subscribe topics, inferential reasoning using the learned network, inferential reasoning using logical propositions, and learning process. From our experiment, the BBN training and prediction perfection grow up with the increase in the number of training data. Future work focuses on obtaining the optimal number of samples needed for effective prediction, effective agents' beliefs merging, communication protocol, and bandwidth utilization.

In this paper, we propose the use of Bayesian inference and learning to solve DCOP in dynamic and uncertain environments. We categorize the agents Bayesian learning process into local learning or centralized learning. That is, the agents learn individually or collectively to make optimal predictions and share learning data. The agents' mission data is subjected to gradient descent or expectation-maximization algorithms for training purposes. The outcome of the training process is the learned network used by the agents for making predictions, estimations, and conclusions to reduce communication load. Surprisingly, results indicate that the algorithms are capable of producing accurate predictions using uncertain data. Simulation experiment result of a multiagent mission for wildfire monitoring suggest robust performance by the learning algorithms using uncertain data. We argue that Bayesian learning could reduce the communication load and improve DCOP algorithms scalability.

Abstract Knowledge monitoring is a component of metacognition which can help students regulate their own learning. In adaptive learning software, the system’s model of the student can be presented as an open learner model (OLM) which is intended to enable monitoring processes. We explore how presenting alignment, between students’ self-assessed confidence and the system’s model of the student, supports knowledge monitoring. When students can see their confidence and their performance (either combined within one skill meter or expanded as two separate skill meters), their knowledge monitoring and performance improves, particularly for low-achieving students. These results indicate the importance of communicating the alignment between the system’s evaluation of student performance and student confidence in the correctness of their answers as a means to support metacognitive skills.

In this paper, two user interfaces (UIs) are compared for supporting decision making in a road traffic management activity. Both UIs provide the information needed to make critical decisions related to traffic management and are designed according to ecological interface design (EID) concepts. One user interface design also incorporates concepts from the proximity compatibility principle (PCP). In addition to comparing the two UI designs, this paper considers the impact of the reliability of automation on user decision time and accuracy. It is shown that the UI designed with a combined of EID and PCP leads to significantly faster total task time, due to faster performance on the information gathering phase of the task. Contrary to expectations, time to gather information is not affected by automation reliability. Participants adapted their decisions according to automation reliability, even without explicit feedback to indicate reliability. Implications for UI design and decision making are discussed.

Credit card fraud analysis is almost entirely automated. However, there may be occasions when a human analyst is required to intervene. In this paper, we consider situations in which a transaction triggers an automated alert but not sufficiently to allow automated response. On such occasions, automated analysis makes a recommendation as to the fraud pattern that has been identified and the human analyst decides, if this recommendation is correct and what action to take. In order to support the analyst, a “dashboard” can be used to display information that is relevant to the fraud pattern. Thus, a computer could analyze transaction data, define this as a known fraud pattern, and then present the analyst with a dashboard to illustrate how the transaction might fit the pattern. We explore the efficiency with which people respond to the computer's recommendation, and whether computer confidence has an impact on this response. We define efficiency in terms of information search: the user will either have the relevant information on screen or will need to drill-into a dashboard (i.e., open additional windows for information). The results show that participants adapt their decision making to the confidence of the automated support, and, although they drill-down even when not required, are efficient in terms of time spent looking at relevant information.

In this paper we report a cognitive model of how people make decisions through interaction. The model is based on the assumption that interaction for decision making is an example of a Partially Observable Markov Decision Process (POMDP) in which observations are made by limited perceptual systems that model human foveated vision and decisions are made by strategies that are adapted to the task. We illustrate the model by applying it to the task of determining whether to block a credit card given a number of variables including the location of a transaction, its amount, and the customer history. Each of these variables have a different validity and users may weight them accordingly. The model solves the POMDP by learning patterns of eye movements (strategies) adapted to different presentations of the data. We compare the model behavior to human performance on the credit card transaction task.

In this paper, we present preliminary results for the analysis of movement variability in human-humanoid imitation activities. We applied the state space reconstruction's theorem which help us to have better understanding of the movement variability than other techniques in time or frequency domains. In our experiments, we tested our hypothesis where participants, even performing the same arm movement, presented slight differences in the way they moved. With this in mind, we asked eighteen participants to copy NAO's arm movements while we collected data from inertial sensors attached to the participants' wrists and estimated the head pose using the OpenFace framework. With the proposed metric, we found that sixteen out of eighteen participants imitate the robot well by moving their arms symmetrically and by keeping their heads static; two participants however moved their head in a synchronous way even when the robot's head was completely static and two different participants moved their arms asymetrically to the robot. Although the work is in its early stage, we believe that such preliminary results are promising for applications in rehabilitation, sport science, entertainment or education.

This paper describes the design and evaluation of a branching comic to compare how readers recall a visual narrative when presented as an interactive, digital program, or as a linear sequence on paper. The layout of the comic is used to visualize this data as heat maps and explore patterns of users' recollections. We describe the theoretical justification for this based upon previous work in narrative visualizations, interactive stories and comics. Having tested the comic with school boys aged 11-12; we saw patterns in the data that complement other research in both interactive stories and visualizations. We argue that the heat maps helped identify these patterns, which have implications for future designs and analyses of interactive visual and/or narrative media.

In this paper we describe the design and development of a multi-touch surface and software that challenges current approaches to the production and consumption of comics. Authorship of the comics involves drawing the 'top level' of the story directly onto paper and projecting lower-level narrative elements, such as objects, characters, dialogue, descriptions and/or events onto the paper via a multi-touch interface. In terms of the impact this has upon the experience of reading and writing, the implementation of paper is intended to facilitate the creation of high-level overviews of stories, while the touch surface allows users to generate branches through the addition of artifacts in accordance with certain theories about interactive narratives. This provides the opportunity to participate in the reading and authoring of both traditional, paper-based texts and interactive, digital scenarios. Prototype comics are used to demonstrate this approach to reading and writing top-level and low-level narratives.

Close Target Reconnaissance (CTR) patrols can be characterized by activities that involve naturalistic decision making to develop situation awareness through processes of sense-making. Any technology that is introduced into such activity needs to be sufficiently “invisible” so as not to disrupt or otherwise interfere with the activities of search and interpretation. It is proposed that some technologies, such as imaging devices, can support recognition-primed decision making (RPDM). It is suggested that providing CTR patrols with cameras could supplement existing technologies, such as night vision goggles or binoculars, and provide an opportunity to capture useful intelligence. The concept of location-based photography (in which metadata are collected in parallel with images) provides a means of effectively producing patrol reports in real time. A prototype system is described and trialled comparing conventional practices surrounding note-taking and report writing with the use of location-based photography. The results show little difference in time spent patrolling under the two conditions but significant improvement in reporting under the location-based photography condition. One explanation of these differences relate to the manner in which note taking provides support for sense-making but could interfere with RPDM (through the need to analyze the situation in sufficient detail to make notes), whereas imaging can be performed as part of the RPDM activity. Tagged images change the nature in which the reports are written in that sense-making is performed largely post hoc, which allows flexibility in interpretation and analysis.

This paper describes a novel tabletop interface, MapTable, which can be used as a tactical command and control interface. It is designed and implemented to explore more intelligent and intuitive interaction in a distributed environment that can support remote collaboration. MapTable offers a common space for planners to work, which retains the intuitive feel of a "sandbox" around which discussion can take place and plans easily displayed, whilst the automated navigation command system means that both planning and the issuing of directions can effectively be merged into a single activity using a single user interface that embeds the tasks into the interaction. Empirical studies were conducted to test this tactical interface in a remote searching task environment. Compared to the traditional desktop command and control interface, MapTable can lead to significant differences in performance. This fusion of planning, command and control means that planners can be expected have a high level of situational awareness with regard to where those they are directing are, where they will be and what others in the team are doing.

Digital communities are often portrayed as operating entirely within the confines of a single technological domain, e.g. a group of people in a web-based chatroom. In this study an integrated approach to digital communities is explored. It is proposed that members of communities employ a range of digital technologies to support their activities and sense of community. This paper describes a study that develops a community using combined text messaging or Short Messaging Service (SMS) with web-based interaction. An application based on this specification is built in order to develop and support a digital community based around the 2002 World Cup. Participants interacted with the system during the soccer tournament, chatting with other members and taking part in competitions. They successfully integrated SMS with the website. Participants reported feeling like a cohesive group and showed increased interest in soccer and the World Cup.

We describe a study that investigates the use of a multimodal technology in a learning task. In relation to the recent interest in learning as a multimodal experience, the use of a technology to acquire and visualize data while in a learning task in real time is argued to be beneficial and enjoyable for pupils. A study is described which investigates whether using the learner's own movements or viewing someone else's movements produces different outcomes when learning about kinematics graphs.

Augmented reality (AR) systems need to be evaluated for their appropriateness for a given task. Three approaches are used to evaluate the waterpump augmented reality tool (WART). The system is assessed using verbal protocol, performance time and a questionnaire. The WART system is compared with a paper version of the assembly instructions. The verbal protocol revealed that there was more cognitive processing for the paper version of task than WART, it took longer to complete the task with WART than the paper version and the questionnaire revealed the effect on performance of problems associated with AR. All of the participants in the evaluation, however, were positive about WART, and appreciated its usefulness.

In this article, wearable computers are considered from the perspective of human factors. Three approaches to the development of this technology are presented: computers that can be worn, information appliances that can be worn, and computers as clothing. The implications for how people will wear and interact with computers in these forms are considered. In particular, in this article a discussion of forms of dialogue to demonstrate how wearable computers require fundamental revision of the way in which we consider human-computer interaction is presented. This article concludes with reports of work on human interaction with wearable computers, both in terms of task performance and physical effects of wearing technology.

In this paper we investigate whether virtual reality (VR) and augmented reality (AR) offer potential for the training of manual skills, such as for assembly tasks, in comparison to conventional media. We present results from experiments that compare assembly completion times for a number of different conditions. We firstly investigate completion times for a task where participants can study an engineering drawing and an assembly plan and then conduct the task. We then investigate the task under various VR conditions and context-free AR. We discuss the relative advantages and limitations of using VR and AR as training media for investigating assembly operations, and we present the results of our experimental work.

The capability profiles of commercial automatic speech recognition (ASR) systems are rapidly improving in terms of vocabulary size, noise robustness and user population. Most contemporary applications of ASR use interfaces relying solely on the speech mode of interaction (over telephone channels for example). Many applications will, however, benefit from using speech input in conjunction with other interaction devices such as trackballs, keyboards and touch-screens. In this paper, we present an interface modelling approach based on a critical path analysis of the interface design. The approach has been developed to model multi-modal interactions using combinations of input devices. Degradation of unit performances allow the effects of environmental factors on the overall interface performance to be predicted. The model is verified by comparison with experimental trials carried out on a number of multi-modal applications. It is demonstrated that the model is able to predict the main performance metric (task completion time) to within 10% of the experimental values.

An experiment was performed which investigated the input and error correction strategies used by subjects when entering digit strings with the Aurix recogniser. Subjects were allowed to control the point at which they received feedback and corrected misrecognitions throughout the trial. It was predicted that they would alter their input strategies as a result of an entry being misrecognised. Specifically it was hypothesised that following a misrecognition, further attempts at entry of that data would use smaller digit input chunks, which should increase the likelihood of success. Comparison of the mean chunk sizes used in error-full compared to error-fi-ee interactions showed this to be the case. Some preliminary modelling work was also undertaken based on the experimental results. This suggests that the choice of input chunk size used by subjects approaches the optimal value based on the recognition accuracy obtained with the system.