Machines have traditionally served as tools to fulfill human requirements; however, the rapid advancement of artificial intelligence has enabled the development of autonomous systems capable of functioning as fully integrated teammates. These agents can share information, assume roles, and execute tasks within collaborative environments. Effective Human–Artificial Agent collaboration, achieved through the integration of complementary cognitive and operational capabilities, has demonstrated improvements in overall team performance across multiple domains, including industrial robotics, healthcare, and augmented reality. Nevertheless, achieving both optimal performance and interaction fluency remains a significant challenge. Real-time monitoring of tasks, intentions, and constraints of human and artificial partners is still limited, and the application of quantifiable online metrics for this purpose is underexplored. This narrative review systematically examines online metrics derived from behavioral, physiological, and interaction-based approaches, discussing their potential to enhance adaptive mechanisms and optimize team fluency in H–AA collaboration.

Currently, flight safety is ensured by the collaboration of at least two pilots in the cockpit. Thanks to progress in automation and telecommunication, aircraft manufacturers and aviation companies envision that a single pilot in the cockpit assisted by a pilot on the ground (i.e Single-Pilot Operation) could ensure flight operations while requiring less human resources. However, without appropriate collaboration tools, this situation of remote collaboration may lead to a degradation of the awareness of actions and attitudes between the two pilots (i.e. mutual awareness). In this paper, we propose to enrich the understanding of the remote collaboration problems of two pilots through a fine-grained analysis of mutual awareness needs. First, we describe awareness frameworks from the literature. Second, we identify awareness issues during a case study involving a crew of pilots in two distant flight simulators. Third, we refine the relevant awareness concepts through exploratory prototyping of collaborative tools. These prototypes are based on three scenarios involving specific awareness requirements including 1) visualizing the physiological state of the pilot on board during a non stabilized approach, 2) an emergency decision making, and 3) global awareness during a whole flight for a better efficiency of the ground assistant operator at the arrival. In this article our contribution is a refined study of the awareness needs adapted to the context of remote collaborative piloting, with the final objective of designing more appropriate tools.

This work explores the use of functional Near Infrared Spectroscopy (fNIRS) to assess cognitive activity during videogame play, and compare it to cognitive activity during cognitive tasks that assess executive control. To this end, we assessed haemodynamic response to videogame and cognitive tasks in the prefrontal cortex, each manipulated on a spectrum of difficulty. In our study (n = 37), we find that mental effort expended during videogame play did not differ from mental effort expended during cognitive tasks---and speculate that regional cognitive activity during gameplay is indicative of functions pertaining to memory encoding and retrieval, planning, and sustainment of attention. Our findings suggest the utility of fNIRS as a means to understand challenge as part of the player experience, and contest the popular conception of videogame play as cognitively undemanding entertainment. Further, we were successful in distinguishing between difficulty levels in the gameplay tasks, situating fNIRS as broadly useful for granular assessment of gameplay difficulty. As such, we contend that fNIRS is an effective and useful tool for generating high-resolution insights regarding cognition (and particularly the experience of difficulty) during gameplay.

An increasing number of domains, including aeronautics, are adopting touchscreens. However, several drawbacks limit their operational use, in particular, eyes-free interaction is almost impossible making it difficult to perform other tasks simultaneously. We introduce GazeForm, an adaptive touch interface with shape-changing capacity that offers an adapted interaction modality according to gaze direction. When the user's eyes are focused on interaction, the surface is flat and the system acts as a touchscreen. When eyes are directed towards another area, physical knobs emerge from the surface. Compared to a touch only mode, experimental results showed that GazeForm generated a lower subjective mental workload and a higher efficiency of execution (20% faster). Furthermore, GazeForm required less visual attention and participants were able to concentrate more on a secondary monitoring task. Complementary interviews with pilots led us to explore timings and levels of control for using gaze to adapt modality.

In this research, we use eye tracking to monitor the attentional behavior of pilots in the cockpit. We built a cockpit monitoring database that serves as a reference for real-time assessment of the pilot's monitoring strategies, based on numerous flight simulator sessions with eye-tracking recordings. Eye tracking may also be employed as a passive input for assistive system, future studies will also explore the possibility to adapt the notifications' modality using gaze.

The allocation of visual attention is a key factor for the humans when operating complex systems under time pressure with multiple information sources. In some situations, attentional tunneling is likely to appear and leads to excessive focus and poor decision making. In this study, we propose a formal approach to detect the occurrence of such an attentional impairment that is based on machine learning techniques. An experiment was conducted to provoke attentional tunneling during which psycho-physiological and oculomotor data from 23 participants were collected. Data from 18 participants were used to train an adaptive neuro-fuzzy inference system (ANFIS). From a machine learning point of view, the classification performance of the trained ANFIS proved the validity of this approach. Furthermore, the resulting classification rules were consistent with the attentional tunneling literature. Finally, the classifier was robust to detect attentional tunneling when performing over test data from four participants.