Pilots operating modern cockpits often face high cognitive demands due to complex interfaces and multitasking requirements, which can lead to overload and decreased performance. This study introduces AdaptiveCoPilot, a neuroadaptive guidance system that adapts visual, auditory, and textual cues in real time based on the pilot’s cognitive workload, measured via functional Near-Infrared Spectroscopy (fNIRS). A formative study with expert pilots (N=3) identified adaptive rules for modality switching and information load adjustments during preflight tasks. These insights informed the design of AdaptiveCoPilot, which integrates cognitive state assessments, behavioral data, and adaptive strategies within a context-aware Large Language Model (LLM). The system was evaluated in a virtual reality (VR) simulated cockpit with licensed pilots (N=8), comparing its performance against baseline and random feedback conditions. The results indicate that the pilots using AdaptiveCoPilot exhibited higher rates of optimal cognitive load states on the facets of working memory and perception, along with reduced task completion times. Based on the formative study, experimental findings, qualitative interviews, we propose a set of strategies for future development of neuroadaptive pilot guidance systems and highlight the potential of neuroadaptive systems to enhance pilot performance and safety in aviation environments.

Current voice assistants present messages in a predefined format without considering users’ mental states. This paper presents an optimization-based approach to alleviate this issue which adjusts the level of details and speech speed of the voice messages according to the estimated cognitive load of the user. In the first user study (N = 12), we investigated the impact of cognitive load on user performance. The findings reveal significant differences in preferred message formats across five cognitive load levels, substantiating the need for voice message adaptation. We then implemented AdaptiveVoice, an algorithm based on combinatorial optimization to generate adaptive voice messages in real time. In the second user study (N = 30) conducted in a VR-simulated driving environment, we compare AdaptiveVoice with a fixed format baseline, with and without visual guidance on the Heads-up display (HUD). Results indicate that users benefit from AdaptiveVoice with reduced response time and improved driving performance, particularly when it is augmented with HUD.

As urban landscapes increasingly transform into smart cities, there is an increasing emphasis on smart buildings for sustainable and efficient management of resources. This evolution necessitates innovative approaches to efficiently manage energy resources, enhance occupant comfort, and ensure environmental sustainability. In response to these challenges, this study introduces a practical and novel smart building management system, which integrates Digital Twin (DT) technology with machine learning, primarily aimed at enhancing thermal comfort in buildings. The system, underpinned by a sensor network-based DT and real-time data visualization using the Unreal Engine, employs a Deep Neural Network (DNN) to predict thermal comfort. The empirical validation, conducted in a controlled laboratory setting, involves a comparative analysis of the DNN’s performance against traditional models and user experience evaluation through the USE Questionnaire. Results demonstrate the DNN’s superior predictive accuracy and high user satisfaction levels in usability and effectiveness. This research highlights the significant role of DT and machine learning in revolutionizing smart building operations, setting a foundation for future advancements in creating sustainable, efficient, and occupant-friendly smart cities.