Generative AI is increasingly adopted in sound practices, yet its use in professional production-oriented workflows remains insufficiently understood. This paper addresses this gap through an empirical study of game sound design, a structured creative practice, examining how game sound practitioners engage with generative AI tools and where current systems fail to support established workflows. We conducted a screening survey (n = 58) and semi-structured interviews involving hands-on use of two generative AI tools (n = 13) with professional game sound practitioners. Findings reveal that practitioners primarily use generative AI for early-stage ideation and rapid prototyping rather than production-ready outputs, and identify a structural misalignment between how current AI systems operate and how professional sound work is organized. We contribute a workflow model of professional game sound production, an analysis of audio teams’ organizational working conditions, and an initial framework for evaluating future AI-assisted game sound design tools grounded in practitioner interaction preferences.

Earthquakes significantly impact societies and economies, underscoring the need for effective search and rescue strategies. As AI and robotics increasingly support these efforts, the demand for high-fidelity, real-time simulation environments for training has become pressing. Earthquake simulation can be considered as a complex system. Traditional simulation methods, which primarily focus on computing intricate factors for single buildings or simplified architectural agglomerations, often fall short in providing realistic visuals and real-time structural damage assessments for urban environments. To address this deficiency, we introduce a real-time, high visual fidelity earthquake simulation platform based on the Chaos Physics System in Unreal Engine, specifically designed to simulate the damage to urban buildings. Initially, we use a genetic algorithm to calibrate material simulation parameters from Ansys into the Unreal Engine’s fracture system , based on real-world test standards. This alignment ensures the similarity of results between the two systems while achieving real-time capabilities. Additionally, by integrating real earthquake waveform data, we improve the simulation’s authenticity, ensuring it accurately reflects historical events. All functionalities are integrated into a visual user interface, enabling zero-code operation, which facilitates testing and further development by cross-disciplinary users. We verify the platform’s effectiveness through three AI-based tasks: similarity detection, path planning , and image segmentation. This paper builds upon the preliminary earthquake simulation study we presented at IMET 2023, with significant enhancements, including improvements to the material calibration workflow and the method for binding building foundations.

Despite the enhanced realism and immersion provided by VR headsets, users frequently encounter adverse effects such as digital eye strain (DES), dry eye, and potential long-term visual impairment due to excessive eye stimulation from VR displays and pressure from the mask. Recent VR headsets are increasingly equipped with eye-oriented monocular cameras to segment ocular feature maps. Yet, to compute the incident light stimulus and observe periocular condition alterations, it is imperative to transform these relative measurements into metric dimensions. To bridge this gap, we propose a lightweight framework derived from the U-Net 3 + deep learning backbone that we re-optimised, to estimate measurable periocular depth maps. Compatible with any VR headset equipped with an eye-oriented monocular camera, our method reconstructs three-dimensional periocular regions, providing a metric basis for related light stimulus calculation protocols and medical guidelines. Navigating the complexities of data collection, we introduce a Dynamic Periocular Data Generation (DPDG) environment based on UE MetaHuman, which synthesises thousands of training images from a small quantity of human facial scan data. Evaluated on a sample of 36 participants, our method exhibited notable efficacy in the periocular global precision evaluation experiment, and the pupil diameter measurement.

This artwork presents an intelligent chatbot called Wander. This work used knowledge-based story generation to facilitate a narrative AI chatbot on daily communication platforms, producing interactive fiction with the most accessible natural language input: text messages. On social media platforms such as Discord and WeChat, Wander can generate a science-fiction style travelogue about the future earth, including text, images and global coordinates (GPS) based on real-world locations (e.g. Paris). The journeys are visualised in real-time on an interactive map that can be updated with participants' data. Based on Viktor Shklovsky's defamiliarization technique, we present how an AI agent can become a storyteller through common messages in daily life and lead participants to see the world from new perspectives. The website of this work is: https://wander001.com/

We developed Wander[00l] as an experiment to discuss several visions toward the metaverse: through crowd contribution, how an AI agent can become a highly accessible storyteller, and how to link the virtual and physical world through AI-generated content (AIGC). In this artwork, we implement a hybrid AIGC and user-generated content (UGC) system to facil-itate a narrative AI chatbot, Wander, that produces interactive fiction through knowledge graphs with text messages input by users on instant messaging social platforms. On Discord and WeChat, Wander can generate science-fiction-style travelogues about the future earth, including text, style-transferred images and global coordinates (GPS) based on real-world locations (e.g. Paris). The crowd interactions with Wander are visualised on an interactive globe map in real time, documenting the players' asynchronous contributions to exploring the speculative future earth. This paper presents Wander's concept, development and user study to demonstrate how people would interact with an AI agent in a narrative context for the future metaverse.

Virtual Environments (VEs) provide the opportunity to simulate a wide range of applications, from training to entertainment, in a safe and controlled manner. For applications which require realistic representations of real world environments, the VEs need to provide multiple, physically accurate sensory stimuli. However, simulating all the senses that comprise the human sensory system (HSS) is a task that requires significant computational resources. Since it is intractable to deliver all senses at the highest quality, we propose a resource distribution scheme in order to achieve an optimal perceptual experience within the given computational budgets. This paper investigates resource balancing for multi-modal scenarios composed of aural, visual and olfactory stimuli. Three experimental studies were conducted. The first experiment identified perceptual boundaries for olfactory computation. In the second experiment, participants ( N=25) were asked, across a fixed number of budgets ( M=5), to identify what they perceived to be the best visual, acoustic and olfactory stimulus quality for a given computational budget. Results demonstrate that participants tend to prioritize visual quality compared to other sensory stimuli. However, as the budget size is increased, users prefer a balanced distribution of resources with an increased preference for having smell impulses in the VE. Based on the collected data, a quality prediction model is proposed and its accuracy is validated against previously unused budgets and an untested scenario in a third and final experiment.

Serious games are accepted as an effective approach to deliver augmented feedback in motor (re-)learning processes. The multi-modal nature of the conventional computer games (e.g. audiovisual representation) plus the ability to interact via haptic-enabled inputs provides a more immersive experience. Thus, particular disciplines such as medical education in which frequent hands on rehearsals play a key role in learning core motor skills (e.g. physical palpations) may benefit from this technique. Challenges such as the impracticality of verbalising palpation experience by tutors and ethical considerations may prevent the medical students from correctly learning core palpation skills. This work presents a new data glove, built from off-the-shelf components which captures pressure sensitivity designed to provide feedback for palpation tasks. In this work the data glove is used to control a serious game adapted from the infinite runner genre to improve motor skill acquisition. A comparative evaluation on usability and effectiveness of the method using multimodal visualisations, as part of a larger study to enhance pressure sensitivity, is presented. Thirty participants divided into a game-playing group ( $$n=15$$ n = 15 ) and a control group ( $$n=15$$ n = 15 ) were invited to perform a simple palpation task. The game-playing group significantly outperformed the control group in which abstract visualisation of force was provided to the users in a blind-folded transfer test. The game-based training approach was positively described by the game-playing group as enjoyable and engaging.