Secondary school systems often reward predictable, mark-bearing performance, making experimentation feel risky, especially in under-resourced contexts, where time, attention, and materials are tightly managed. We report on an exploratory case study of a charity-led digital arts programme delivered in two low-socioeconomic secondary schools in which immersive (i.e., Virtual Reality) and non-immersive (i.e., tablet-based) digital tools were introduced into Year 7 (400+ pupils, typically aged 11–12) curriculum. Through thematic analysis of the lesson observation notes and student surveys, we show that pupils take more creative risks, value exploration, iterate more, and increasingly learn through and with peers. We characterise shifts in classroom ecology, including shared talk, turn-taking, micro-tutorials, and practices of care and repair that help sustain learning-in-process. Finally, we analyse the breakdowns and trade-offs of deploying off-the-shelf creative technologies in low-resource school routines, and derive deployment lessons for resilient, bottom-up creative technology curricula that fit everyday classrooms.

In underserved coastal communities, the adoption of digital technology is deeply stratified. While schools with high socio-economic status (SES) leverage technology for creative agency, under-resourced schools often face a second-level digital divide, where hardware is treated as a disruption rather than a tool for inquiry. We investigate this dynamic through an exploratory study of a digital arts programme in low-SES UK secondary schools, in which Human-Computer Interaction (HCI) researchers served as advisors and observers of a co-created partnership between schools and a local digital arts charity. Our findings reveal a pedagogical chasm. Systemic constraints and assessment pressures force teachers into the role of risk-averse behavioural gatekeepers, while external facilitators manage the creative play. However, we identify how students bridge this gap by establishing peer infrastructures, shared talk and repair work that transform technical breakdowns into possibility spaces for learning. We argue that sustainable integration requires designing for school settings to empower the human relationships central to creative technology use.

Communicating slow, invisible environmental processes such as corrosion remains challenging in public contexts. We explore how smell can function as a creative design material in a multisensory VR experience for science communication. Using Research-through-Design (RtD), we developed and iteratively refined a 5-minute VR experience across co-design, prototyping, and public deployment. Olfaction was introduced as an embodied layer to translate imperceptible chemical processes into experiential form. Findings from a festival deployment with 73 participants show that smell enhanced immersion and memorability for some users but was perceived inconsistently and often amplified already salient visual moments. We characterise smell as an unruly material due to variability in perception, diffusion, and context, and contribute an RtD account of multisensory VR design for science communication, outlining implications for designing with olfaction in public settings.

Adults with intellectual disabilities (ID) face systemic social exclusion that narrows autonomy and life opportunities. While social virtual reality (VR) offers a powerful medium for identity expression and community belonging, research often adopts a remedial paradigm, focusing on training functional skills in scripted environments. This paper challenges this deficit-based model by treating social VR as an open world for participation. Following 11 adults with ID across multi-session engagements with VRChat, we employed an adaptive, relational method to scaffold participant leadership. Findings reveal that participants used the platform for interest-driven discovery, sustained through interdependent care webs. Crucially, the study demonstrates how social VR supports transferable confidence and emerging digital citizenship, enabling some users to transition from novices to community leaders. We contribute six Disability Justice-aligned design principles articulating a world-making paradigm that reorients Human-Computer Interaction toward supporting personhood and self-determination in mainstream digital publics.

Digital health interventions in the Global South often rely on transmission models, assuming that delivering correct medical information yields better care. We challenge this view through an analysis of a multi-platform social media intervention for Community-Based stroke Rehabilitation(CBR) in rural Thailand. Following a collaborative development process with clinicians and a deployment across roughly 2,000 villages, we interviewed 28 caregivers, patients, and health volunteers. We found that communities appropriated the technology in unexpected ways, such as using videos as “social objects” to manage family hierarchies, integrating rehabilitation into Buddhist merit-making, and prioritising offline peer networks over online discussion. Our findings suggest that effective Human-Computer-Interaction (HCI) for digital health in Low- and Middle-Income Countries (LMICs) should look beyond engagement metrics to support the appropriation of digital tools, enabling communities to integrate clinical protocols into their existing cultural and relational fabrics.

Music is increasingly performed and experienced in Social Virtual Reality (Social VR), from VRChat raves to high-production concerts on bespoke platforms. Yet Human-Computer-Interaction (HCI) research still focuses mainly on building new VR systems rather than examining the communities that already create and sustain these practices. We present a cultural mapping of the Social VR music scene based on 84 survey responses, 27 interviews, and 17 event observations with diverse stakeholders, including audience members, musicians, developers, platform owners, and event organisers. We found that the scene operates as a fragmented cross-platform ecosystem sustained by user-generated infrastructure and continuous community labour. The bottom-up organisation produces role fluidity with individuals dynamically shifting between roles as performers, world builders, organisers, and audience members. However, the openness that enables this creativity also creates tensions between expectations of free access and the financial and emotional labour required to keep events running. Taken together, our findings reveal the vibrant cultural practices that continue to flourish in Social VR, even as corporate narratives declare the “metaverse” dead.

Social virtual reality (VR) encompasses a growing network of three-dimensional virtual worlds where users interact in a shared, embodied way. While research has focused on the social interactions between the users themselves, less is known about how the design of virtual spaces influences these interactions. Our study combines interviews with 15 social VR users logging over 1,000 hours and a 20-hour spatial protocol of a purposeful sampling of VR worlds. We analysed how spatial characteristics (including proportion, sightlines, materiality, atmosphere, and navigation) influence meaningful user interaction to turn space into place. We synthesised four place types for a new social VR typology: Cuddle worlds that encourage cosy conversations; Mingle worlds that facilitate new encounters; Venture worlds that promote exploration; and Guided worlds that elicit a sense of belonging with the online community. By relating architectural affordances to social patterns, we contribute insights towards the purposeful design of social VR places.

Figure 1: First page previews of our Facebook, YouTube, TikTok, and LINE platforms, aligned from left to right

This paper explores the integration of co-design and art-making in developing technologies that support personhood in dementia care. While technologies for dementia care have advanced, there remains a gap in creating solutions that are directly informed by the experiences of people living with dementia and support their individuality. In collaboration with the specialist arts organisation Bright Shadow CIO, our work involves engaging people living with dementia in the design process. Over five weeks of co-design sessions, 44 participants worked alongside artists to craft four physical boxes that represent “meaningful places.” The physical boxes were then transformed into VR environments, allowing participants to immerse themselves in and interact with their creations from a first-person perspective. Our findings demonstrate that VR alone is insufficient in dementia care. For VR to be meaningful, it must be be part of a broader intervention that includes trust-building, sensory engagement, and creative involvement. Within this process, art-making serves as both a method and medium, providing a means of self-expression and connection to identity. Our findings challenge conventional approaches to dementia-focused VR, advocating for a shift toward inclusive and care-driven technology design.

ABSTRACT Virtual reality (VR) immerses users in digital environments and is used in various applications. VR content is created using either computer‐generated or conventional imaging. However, conventional imaging captures only 2D spatial information, which limits the realism of VR content. Advanced technologies like light field (LF) imaging can overcome this limitation by capturing both 2D spatial and 2D angular information in 4D LF images. This paper proposes a depth reconstruction model through LF imaging to aid in creating realistic VR content. Comprehensive calibrations are performed, including adjustments for camera parameters, depth calibration, and field of view (FOV) estimation. Aberration corrections, like distortion and vignetting effect correction, are conducted to enhance the quality of the reconstruction. To achieve realistic scene reconstruction, experiments were conducted by setting up a scenario with multiple objects positioned at three different depths. Quality assessments were carried out to evaluate the reconstruction quality across these varying depths. The results demonstrate that depth reconstruction quality improves with the proposed method. It also indicates that the model reduces LF image size and processing time. The depth images reconstructed by the proposed model have the potential to generate realistic VR content and can also facilitate the integration of refocusing capabilities within VR environments.

One factor leading to compromised Quality of Life (QoL) for people with dementia in long-term care settings is the significant barriers they face in accessing experiences beyond their physical premises. Although, in recent years, research within HCI has investigated the use of Virtual Reality (VR) in providing enriched experiences for people with dementia, few studies have looked into the practical issues relating to integrating and deploying VR into care settings and how in-depth understanding of such issues can inspire VR design for dementia care. Considering the perspective of key stakeholders, we demonstrate VR’s potential to create a rich conversation space with family members, preserve the identity and personhood of people with dementia through VR, and consider organisational issues of VR deployment in care settings. As such, this paper contributes to the HCI research community with future design directions to enhance the deployment and adoption of VR in dementia care settings.

Our work explores how audiences perceive and engage with remote music events. The shift to digital platforms has transformed the live music experience, leading to the rise of remote music performances. Following the COVID-19 pandemic, platforms such as Twitch have been repurposed and surged in popularity as meaningful outlets for live music dissemination. Concurrently, novel platforms specifically crafted for music events have begun to emerge. Understanding the preferences and challenges of remote music audiences is crucial because it guides the development of platforms that effectively answer varied user needs, which could enhance overall engagement and shape the future of music consumption. For this, we employed a qualitative multi-method approach, including a survey, interviews, and event observations, to start mapping the design space of remote music performances and capture a wide spectrum of audience perspectives. Our findings reveal audience engagement patterns in remote music events across different digital platforms, highlighting unmet needs, desires as well as opportunities for future HCI work in this area.

This paper examines the design of technology for remote music performances, from the perspective of their audiences. In this process, we involved a total of 104 participants across the different stages of our project. Initially, we collected qualitative data from remote audiences using several methods, including surveys, interviews, and observations. Through the thematic analysis of this data, we identified four design dimensions consisting of 17 key elements that illustrate what audiences value in remote music spaces. We applied these insights in a participatory design workshop with diverse stakeholders, contributing to the development of speculative design ideas in this field. The paper concludes by presenting key design insights for future technology advancements in remote music performances. The research contributes to the evolving design space of remote music performances, offering valuable perspectives for researchers, designers, and industry stakeholders.

People living with dementia are at risk of social isolation, and conversational AI agents can potentially support such individuals by reducing their loneliness. In our study, a conversational AI agent, called MindTalker, co-designed with therapists and utilizing the GPT-4 Large Language Model (LLM), was developed to support people with early-stage dementia, allowing them to experience a new type of “social relationship” that could be extended to real life. Eight PwD engaged with MindTalker for one month or even longer, and data was collected from interviews. Our findings emphasized that participants valued the novelty of AI, but sought more consistent, deeper interactions. They desired a personal touch from AI, while stressing the irreplaceable value of human interactions. The findings underscore the complexities of AI engagement dynamics, where participants commented on the artificial nature of AI, highlighting important insights into the future design of conversational AI for this population.

Abstract The light field (LF) imaging technique can capture 3D scene information in 4D by recording both 2D intensity and 2D direction of incoming light rays. Due to this capability, LF has shown a great interest in virtual reality (VR) and augmented reality (AR) for enhanced immersion, improved depth perception and reconstruction of realistic 3D environments. This paper presents a comprehensive review of LF imaging technology and other approaches used for VR content creation. The applications of LF technology beyond VR and AR are also discussed. The challenges and limitations of other approaches for VR content creation are examined. State‐of‐the‐art research has focused on how VR experiences benefit from LF technology and identified the challenges to creating comfortable, immersive and realistic VR content such as (1) image size and resolution, (2) processing speed, (3) precise calibration and (4) depth reconstruction. Recommendations that can be considered for creating immersive VR content are provided to enhance user experience. These recommendations aim to contribute to developing more comfortable and realistic VR content, extending the potential applications of LF imaging technology in diverse fields.

The development of AI to function as communicators (i.e., conversational agents), has opened the opportunity to rethink AI’s place within people’s social worlds, and the process of sense-making between humans and machines, especially for people with autism who may stand to benefit from such interactions. The current study aims to explore the interactions of six autistic and six non-autistic adults with a conversational virtual human (CVH/conversational agent/chatbot) over 1–4 weeks. Using semi-structured interviews, conversational chatlogs and post-study online questionnaires, we present findings related to human-chatbot interaction, chatbot humanization/dehumanization and chatbot’s autistic/non-autistic traits through thematic analysis. Findings suggest that although autistic users are willing to converse with the chatbot, there are no indications of relationship development with the chatbot. Our analysis also highlighted autistic users’ expectations of empathy from the chatbot. In the case of the non-autistic users, they tried to stretch the conversational agent’s abilities by continuously testing the AI conversational/cognitive skills. Moreover, non-autistic users were content with Kuki’s basic conversational skills, while on the contrary, autistic participants expected more in-depth conversations, as they trusted Kuki more. The findings offer insights to a new human-chatbot interaction model specifically for users with autism with a view to supporting them via companionship and social connectedness.

Abstract Passive haptics have long been used to enhance the user’s experience in virtual reality (VR). However, creating props to be used in a virtual environment can be a complicated and lengthy process. Current research looks to create passive haptic props based on the layout of, or objects in, the user’s real environment. However, we identify three key limitations of current research. Firstly, procedural generation introduces many unknown variables into the design process, which complicates applying such techniques to scenarios requiring knowledge of the virtual environment’s layout ahead of time. Furthermore, such techniques limit the size and dimensions of the virtual space to that of the real space. Lastly, current research necessitates pre-scanning or real-time scanning of the user’s real environment, often requiring specialist equipment and expertise, thus limiting its generalisability. This research proposes P rop O riented W orld R otation , a technique that attempts to answer the aforementioned limitations and simplify the process of adding haptic feedback to VR applications. We implemented this technique in a demonstration game and give an overview of the steps taken to apply the technique in a real context. We analysed the demonstration system’s performance and conducted an initial user evaluation in three different physical environments. While our stress test of the system’s performance highlights the necessity for certain optimisations in complex environments, our initial user feedback suggests that users experienced a stronger sense of presence and feelings of safety in our passive haptics-enhanced environment. Hence, we conclude that our proposal has the potential to enhance experiences in VR with haptic feedback.

Human behaviour and habits co-evolve with technology, and the metaverse is poised to become a key player in reshaping how we live our everyday life. Given the importance of food in our daily lives, we ask: how will our relationships with food be transformed by the metaverse, and what are the promises and pitfalls of this technology? To answer this, we propose a co-design study that reveals the important elements people value in their daily interactions with food. We then present a speculative catalogue of novel metaverse food experiences, and insights from discussing these ideas with food designers, anthropologists and metaverse experts. Our work aims to provide designers with inspirations for building a metaverse that: provides inclusive opportunities for the future of food; helps re-discover the forgotten or lost knowledge about food; facilitates the exploration, excitement and joy of eating; and reinvigorates the ways that food can soothe and heal.

While conversational agents have traditionally been used for simple tasks such as scheduling meetings and customer service support, recent advancements have led researchers to examine their use in complex social situations, such as to provide emotional support and companionship. For mourners who could be vulnerable to the sense of loneliness and disruption of self-identity, such technology offers a unique way to help them cope with grief. In this study, we explore the potential benefits and risks of such a practice, through semi-structured interviews with 10 mourners who actively used chatbots at different phases of their loss. Our findings indicated seven approaches in which chatbots were used to help people cope with grief, by taking the role of listener, acting as a simulation of the deceased, romantic partner, friend and emotion coach. We then highlight how interacting with the chatbots impacted mourners’ grief experience, and conclude the paper with further research opportunities.

Abstract In order to create immersive experiences in virtual worlds, we need to explore different human senses (sight, hearing, smell, taste, and touch). Many different devices have been developed by both industry and academia towards this aim. In this paper, we focus our attention on the researched area of thermal and wind devices to deliver the sensations of heat and cold against people’s skin and their application to human-computer interaction (HCI). First, we present a review of devices and their features that were identified as relevant. Then, we highlight the users’ experience with thermal and wind devices, highlighting limitations either found or inferred by the authors and studies selected for this survey. Accordingly, from the current literature, we can infer that, in wind and temperature-based haptic systems (i) users experience wind effects produced by fans that move air molecules at room temperature, and (ii) there is no integration of thermal components to devices intended for the production of both cold or hot airflows. Subsequently, an analysis of why thermal wind devices have not been devised yet is undertaken, highlighting the challenges of creating such devices.

Omnidirectional videos have become a leading multimedia format for Virtual Reality applications. While live 360$^\circ$videos offer a unique immersive experience, streaming of omnidirectional content at high resolutions is not always feasible in bandwidth-limited networks. While in the case of flat videos, scaling to lower resolutions works well, 360$^\circ$video quality is seriously degraded because of the viewing distances involved in head-mounted displays. Hence, in this article, we investigate first how quality degradation impacts the sense of presence in immersive Virtual Reality applications. Then, we are pushing the boundaries of 360$^\circ$technology through the enhancement with multisensory stimuli. 48 participants experimented both 360$^\circ$scenarios (with and without multisensory content), while they were divided randomly between four conditions characterised by different encoding qualities (HD, FullHD, 2.5K, 4K). The results showed that presence is not mediated by streaming at a higher bitrate. The trend we identified revealed however that presence is positively and significantly impacted by the enhancement with multisensory content. This shows that multisensory technology is crucial in creating more immersive experiences.

Location-aware mobile guides are a popular technology for enhancing the experience of heritage walks in historical outdoor sites. Smartphones, as typical mediums for such systems, have been criticised for limiting users’ opportunities for embodied engagement with the environment. In this work, we investigate how display technologies beyond the traditional personal mobile screen can facilitate embodied experiences during outdoor heritage walks. To do this, we revisit the use of portable projected displays as a medium that allows us to explore the effects of overt and blended displays in this context. We conducted a study with 42 participants on an outdoor heritage walk, using two display modalities: smartphone and projected display. We discovered that besides the display modality, users’ attitude toward technology, their embodied relationship with the device, and incorporation of spatial aspects in interaction play a key role in generating engagement and shaping the experience of heritage walks.

The acoustic and visual experiences of musicians in the spaces they perform in are complex and organic in nature, entailing a continuous interaction with the environment. With this project, we leverage the power of Virtual Reality (VR) to support musicians in their creative practice by transporting them to novel sonic and visual worlds. For this, we developed a musician-centred VR system, featuring various acoustic and visual virtual environments, VR Rehearse & Perform, based on design requirements gathered with musicians and performance experts. To investigate how VR can be designed to support music-makers in their creative musical practice, we performed iterative tests with 19 musicians followed by semi-structured interviews. Our findings suggest that VR has the potential to support different aspects of the creative musical practice, such as rehearsing, performing and improvising. Our research provides insights and inspirations toward designing musician-centred VR experiences for various musical activities.

With one-person households being increasingly common and Covid-19 lockdown policies forcing people to stay home, remote dining has become common practice for many, who take it as an opportunity to connect with others in times of loneliness. Sharing meals online, also known as digital commensality, is a rich form of interaction, where people leverage technology to achieve a sense of connectedness and belonging while eating. In this paper, we look at digital commensality and we explore its inherent playful potential with the aim to inspire the design of engaging technologies that can support, enhance and augment this form of interaction. For this, we used a situated play design approach to document and analyze the behavior of 36 people (including pairs of friends and strangers) sharing meals online. Our analysis surfaced a set of play potentials of remote dining -- i.e., playful things people already do and enjoy spontaneously while sharing meals online. We present those play potentials as inspirational material: they can motivate and enrich the design of future digital commensality technologies by responding to people's desire for playful and social interaction with, through, and around food.

In this paper, we propose VR Rehearse & Perform - a Virtual Reality application for enhancing the rehearsal efforts of performers by providing them access to accurate recreations - both visual and acoustical - of iconic concert venues.

In this paper, we present a system that allows developers to add passive haptic feedback into their virtual reality applications by making use of existing physical objects in the user’s real environment. Our approach has minimal dependence on procedural generation and does not limit the virtual space to the dimensions of the physical play-area.

Abstract Quality of Experience (QoE) is inextricably linked to the human side of the multimedia experience. Whilst there has been a considerable amount of research undertaken to explore the various dimensions of QoE, one facet which been relatively unexplored is the role of individual differences in determining an individual’s QoE. Whereas this is certainly true of multimedia applications, when it comes to mulsemedia (multiple media engaging three or more human senses) this is even more so, given its emerging and novel nature. Accordingly, in this paper we report the results of a study which investigated the role that individual differences (such as age, gender, education, and smell sensitivity) have on QoE, when mulsemedia incorporating olfactory and haptic stimuli is experienced in cross-modal environments. Our results reveal that whilst users had a satisfying overall mulsemedia experience the specific use of cross modally matched odours did not result in significantly higher QoE levels than when a control scent (rosemary) was employed. However, aspects of QoE are impacted upon by all individual differences dimensions considered in our study.

Abstract A great deal of research effort has been put in exploring crossmodal correspondences in the field of cognitive science which refer to the systematic associations frequently made between different sensory modalities (e.g. high pitch is matched with angular shapes). However, the possibilities cross-modality opens in the digital world have been relatively unexplored. Therefore, we consider that studying the plasticity and the effects of crossmodal correspondences in a mulsemedia setup can bring novel insights about improving the human-computer dialogue and experience. Mulsemedia refers to the combination of three or more senses to create immersive experiences. In our experiments, users were shown six video clips associated with certain visual features based on color, brightness, and shape. We examined if the pairing with crossmodal matching sound and the corresponding auto-generated haptic effect, and smell would lead to an enhanced user QoE. For this, we used an eye-tracking device as well as a heart rate monitor wristband to capture users’ eye gaze and heart rate whilst they were experiencing mulsemedia. After each video clip, we asked the users to complete an on-screen questionnaire with a set of questions related to smell, sound and haptic effects targeting their enjoyment and perception of the experiment. Accordingly, the eye gaze and heart rate results showed significant influence of the cross-modally mapped multisensorial effects on the users’ QoE. Our results highlight that when the olfactory content is crossmodally congruent with the visual content, the visual attention of the users seems shifted towards the correspondent visual feature. Crosmodally matched media is also shown to result in an enhanced QoE compared to a video only condition.

Sensory studies emerged as a significant influence upon Human Computer Interaction and traditional multimedia. Mulsemedia is an area that extends multimedia addressing issues of multisensorial response through the combination of at least three media, typically a non-traditional media with traditional audio-visual content. In this paper, we explore the concepts of Quality of Experience and crossmodal correspondences through a case study of different types of mulsemedia setups. The content is designed following principles of crossmodal correspondence between different sensory dimensions and delivered through olfactory, auditory and vibrotactile displays. The Quality of Experience is evaluated through both subjective (questionnaire) and objective means (eye gaze and heart rate). Results show that the auditory experience has an influence on the olfactory sensorial responses and lessens the perception of lingering odor. Heat maps of the eye gazes suggest that the crossmodality between olfactory and visual content leads to an increased visual attention on the factors of the employed crossmodal correspondence (e.g., color, brightness, shape).

Previous research has shown that adding multisensory media-mulsemedia-to traditional audiovisual content has a positive effect on user Quality of Experience (QoE). However, the QoE impact of employing mulsemedia in 360° videos has remained unexplored. Accordingly, in this paper, a QoE study for watching a 360° video-with and without multisensory effects-in a full free-viewpoint VR setting is presented. The parametric space we considered to influence the QoE consists of the encoding quality and the motion level of the transmitted media. To achieve our research aim, we propose a wearable VR system that provides multisensory enhancement of 360° videos. Then, we utilise its capabilities to systematically evaluate the effects of multisensory stimulation on perceived quality degradation for videos with different motion levels and encoding qualities. Our results make a strong case for the inclusion of multisensory effects in 360° videos, as they reveal that both user-perceived quality, as well as enjoyment, are significantly higher when mulsemedia (as opposed to traditional multimedia) is employed in this context. Moreover, these observations hold true independent of the underlying 360° video encoding quality-thus QoE can be significantly enhanced with a minimal impact on networking resources.

Different senses provide us with information of various levels of precision and enable us to construct a more precise representation of the world. Rich multisensory simulations are thus beneficial for comprehension, memory reinforcement, or retention of information. Crossmodal mappings refer to the systematic associations often made between different sensory modalities (e.g., high pitch is matched with angular shapes) and govern multisensory processing. A great deal of research effort has been put into exploring cross-modal correspondences in the field of cognitive science. However, the possibilities they open in the digital world have been relatively unexplored. Multiple sensorial media (mulsemedia) provides a highly immersive experience to the users and enhances their Quality of Experience (QoE) in the digital world. Thus, we consider that studying the plasticity and the effects of cross-modal correspondences in a mulsemedia setup can bring interesting insights about improving the human computer dialogue and experience. In our experiments, we exposed users to videos with certain visual dimensions (brightness, color, and shape), and we investigated whether the pairing with a cross-modal matching sound (high and low pitch) and the corresponding auto-generated vibrotactile effects (produced by a haptic vest) lead to an enhanced QoE. For this, we captured the eye gaze and the heart rate of users while experiencing mulsemedia, and we asked them to fill in a set of questions targeting their enjoyment and perception at the end of the experiment. Results showed differences in eye-gaze patterns and heart rate between the experimental and the control group, indicating changes in participants’ engagement when videos were accompanied by matching cross-modal sounds (this effect was the strongest for the video displaying angular shapes and high-pitch audio) and transitively generated cross-modal vibrotactile effects.<?vsp -1pt?>

Media content is consumed more and more on digital and mobile devices. Internet of Things enables media providers to move beyond the traditional content and entertainment space and offer a plethora of new services that creates an engaging user experience around content. However, one of the questions that remain open is how to design content that enhances users' quality of experience by widening the classic audio-visual multimedia content spectrum to stimulate also other human senses such as olfaction or touch. In this paper we look at systematic associations made between different sensory modalities (crossmodal correspondences) in a digital setup and on their influence on the quality of experience. Our experiments target various senses: audio, visual, olfaction and tactile and show that creating untraditional content that matches the visual dominants of the video does not contribute to an improved experience of mulsemedia.

Serious games are becoming an alternative educational method in a variety of fields because of their potential to improve the quality of learning experiences and to facilitate knowledge acquisition and content understanding. Moreover, entertainment-driven learners are more easily motivated to benefit from the learning process through meaningful activities defined in a game context. Interfacing educational computer games with multisensorial interfaces allows for a seamless integration between virtual and physical environments. Multisensorial cues can improve memory and attention and increase the cognitive and sensory-motor performance. Despite of the increasing knowledge in sensory processes, multisensory experiences and interactions in computer based instruction remain insufficiently explored and understood. In this paper, we present a multisensory educational game - Fragrance Channel - and we investigate how enabling olfaction can contribute to users' learning performance, engagement and quality of experience. We compare results obtained after experiencing Fragrance Channel in the presence and absence of olfactory feedback on both a mobile and a PC. A knowledge test administered before and immediately after showed that our proposed educational game led to an improvement of performance in all the explored conditions. Subjective measurements carried out after the olfactory experience showed that students enjoyed the scenario and appreciated it as being relevant.

The aim of this paper is to compare the learning effectiveness and learning motivation of digital game-based learning between the competitive and the cooperative mode. Forty-seven junior high school students were invited to participate in the learning experiment. The students were divided into two groups and played Fragrance Channel in two different modes. Pre-and post-tests were used to evaluate students' learning effectiveness under different conditions and an IMMS questionnaire was used to evaluate learning motivation. The results show no significant differences for both groups when evaluating learning effectiveness and motivation. However, both groups presented significant differences between their pre-and post-tests conditions when it comes to the acquired knowledge. We identified significant differences also for the satisfaction level showing that students enjoyed it more to play in the competitive mode.

In recent years, the emerging immersive technologies (e.g. Virtual/Augmented Reality, multisensorial media) bring brand-new multi-dimensional effects such as 3D vision, immersion, vibration, smell, airflow, etc. to gaming, video entertainment and other aspects of human life. This paper reports results from an European Horizon 2020 research project on the impact of multisensoral media (mulsemedia) on educational learner experience. A mulsemedia-enhanced test-bed was developed to perform delivery of video content enhanced with haptic, olfaction and airflow effects. The results of the quality rating and questionnaires show significant improvements in terms of mulsemedia-enhanced teaching.

People with cognitive disabilities are often socially excluded. We propose a system based on Virtual and Augmented Reality that has the potential to act as an educational and support tool in everyday tasks for people with cognitive disabilities. Our solution consists of two components: the first that enables users to train for several essential quotidian activities and the second that is meant to offer real time guidance feedback for immediate support. In order to illustrate the functionality of our proposed system, we chose to train and support navigation skills. Thus, we conducted a preliminary study on people with Down Syndrome (DS) based on a navigation task. Our experiment was aimed at evaluating the visual and spatial perception of people with DS when interacting with different elements of our system. We provide a preliminary evaluation that illustrates how people with DS perceive different landmarks and types of visual feedback, in static images and videos. Although we focused our study on people with DS, people with different cognitive disabilities could also benefit from the features of our solution. This analysis is mandatory in the design of a virtual intelligent system with several functionalities that aims at helping disabled people in developing basic knowledge in every day tasks.

The ability to commute and travel alone is an important skill that enables people to be more independent, and integrated with society. People with Down's Syndrome often experience low social integration, and low degree of independence. As part of the European Commission funded POSEIDON project, we want to explore how context-aware, and assistive technology can enable users with Down's Syndrome be more independent, including the ability to commute alone to a place of interest. In this paper, we report on our current progress in developing navigational services within the context of the POSEIDON project. We carried out a semi-structured qualitative evaluation of an early version of our navigational services with 6 individuals with Down's Syndrome, and report on our findings.

The use of Virtual Reality (VR) in sports training is now widely studied with the perspective to transfer motor skills learned in virtual environments (VEs) to real practice. However precision motor tasks that require high accuracy have been rarely studied in the context of VE, especially in Large Screen Image Display (LSID) platforms. An example of such a motor task is the basketball free throw, where the player has to throw a ball in a 46cm wide basket placed at 4.2m away from her. In order to determine the best VE training conditions for this type of skill, we proposed and compared three training paradigms. These training conditions were used to compare the combinations of different user perspectives: first (1PP) and third-person (3PP) perspectives, and the effectiveness of visual guidance. We analysed the performance of eleven amateur subjects who performed series of free throws in a real and immersive 1:1 scale environment under the proposed conditions. The results show that ball speed at the moment of the release in 1PP was significantly lower compared to real world, supporting the hypothesis that distance is underestimated in large screen VEs. However ball speed in 3PP condition was more similar to the real condition, especially if combined with guidance feedback. Moreover, when guidance information was proposed, the subjects released the ball at higher - and closer to optimal - position (5-7% higher compared to no-guidance conditions). This type of information contributes to better understand the impact of visual feedback on the motor performance of users who wish to train motor skills using immersive environments. Moreover, this information can be used by exergames designers who wish to develop coaching systems to transfer motor skills learned in VEs to real practice.