We present a real-time method for generating 3D Gaussian head avatars directly from monocular 2D video streams, without per-subject training or 3D morphable model fitting. We train a single model on a large collection of diverse video clips of human heads to generalize across identities, expressions, and poses. Input images are encoded with a pre-trained vision transformer into feature maps, from which both the static and dynamic components are inferred without relying on facial keypoints or parametric models. Dynamic information is injected into the avatar generation process via cross-attention, enabling both self- and cross-identity reenactment. Experiments on the VFHQ dataset show accurate and robust reconstruction performance and improved view point stability compared to the state-of-the-art. We propose a full end-to-end pipeline which is able to create, animate, and render avatars at over 20 fps, enabling new applications in live telepresence and interactive virtual characters.

This work-in-progress paper presents findings from a pilot study (N = 30), conducted as part of the Horizon Europe AMPLIFY project, investigating physiological responses to music. Participants listened to two counterbalanced musical excerpts while heart rate, electrodermal activity (EDA) and skin temperature were recorded via EmotiBit™ wearable sensors. Of 489 statistically significant individual deviations from baseline (Bonferroni-corrected Mann-Whitney U), 33 were shared across multiple participants. To check whether these occurred at the same instant, temporal analysis of the shared deviations identified 82 different episodes in which multiple participants deviated simultaneously. Comparing audio features during these episodes to the remainder of each musical excerpt yielded 27 Bonferroni-significant associations (Cohen's d up to 1.62). Broadband EDA and skin conductance response frequency emerged as the most responsive physiological measures, showing strong correlations with spectral brightness and timbral features of the musical passages.

Whiplash is a serious neck injury that can be caused by sudden movements of the neck such as a car accident. Recovery is based on physiotherapy designed to restore cervical movement and reduce pain. Long term engagement is difficult to achieve once patients are no longer in a clinical setting, which in turn affects overall recovery. The proposed research investigates how remote sensor based and gamification can support sustained engagement in whiplash recovery in a home based setting. The research will focus on the development of an AR application that incorporates non invasive sensors to observe neck and head movement during therapy. Acting as a virtual coach, this system will provide guidance, feedback, and motivational elements to encourage correct and consistent performance.

Functional recovery during upper-limb rehabilitation is often slow and difficult for patients to interpret, particularly when visible improvements in movement remain limited in the short term. This work-in-progress study explores whether a Mixed Reality (MR) approach to mirror therapy can strengthen a patient’s perceived sense of recovery during hand rehabilitation, and whether that perceptual shift may in turn support motivation, confidence, and engagement with prescribed physiotherapy exercises. We propose a Meta Quest 3 MR system in which a patient’s healthy hand is tracked and mirrored to visually represent the affected hand during guided rehabilitation tasks. Rather than positioning mirror therapy only as a motor rehabilitation technique, we frame it here as a perceptual intervention aimed at strengthening perceived recovery during periods when objective progress may remain difficult for patients to recognise. This paper presents the concept, motivation, and system design for exploring perceived recovery as a meaningful factor in MR upper-limb rehabilitation.

This paper investigates immersive Extended Reality (XR) broadcasting as a means of enhancing audience experience within production-realistic workflows. An XR broadcast system was iteratively refined through broadcaster-in-the-loop evaluations to improve usability and production stability. A controlled audience study then compared an immersive XR broadcast with a conventional 2D Twitch stream presenting identical content. Participants in the XR condition viewed the broadcast using head-mounted displays within a shared virtual studio, while participants in the 2D condition viewed the stream on desktop displays while physically co-located in the same classroom environment, providing a strong baseline that preserved interpersonal awareness.

Extended Reality (XR) refers to a group of computer-generated technologies, including Augmented Reality (AR), Mixed Reality (MR), and Virtual Reality (VR). It is a rapidly developing field that has found applications across various high-performance sectors such as rehabilitation, manufacturing, and medicine. Despite this growth, its adoption in sports is still relatively limited and not yet fully explored.

Virtual reality (VR) has shown increasing potential in supporting the learning and development of autistic children. However, the design of VR scenes remains unexplored compared to the technology’s implementation and intervention effectiveness. This study aims to fill this gap by focusing on how to design VR environments to better meet the needs of autistic children. We used a mixed approach, collecting feedback from six experts and six parents regarding their expectations for scene design, as well as eye-tracking data from VR experiences. The findings highlight the importance of different types of stakeholder involvement in design decisions and identify key elements in VR scene design. Based on these insights, future work proposes developing and validating a structured set of VR scene design guidelines, a process that combines systematic review of the literature with expert evaluation. This approach ensures that the final guidelines are theoretically sound and practically applicable. The anticipated contribution is to provide a validated framework by experts to support the design of user-centered VR environments for autistic children, thereby advancing research in the field of immersive and interactive media.

It is well accepted that the lack of design guidelines for virtual reality (VR) experiences for autistic children is limiting VR as a support or educational tool.Autistic children may have unique sensory and cognitive needs making typically designed VR applications difficult to experience and tolerate.To better understand factors that should be considered within this context, this paper presents the results of an expert evaluation of a VR application that was developed to introduce the concept of sharing.Six experts working in the field of autism were invited to take part in an experimental evaluation of an immersive VR social story application.The evaluation methods included questionnaires (open-ended) and some behavior analysis (e.g.eye tracking).The experts provided valuable feedback and detailed suggestions on the VR scene design, including lighting, subtitles, and animation design.By capturing the gaze data of the experts, it was found that there were significant differences in the duration of their gazes at different objects of interest in the VR environment.This provided valuable insights in terms of the VR production of social stories.In the future, we plan to expand these insights by incorporating additional stakeholder comments and behavior analysis.This ongoing research aims to develop more inclusive and effective VR design principles for applications for autistic children.

Although audio-augmented reality (AAR) has known applications in music, the use of wearables such as augmented reality (AR) glasses for egocentric audio data capture for music has not been investigated. Current egocentric datasets are mostly focused on speech research, neglecting music's unique demands for tasks such as real-time optimisation or assistive listening. This paper introduces EgoMusic, a multimodal dataset featuring synchronised egocentric audio-visual data captured with AR glasses during live performances, alongside studio-quality audio references. We investigate AR glasses' utility for music and baseline artificial intelligence (AI) approaches for hearing enhancement, positioning EgoMusic as the first dataset that enables research for egocentric music AAR.

As immersive 360° video experiences through head-mounted displays (HMDs) gain widespread adoption, the need for real-time, fine-grained assessment of Quality of Experience (QoE) becomes increasingly critical for optimising user engagement and system performance. This paper introduces RCQoEA-360VR, a novel multi-modal dataset designed for continuous QoE evaluation in virtual reality (VR) environments. In a controlled study (N=32), participants watched five selected 360° video sequences across eight different video quality configurations (from the VQEG database) using a Vive Pro Eye while providing continuous QoE annotations via a touchpad-based input method, enhanced by the DotMorph peripheral visualisation technique. The dataset also includes synchronised physiological signals (electrocardiogram and galvanic skin response), behavioural data (eye and head movements) and post-viewing QoE ratings gathered through a within-VR interface. RCQoEA-360VR addresses a critical gap in existing public datasets by providing a fine-grained, synchronised multimodal data for immersive QoE analysis. It offers a unique and valuable resource for the research community, supporting a wide range of research applications, including QoE prediction, behavioural modelling, adaptive streaming, and implicit perceptual analysis.

Individuals who experience a significant loss of vision or blindness often face challenges navigating the world. In such scenarios, there is often a dependence on various types of support or aids to enable individuals to move through their surroundings. Examples of this include guide dogs and assistive white canes. Those who suffer from blindness also rely on a heightened sense of auditory processing. Despite these solutions, navigating the modern world is challenging from a safety perspective.

This paper presents a no-code eXtended Reality (XR) workflow that allows non technical people to create XR experiences as part of equipment design review (EDR) processes. The demo showcases how the workflow can enable seamless visualization, interaction, and evaluation of complex CAD models in XR. By combining PiXYZ for CAD-to-mesh translation with Unity's XR Interaction Toolkit and Meta XR SDK, the system delivers immersive experiences that enhance spatial perception and reduce cognitive load for mechanical engineers. The demo also incorporates a stylus based approach for adding hand-drawn notes in real-time on top of imported CAD models. Attendees will witness the complete XR creation pipeline, from CAD import to a live Mixed Reality based design review, demonstrating efficiency, scalability, and potential to revolutionize XR-driven engineering workflows.

Immersive extended reality (XR) experiences typically rely on audio and visual cues, which can unintentionally exclude deaf and hard-of-hearing (DHH) individuals due to their limited access to auditory feedback. In response, this demo introduces a system that converts audio into haptic feedback, enabling DHH users to perceive sound through tactile sensations. The goal of this research is to investigate how multisensory technologies can enhance accessibility in immersive environments. By stimulating senses beyond the traditional audio-visual modalities, the system serves as an initial platform for creating inclusive experiences across a range of applications, including gaming, cinema, and live events. This approach aims to promote more equitable participation in XR by addressing sensory barriers through innovative design.

In recent years, sensor-based technologies have significantly expanded the scope of what is possible with Extended Reality (XR) systems offering not only new modes of interaction but also new ways to enhance user experience. This has resulted in significant impact in different application domains, with long-term healthcare one such example. Remote healthcare and specifically in physical rehabilitation bring many interesting challenges both from clinical and research perspectives. Conditions like Ulnar Nerve Palsy, which is characterised by degradation of a patient’s nerve causing the loss of sensation and muscle weakness in a hand, calls for precise and continuous motor therapy. Immersive solutions can potentially be a solution that can extend beyond the care of clinical settings. This demand highlights not only the clinical need, but also the importance of how systems are designed to foster patient engagement, comfort and confidence during therapy.In this paper a sensor-based glove system tailored for the needs of ulnar nerve recovery is presented. It is designed not just to support functional rehabilitation in the hope of achieving long-term Quality of Life (QoL) improvements but is informed by the patients Quality of Experience (QoE) through clear feedback, responsive interaction, and a comfortable wearable design. By enhancing the perception of quality during the rehabilitative process, the system aims to improve overall adherence, increase motivation and support improvements in patients’ functional independence and quality of life.

Recent advancements in Virtual Reality (VR) technologies have significantly enhanced immersive experiences, bridging the gap between physical and digital worlds.Modern Head-Mounted Displays (HMDs) and motion capture systems such as the KAT VR treadmill have transformed VR into a dynamic platform for applications ranging from entertainment to fitness.These innovations offer opportunities to integrate physical activity within virtual environments, promoting engaging and interactive experiences.This paper presents an overview of a work-in-progress quality of experience (QoE) evaluation of a VR fitness system that explores the utility of a VR locomotion system.Explicit measures will be captured using traditional post-experience questionnaires, whilst implicit measures of QoE in the form of heart rate, electrodermal activity, and skin temperature will be captured using the Emotibit wristband.In parallel, the system will monitor objective metrics which aim to understand the number of calories burned, and distance travelled.The findings aim to highlight the potential of VR as a tool for promoting physical health while contributing to the growing field of fitness-focused social VR.

Auditory attention is a fundamental cognitive process essential for effective communication and interaction. Auditory stimuli are often accompanied by distractors that can significantly impact task performance by means of reducing attention. This study investigates the influence of auditory and audiovisual distractors on an auditory attention task within a Virtual Reality classroom. As part of the user evaluation, participants had to listen to two short stories. They performed two tasks: (i) “DISTRACTORS”, where participants had to identify an auditory or an audiovisual stimulus and (ii) “KEYWORDS,” where participants had to identify a specific keyword in the story by pressing a button on the controller. During the experiment, the participants’ physiological (e.g. skin conductance levels, gaze data, etc.) and subjective data (i.e. questionnaires) were collected. The results revealed that the interval between the presentation of the keyword and the presentation of the distractors impacted task performance by negatively affecting auditory attention. Also, it was observed that the time spent looking at the speaker telling the story positively correlated with task performance, whereas the time spent looking at distractors was found to be negatively correlated with task performance. Finally, this study gives insights into how physiological metrics can be used to infer auditory attention in VR experiences.

Limited research has explored the impact of spatial audio on viewer experience in 360° videos. This research aims to address this gap by investigating the effects of spatial versus non-spatial audio configurations on the Quality of Experience (QoE) and visual attention. Spatial audio simulates real-world soundscapes, while non-spatial audio lacks contextual depth. A robust methodology combining subjective assessments with head pose and eye tracking and physiological response analysis provides insights into the role of audio spatiality. Preliminary results suggest that spatial audio significantly enhances immersion and directs visual attention. These findings offer actionable guidelines for content creators and contribute to the understanding of audio-visual integration in immersive content. Additionally, the research informs potential advancements in content delivery optimization. Spatial audio understanding can guide adaptive streaming techniques to prioritize relevant audio streams, potentially reducing bandwidth requirements. This can improve viewer experience in bandwidth-constrained environments.

In recent years, the Virtual Reality (VR) sector has blurred boundaries between the the real world, and virtual world. Key advances in display technologies have led to greater levels of immersion within VR. HMD’s are now lighter and allow for more interactive experiences. Despite these impressive changes, VR enthusiasts continue to tighten the gap between physical and digital spaces by stimulating additional senses using immersive accessories. Haptic feedback is a new and emerging consumer area within this space.Although technologies exist to stimulate haptic feedback within VR, they are often expensive and therefore out of reach for many consumers. As a result, a new generation of eXtended Reality (XR) hardware enthusiasts have emerged, establishing workflows to create low costs systems to stimulate the senses beyond audiovisual. In this paper, we investigate the process and challenges of building one such low-cost solution taking inspiration from LucidVR and OpenGlove SDK.

In order to understand the sentiments between users as they worked together as part of a collaborative Virtual Reality (VR) task, this research analyzed voice interaction using Assembly AI. The real-time conversations between 11 pairs of users were captured as they performed a joint puzzle task. The users were in separate physical spaces but connected in a shared virtual setting. The extracted voice sentiments were correlated with self-reported frustration levels and task completion times. The findings reveal a significant positive correlation between negative sentiments and frustration (r=0.73, p≤0.05) and a negative correlation between positive sentiments and task completion time (r=-0.87, p≤0.001). This shows the relationship between sentiment (positive and negative) and task completion in collaborative VR. These results highlight the potential benefits of analyzing voice sentiment in collaborative VR applications.

The field of user quality of experience (QoE) in multimedia communications has become increasingly important due to the widespread use of digital technology in our everyday lives. The ability to accurately predict user QoE by processing physiological signals has significant applications. This study proposes a machine learning (ML) approach for predicting user QoE using physiological signals. It focuses on perceived overall quality and perceived audio quality by processing electrocardiogram (ECG) and respiration signals from the SoPMD Dataset 2. The study evaluated various ML models on individual and fused modalities while implementing dimensionality reduction, feature selection, and hyperparameter tuning. The SHapley Additive exPlanations (SHAP) method was used to interpret the ML models' outputs, which helped identify the features providing the most utility. The results show that the random forest model provided the best performance, with an F1-score of 87.91% for fusion data and 80.49% for ECG data in classifying perceived audio quality and overall quality, respectively. These results imply that physiological signals, such as ECG and respiration, hold great potential for predicting user QoE in multimedia experiences.

Social interactions hold immense significance in the lives of individuals. Children with a diagnosis of autism spectrum disorder (ASD) may face difficulties when engaging in social situations. These challenges often manifest as neurodivergent behaviours and a reduced willingness to engage in social sharing. Sharing is an integral aspect of social interaction. A lack of awareness or a limited inclination to share can hinder the acquisition of vital social skills. In this paper, a virtual reality (VR) application inspired by the Social Story™ intervention is presented. The aim of the work is to help children with ASD understand social scenarios and gain sharing skills. The application can be used in two configurations: 2D screen and head-mounted display (HMD). Since the system is being designed for a vulnerable group, as part of the iterative application design process, we solicited opinions from six expert professionals who work with children with ASD. They tested, evaluated (questionnaires and physiological data capture) and provided open feedback on the system. The key recommendation from the expert assessors is that the HMD-based interventions are more suitable as an advanced teaching tool for showing skills and knowledge, whereas screen-based interventions could be used for daily practice in children with ASD. Experts evaluated the two configurations from three factors: immersion, enjoyment, and comprehension. In terms of physiological responses, there was a greater increase in electrodermal activity (EDA) responses for the HMD-based intervention compared to the 2D intervention. The significant potential of such a system was highlighted by the expert users.

This article presents the results of an empirical study that aimed to investigate the influence of various types of audio (spatial and non-spatial) on the user quality of experience (QoE) of and visual attention in 360° videos. The study compared the head pose, eye gaze, pupil dilations, heart rate, and subjective responses of 73 users who watched ten 360° videos with different sound configurations. The configurations evaluated were no sound; non-spatial (stereo) audio; and two spatial sound conditions (first- and third-order ambisonics). The videos covered various categories and presented both indoor and outdoor scenarios. The subjective responses were analyzed using an ANOVA (Analysis of Variance) to assess mean differences between sound conditions. Data visualization was also employed to enhance the interpretability of the results. The findings reveal diverse viewing patterns, physiological responses, and subjective experiences among users watching 360° videos with different sound conditions. Spatial audio, in particular third-order ambisonics, garnered heightened attention. This is evident in increased pupil dilation and heart rate. Furthermore, the presence of spatial audio led to more diverse head poses when sound sources were distributed across the scene. These findings have important implications for the development of effective techniques for optimizing processing, encoding, distributing, and rendering content in virtual reality (VR) and 360° videos with spatialized audio. These insights are also relevant in the creative realms of content design and enhancement. They provide valuable guidance on how spatial audio influences user attention, physiological responses, and overall subjective experiences. Understanding these dynamics can assist content creators and designers in crafting immersive experiences that leverage spatialized audio to captivate users, enhance engagement, and optimize the overall quality of VR and 360° video content. The dataset, scripts used for data collection, ffmpeg commands used for processing the videos, and the subjective questionnaire and its statistical analysis are publicly available.

The current definition of quality of experience (QoE) designates delight and annoyance as diametrically opposing indicators of the degree of fulfilment of an application, service or system user's pragmatic and hedonic needs and expectations. However, these inherited emotion terms are rarely used to describe emotions of equal amounts of arousal or opposing amounts of valence in the literature. This work assesses the significance of this asymmetry to the definition of QoE by determining the utility of emotion terms to communicate the emotion component of QoE. This was done in the context of a QoE evaluation of augmented reality training instruction formats. Correlates were sought between various measures of emotional state. This included physiological ratings, facial expressions and eye gaze. Emotional state was subjectively reported using three distinct methods: self-assessment manikin questionnaire; 2D emotion space terms; and open-ended terms. Regression analysis showed multiple significant correlations between implicit and explicit metrics, but not to the emotion terms used by the participants. This calls into question the utility of such vaguely understood terms. The use of more symmetrically opposing emotions in the definition of QoE may benefit consensual interdisciplinary communication.

This paper addresses the need to understand the social dynamics of multiple users collaborating in a Virtual Reality (VR) environment. Specifically, we investigate whether the physiological synchrony between leader-follower task roles in Collaborative Virtual Environments is influenced by the user's previous experience and background. Physiological synchrony is a measure of spontaneous temporal coordination or interdependency between the physiological activity of two or more individuals. Electrodermal activity signals from 8 pairs of users were collected and processed using two different algorithms to measure their synchronization. The results show that physiological synchrony has a moderate correlation with task completion time in VR. Also, the Dynamic Time Warping (DTW) algorithm is more effective in computing synchrony. The findings show correlations between DTW distance, and task performance measures (completion time, and pre-construct scores).

Virtual Reality has gained significant interest with the advancement of display and processing technologies in recent years. Traditionally, visual stimuli were the main point of interest in the development of new VR applications. However, audio is also key to the success of such experiences. Audio technology is responsible for making the VR environment more immersive as it is directly correlated with how we perceive the world. Our real-world environments are multimodal and multisensory. In this work, we investigate the effect of two types of distractors in an auditory selective attention task. Users were immersed in a virtual classroom with spatialised audio enabled. The task itself was divided into two subtasks. First, participants had to identify different types of auditory and audiovisual stimuli in the scene. Second, they had to focus their attention on a speaker in front of them and identify a keyword from the story each time they heard it. Furthermore, participants were divided into three groups that differ on when the distractor is presented in the second subtask: (a) before, (b) same time, and (c) after the keyword. Findings from this study show that there are significant gender differences for listeners immersed in an environment with competing sounds in recalling a story. Moreover, the time when distractors are presented significantly affected the response time, being inversely proportional to the mean response time.

In recent years, museums have become more interactive and immersive through the adaptation of technology within large scale art exhibitions. Due to these changes, new types of cultural experiences are more appealing to a younger audience. Despite these positive changes, some museum experiences are still primarily focused on visual art experiences, which remain out of reach to those with visual impairments. Such unimodal and visual dominated experiences restrict these users who depend on sensory feedback to experience the world around them.

Social interactions and communication play a crucial role in people’s lives. Those with autism spectrum disorder (ASD), especially children, may have difficulties participating in social interactions. Such challenges can be characterised by displaying atypical behaviours and limited sharing intention in social settings. Sharing is an important part of social interaction, and a lack of awareness or limited willingness to share undermines the development of social skills. These characteristics may be related to the impaired theory of mind (ToM). This means that it is difficult to understand people’s wishes and feelings. A range of interventions have been created to help develop social communication skills. The Social Story™ intervention is one such example, and it provides clear visual narratives to explain social situations and concepts to help children with ASD. The narratives provide a mechanism to visually communicate typical communication behaviours. The social story intervention approach is book-based. As such, it is dependent on a reader to communicate well the concepts and demands a certain level with respect to the listener’s imagination capacity. With the limitation of the paper-based medium in mind, this work-in-progress paper outlines the steps, approach, and end application to translate the Social Story™ into a virtual reality (VR) experience. The Social Story™ experience in VR potentially offers a more interactive, immersive and flexible intervention.

Exactly-once data processing/delivery can be guaranteed in traditional big data processing systems, e.g. Apache Flink. Checkpoint is commonly used as the solution. Each operator in these systems can restart from the last successfully saved state whenever a failure happens. It is not necessary to restore the logical job graph onto the same device(s) in traditional datacentre scenarios with powerful servers close to each other. However, the datacentre oriented solutions are not suitable for IoT collaborative edge computing scenarios. The logical job graph is tightly coupled to the physical topology in IoT networks. Data processing task(s) cannot be placed at a random edge device to recover from a network failure as it needs to evaluate the benefits of transmitting data versus processing/aggregating the data. To address the above challenges, this paper proposes an Information Centric Networking based solution and correspondent protocols to provide Exactly-once-computation for the Collaborative Edge in IoT (ECE). It contains a job execution scheme to deliver IoT jobs with exactly once data computation guarantee and a recovery procedure to dynamically change the IoT job execution graph while experiencing link failures. The protocol also provides a checking procedure on data state (received/un-received and computed/un-computed) to prevent any data loss or duplicated data processing due to the updated job graph. A data identification approach based on the job graph is devised to support the ECE functionality. A testbed has been developed on ndnSIM and the simulation results have verified the feasibility and scalability of ECE design. It also evaluates the overhead incurred by the ECE protocol to guarantee exactly once data computation.

Conditional automation may require drivers of Autonomous Vehicles (AVs) to turn their attention away from the roads by taking part, for instance, in Non-Driving Related Tasks (NDRTs). That might cause them a lack of Situation Awareness (SA) when resuming to manual control. Consequently, Take-Over Requests (TORs) are system events intended to inform drivers when the vehicle is unable to handle an upcoming situation that is outside its Operational Design Domain (ODD) and the automated system might get disengaged, requiring driver's attention. The short time frame of time-critical TOR events impacts on the performance of the machine-to-human transition, especially when the driver is engaged with NDRTs. This can lead to dangerous driving conditions. In that context, this work proposes the demonstration of a device called Adaptive Tactile Device (ATD), capable of continuously adjust itself according to the driving conditions and smooth control transition by constantly informing the driver about road and system state based on force haptic feedback. The Continuous-Time (CT) nature of the proposed device is intended to provide adaptive feedback during automated vehicle conditions, promoting a feeling of control and possibly improving driver's Situation Awareness (SA) during TOR events. Future work will consider implementing this device on the steering wheel or driver's seat and collect the user's Quality of Experience (QoE) when using it in Virtual Reality (VR) simulations, to be compared with the user's objective and subjective metrics when receiving Discrete-Time (DT) feedback warnings previously applied in TOR research.

Immersive technologies are rapidly gaining traction across a variety of application domains. 360° video is one such technology, which can be captured with an omnidirectional multi-camera arrangement. With a Virtual Reality (VR) Head Mounted Display (HMD), users have the freedom to look in any direction they wish within the scene. While there is a plethora of work focused on modeling visual attention (VA) in VR, little research has considered the influence of the audio modality on VA in VR. It is well known that audio has an important role in VR experiences. Listeners can experience sound in all directions with high quality spatial audio. One such technique, Ambisonics or 3D audio, provides a full 360° sound soundscape.

Augmented reality (AR) has significant potential as a training platform. The pedagogical purpose of training is learning or transfer. Learning is the acquisition of an ability to perform a procedure as taught while transfer involves generalising that knowledge to similar procedures in the same domain. Quality of experience (QoE) concerns the fulfilment of the application, system or service user's pragmatic and hedonic needs and expectations. Learning or transfer fulfil the AR trainee's pragmatic needs. Training instructions can be presented in procedural, and example formats. Procedural instructions tell the trainee what to do while examples show the trainee how to do it. These two different instruction formats can influence learning, transfer, and hardware resource availability differently. The AR trainee's hedonic needs and expectations may be influenced by the impact of instruction format resource consumption on system performance. Efficient training efficacy is a design concern for mobile AR training applications. This work aims to inform AR training application design by evaluating the influence of procedural and example instruction formats on AR trainee QoE.

Wearable sensors are worn by subjects to allow for continuous physiological monitoring. The use of wearable sensors for the quantification of movement within research communities has increased in recent years, with the purpose of objectively assessing and diagnosing the progression of Parkinson's Disease (PD). Most studies taking this approach for PD have stated that there is a need for a long-term solution, due to individuals having varying symptoms at different stages of the disease. Furthermore, a preference for home-based care has increased in recent times due to COVID-19, with clinical care being highly effected due to cancellations, delayed appointments, or a reduction of time spent with patients. The necessity for a system for patients with Parkinson's is extremely significant. There is no clinically available long-term assessment for tremors, and how these systems can be used to assess and aid in a clinical environment is still underdeveloped. The proposed system which includes wireless sensors, and results based off the clinical scale used currently for tremor assessment, may allow for constant, real-time, and accurate monitoring of a subject with tremors. This will provide more detailed medical data to enable long-term assessment, diagnosis, as well as person-centered physical therapy.

This paper contains the research proposal of Samantha O'Sullivan that was presented at the MMSys 2022 doctoral symposium. The use of wearable sensors for the understanding and quantification of movement within research communities working on Parkinson's Disease (PD) has increased significantly in recent years with a motivation to objectively diagnose, assess and then understand the progression of the disease. Most studies taking this approach for PD have stated that there is a need for a long-term solution, due to varying symptoms at different stages of the disease. COVID-19 has brought further limitations in the delivery of clinical care, reducing time with therapists and doctors whilst increasing the preference for at-home care. The necessity for a system for patients with PD is extremely significant. There is no clinically available long-term assessment for tremors, which is an issue highlighted in the literature. By using wireless sensors to track tremor severity continuously, and telehealth to create communication between patient and clinician, this proposed system will allow for better targeted therapy, accurate statistics, and constant accessible data. In this context, this work will design, build, and evaluate a novel system that would allow for constant monitoring of a patient with tremors. By using wireless sensors and telehealth, it will provide more detailed data that may enable directed and informed physical therapy. It will also improve communication creating a data flow constantly between clinician and patient to improve person-centered feedback, and aid towards the diagnosis and assessment of disease progression. The incorporation of a mobile/cloud-based application to assist this is due to the current heightened preference for home-based healthcare, long-term evaluation of tremors and personalized physical therapy. The primary focus of the PhD will be on capturing tremor activity and progression through a telehealth-based system. This proposed system will obtain real-time readings of tremors using wireless sensors and an application that will communicate consistently with healthcare professionals. The aim will be to provide better home-based care, person-centered physical therapy and improve quality of life.

This paper contains the research proposal of Mary Pidgeon that was presented at the MMSys 2022 doctoral symposium. Emotion recognition from physiological signals has seen a huge growth in recent decades. Wearables such as smart watches now have sensors to accurately measure physiological signals such as electrocardiography (ECG), blood volume pressure (BVP), galvanic skin response (GSR), and skin temperature (ST). These sensors have also been embedded in textiles. Collaborative body sensor networks (CBSN) have been used to analyse emotion reactions in a social setting from heart rate sensors. Federated learning, a recently proposed machine learning paradigm, protects user's private information while using information from several users to train a global machine learning model. Federated learning has several categorisations based on data partitioning, the privacy mechanisms, machine learning models and methods for solving heterogeneity. In this doctoral thesis, we propose using a smart clothing body sensor network to collect peripheral physiological data while protecting the user's privacy using federated machine learning. We present three primary research questions to address the challenges in emotion prediction, data collection from e-textile sensors and federated (FL) learning.

Immersive multimedia technologies such as virtual reality (VR) are now finding potential applications in domains outside of entertainment and gaming in areas such as health, education, and tourism, to name a few. This article presents a Quality of Experience (QoE) evaluation of an immersive haptic-based VR wheelchair simulator. The paper presents the results of an explicit and implicit (physiology-based) QoE evaluation of the Immersive Simulator in three different configurations: (a) desktop group (non-immersive); (b) headset 1 group (immersive with a high rate of motion acceleration); and (c) headset 2 group (immersive with a lower rate of motion acceleration). As part of the user evaluations, participants in each of the groups completed several questionnaires, including: an emotion questionnaire (SAM), cognitive task load (NASA-TLX), user expectations usability (SUS), and presence (IPQ). In addition, during the experience, physiological responses such as electrodermal activity (EDA) and heart rate variability (HRV) were recorded. The self-reported findings suggest that both headset groups had higher usability and presence levels than the desktop group. The two headset groups also had greater pleasant and exciting emotions than the desktop group. The NASA-TLX findings indicate that the headset 1 group presented the highest task cognitive load. The performance evaluation shows that both headset groups had a better results than the desktop group in terms of task completion.

Industry 4.0 is aimed at the full manufacturing domain automatization and digitalization. Humans and robots working together are being discussed widely both in the academic and industrial sectors. As being discussed, there is a need for a more novel type of interaction method between humans and robots. This demonstrational paper is presenting the technical advancement and prototype of remote control and re-programming of the Industrial robot. This development is safe and efficient and allows the operator to focus rather on the final task than the programming process.

This paper contains the research proposal of Sowmya Vijayakumar that was presented at the MMSys 2022 doctorial symposium. Multimedia applications can now be found across many application domains including but not limited to entertainment, communication, health, business, and education. It is becoming more and more important to understand the factors that influence user perceptual quality, and hence monitoring user quality of experience (QoE) for improving multimedia interaction and services is essential. In this PhD work, we propose advanced machine learning techniques to predict QoE from physiological signals for immersive multimedia experiences. The aim of this doctoral study is to investigate the utility of physiological responses for QoE assessment for different multimedia technologies. Here, the research questions and solutions proposed to address this challenge are presented. A multimodal QoE prediction model is being developed that integrates several physiological measurements to improve QoE prediction performance.

This paper presents an evaluation of a deep learning (DL) model to predict the user quality of experience (QoE) from physiological signals using a publicly available multimodal dataset, SoPMD. The subjective scores related to QoE factors, namely, perceptual video quality, immersion level, surrounding awareness, interest in video content and audio content are evaluated. A DL model, Bidirectional Long-short-term memory (BiLSTM), is trained on the fusion of electrocardiogram (ECG) and respiration features to predict subjective scores for the five QoE factors. This study achieved classification accuracies and Fl-scores ranging between 58% and 67% for different QoE factors. The results of the BiLSTM model were compared with machine learning techniques. The experimental results demonstrated that the proposed BiLSTM network has the potential to predict QoE from physiological signals.

The conventional smart gloves present a challenge regarding their portability as most work on gesture recognition techniques based on vision sensing and image processing. The multiple algorithms and signal filtering further make the overall process cumbersome. This work proposes a Shape Memory Alloy (SMA) integrated sensing mechanism in a smart glove for autonomous control. A novel hand gesture recognition technology is developed using kinaesthetic feedback from the finger joint movements. The paper presents a smart glove with an external SMA embedded tubing attachment for the thumb, index, and middle fingers. The motion of the SMA wires is constrained between a fixed end on the tip of the fingers, and the other end is connected to a linear position sensor with spring feedback. The SMA wires in this design exist in their Austenite phase at room temperature, thus exhibiting superelastic or pseudoelastic behavior. The tension in the SMA wire is observed and measured upon bending the fingers, corresponding to the mechanical travel in the linear position sensor. The individual and a combination of position sensor readings are then used as commands for actuating interactive toys. Using a three-finger approach, one can extract seven commands depending upon single or multiple finger movements. This data is further used to actuate the toys, and a use-case for cobotic application is proposed to help better understand interactive play, hand-eye coordination, and thus early cognitive development in children with Autism Spectrum Disorder (ASD). The discrete data output with binary data is independent of other devices or heavy data processing requirements, thus making the proposed novel SM-EXO a better alternative for non-portable and complex smart gloves.

Due to COVID-19, crowdsourcing has gained momentum as an alternative methodology for continuing research and for Quality of Experience (QoE) assessment. Employing this approach, we remotely evaluated the user perceived QoE of two different visual rendering formats as part of an Autonomous Vehicles (AVs) simulation. The aim was to investigate the participant's QoE when testing AV technology in distinct visual rendering qualities (lowpoly vs high-poly) of an online streamed 360° car riding experience. In addition, a scoring model based on the expected reliability of each level of the remote assessment was designed. Findings suggest that the consumer's preferences towards the adoption of AV technology is highly determined by the system and human effects on Influence Factors (IFs). Moreover, the adequacy of reliability into a mathematical model is highlighted as a potential turning point for QoE assessment, by carrying out the evaluation tasks from the laboratory environment into the internet, particularly relevant in pandemic times.

The availability of affordable head-mounted display technology has facilitated new, potentially more immersive, interactive multimedia experiences. These technologies were traditionally focused on entertainment; however, academia and industry are now exploring applications in other domains such as health, learning and training. Key to the success of these new multimedia experiences is the understanding of a user's perceived quality of experience (QoE). Subjective user ratings have been the primary mechanism to capture insights into a user's experience. Such ratings have generally been captured post experience and reflected using a mean opinion score (MOS). However, user perception is multifactorial and subjective ratings alone do not express the true measure of an experience. As a result, recent efforts to capture QoE have included exploring the use of implicit metrics (e.g., physiological measures). This article presents the results of an experimental QoE evaluation and comparison of immersive applications delivered across three multimedia platforms. The platforms compared were augmented reality, tablet and virtual reality. The QoE methodology employed considered explicit (post-test questionnaire) and implicit (heart rate and electrodermal activity) assessment methods. The results indicate comparatively higher levels of QoE for users of the augmented reality and tablet platforms.

Real world virtual world communication and interaction will be a cornerstone of future intelligent manufacturing ecosystems. Human robotic interaction is considered to be the basic element of factories of the future. Despite the advancement of different technologies such as wearables and Augmented Reality (AR), human-robot interaction (HRI) is still extremely challenging. Whilst progress has been made in the development of different mechanisms to support HRI, there are issues with cost, naturalistic and intuitive interaction, and communication across heterogeneous systems. To mitigate these limitations, RoSTAR is proposed. RoSTAR is a novel open-source HRI system based on the Robot Operating System (ROS) and Augmented Reality. An AR Head Mounted Display (HMD) is deployed. It enables the user to interact and communicate through a ROS powered robotic arm. A model of the robot arm is imported directly into the Unity Game engine, and any interactions with this virtual robotic arm are communicated to the ROS robotic arm. This system has the potential to be used for different process tasks, such as robotic gluing, dispensing and arc welding as part of an interoperable, low cost, portable and naturalistically interactive experience.

The driver/passenger role in autonomous driving experiences is an important subject that is concerned with how these systems will interact and communicate with the human stakeholder. Even though self-driving technologies have made rapid progress in recent years, their success will depend on ethics, safe capabilities and user acceptance. The overall satisfaction resulting from the interaction between humans, these technologies and their evaluation in simulators can help industry to improve their design. Such evaluations can be based on user's perceived Quality of Experience (QoE). The impact of the quality features that compound VR environments is an important factor to understand. Sense of depth, texture quality and resolution are some examples that influence the perceived quality of the simulation and immersion levels. This demonstration promotes a VR autonomous driving experience in one street of Athlone, Ireland. The application will be presented in two different formats: photogrammetry, a technique that uses photos to provide realistic 3D content, and secondly non-photorealistic simulated environment. Further investigations will be carried out to understand factors that enable the highest immersive experience in autonomous vehicles considering feedback modalities between the car and its users.

Augmented reality (AR) is a key technology to enhance worker effectiveness during increasing automation of repetitive jobs in the workplace. AR will achieve this by assisting the user to successfully perform complex and frequently changing procedures. The design of AR applications for these roles is critical to their acceptability and utility. User quality of experience (QoE) will inform these design decisions. A user arrives at a quality judgment upon post-experience reflection of their degree of delight or annoyance relating to the degree of fulfilment of pragmatic and hedonic needs and expectations of the medium under consideration. QoE researchers have largely depended upon post-experience subjective reports to determine the user's QoE. Subjective reports have been shown to be biased by primacy, recency and maxima of experience stimuli. Recent research involves the identification of implicit metrics that can be used to determine user QoE continuously during a multimedia experience. This work evaluates head rotation frequency as an objective metric of user QoE. The literature shows that emotion is expressed in the frequency of a person's head rotation around three axes of movement (pitch, yaw and roll). Low frequency head rotation has been shown to include expression of happy emotion while high frequency exclusively expresses anger emotion. These emotions are analogous to those reflected on by a user during the quality formation process. This demo paper analyses the amount of high frequency head rotation exhibited by the user upon task completion using an AR procedure assistance application or a paper-based control. An optimal Rubik's Cube solving AR application was used as a proof of concept for AR-based procedure assistance. Preliminary results showed that the AR environment yielded higher task success rates and significantly shorter task completion durations. The AR users exhibited significantly lower amplitudes of anger frequencies in their head rotations than the control group.

Virtual reality (VR) has proven to be a powerful tool enabling the development of immersive multimedia experiences. Initially focused on entertainment, industry and academia have begun to adapt and develop immersive applications for the healthcare domain, with opportunities in terms of condition assessment, diagnosis and intervention. In the context of immersive applications, audio, and in particular spatial audio, plays an important role on the immersion level. In order to process this information, the auditory cortex uses spatial cues encoded in the sound to provide relevant information about the distance, intensity and direction of the sound source. However, many different types of listening disorders can affect this capability. One condition, central auditory processing disorder (CAPD), significantly affects a user's ability to discriminate between different sound sources. People who suffer with this condition, are incapable of processing sounds properly, which may be stressful and frustrating when doing tasks with complex sounds or in noisy environments. This can have a significant impact on a person's quality of life. In this paper, an immersive VR spatial audio application is presented. It enables us to evaluate the ability of users to specify or localise the source of a sound. An integrated sensing system continuously collects relevant data from the user in order to fully understand how to quantify and evaluate spatial auditory skills from a quality of experience (QoE) perspective. QoE gives insight into a user's state and behaviour. To perform a detailed QoE evaluation of the listening task, implicit and explicit metrics were collected from the user. These included: self-reporting questionnaires, localisation performance, and physiological metrics. Data collected from this QoE evaluation gives an insight into a user's abilities to localise sound sources in VR, and also provides information on behaviour and effort (workload) in performing the task.

This paper presents a quality of experience (QoE) and cybersickness study of an immersive and interactive wheelchair training simulator. The study had two independent groups and each group experienced the virtual wheelchair with either high or low jerk effect levels. Jerk is the rate of change of an object's acceleration over time and it is well accepted as an important consideration in dynamic immersive experiences. The QoE influencing factors (content, human and system) were considered as part of the comparison between groups. Content influencing factors were simulator configurations (high/low jerk effect) and presence levels (Igroup presence questionnaire). Human influencing factors were cognitive task load (NASA task load assessment), emotional (self-assessment manikin) and physiological response (electrodermal activity and heart rate variability). System influencing factors were device-related (immersive headset) and usability levels (system usability scale). The key focus of this work was to understand the influence of jerk on cybersickness and this was captured by using the simulator sickness questionnaire. The current findings indicate that the simulator with low jerk effect resulted in a higher arousal response and reduced simulator sickness symptoms, based on the self-assessment manikin and simulator sickness questionnaire. In addition, in terms of implicit metrics, the same group also showed more positive electrodermal activity responses and lower heart rates compared to the group that experienced the simulator with a high jerk effect. A significant correlation was found between heart rates observed during collision periods and simulator sickness scores. This correlation is moderately positive (r=0.558), which suggests a higher heart rate variability after collisions may be indicative of a higher tendency towards cybersickness.

360° video, also known as immersive video, is the recording of video content which simultaneously captures scene information in every direction, using an omnidirectional camera. Due to their immersive nature, the popularity of 360° videos has grown significantly. Understanding user Visual Attention when watching 360° videos is very important. This knowledge can help develop effective techniques for processing, encoding, distributing, and rendering 360° content. Whilst major efforts have concentrated on the visual element of immersive experiences, recently there has been growing interest in different forms of audio and in particular high-quality spatial audio. Spatial audio allows listeners to experience sound in all directions. Ambisonics or 3D audio is one such technique which offers a complete 360° soundscape. Although several models of visual and audio-visual attention have been proposed, very few have investigated the role of spatial audio in guiding attention in 360° videos. This demo shows our dataset and our methodological approach to understanding the user's audio-visual attention and QoE when experiencing 360° videos enhanced with spatial sound (first and third order ambisonic). Our research focus is to understand how audio affects Visual Attention in 360° videos and to evaluate its impact on the user's Quality of Experience (QoE).

Human-Robot Interaction (HRI) will be a crucial component of smart factories of the future (FoF). This demo presents a reliable and cost-effective HRI system based on Augmented Reality (AR). In this demonstration, we offer concepts and methods currently being developed to enable high-level human-robot collaboration and interaction. The user interaction and instructions are captured via AR and communicated to a Robot Operating System (ROS) powered robotic arm.

Eye-based information channels include the pupils, gaze, saccades, fixational movements, and numerous forms of eye opening and closure. Pupil size variation indicates cognitive load and emotion, while a person's gaze direction is said to be congruent with the motivation to approach or avoid stimuli. The eyelids are involved in facial expressions that can encode basic emotions. Additionally, eye-based cues can have implications for human annotators of affect. Despite these facts, the use of eye-based cues in affective computing is in its infancy and this work is intended to start to address this. Eye-based feature sets, incorporating data from all of the aforementioned information channels, that can be estimated from video are proposed. Feature set refinement is provided by way of continuous arousal and valence learning and prediction experiments on the RECOLA validation set. The eye-based features are then combined with a speech feature set to provide confirmation of their usefulness and assess affect prediction performance compared with group-of-humans-level performance on the RECOLA test set. The core contribution of this paper, a refined eye-based feature set, is shown to provide benefits for affect prediction. It is hoped that this work stimulates further research into eye-based affective computing.

Edge computing is proposed to remedy the Cloud-only processing architecture for Internet of Things (IoT) because of the massive amounts of IoT data. The challenge is how to deploy and execute data processing tasks on heterogeneous IoT edge network. As MapReduce is a well-known model in Cloud computing for distributed processing of big data, this paper aims to devise a MapReduce-based protocol to achieve IoT edge computing. Our design is built upon the novel Information Centric Networking (ICN), which supports function naming and forwarding so as to facilitate task distribution among edge devices. To guarantee the correctness of task execution, a tree topology is formed in our approach to establish the logical connection between different types of edge devices, namely processing-capable nodes and forward-only ones. Moreover, the proposed protocol includes a task maintenance scheme that enables the coexistence of multiple IoT computation jobs. A testbed is implemented on ndnSIM to verify the feasibility of our design. The results show our approach could significantly decrease the network traffic compared with centralized data processing.

Augmented reality (AR) can support a user in performing an expert task by overlaying real world objects with the domain specific information required to complete the task. Understanding how users can process and use such information is very important for informing the design of AR technologies and applications. In this paper, the results of a quality of experience (QoE) evaluation of an AR application for the task of solving a Rubik's Cube are presented. The Rubik's Cube was selected based on its familiarity and the expertise needed to solve it unaided. An empirical approach was taken to identify the QoE features that affect the usability and utility of an AR head-mounted display (HMD) compared with paper-based instruction. The QoE evaluation methodology involved the capture and analysis of implicit and explicit QoE metrics. The utility (in terms of performance) of each mode of instruction was objectively measured using: (a) cube completion success rates; and (b) time-to-completion. The implicit metrics of electrodermal activity (EDA), skin temperature, heart rate and the novel use of facial action units (AUs) were recorded to infer emotional state during the task completion. Finally, with respect to explicit metrics, the test subjects completed a Likert scale questionnaire post the experience to subjectively report QoE as well as a self-assessment manikin (SAM) questionnaire to self-report emotional state upon task completion. The results show that AR yielded higher success rates and significantly lower time-to-completion rates. The AR group explicitly reported higher levels of positive valance (affective state) than the paper-based group. The physiological data showed that the AR group were less stressed (via EDA) than the paper-based group. Finally, analysis of the AU data reflected a greater than chance (total: 21.85%) accuracy when predicting affective state based on SAM questionnaires as ground-truth.

Motion analysis is a technique used by clinicians (among many others) that quantifies human movement by using camera-based systems. Marker-based motion analysis systems have been used across a variety of application domains, from Interactive 3D Tele-Immersion (i3DTI) environments to the diagnosis of neuromuscular and musculoskeletal diseases. Although such analysis is performed in several laboratories in many countries, numerous issues exist: (1) the high cost of precise motion capture systems; (2) scarcity of qualified personnel to operate them; (3) expertise required to interpret their results; (4) space requirements to install and store these systems; (5) complexity in terms of measurement protocol required for such systems; (6) limited availability; (7) and in some situations the use of markers means they are unsuitability for certain clinical use cases (e.g. for patients recovering from orthopaedic surgery). In this paper, we present, from a system perspective, an alternative, cheaper, and more accessible system for motion analysis. The ultimate aim is to use the output of this multimodal marker-less system as part of an immersive multimedia gait re-education tool. In real-time, it will advise the user on their gait performance (as well as potentially providing accurate clinical data to clinicians). With the initial focus on the capture system, we have developed and evaluated a novel multimodal system which integrates Multiple Microsoft Kinects (which employ RGB-D cameras) with multiple Shimmer Inertial Measurement Unit (IMU) sensors. We have compared this system with the VICON system (the gold standard in motion capture). Our marker-less motion capture system combines data from 4 skeletons generating 3D and complete 360 degrees in view skeleton. The system combines unit quaternions from each Kinect joint with quaternions from 4 inertial measurement units to promote integration. We used our system to measure 3D points of 12 joints from the Kinect fused skeleton and flexion-extension angles of the knee and hip in a walking trial in 8 participants with 8--10 trials per participant. The analysis found component similarity of 0.97 for knee angles and 0.98 for hip angles. These results show that our system, through combination of Multi Kinect system and Shimmer IMUs, offers a cheaper, sufficiently accurate and more accessible human motion analysis system.

No abstract available.

The number of devices connected by the Internet of Things (IoT) has exceeded billions today. As IoT grows, so do the volumes of data it produces. Distributed data process (such as: MapReduce) performs better than a single central server by considering the big data set. Moreover, many IoT network spans widely in geographical areas, such as smart cities and supply chain management. Thus, the collected data can be processed in distributed way before transmission. This paper studies how to develop a MapReduce framework to process massive IoT data. Because in many cases IoT consumers desire more to get the meaningful knowledge than to build connections with multiple devices. The proposed framework should also fits well with the information-centric nature of IoT applications. As a result, our design (MR-IoT) is built upon a novel Information Centric Networking (ICN) architecture - NDN. It defines two schemes to execute MapReduce tasks on IoT: computational tree construction and computational task dissemination. A testbed is built on ndnSIM to verify the design and the result shows the defined schemes work correctly and the network traffic is significantly decreased.

The1 key aim of various assistive technology (AT) systems is to augment an individual's functioning whilst supporting an enhanced quality of life (QoL). In recent times, we have seen the emergence of Virtual Reality (VR) based assistive technology systems made possible by the availability of commercially available Head Mounted Displays (HMDs). The use of VR for AT aims to support levels of interaction and immersion not previously possibly with more traditional AT solutions. Crucial to the success of these technologies is understanding, from the user perspective, the influencing factors that affect the user Quality of Experience (QoE). In addition to the typical QoE metrics, other factors to consider are human behavior like mental and emotional state, posture and gestures. In terms of trying to objectively quantify such factors, there are wide ranges of wearable sensors that are able to monitor physiological signals and provide reliable data. In this demo, we will capture and present the users EEG, heart Rate, EDA and head motion during the use of AT VR application. The prototype is composed of the sensor and presentation systems: for acquisition of biological signals constituted by wearable sensors and the virtual wheelchair simulator that interfaces to a typical LCD display.

Affective computing research traditionally focused on labeling a person's emotion as one of a discrete number of classes e.g. happy or sad. In recent times, more attention has been given to continuous affect prediction across dimensions in the emotional space, e.g. arousal and valence. Continuous affect prediction is the task of predicting a numerical value for different emotion dimensions. The application of continuous affect prediction is powerful in domains involving real-time audio-visual communications which could include remote or assistive technologies for psychological assessment of subjects. Modalities used for continuous affect prediction may include speech, facial expressions and physiological responses. As opposed to single modality analysis, the research community have combined multiple modalities to improve the accuracy of continuous affect prediction. In this context, this paper investigates a continuous affect prediction system using the novel combination of speech and eye gaze. A new eye gaze feature set is proposed. This novel approach uses open source software for real-time affect prediction in audio-visual communication environments. A unique advantage of the human-computer interface used here is that it does not require the subject to wear specialized and expensive eye-tracking headsets or intrusive devices. The results indicate that the combination of speech and eye gaze improves arousal prediction by 3.5% and valence prediction by 19.5% compared to using speech alone.

Named Data Networking (NDN) is a promising communication paradigm to support content distribution for the Future Internet. The objective of this paper is to maximize the consumer downloading rate by retrieving content via multiple paths concurrently. This is supported by adaptive forwarding in NDN. The majority of solutions for selecting the forwarding interfaces do so based on latency. However, this can overload the low-latency paths quickly. This can occur as users reduce requesting rates based on congestion signals, from the low-latency paths. Hence, the high-latency paths are not fully utilized. This paper solves this problem by introducing Backpressure Interest Control Protocol (B-ICP). In B-ICP, routers estimate the forwarding capabilities of interfaces based on congestion signals and limit the forwarding rates to interfaces accordingly. Thus, B-ICP avoids congesting certain paths earlier than others, with the aim being evenly distributed path utilization. Simulation-based evaluations show that B-ICP improves throughput, converges to equilibriums quickly and supports the producers that join the network dynamically in comparison with existing solutions.

Named Data Networking (NDN) is a promising Future Internet architecture to support efficient content distribution. Specifically, P2P may gain benefits from NDN, as NDN inherently provides a flexible forwarding plane for multi-source and multi-path communications. Existing studies in this area have proposed solutions, but these are adversely affected by link latency. This leads to illogical resource allocation and low link utilization for P2P. In this paper, we propose a new Heterogeneous-Latency Adaptive Forwarding (HLAF) strategy for peer-assisted video streaming in NDN. In peer-assisted video streaming, users (peers) proactively share the available content to others. By measuring the performance of forwarding interfaces, using both the level of congestion and the round-trip time, HLAF enables efficient P2P communication, which minimizes the latency and enhances the throughput. The experimental results show that the proposed strategy can enhance the peers' Quality of Experience (QoE).

The availability of affordable head-mounted displays has resulted in renewed interest in augmented reality (AR) and virtual reality (VR). This in turn has led to research into users' perceptions of quality in AR and VR environments. In this paper, the authors report the results of an experimental study that compared the user quality of experience (QoE) of an interactive and immersive speech and language assessment implemented in both AR and VR. To the best of the authors' knowledge, this is the first work that compares user QoE of VR and AR applications, in particular with a focus on applications in the speech and language domain. Subjective and objective data was captured to assess a user's QoE. Objective data, such as heart rate and electrodermal activity (EDA) was collected using consumer electronic devices such as the Fitbit and the PIP biosensor. Subjective data was captured using a user's self-reported ratings in a post-test questionnaire. The findings of this study demonstrate similar QoE ratings for both the AR and VR environments. The findings also suggest that users acclimatized to the AR environment more quickly than the VR environment.

Recently, the concept of adding multisensory media components to complement and extend user Quality of Experience (QoE) of traditional media has gained attention from both academia and industry. Research works stimulating additional senses like olfaction (sense of smell), haptic (sense of touch) and gustation (sense of taste) have emerged. In particular in theme parks, multisensory experiences that also offer ambient lighting effects, vibrating seats, wind generators, mist effects, heaters/coolers, etc. are appearing. Considering this growing awareness and popularity, a key research challenge is to experimentally evaluate if and how these different effects affect user QoE. In this context, there is a lack of common test content and raw data results to support reproducible research and cross research team verification. This paper fills this gap. We share: the data from the empirical study; the video content; the olfactory components employed to enrich the video; the methodologies employed and various other aspects found through experience to be important. Uniquely, this work is complemented by two datasets, obtained in two separate but related empirical studies, one conducted in the UK, and the other in Ireland.

In the quest to increase user perceived quality of experience (QoE), the classic audio-visual content paradigm can be extended to include media components that stimulate other human senses. Among these, olfaction-enhanced multimedia has attracted significant attention, as it is both attractive from user point of view and challenging from research perspective. This paper presents the results of two subjective studies which analyzed user QoE of olfaction-enhanced multimedia. Diverse scent types and video content were considered. In particular, QoE levels were studied when one and two olfaction stimuli enhanced audiovisual media. The results presented show that scent type influences user QoE. Statistically significant differences between pleasant and unpleasant scent types existed. Also, in certain cases, users were prepared to forgive the presence of unpleasant scent types with respect to QoE. Finally, users reported a clear preference for olfaction presented after the video sequence with which the olfaction effect should be synchronized, as opposed to before the video sequence.

The Internet of Things (IoT) places significant demands on network infrastructure in order to process data captured by ubiquitous sensor devices. One existing technique to support this sensor data processing involves transporting captured data to cloud servers. This approach suffers from numerous issues such as increased transmission costs i.e. bandwidth consumption and delays. To help resolve these issues, this paper proposes Programmable IoT (PIoT), a novel IoT data processing architecture. It is an application layer design which operates over Named Data Networking (NDN) to enable the execution of reconfigurable processing-logic in the network. In addition, a novel naming scheme and computation service for IoT is presented to describe the processing requirements using Lambda Expressions. To verify the feasibility of our design, a real-world implementation was created and evaluated. It compares efficiency of the in-network versus out-network approaches.

Recently, we have seen an emergence of affordable Head Mounted Displays (HMD) such as the Oculus Rift, HTC Vive, and the PS4 Project Morpheus which allow users to experience 3D virtual reality (VR). These types of hardware aim to facilitate new and novel experiences for users above and beyond what is possible with traditional audiovisual displays. However, a very limited number of studies exist in the literature to determine the influence of these technologies on user Quality of Experience (QoE). In order to evaluate QoE as users consume VR content, this paper proposes the use of affordable consumer electronics to capture objective physiological metrics: Heart Rate (HR) and ElectroDermal Activity (EDA). Our findings indicate different HR and EDA dependent on VR and non-VR environments. Additionally, we examine the relationship between these objective metrics and user QoE captured via a post-test questionnaire. To the best of the authors knowledge, this is the first work which demonstrates a tangible relationship between the EDA/HR combination and user QoE of immersive VR environments.

With the aim to enrich users' perceived multimedia experience, the authors present the results of an empirical study which looked at user perception of olfaction based mulsemedia. The goal is to evaluate the influence of users' age and gender on user quality of experience (QoE) considering various scent types and categories (pleasant or not). The results present a complex relationship between these variables and how they influence user QoE. They indicate that different user groups report different perception of content level factors for olfaction based mulsemedia.

Media-rich content plays a vital role in consumer applications today, as these applications try to find new and interesting ways to engage their users. Video, audio, and the more traditional forms of media content continue to dominate with respect to the use of media content to enhance the user experience. Tactile interactivity has also now become widely popular in modern computing applications, while our olfactory and gustatory senses continue to have a limited role. However, in recent times, there have been significant advancements regarding the use of olfactory media content (i.e., smell), and there are a variety of devices now available to enable its computer-controlled emission. This paper explores the impact of the audio stream on user perception of olfactory-enhanced video content in the presence of skews between the olfactory and video media. This research uses the results from two experimental studies of user-perceived quality of olfactory-enhanced multimedia, where audio was present and absent, respectively. Specifically, the paper shows that the user Quality of Experience (QoE) is generally higher in the absence of audio for nearly perfect synchronized olfactory-enhanced multimedia presentations (i.e., an olfactory media skew of between {−10,+10s}); however, for greater olfactory media skews (ranging between {−30s;−10s} and {+10s, +30s}) user QoE is higher when the audio stream is present. It can be concluded that the presence of the audio has the ability to mask larger synchronization skews between the other media components in olfaction-enhanced multimedia presentations.

Many existing algorithms manage network handover using static thresholds or weights applied to performance metrics. Such approaches are performance limited as they require knowledge of prior network performance. Performance metric thresholds and weights are often preconfigured based on the experience of network personnel. Static weightings are often configured for ideal network scenarios and are not able to adapt to changing environmental conditions. Previous studies have illustrated how the combination of foliage and weather can introduce interference at the receiver. This paper proposes an Algorithm for Mitigating ENvironmental Degradations. (AMEND). AMEND is a pluggable directed feed-forward neural network designed for vehicular environments. The handover decisions implemented by AMEND are based on predicted weather conditions, historic network performance and dynamic performance characteristics. Results illustrate that in varying weather conditions AMEND has improved overall performance over existing approaches. In poor weather conditions, implementation of AMEND has led to a performance improvement of over 500% in comparison to existing approaches.

Handover algorithms typically operate by assigning preconfigured threshold or weight values onto network performance metrics such as delay, data loss and signal strength. Such approaches are performance limited as they do not consider external factors that affect the network such as the physical environment and current weather conditions. Previous research illustrates that foliage density combined with detrimental weather conditions can have degrading effects on wireless links. The changes to these environmental factors over long vehicular-based mobile user sessions can lead to sub-optimal handover decisions and a negative impact on a user's Quality of Experience during mobile video streaming. There is need for a handover approach that adapts to these factors and mitigates any negative effects that occur. This paper proposes a method for Environmental factor Mitigation on mULtimedia StreamIng Networks (EMULSIoN). EMULSIoN uses a perceptron artificial neural network approach to mitigate the latency and delays caused by environmental factors. Using dynamic network performance metrics and with known topographical data, the EMULSIoN directed learning approach can learn from previous user sessions to mitigate these environmental effects. EMULSIoN further uses GPS and topographical data to divide vehicular routes into small sub-areas for optimal performance in varied terrain. Results illustrate that EMULSIoN has significant video quality improvements in comparison to pre-configured weight handover strategies.

This study looked at users' perception of interstream synchronization between audiovisual media and two olfactory streams. The ability to detect skews and the perception and impact of skews on user Quality of Experience (QoE) is analyzed. The olfactory streams are presented with the same skews (i.e., delay) and with variable skews (i.e., jitter and mix of scents). This article reports the limits beyond which desynchronization reduces user-perceived quality levels. Also, a minimum gap between the presentations of consecutive scents is identified, necessary to ensuring enhanced user-perceived quality. There is no evidence (not considering scent type) that overlapping or mixing of scents increases user QoE levels for olfaction-enhanced multimedia.

As a step towards enhancing users' perceived multimedia quality levels, this article presents the results of a study which looked at user's perception of inter-stream synchronization between scent and video. The ability to detect and the perception of and impact of skew on user's quality of experience is analyzed considering user's age, sex, and culture (user profile). The results indicate that skews beyond a certain level between olfaction and video have a negative impact on user-perceived experience. Olfaction before video is more noticeable to users than olfaction after video, and assessors are more tolerable of olfactory data presented after video.

Lately, significant efforts have being put into proposing various solutions for increasing multimedia viewers' perceived quality levels. One innovative avenue is to enhance users' quality of experience (QoE) by extending the classic audio-visual multimedia content to stimulate also other human senses such as olfaction, tactile, etc. In this context, this paper focuses on olfaction-enhanced multimedia content and presents the results of an experimental study which looked at user perception of inter-stream synchronization between olfactory data and video, whereby the audio used provides no contextual information. The study investigates how age and gender influence users' perception of the temporal boundaries within which they perceive olfactory data and video to be synchronized. The impact on user QoE levels (considering sense of enjoyment, relevance and reality) during synchronous and asynchronous presentations of olfactory and video media is also analyzed and discussed. The results show that there are significant differences in terms of how users of various gender and age groups perceive the skew between olfaction and video content and in their QoE levels.

As a step towards enhancing users' perceived multimedia quality levels beyond the level offered by the classic audiovisual systems, the authors present the results of an experimental study which looked at user's perception of inter-stream synchronization between olfactory data (scent) and video (without relevant audio). The impact on user's quality of experience (by considering enjoyment, relevance and reality) comparing synchronous with asynchronous presentation of olfactory and video media is analyzed and discussed. The aim is to empirically define the temporal boundaries within which users perceive olfactory data and video to be synchronized. The key analysis compares the user detection and perception of synchronization error. State of the art works have investigated temporal boundaries for olfactory data with audiovisual media, but no works document the integration of olfactory data and video (with no related audio). The results of this work show that the temporal boundaries for olfactory and video only are significantly different from olfactory, video and audio. The authors conclude that the absence of contextual audio reduces considerably the acceptable temporal boundary between the scent and video. The results also indicate that olfaction before video is more noticeable to users than olfaction after video and that users are more tolerable of olfactory data after video rather than olfactory data before video. In addition the results show the presence of two main synchronization regions. This work is a step towards the definition of synchronization specifications for multimedia applications based on olfactory and video media.