Ronan Flynn

dblp:41/1480 · DBLP profile ↗
← Back
37ranked-venue papers
3as first author
20since 2021 · last 2026
0000-0002-6475-005XORCID · verified

Domains — the database's venue-derived domains; a paper can count in several

Graphics, computer vision, multimedia, augmented reality and games · 24 · 3 first-author · 14 since 2021Human-computer interaction and ubiquitous computing · 14 · 8 since 2021Computer networks · 6 · 3 since 2021Software engineering, systems software and programming languages · 1 · 1 since 2021Applied, interdisciplinary, general and emerging computing · 1
YearPublicationVenuePosition
2026 Analysing Infant Physiological Responses to Live Musical Performance Using Wearable Sensors
Eoghan Hynes, Sowmya Vijayakumar, Bryan Dunphy, Lara Matos, Paulo Lameiro, Niall Murray, Ronan Flynn
QoMEX7
2026 Physiological Responses to Musical Stimuli - an Initial Investigation
abstract
This work-in-progress paper presents findings from a pilot study (N = 30), conducted as part of the Horizon Europe AMPLIFY project, investigating physiological responses to music. Participants listened to two counterbalanced musical excerpts while heart rate, electrodermal activity (EDA) and skin temperature were recorded via EmotiBit™ wearable sensors. Of 489 statistically significant individual deviations from baseline (Bonferroni-corrected Mann-Whitney U), 33 were shared across multiple participants. To check whether these occurred at the same instant, temporal analysis of the shared deviations identified 82 different episodes in which multiple participants deviated simultaneously. Comparing audio features during these episodes to the remainder of each musical excerpt yielded 27 Bonferroni-significant associations (Cohen's d up to 1.62). Broadband EDA and skin conductance response frequency emerged as the most responsive physiological measures, showing strong correlations with spectral brightness and timbral features of the musical passages.
Eoghan Hynes, Bryan Dunphy, Sowmya Vijayakumar, Niall Murray, Ronan Flynn
IMX5
2025 RCQoEA-360VR: Real-time Continuous QoE Scores for HMD-based 360° VR Dataset
abstract
As immersive 360° video experiences through head-mounted displays (HMDs) gain widespread adoption, the need for real-time, fine-grained assessment of Quality of Experience (QoE) becomes increasingly critical for optimising user engagement and system performance. This paper introduces RCQoEA-360VR, a novel multi-modal dataset designed for continuous QoE evaluation in virtual reality (VR) environments. In a controlled study (N=32), participants watched five selected 360° video sequences across eight different video quality configurations (from the VQEG database) using a Vive Pro Eye while providing continuous QoE annotations via a touchpad-based input method, enhanced by the DotMorph peripheral visualisation technique. The dataset also includes synchronised physiological signals (electrocardiogram and galvanic skin response), behavioural data (eye and head movements) and post-viewing QoE ratings gathered through a within-VR interface. RCQoEA-360VR addresses a critical gap in existing public datasets by providing a fine-grained, synchronised multimodal data for immersive QoE analysis. It offers a unique and valuable resource for the research community, supporting a wide range of research applications, including QoE prediction, behavioural modelling, adaptive streaming, and implicit perceptual analysis.
Sowmya Vijayakumar, Tong Xue, Abdallah El Ali, Irene Viola 0001, Ronan Flynn, Peter Corcoran 0001, Pablo César, Niall Murray
ACM Multimedia5
2024 Impact of Auditory and Audiovisual Distractors on Task Performance in a VR-based Auditory Attention Task
abstract
Auditory attention is a fundamental cognitive process essential for effective communication and interaction. Auditory stimuli are often accompanied by distractors that can significantly impact task performance by means of reducing attention. This study investigates the influence of auditory and audiovisual distractors on an auditory attention task within a Virtual Reality classroom. As part of the user evaluation, participants had to listen to two short stories. They performed two tasks: (i) “DISTRACTORS”, where participants had to identify an auditory or an audiovisual stimulus and (ii) “KEYWORDS,” where participants had to identify a specific keyword in the story by pressing a button on the controller. During the experiment, the participants’ physiological (e.g. skin conductance levels, gaze data, etc.) and subjective data (i.e. questionnaires) were collected. The results revealed that the interval between the presentation of the keyword and the presentation of the distractors impacted task performance by negatively affecting auditory attention. Also, it was observed that the time spent looking at the speaker telling the story positively correlated with task performance, whereas the time spent looking at distractors was found to be negatively correlated with task performance. Finally, this study gives insights into how physiological metrics can be used to infer auditory attention in VR experiences.
Adrielle Nazar Moraes, Eoghan Hynes, Ronan Flynn, Andrew Hines, Niall Murray
ISMAR3
2024 The Role of ECG and Respiration in Predicting Quality of Experience
abstract
The field of user quality of experience (QoE) in multimedia communications has become increasingly important due to the widespread use of digital technology in our everyday lives. The ability to accurately predict user QoE by processing physiological signals has significant applications. This study proposes a machine learning (ML) approach for predicting user QoE using physiological signals. It focuses on perceived overall quality and perceived audio quality by processing electrocardiogram (ECG) and respiration signals from the SoPMD Dataset 2. The study evaluated various ML models on individual and fused modalities while implementing dimensionality reduction, feature selection, and hyperparameter tuning. The SHapley Additive exPlanations (SHAP) method was used to interpret the ML models' outputs, which helped identify the features providing the most utility. The results show that the random forest model provided the best performance, with an F1-score of 87.91% for fusion data and 80.49% for ECG data in classifying perceived audio quality and overall quality, respectively. These results imply that physiological signals, such as ECG and respiration, hold great potential for predicting user QoE in multimedia experiences.
Sowmya Vijayakumar, Ronan Flynn, Peter Corcoran 0001, Niall Murray
QoMEX2
2023 Chatbot-based Feedback for Dynamically Generated Workflows in Docker Networks
abstract
This paper presents an implementation of a feedback mechanism for a workflow management framework. A chatbot that uses natural language processing (NLP) is central to the proposed feedback mechanism. NLP is used to transform text-based plain language input, both human-written and machine-generated, into a form that the framework can use to generate a workflow for execution in an environment of interest. The example environment described here is containerized network management, in which the workflow management framework, using feedback, can detect anomalies and mitigate potential incidents.
Andrzej Jasinski, Yuansong Qiao, Enda Fallon, Ronan Flynn
NetSoft4
2023 Towards a symmetrical definition of QoE: An Evaluation of Emotion Semantics in Augmented Reality Training
abstract
The current definition of quality of experience (QoE) designates delight and annoyance as diametrically opposing indicators of the degree of fulfilment of an application, service or system user's pragmatic and hedonic needs and expectations. However, these inherited emotion terms are rarely used to describe emotions of equal amounts of arousal or opposing amounts of valence in the literature. This work assesses the significance of this asymmetry to the definition of QoE by determining the utility of emotion terms to communicate the emotion component of QoE. This was done in the context of a QoE evaluation of augmented reality training instruction formats. Correlates were sought between various measures of emotional state. This included physiological ratings, facial expressions and eye gaze. Emotional state was subjectively reported using three distinct methods: self-assessment manikin questionnaire; 2D emotion space terms; and open-ended terms. Regression analysis showed multiple significant correlations between implicit and explicit metrics, but not to the emotion terms used by the participants. This calls into question the utility of such vaguely understood terms. The use of more symmetrically opposing emotions in the definition of QoE may benefit consensual interdisciplinary communication.
Eoghan Hynes, Ronan Flynn, Brian Lee 0001, Niall Murray
QoMEX2
2023 A Comparison of Gender Differences and Performance Metrics in a VR-Based Auditory Selective Task
abstract
Virtual Reality has gained significant interest with the advancement of display and processing technologies in recent years. Traditionally, visual stimuli were the main point of interest in the development of new VR applications. However, audio is also key to the success of such experiences. Audio technology is responsible for making the VR environment more immersive as it is directly correlated with how we perceive the world. Our real-world environments are multimodal and multisensory. In this work, we investigate the effect of two types of distractors in an auditory selective attention task. Users were immersed in a virtual classroom with spatialised audio enabled. The task itself was divided into two subtasks. First, participants had to identify different types of auditory and audiovisual stimuli in the scene. Second, they had to focus their attention on a speaker in front of them and identify a keyword from the story each time they heard it. Furthermore, participants were divided into three groups that differ on when the distractor is presented in the second subtask: (a) before, (b) same time, and (c) after the keyword. Findings from this study show that there are significant gender differences for listeners immersed in an environment with competing sounds in recalling a story. Moreover, the time when distractors are presented significantly affected the response time, being inversely proportional to the mean response time.
Adrielle Nazar Moraes, Ronan Flynn, Andrew Hines, Niall Murray
QoMEX2
2022 Continuous-time feedback device to enhance situation awareness during take-over requests in automated driving conditions
abstract
Conditional automation may require drivers of Autonomous Vehicles (AVs) to turn their attention away from the roads by taking part, for instance, in Non-Driving Related Tasks (NDRTs). That might cause them a lack of Situation Awareness (SA) when resuming to manual control. Consequently, Take-Over Requests (TORs) are system events intended to inform drivers when the vehicle is unable to handle an upcoming situation that is outside its Operational Design Domain (ODD) and the automated system might get disengaged, requiring driver's attention. The short time frame of time-critical TOR events impacts on the performance of the machine-to-human transition, especially when the driver is engaged with NDRTs. This can lead to dangerous driving conditions. In that context, this work proposes the demonstration of a device called Adaptive Tactile Device (ATD), capable of continuously adjust itself according to the driving conditions and smooth control transition by constantly informing the driver about road and system state based on force haptic feedback. The Continuous-Time (CT) nature of the proposed device is intended to provide adaptive feedback during automated vehicle conditions, promoting a feeling of control and possibly improving driver's Situation Awareness (SA) during TOR events. Future work will consider implementing this device on the steering wheel or driver's seat and collect the user's Quality of Experience (QoE) when using it in Virtual Reality (VR) simulations, to be compared with the user's objective and subjective metrics when receiving Discrete-Time (DT) feedback warnings previously applied in TOR research.
Guilherme Daniel Gomes, Ronan Flynn, Niall Murray
MMSys2
2022 A QoE evaluation of procedural and example instruction formats for procedure training in augmented reality
abstract
Augmented reality (AR) has significant potential as a training platform. The pedagogical purpose of training is learning or transfer. Learning is the acquisition of an ability to perform a procedure as taught while transfer involves generalising that knowledge to similar procedures in the same domain. Quality of experience (QoE) concerns the fulfilment of the application, system or service user's pragmatic and hedonic needs and expectations. Learning or transfer fulfil the AR trainee's pragmatic needs. Training instructions can be presented in procedural, and example formats. Procedural instructions tell the trainee what to do while examples show the trainee how to do it. These two different instruction formats can influence learning, transfer, and hardware resource availability differently. The AR trainee's hedonic needs and expectations may be influenced by the impact of instruction format resource consumption on system performance. Efficient training efficacy is a design concern for mobile AR training applications. This work aims to inform AR training application design by evaluating the influence of procedural and example instruction formats on AR trainee QoE.
Eoghan Hynes, Ronan Flynn, Brian Lee 0001, Niall Murray
MMSys2
2022 A questionnaire-based and physiology-inspired quality of experience evaluation of an immersive multisensory wheelchair simulator
abstract
Immersive multimedia technologies such as virtual reality (VR) are now finding potential applications in domains outside of entertainment and gaming in areas such as health, education, and tourism, to name a few. This article presents a Quality of Experience (QoE) evaluation of an immersive haptic-based VR wheelchair simulator. The paper presents the results of an explicit and implicit (physiology-based) QoE evaluation of the Immersive Simulator in three different configurations: (a) desktop group (non-immersive); (b) headset 1 group (immersive with a high rate of motion acceleration); and (c) headset 2 group (immersive with a lower rate of motion acceleration). As part of the user evaluations, participants in each of the groups completed several questionnaires, including: an emotion questionnaire (SAM), cognitive task load (NASA-TLX), user expectations usability (SUS), and presence (IPQ). In addition, during the experience, physiological responses such as electrodermal activity (EDA) and heart rate variability (HRV) were recorded. The self-reported findings suggest that both headset groups had higher usability and presence levels than the desktop group. The two headset groups also had greater pleasant and exciting emotions than the desktop group. The NASA-TLX findings indicate that the headset 1 group presented the highest task cognitive load. The performance evaluation shows that both headset groups had a better results than the desktop group in terms of task completion.
Débora Pereira Salgado, Ronan Flynn, Eduardo L. M. Naves, Niall Murray
MMSys2
2022 AI-derived quality of experience prediction based on physiological signals for immersive multimedia experiences: research proposal
abstract
This paper contains the research proposal of Sowmya Vijayakumar that was presented at the MMSys 2022 doctorial symposium. Multimedia applications can now be found across many application domains including but not limited to entertainment, communication, health, business, and education. It is becoming more and more important to understand the factors that influence user perceptual quality, and hence monitoring user quality of experience (QoE) for improving multimedia interaction and services is essential. In this PhD work, we propose advanced machine learning techniques to predict QoE from physiological signals for immersive multimedia experiences. The aim of this doctoral study is to investigate the utility of physiological responses for QoE assessment for different multimedia technologies. Here, the research questions and solutions proposed to address this challenge are presented. A multimodal QoE prediction model is being developed that integrates several physiological measurements to improve QoE prediction performance.
Sowmya Vijayakumar, Peter Corcoran 0001, Ronan Flynn, Niall Murray
MMSys3
2022 Natural Language Processing Applied to Dynamic Workflow Generation for Network Management
abstract
Workflow implementation in network management is a challenge, especially when it depends on human interaction. Traditional workflow generation, which was primarily focused on business management, employed a graphical representation of managed elements, supported using a human-understandable explanation. The difficulty with this approach is workflow adaptation to avoid execution problems when unexpected errors occur. In such circumstances, human intervention is necessary to resolve the problem. Current workflow generation solutions are inflexible and inefficient with regards to self-checking and self-healing, and they cannot be easily scaled or adapted. This paper presents a novel workflow management framework that controls an environment using dynamic workflow generation. The framework detects inconsistencies in workflow execution and reacts automatically to prevent a workflow termination or to replace a corrupted workflow if execution is not possible. A language-based text-oriented ticket input is used to describe the problem to be solved. This text is translated using natural language processing to a syntax understandable by the framework in order to generate the required workflow. The functionality of the workflow management framework is demonstrated in a software-defined networking environment.
Andrzej Jasinski, Yuansong Qiao, Enda Fallon, Ronan Flynn
NOMS4
2022 Natural Language Processing for the Dynamic Generation of Network Management Workflows
abstract
This paper presents a workflow management framework in which natural language processing is used to support workflow creation. This work demonstrates an application of the framework for network management. Advantages of the framework include ease of implementation, low power consumption, flexibility, and scalability. Central to the framework is a distributed management structure. As input to the framework, a text-based ticket, written using plain language, is translated by NLP to a form that the framework can understand and act on. The message in the ticket input is used to create a dynamic workflow that can be executed in a controlled environment. Executed workflows are monitored for incident detection and mitigation.
Andrzej Jasinski, Yuansong Qiao, Enda Fallon, Ronan Flynn
NOMS4
2022 BiLSTM-based Quality of Experience Prediction using Physiological Signals
abstract
This paper presents an evaluation of a deep learning (DL) model to predict the user quality of experience (QoE) from physiological signals using a publicly available multimodal dataset, SoPMD. The subjective scores related to QoE factors, namely, perceptual video quality, immersion level, surrounding awareness, interest in video content and audio content are evaluated. A DL model, Bidirectional Long-short-term memory (BiLSTM), is trained on the fusion of electrocardiogram (ECG) and respiration features to predict subjective scores for the five QoE factors. This study achieved classification accuracies and Fl-scores ranging between 58% and 67% for different QoE factors. The results of the BiLSTM model were compared with machine learning techniques. The experimental results demonstrated that the proposed BiLSTM network has the potential to predict QoE from physiological signals.
Sowmya Vijayakumar, Ronan Flynn, Peter Corcoran 0001, Niall Murray
QoMEX2
2021 A Workflow Management Framework for the Dynamic Generation of Workflows that is Independent of the Application Environment
Andrzej Jasinski, Yuansong Qiao, Enda Fallon, Ronan Flynn
IM4
2021 A Framework for the Dynamic Generation of Workflows for Network Management
Andrzej Jasinski, Yuansong Qiao, Enda Fallon, Ronan Flynn
IM4
2021 "A Crowdsourcing-based QoE Evaluation of an Immersive VR Autonomous Driving Experience"
abstract
Due to COVID-19, crowdsourcing has gained momentum as an alternative methodology for continuing research and for Quality of Experience (QoE) assessment. Employing this approach, we remotely evaluated the user perceived QoE of two different visual rendering formats as part of an Autonomous Vehicles (AVs) simulation. The aim was to investigate the participant's QoE when testing AV technology in distinct visual rendering qualities (lowpoly vs high-poly) of an online streamed 360° car riding experience. In addition, a scoring model based on the expected reliability of each level of the remote assessment was designed. Findings suggest that the consumer's preferences towards the adoption of AV technology is highly determined by the system and human effects on Influence Factors (IFs). Moreover, the adequacy of reliability into a mathematical model is highlighted as a potential turning point for QoE assessment, by carrying out the evaluation tasks from the laboratory environment into the internet, particularly relevant in pandemic times.
Guilherme Daniel Gomes, Ronan Flynn, Niall Murray
QoMEX2
2021 A Physiology-Based QoE Comparison of Interactive Augmented Reality, Virtual Reality and Tablet-Based Applications
abstract
The availability of affordable head-mounted display technology has facilitated new, potentially more immersive, interactive multimedia experiences. These technologies were traditionally focused on entertainment; however, academia and industry are now exploring applications in other domains such as health, learning and training. Key to the success of these new multimedia experiences is the understanding of a user's perceived quality of experience (QoE). Subjective user ratings have been the primary mechanism to capture insights into a user's experience. Such ratings have generally been captured post experience and reflected using a mean opinion score (MOS). However, user perception is multifactorial and subjective ratings alone do not express the true measure of an experience. As a result, recent efforts to capture QoE have included exploring the use of implicit metrics (e.g., physiological measures). This article presents the results of an experimental QoE evaluation and comparison of immersive applications delivered across three multimedia platforms. The platforms compared were augmented reality, tablet and virtual reality. The QoE methodology employed considered explicit (post-test questionnaire) and implicit (heart rate and electrodermal activity) assessment methods. The results indicate comparatively higher levels of QoE for users of the augmented reality and tablet platforms.
Conor Keighrey, Ronan Flynn, Siobhan Murray, Niall Murray
IEEE Trans. Multim.2
2021 Delay-Based Network Utility Maximization Modelling for Congestion Control in Named Data Networking
abstract
Content replication and name-based routing lead to a natural multi-source and multipath transmission paradigm in NDN. Due to the unique connectionless characteristic of NDN, current end-to-end multipath congestion control schemes (e.g. MPTCP) cannot be used directly on NDN. This paper proposes a Network Utility Maximization (NUM) model to formulate multi-source and multipath transmission in NDN with in-network caches. From this model, a family of receiver-driven transmission solutions can be derived, named as path-specified congestion control. The path-specified congestion control enables content consumers to separate the traffic control on each path, which consequently facilitates fair and efficient bandwidth sharing amongst all consumers. As a specific instance, a Delay-based Path-specified Congestion Control Protocol (DPCCP) is presented, which utilizes queuing delays as signals to measure and control congestion levels of different bottlenecks. In addition, a set of high-performance congestion control laws are designed to accelerate bandwidth and fairness convergence towards the optimum defined by the NUM model. Finally, DPCCP is compared with state-of-the-art solutions. The experimental evaluations show that DPCCP outperforms existing solutions in terms of bandwidth utilization, convergence time and packet loss.
Yuhang Ye 0001, Brian Lee 0001, Ronan Flynn, Jin Xu 0005, Guiming Fang, Yuansong Qiao
IEEE/ACM Trans. Netw.3
2020 A QoE Evaluation of an Immersive Virtual Reality Autonomous Driving Experience
abstract
The driver/passenger role in autonomous driving experiences is an important subject that is concerned with how these systems will interact and communicate with the human stakeholder. Even though self-driving technologies have made rapid progress in recent years, their success will depend on ethics, safe capabilities and user acceptance. The overall satisfaction resulting from the interaction between humans, these technologies and their evaluation in simulators can help industry to improve their design. Such evaluations can be based on user's perceived Quality of Experience (QoE). The impact of the quality features that compound VR environments is an important factor to understand. Sense of depth, texture quality and resolution are some examples that influence the perceived quality of the simulation and immersion levels. This demonstration promotes a VR autonomous driving experience in one street of Athlone, Ireland. The application will be presented in two different formats: photogrammetry, a technique that uses photos to provide realistic 3D content, and secondly non-photorealistic simulated environment. Further investigations will be carried out to understand factors that enable the highest immersive experience in autonomous vehicles considering feedback modalities between the car and its users.
Guilherme Daniel Gomes, Ronan Flynn, Niall Murray
QoMEX2
2020 A QoE Evaluation of an Augmented Reality Procedure Assistance Application
abstract
Augmented reality (AR) is a key technology to enhance worker effectiveness during increasing automation of repetitive jobs in the workplace. AR will achieve this by assisting the user to successfully perform complex and frequently changing procedures. The design of AR applications for these roles is critical to their acceptability and utility. User quality of experience (QoE) will inform these design decisions. A user arrives at a quality judgment upon post-experience reflection of their degree of delight or annoyance relating to the degree of fulfilment of pragmatic and hedonic needs and expectations of the medium under consideration. QoE researchers have largely depended upon post-experience subjective reports to determine the user's QoE. Subjective reports have been shown to be biased by primacy, recency and maxima of experience stimuli. Recent research involves the identification of implicit metrics that can be used to determine user QoE continuously during a multimedia experience. This work evaluates head rotation frequency as an objective metric of user QoE. The literature shows that emotion is expressed in the frequency of a person's head rotation around three axes of movement (pitch, yaw and roll). Low frequency head rotation has been shown to include expression of happy emotion while high frequency exclusively expresses anger emotion. These emotions are analogous to those reflected on by a user during the quality formation process. This demo paper analyses the amount of high frequency head rotation exhibited by the user upon task completion using an AR procedure assistance application or a paper-based control. An optimal Rubik's Cube solving AR application was used as a proof of concept for AR-based procedure assistance. Preliminary results showed that the AR environment yielded higher task success rates and significantly shorter task completion durations. The AR users exhibited significantly lower amplitudes of anger frequencies in their head rotations than the control group.
Eoghan Hynes, Ronan Flynn, Brian Lee 0001, Niall Murray
QoMEX2
2020 Evaluating the User in a Sound Localisation Task in a Virtual Reality Application
abstract
Virtual reality (VR) has proven to be a powerful tool enabling the development of immersive multimedia experiences. Initially focused on entertainment, industry and academia have begun to adapt and develop immersive applications for the healthcare domain, with opportunities in terms of condition assessment, diagnosis and intervention. In the context of immersive applications, audio, and in particular spatial audio, plays an important role on the immersion level. In order to process this information, the auditory cortex uses spatial cues encoded in the sound to provide relevant information about the distance, intensity and direction of the sound source. However, many different types of listening disorders can affect this capability. One condition, central auditory processing disorder (CAPD), significantly affects a user's ability to discriminate between different sound sources. People who suffer with this condition, are incapable of processing sounds properly, which may be stressful and frustrating when doing tasks with complex sounds or in noisy environments. This can have a significant impact on a person's quality of life. In this paper, an immersive VR spatial audio application is presented. It enables us to evaluate the ability of users to specify or localise the source of a sound. An integrated sensing system continuously collects relevant data from the user in order to fully understand how to quantify and evaluate spatial auditory skills from a quality of experience (QoE) perspective. QoE gives insight into a user's state and behaviour. To perform a detailed QoE evaluation of the listening task, implicit and explicit metrics were collected from the user. These included: self-reporting questionnaires, localisation performance, and physiological metrics. Data collected from this QoE evaluation gives an insight into a user's abilities to localise sound sources in VR, and also provides information on behaviour and effort (workload) in performing the task.
Adrielle Nazar Moraes, Ronan Flynn, Andrew Hines, Niall Murray
QoMEX2
2020 The Impact of Jerk on Quality of Experience and Cybersickness in an Immersive Wheelchair Application
abstract
This paper presents a quality of experience (QoE) and cybersickness study of an immersive and interactive wheelchair training simulator. The study had two independent groups and each group experienced the virtual wheelchair with either high or low jerk effect levels. Jerk is the rate of change of an object's acceleration over time and it is well accepted as an important consideration in dynamic immersive experiences. The QoE influencing factors (content, human and system) were considered as part of the comparison between groups. Content influencing factors were simulator configurations (high/low jerk effect) and presence levels (Igroup presence questionnaire). Human influencing factors were cognitive task load (NASA task load assessment), emotional (self-assessment manikin) and physiological response (electrodermal activity and heart rate variability). System influencing factors were device-related (immersive headset) and usability levels (system usability scale). The key focus of this work was to understand the influence of jerk on cybersickness and this was captured by using the simulator sickness questionnaire. The current findings indicate that the simulator with low jerk effect resulted in a higher arousal response and reduced simulator sickness symptoms, based on the self-assessment manikin and simulator sickness questionnaire. In addition, in terms of implicit metrics, the same group also showed more positive electrodermal activity responses and lower heart rates compared to the group that experienced the simulator with a high jerk effect. A significant correlation was found between heart rates observed during collision periods and simulator sickness scores. This correlation is moderately positive (r=0.558), which suggests a higher heart rate variability after collisions may be indicative of a higher tendency towards cybersickness.
Débora Pereira Salgado, Ronan Flynn, Eduardo L. M. Naves, Niall Murray
QoMEX2
2019 Eye-based Continuous Affect Prediction
abstract
Eye-based information channels include the pupils, gaze, saccades, fixational movements, and numerous forms of eye opening and closure. Pupil size variation indicates cognitive load and emotion, while a person's gaze direction is said to be congruent with the motivation to approach or avoid stimuli. The eyelids are involved in facial expressions that can encode basic emotions. Additionally, eye-based cues can have implications for human annotators of affect. Despite these facts, the use of eye-based cues in affective computing is in its infancy and this work is intended to start to address this. Eye-based feature sets, incorporating data from all of the aforementioned information channels, that can be estimated from video are proposed. Feature set refinement is provided by way of continuous arousal and valence learning and prediction experiments on the RECOLA validation set. The eye-based features are then combined with a speech feature set to provide confirmation of their usefulness and assess affect prediction performance compared with group-of-humans-level performance on the RECOLA test set. The core contribution of this paper, a refined eye-based feature set, is shown to provide benefits for affect prediction. It is hoped that this work stimulates further research into eye-based affective computing.
Jonny O'Dwyer, Niall Murray, Ronan Flynn
ACII3
2019 A Quality of Experience Evaluation Comparing Augmented Reality and Paper Based Instruction for Complex Task Assistance
abstract
Augmented reality (AR) can support a user in performing an expert task by overlaying real world objects with the domain specific information required to complete the task. Understanding how users can process and use such information is very important for informing the design of AR technologies and applications. In this paper, the results of a quality of experience (QoE) evaluation of an AR application for the task of solving a Rubik's Cube are presented. The Rubik's Cube was selected based on its familiarity and the expertise needed to solve it unaided. An empirical approach was taken to identify the QoE features that affect the usability and utility of an AR head-mounted display (HMD) compared with paper-based instruction. The QoE evaluation methodology involved the capture and analysis of implicit and explicit QoE metrics. The utility (in terms of performance) of each mode of instruction was objectively measured using: (a) cube completion success rates; and (b) time-to-completion. The implicit metrics of electrodermal activity (EDA), skin temperature, heart rate and the novel use of facial action units (AUs) were recorded to infer emotional state during the task completion. Finally, with respect to explicit metrics, the test subjects completed a Likert scale questionnaire post the experience to subjectively report QoE as well as a self-assessment manikin (SAM) questionnaire to self-report emotional state upon task completion. The results show that AR yielded higher success rates and significantly lower time-to-completion rates. The AR group explicitly reported higher levels of positive valance (affective state) than the paper-based group. The physiological data showed that the AR group were less stressed (via EDA) than the paper-based group. Finally, analysis of the AU data reflected a greater than chance (total: 21.85%) accuracy when predicting affective state based on SAM questionnaires as ground-truth.
Eoghan Hynes, Ronan Flynn, Brian Lee 0001, Niall Murray
MMSP2
2018 A QoE assessment method based on EDA, heart rate and EEG of a virtual reality assistive technology system
abstract
The1 key aim of various assistive technology (AT) systems is to augment an individual's functioning whilst supporting an enhanced quality of life (QoL). In recent times, we have seen the emergence of Virtual Reality (VR) based assistive technology systems made possible by the availability of commercially available Head Mounted Displays (HMDs). The use of VR for AT aims to support levels of interaction and immersion not previously possibly with more traditional AT solutions. Crucial to the success of these technologies is understanding, from the user perspective, the influencing factors that affect the user Quality of Experience (QoE). In addition to the typical QoE metrics, other factors to consider are human behavior like mental and emotional state, posture and gestures. In terms of trying to objectively quantify such factors, there are wide ranges of wearable sensors that are able to monitor physiological signals and provide reliable data. In this demo, we will capture and present the users EEG, heart Rate, EDA and head motion during the use of AT VR application. The prototype is composed of the sensor and presentation systems: for acquisition of biological signals constituted by wearable sensors and the virtual wheelchair simulator that interfaces to a typical LCD display.
Débora Pereira Salgado, Felipe Roque Martins, Thiago Braga Rodrigues, Conor Keighrey, Ronan Flynn, Eduardo L. M. Naves, Niall Murray
MMSys5
2018 PTP: Path-specified transport protocol for concurrent multipath transmission in named data networks
Yuhang Ye 0001, Brian Lee 0001, Ronan Flynn, Niall Murray, Guiming Fang, Jianwen Cao 0001, Yuansong Qiao
Comput. Networks3
2017 Continuous affect prediction using eye gaze and speech
abstract
Affective computing research traditionally focused on labeling a person's emotion as one of a discrete number of classes e.g. happy or sad. In recent times, more attention has been given to continuous affect prediction across dimensions in the emotional space, e.g. arousal and valence. Continuous affect prediction is the task of predicting a numerical value for different emotion dimensions. The application of continuous affect prediction is powerful in domains involving real-time audio-visual communications which could include remote or assistive technologies for psychological assessment of subjects. Modalities used for continuous affect prediction may include speech, facial expressions and physiological responses. As opposed to single modality analysis, the research community have combined multiple modalities to improve the accuracy of continuous affect prediction. In this context, this paper investigates a continuous affect prediction system using the novel combination of speech and eye gaze. A new eye gaze feature set is proposed. This novel approach uses open source software for real-time affect prediction in audio-visual communication environments. A unique advantage of the human-computer interface used here is that it does not require the subject to wear specialized and expensive eye-tracking headsets or intrusive devices. The results indicate that the combination of speech and eye gaze improves arousal prediction by 3.5% and valence prediction by 19.5% compared to using speech alone.
Jonny O'Dwyer, Ronan Flynn, Niall Murray
BIBM2
2017 B-ICP: Backpressure Interest Control Protocol for Multipath Communication in NDN
abstract
Named Data Networking (NDN) is a promising communication paradigm to support content distribution for the Future Internet. The objective of this paper is to maximize the consumer downloading rate by retrieving content via multiple paths concurrently. This is supported by adaptive forwarding in NDN. The majority of solutions for selecting the forwarding interfaces do so based on latency. However, this can overload the low-latency paths quickly. This can occur as users reduce requesting rates based on congestion signals, from the low-latency paths. Hence, the high-latency paths are not fully utilized. This paper solves this problem by introducing Backpressure Interest Control Protocol (B-ICP). In B-ICP, routers estimate the forwarding capabilities of interfaces based on congestion signals and limit the forwarding rates to interfaces accordingly. Thus, B-ICP avoids congesting certain paths earlier than others, with the aim being evenly distributed path utilization. Simulation-based evaluations show that B-ICP improves throughput, converges to equilibriums quickly and supports the producers that join the network dynamically in comparison with existing solutions.
Yuhang Ye 0001, Brian Lee 0001, Ronan Flynn, Niall Murray, Yuansong Qiao
GLOBECOM3
2017 HLAF: Heterogeneous-Latency Adaptive Forwarding strategy for Peer-Assisted Video Streaming in NDN
abstract
Named Data Networking (NDN) is a promising Future Internet architecture to support efficient content distribution. Specifically, P2P may gain benefits from NDN, as NDN inherently provides a flexible forwarding plane for multi-source and multi-path communications. Existing studies in this area have proposed solutions, but these are adversely affected by link latency. This leads to illogical resource allocation and low link utilization for P2P. In this paper, we propose a new Heterogeneous-Latency Adaptive Forwarding (HLAF) strategy for peer-assisted video streaming in NDN. In peer-assisted video streaming, users (peers) proactively share the available content to others. By measuring the performance of forwarding interfaces, using both the level of congestion and the round-trip time, HLAF enables efficient P2P communication, which minimizes the latency and enhances the throughput. The experimental results show that the proposed strategy can enhance the peers' Quality of Experience (QoE).
Yuhang Ye 0001, Brian Lee 0001, Ronan Flynn, Niall Murray, Yuansong Qiao
ISCC3
2017 A QoE evaluation of immersive augmented and virtual reality speech & language assessment applications
abstract
The availability of affordable head-mounted displays has resulted in renewed interest in augmented reality (AR) and virtual reality (VR). This in turn has led to research into users' perceptions of quality in AR and VR environments. In this paper, the authors report the results of an experimental study that compared the user quality of experience (QoE) of an interactive and immersive speech and language assessment implemented in both AR and VR. To the best of the authors' knowledge, this is the first work that compares user QoE of VR and AR applications, in particular with a focus on applications in the speech and language domain. Subjective and objective data was captured to assess a user's QoE. Objective data, such as heart rate and electrodermal activity (EDA) was collected using consumer electronic devices such as the Fitbit and the PIP biosensor. Subjective data was captured using a user's self-reported ratings in a post-test questionnaire. The findings of this study demonstrate similar QoE ratings for both the AR and VR environments. The findings also suggest that users acclimatized to the AR environment more quickly than the VR environment.
Conor Keighrey, Ronan Flynn, Siobhan Murray, Niall Murray
QoMEX2
2015 AMEND - An Algorithm for Mitigating ENvironmental Degradations in heterogeneous networks
abstract
Many existing algorithms manage network handover using static thresholds or weights applied to performance metrics. Such approaches are performance limited as they require knowledge of prior network performance. Performance metric thresholds and weights are often preconfigured based on the experience of network personnel. Static weightings are often configured for ideal network scenarios and are not able to adapt to changing environmental conditions. Previous studies have illustrated how the combination of foliage and weather can introduce interference at the receiver. This paper proposes an Algorithm for Mitigating ENvironmental Degradations. (AMEND). AMEND is a pluggable directed feed-forward neural network designed for vehicular environments. The handover decisions implemented by AMEND are based on predicted weather conditions, historic network performance and dynamic performance characteristics. Results illustrate that in varying weather conditions AMEND has improved overall performance over existing approaches. In poor weather conditions, implementation of AMEND has led to a performance improvement of over 500% in comparison to existing approaches.
Sean Hayes, Enda Fallon, Ronan Flynn, Niall Murray
CCNC3
2015 EMULSIoN: Environment Mitigation on mULtimedia StreamIng Networks
abstract
Handover algorithms typically operate by assigning preconfigured threshold or weight values onto network performance metrics such as delay, data loss and signal strength. Such approaches are performance limited as they do not consider external factors that affect the network such as the physical environment and current weather conditions. Previous research illustrates that foliage density combined with detrimental weather conditions can have degrading effects on wireless links. The changes to these environmental factors over long vehicular-based mobile user sessions can lead to sub-optimal handover decisions and a negative impact on a user's Quality of Experience during mobile video streaming. There is need for a handover approach that adapts to these factors and mitigates any negative effects that occur. This paper proposes a method for Environmental factor Mitigation on mULtimedia StreamIng Networks (EMULSIoN). EMULSIoN uses a perceptron artificial neural network approach to mitigate the latency and delays caused by environmental factors. Using dynamic network performance metrics and with known topographical data, the EMULSIoN directed learning approach can learn from previous user sessions to mitigate these environmental effects. EMULSIoN further uses GPS and topographical data to divide vehicular routes into small sub-areas for optimal performance in varied terrain. Results illustrate that EMULSIoN has significant video quality improvements in comparison to pre-configured weight handover strategies.
Sean Hayes, Enda Fallon, Ronan Flynn, Gabriel-Miro Muntean, Niall Murray
IWCMC3
2012 Feature selection for reduced-bandwidth distributed speech recognition
Ronan Flynn, Edward Jones
Speech Commun.1
2012 Reducing bandwidth for robust distributed speech recognition in conditions of packet loss
Ronan Flynn, Edward Jones
Speech Commun.1
2008 Combined speech enhancement and auditory modelling for robust distributed speech recognition
Ronan Flynn, Edward Jones
Speech Commun.1