Videogames are persuasive tools that can direct players’ pro-environmental attitudes. However, there is limited understanding of their impacts on attitudes in a climate-themed context. With a focus on message design and framing, this paper investigated the role and significance of videogames in climate communication. We conducted two parallel within-subjects experiments using two climate-themed games, Beyond Blue (N=36) and Plasticity (N=37), to examine the effects of persuasive game design in directing attitudes. We found that, regardless of message design, both games increased players’ cognitive attitudes after gameplay. We also found that Plasticity’s multi-layered message design (central loss-frame with a potential hopeful ending) increased short-term awareness of the climate-change threat and long-term hope for dealing with the issue, balancing fear and empowerment and emphasising the intricacies of message framing. By demonstrating the efficacy of persuasive design and environmental message framing in gameful interventions using explicit and implicit measures, our paper contributes to the application of interactive technology for effective climate-change communication in the short and long-term. • Videogames can be persuasive tools in climate communication through message framing. • Environmental message framing in games can influence cognitive attitudes. • Subtleties of message framing in videogames can affect the reach of the impact. • Mixed message designs can enhance game-based climate communication outcomes.

Vision Large Language Models (VLLMs) exhibit promising potential for multi-modal understanding, yet their application to video-based emotion recognition remains limited by insufficient spatial and contextual awareness. Traditional approaches, which prioritize isolated facial features, often neglect critical non-verbal cues such as body language, environmental context, and social interactions, leading to reduced robustness in real-world scenarios. To address this gap, we propose Set-of-Vision-Text Prompting (SoVTP), a novel framework that enhances zero-shot emotion recognition by integrating spatial annotations (e.g., bounding boxes, facial landmarks), physiological signals (facial action units), and contextual cues (body posture, scene dynamics, others’ emotions) into a unified prompting strategy. SoVTP preserves holistic scene information while enabling fine-grained analysis of facial muscle movements and interpersonal dynamics. Extensive experiments show that SoVTP achieves substantial improvements over existing visual prompting methods, demonstrating its effectiveness in enhancing VLLMs’ video emotion recognition capabilities.

Vision Large Language Models (VLLMs) are transforming the intersection of computer vision and natural language processing.Nonetheless, the potential of using visual prompts for emotion recognition in these models remains largely unexplored and untapped.Traditional methods in VLLMs struggle with spatial localization and often discard valuable global context.To address this problem, we propose a Set-of-Vision prompting (SoV) approach that enhances zero-shot emotion recognition by using spatial information, such as bounding boxes and facial landmarks, to mark targets precisely.SoV improves accuracy in face count and emotion categorization while preserving the enriched image context.Through a battery of experimentation and analysis of recent commercial or open-source VLLMs, we evaluate the SoV model's ability to comprehend facial expressions in natural environments.Our findings demonstrate the effectiveness of integrating spatial visual prompts into VLLMs for improving emotion recognition performance. Question:How many visible faces are there in the image?Analyze each face's emotion and group the face's emotions. Answer:In the image, there are 22 visible faces.Emotions can be grouped into three main categories. Neutral Emotion: Approximately 16Faces.This neutral expression is typical in settings requiring attention to a speaker or event, reflecting an engaged yet reserved emotional state. Mildly Positive Emotion: Approximately 5Faces.This could be a response to something interesting or mildly amusing happening at the event, but their reactions are subdued, in keeping with the formal setting.Happy: 1 Face.This expression stands out as more joyful or enthusiastic about the proceedings.

Videogames have the capacity to change people’s attitudes by engaging players with interactive gameplay, persuasive narratives, and immersive simulated realities. We present a framework that targets attitudinal evaluation in game design by describing how game mechanics, narrative, and animation features form player experiences that influence attitudes. To test this framework, we conducted an interventional study (N=36) using a repeated measures design that tested the effects of a climate-themed simulation game, Beyond Blue, on players’ explicit and implicit pro-environmental attitudes in both the short and long term. We also measured the effects of feature-based design (mechanics, narrative, and animation) and elemental design on attitudes. Our results showed that the climate-themed intervention impacted participants’ short-term cognitive attitudes. We also found that Beyond Blue’s overall mechanics and narrative features were the main significant predictors of pro-environmental cognitive attitudes. Challenge design was the only significant element that predicted participants’ cognitive attitudes. Our results demonstrate that climate communication can benefit from a carefully designed theme-oriented videogame as an effective creative tool.

Videogames can be persuasive assets in climate communication. However, there is insufficient knowledge on how to employ specific game design aspects to influence attitudes in the context of climate change. We developed a novel conceptual Climate-Oriented Persuasive Edutainment (C.O.P.E.) model to describe how game design features and elements wrapped in a climate-themed message frame can contribute to pro-environmental attitudes. We evaluated the design decisions in a loss-framed climate-themed videogame (Plasticity) as our case study. In a repeated measures interventional experiment (N=37), we examined the effects of Plasticity on players’ explicit and implicit pro-environmental attitudes in the short and long term. We also assessed participants’ experiences of the attitudinal capacity of three features of game design (mechanics, narrative, animation) and their relevant elements. We found that playing Plasticity influenced participants’ threat perception and cognitive attitudes in the short term. The overall mechanics and narrative features were predictors of participants’ climate threat perception, while the overall animation design predicted pro-environmental cognitive attitude. The storyline was the only element that predicted both threat perception and cognitive attitude. Also, the challenge design predicted threat perception and the exploration design predicted cognitive attitudes. Our work shows that a theme-pertinent, detail-oriented game design can be effective in enhancing climate communication.

Multi-embodied agents can have both physical and virtual bodies, moving between real and virtual environments to meet user needs, embodying robots or virtual agents alike to support extended human-agent relationships. As a design paradigm, multi-embodiment offers potential benefits to improve communication and access to artificial agents, but there are still many unknowns in how to design these kinds of systems. This paper presents the results of a scoping review of the multi-embodiment research, aimed at consolidating the existing evidence and identifying knowledge gaps. Based on our review, we identify key research themes of: multi-embodied systems, identity design, human-agent interaction, environment and context, trust, and information and control. We also identify 16 key research challenges and 12 opportunities for future research.

The proliferation of synthetic media generation technologies, such as generative AI, has led to a surge of media content generation and consumption. While this progress opens new opportunities, especially in creative industries, it also causes challenges, including piracy, fake media distribution, and concerns about trust and privacy. In the creative sector, media modifications are often part of the production pipelines and in many application domains, creators need or want to declare the type of modifications that were performed on the media asset. The cryptographically signed association of provenance information with the media asset itself provides a trust link between the owner or editor of a media asset and its consumers. The absence of such assertions may reveal the lack of trustworthiness in media assets or worse, the intention to hide the existence of manipulations. This paper describes the JPEG Trust framework (ISO/IEC 21617) that aims to establish trust in digital media creation, modification, annotation, distribution and consumption. The framework provides standardized protocols to extract indicators to assess trustworthiness, means to annotate media provenance, and securely link the assets and associated annotations together.

We propose a new research framework by which the nascent discipline of human-AI teaming can be explored within experimental environments in preparation for transferal to real-world contexts. We examine the existing literature and unanswered research questions through the lens of an Agile approach to construct our proposed framework. Our framework aims to provide a structure for understanding the macro features of this research landscape, supporting holistic research into the acceptability of human-AI teaming to human team members and the affordances of AI team members. The framework has the potential to enhance decision-making and performance of hybrid human-AI teams. Further, our framework proposes the application of Agile methodology for research management and knowledge discovery. We propose a transferability pathway for hybrid teaming to be initially tested in a safe environment, such as a real-time strategy video game, with elements of lessons learned that can be transferred to real-world situations.

Invertible networks have various benefits for image de-noising since they are lightweight, information-lossless, and memory-saving during back-propagation. However, applying invertible models to remove noise is challenging because the input is noisy, and the reversed output is clean, following two different distributions. We propose an invertible denoising network, InvDN, to address this challenge. InvDN transforms the noisy input into a low-resolution clean image and a latent representation containing noise. To discard noise and restore the clean image, InvDN replaces the noisy latent representation with another one sampled from a prior distribution during reversion. The de-noising performance of InvDN is better than all the existing competitive models, achieving a new state-of-the-art result for the SIDD dataset while enjoying less run time. Moreover, the size of InvDN is far smaller, only having 4.2% of the number of parameters compared to the most recently proposed DANet. Further, via manipulating the noisy latent representation, InvDN is also able to generate noise more similar to the original one. Our code is available at: https://github.com/Yang-Liu1082/InvDN.git.

Deep learning (DL) models are widely used to provide a more convenient and smarter life. However, biased algorithms will negatively influence us. For instance, groups targeted by biased algorithms will feel unfairly treated and even fearful of negative consequences of these biases. This work targets biased generative models' behaviors, identifying the cause of the biases and eliminating them. We can (as expected) conclude that biased data causes biased predictions of face frontalization models. Varying the proportions of male and female faces in the training data can have a substantial effect on behavior on the test data: we found that the seemingly obvious choice of 50:50 proportions was not the best for this dataset to reduce biased behavior on female faces, which was 71% unbiased as compared to our top unbiased rate of 84%. Failure in generation and generating incorrect gender faces are two behaviors of these models. In addition, only some layers in face frontalization models are vulnerable to biased datasets. Optimizing the skip-connections of the generator in face frontalization models can make models less biased. We conclude that it is likely to be impossible to eliminate all training bias without an unlimited size dataset, and our experiments show that the bias can be reduced and quantified. We believe the next best to a perfect unbiased predictor is one that has minimized the remaining known bias.

Lying is a common act in daily life and may have various degrees of falsehood. Deception detection has always been a fascinating area of research in which many studies have been conducted using subjects’ facial, verbal or bodily cues to spot potential deceit. However, none of the studies have investigated the physiological responses of observers in response to misleading statements with various degrees of falsehood. In this paper, we investigated this problem by first conducting designed experiments to collect participants’ physiological signals while they were watching stimulus videos with various falsehood levels. Then, the data was analysed using machine learning or deep learning models. Various challenges including relatively small amounts of training data and imbalanced classes have been addressed by implementing data augmentation. The results show that deep learning models, such as ResNet and VAE-LSTM, can predict the degree of falsehood with an F1-measure up to 0.83 from observers’ reactions when compared to the stimuli ground truth. This was attained when the model was trained with the most useful physiological signal in this study, Electrodermal Activity (EDA). This result indicates that observers’ physiological signals can be used as an indicator to determine the degree of falsehood for misleading statements. In the future, this system may be applied to provide an objective evaluation for deception detection.

Recently, researchers in the field of affective neuroscience have taken a keen interest in identifying patterns in brain activities that correspond to specific emotions. The relationship between music stimuli and brain waves has been of particular interest due to music's disputed effects on brain activity. While music can have an anticonvulsant effect on the brain and act as a therapeutic stimulus, it can also have proconvulsant effects such as triggering epileptic seizures. In this paper, we take a computational approach to understand the effects of different types of music on the human brain; we analyse the effects of 3 different genres of music in participants electroencephalograms (EEGs). Brain activity was recorded using a 14-channel headset from 24 participants while they listened to different music stimuli. Statistical features were extracted from the signals and useful features and channels were identified using various feature selecting techniques. Using these features we built classification models based on K-nearest Neighbour (KNN), Support Vector Machine (SVM) and Neural Network (NN). Our analysis shows that NN, along with Genetic Algorithm (GA) feature selection, can reach the highest accuracy of 97.5% in classifying the 3 music genres. The model also reaches 98.6% accuracy in classifying music based on participants' subjective rating of emotion. Additionally, the recorded brain waves identify different gamma wave levels, which are crucial in detecting epileptic seizures. Our results show that these computational techniques are effective in distinguishing music genres based on their effects on human brains.

We live in a world surrounded with `fake news' and manipulated information, so a system assisting people with knowing what information to trust would be beneficial. Our research investigates situations where the presenters themselves have doubts about the information they are delivering, and we detect this via advanced affective computing techniques. To this end we examine the physiological foundations for observer recognition of the doubt effect: the subjective belief or disbelief of a presenter in some information he or she is presenting. Firstly, we construct stimulus videos that display presenters delivering information about which we manipulate their degree of doubt. We then show these stimuli to observers, and record four of their physiological signals. We find that a generalised neural network trained with physiological features is more accurate in differentiating the presenters' doubt/manipulated belief when compared with the same observers' own conscious judgments. The affective recognition performance improves when we analyse the physiological signals using multi-task learning techniques to train personalised and group personalised neural networks. The ability to recognise this doubt effect derives from observers' fundamental emotional reactions to the viewed stimuli, reflected in their physiological responses, and learnt by our neural networks. We believe this system using observer physiological signals collected in real life could reveal accurate and hidden audience distrust, which could in turn lead to enhanced truthfulness in future public- presented statements.

The relationship between music and human physiological signals has been a topic of interest among researchers for many years. Understanding this relationship can not only lead to more enhanced music therapy methods, but it may also help in finding a cure to mental disorders and epileptic seizures that are triggered by certain music. In this paper, we investigate the effects of 3 different genres of music in participants' Electrodermal Activity (EDA). Signals were recorded from 24 participants while they listened to 12 music stimuli. Various feature selection methods were applied to a number of features which were extracted from the signals. A simple neural network using Genetic Algorithm (GA) feature selection can reach as high as 96.8% accuracy in classifying 3 different music genres. Classification based on participants' subjective rating of emotion reaches 98.3% accuracy with the Statistical Dependency (SD) / Minimal Redundancy Maximum Relevance (MRMR) feature selection technique. This shows that human emotion has a strong correlation with different types of music. In the future this system can be used to distinguish music based on their positive of negative effect on human mental health.

Reading online can be difficult due to the distractions of digital environments. In this paper we present a user study in which participants' eye gaze was recorded as they read text in a visually distracting environment. We explore two distraction mitigation signals using real-time eye gaze data to investigate whether the effects help reduce distraction rate as well as aid recovery from distractions. These signals involved adding a signal to the last word read before a distraction occurred to show the reader where they were up to. We compared these experimental conditions on both first (L1) and second (L2) English language readers and for easy and difficult to read texts. The results demonstrate that the mitigation signals helped recovery from a distraction by drawing participants' attention back to the text as well as indicating from where to recommence reading. We conclude with recommendations on implementing distraction mitigation signals in text.