Fear is a multifaceted emotion, challenging to define and assess accurately. Biologically, it is an innate response to threats, whereas psychologically, it is shaped by individual experiences and societal influences, sometimes evolving into specific phobias. One such phobia, arachnophobia (the fear of spiders), is particularly widespread and can vary widely in intensity among individuals. This paper presents a prototypal system that classifies the severity of arachnophobia using Machine Learning (ML) algorithms within a Virtual Reality (VR) game-based environment. The proposed system utilizes a two-stage clustering approach to analyze the behavioral data collected during VR exposure. Additionally, participants' self-reported fear levels are measured using the Spider Phobia Questionnaire (SPQ), to provide a comprehensive assessment. Preliminary results suggest that this method could be effectively used to classify arachnophobia intensity level, thus offering potential applications in both clinical settings and video games.

Attention guides our gaze to fixate the proper location of the scene and holds it in that location for the de-served amount of time given current processing demands, before shifting to the next one. As such, gaze deploy-ment crucially is a temporal process. Existing computational models have made significant strides in predicting spatial aspects of observer's visual scanpaths (where to look), while often putting on the background the tempo-ral facet of attention dynamics (when). In this paper we present TPP-Gaze, a novel and principled approach to model scanpath dynamics based on Neural Temporal Point Process (TPP), that Jointly learns the temporal dynamics of fixations position and duration, integrating deep learning methodologies with point process theory. We conduct ex-tensive experiments across five publicly available datasets. Our results show the overall superior performance of the proposed model compared to state-of-the-art approaches. Source code and trained models are publicly available at: https://github.com/phuselab/tppgaze.

In the realm of video games, achieving a state of flow is paramount for player engagement, striking a delicate balance between boredom and anxiety. Under such circumstances, monitoring player's stress level plays a crucial role. Here, by addressing state-space dynamics, we model stress unfolding over time, allowing for both its continuous and discrete assessment. This study builds upon our previous research, advancing stress assessment techniques to enhance tailored and immersive gaming experiences through Virtual Reality in horror games. We provide detailed insights into our collected data, which includes physiological measurements, motion behavioural data, and participants' self-reported stress levels. Additionally, we conduct an in-depth analysis on individual participants to delve deeper into the dynamics of stress experienced by each player. Results achieved give evidence that the measurement of motion behavioural data, exclusively collected from the headset, well compares to that based on electrodermal activity (EDA), more classically related to stress assessment. Data are made publicly available athttps://zenodo.org/records/15025199.

Touch exploration of fabric is used to evaluate its properties, and it could further be leveraged to understand a consumer’s sensory experience and preference so as to support them in real time to make careful clothing purchase decisions. In this paper, we open up opportunities to explore the use of technology to provide such support with our FabricTouch dataset, i.e., a multimodal dataset of fabric assessment touch gestures. The dataset consists of bilateral forearm movement and muscle activity data captured while 15 people explored 114 different garments in total to evaluate them according to 5 properties (warmth, thickness, smoothness, softness, and flexibility). The dataset further includes subjective ratings of the garments with respect to each property and ratings of pleasure experienced in exploring the garment through touch. We further report baseline work on automatic detection. Our results suggest that it is possible to recognise the type of fabric property that a consumer is exploring based on their touch behaviour. We obtained mean F1 score of 0.61 for unseen garments, for 5 types of fabric property. The results also highlight the possibility of additionally recognizing the consumer’s subjective rating of the fabric when the property being rated is known, mean F1 score of 0.97 for unseen subjects, for 3 rating levels.

In this work, we explored the advantages of dynamic game balancing using players’ affective state introducing StrEx, a plugin designed to modulate the stress level induced by a video game in an unobtrusive way. The proposed plugin is also intended to validate results obtained by [1]. Our system collects motion behavioral data and updates a model of the player’s stress level to guide the transitions of a Finite State Machine which regulates the stress level induced by the game through the generation of game content. We designed and developed a virtual reality horror-survival game, with the aim of testing the system functioning. Our approach shows promising potential to create more immersive and engaging video games exploiting affective-based adaptation.

Abstract In this study we propose an approach to assess the fear of heights through a 3D virtual reality environment. We show that an immersive scenario provides a suitable infrastructure to such purpose, when supported by related behavioural and physiological measurements. Our approach is grounded in the principled framework of constructed emotions. This allows to shape fear detection as a case of categorical perception, which is amenable to be formalised as an unsupervised learning problem. Meanwhile, it paves the way for addressing meaningful physiological parameters for the assessment. Gauging fear of heights in individuals, beyond its theoretical relevance, is cogent for the early discernment of workers who are unsuited for operating at altitude and who may require to undergo specific training or, eventually, to be recruited for different positions.

We present AMHUSE (A Multimodal dataset for HUmour SEnsing) along with a novel web-based annotation tool named DANTE (Dimensional ANnotation Tool for Emotions). The dataset is the result of an experiment concerning amusement elicitation, involving 36 subjects in order to record the reactions in presence of 3 amusing and 1 neutral video stimuli. Gathered data include RGB video and depth sequences along with physiological responses (electrodermal activity, blood volume pulse, temperature). The videos were later annotated by 4 experts in terms of valence and arousal continuous dimensions. Both the dataset and the annotation tool are made publicly available for research purposes.

Within a Music Information Retrieval perspective, the goal of the study presented here is to investigate the impact on sound features of the musician's affective intention, namely when trying to intentionally convey emotional contents via expressiveness. A preliminary experiment has been performed involving 10 tuba players. The recordings have been analysed by extracting a variety of features, which have been subsequently evaluated by combining both classic and machine learning statistical techniques. Results are reported and discussed.

In this paper, we shall consider the problem of deploying attention to the subsets of the video streams for collating the most relevant data and information of interest related to a given task. We formalize this monitoring problem as a foraging problem. We propose a probabilistic framework to model observer's attentive behavior as the behavior of a forager. The forager, moment to moment, focuses its attention on the most informative stream/camera, detects interesting objects or activities, or switches to a more profitable stream. The approach proposed here is suitable to be exploited for multistream video summarization. Meanwhile, it can serve as a preliminary step for more sophisticated video surveillance, e.g., activity and behavior analysis. Experimental results achieved on the UCR Videoweb Activities Data Set, a publicly available data set, are presented to illustrate the utility of the proposed technique.

Visual attention guides our gaze to relevant parts of the viewed scene, yet the moment-to-moment relocation of gaze can be different among observers even though the same locations are taken into account. Surprisingly, the variability of eye movements has been so far overlooked by the great majority of computational models of visual attention. In this paper we present the ecological sampling model, a stochastic model of eye guidance explaining such variability. The gaze shift mechanism is conceived as an active random sampling that the foraging eye carries out upon the visual landscape, under the constraints set by the observable features and the global complexity of the landscape. By drawing on results reported in the foraging literature, the actual gaze relocation is eventually driven by a stochastic differential equation whose noise source is sampled from a mixture of α-stable distributions. This way, the sampling strategy proposed here allows to mimic a fundamental property of the eye guidance mechanism: where we choose to look next at any given moment in time, it is not completely deterministic, but neither is it completely random To show that the model yields gaze shift motor behaviors that exhibit statistics similar to those displayed by human observers, we compare simulation outputs with those obtained from eye-tracked subjects while viewing complex dynamic scenes.

We discuss how a probabilistic interpretation of the output provided by a cascade of boosted classifiers can be exploited for Bayesian tracking in video streams. In particular, real-time face and body detection can be achieved by relying on such a Bayesian framework. Results show that such integrated approach is appealing with respect both to robustness and computational efficiency.

We address the problem of object-based visual attention from a Bayesian standpoint. We contend with the issue of joint segmentation and saliency computation suitable to provide a sound basis for dealing with higher level information related to objects present in dynamic scene. To this end we propose a framework relying on nonparametric Bayesian techniques, namely variational inference on a mixture of Dirichlet processes.

In this paper we consider the problem of sensorimotor coordination in a Bayesian framework. To this end we introduce a novel kind of Dynamic Bayesian Network serving as the core tool to integrate active vision and task-constrained motor behaviors. The proposed system is put into work by addressing the challenging task of realistic drawing performed by a robotic agent, namely a 7-DOF anthropomorphic manipulator. Simulation results are compared to those obtained by eye-tracked human subjects involved in drawing experiments.

In this paper, we discuss how image segmentation can be handled by using Bayesian learning and inference. In particular variational techniques relying on free energy minimization will be introduced. It will be shown how to embed a spatial diffusion process on segmentation labels within the Variational Bayes learning procedure so as to enforce spatial constraints among labels.

In this paper we address the challenging problem of sensorimotor integration, with reference to eye-hand coordination of an artificial agent engaged in a natural drawing task. Under the assumption that eye-hand coupling influences observed movements, a motor continuity hypothesis is exploited to account for how gaze shifts are constrained by hand movements. A Bayesian model of such coupling is presented in the form of a novel Dynamic Bayesian Network, namely an Input–Output Coupled Hidden Markov Model. Simulation results are compared to those obtained by eye-tracked human subjects involved in drawing experiments.

We present a Bayesian model that allows to automatically generate fixations/foveations and that can be suitably exploited for compression purposes. The twofold aim of this work is to investigate how the exploitation of high-level perceptual cues provided by human faces occurring in the video can enhance the compression process without reducing the perceived quality of the video and to validate such assumption with an extensive and principled experimental protocol.

In this paper we address the issue of how form and motion can be integrated in order to provide suitable information to attentively track multiple moving objects. Such integration is designed in a Bayesian framework, and a Belief Propagation technique is exploited to perform coherent form/motion labeling of regions of the observed scene. Experiments on both synthetic and real data are presented and discussed.

In this note it is discussed how face detection and tracking in video can be achieved relying on a detection-tracking loop. Such integrated approach is appealing with respect either to robustness and computational efficiency.

Multiresolution diffused expectation maximisation performs segmentation on vector (e.g. color) images within a multiscale framework; segmentation is carried out via the expectation maximisation algorithm, coupled with anisotropic diffusion on classes, in order to account for the spatial dependencies among pixels.

We view scenes in the real world by moving our eyes three to four times each second and integrating information across subsequent fixations (foveation points). By taking advantage of this fact, in this paper we propose an original approach to partitioning of a video into shots based on a foveated representation of the video. More precisely, the shot-change detection method is related to the computation, at each time instant, of a consistency measure of the fixation sequences generated by an ideal observer looking at the video. The proposed scheme aims at detecting both abrupt and gradual transitions between shots using a single technique, rather than a set of dedicated methods. Results on videos of various content types are reported and validate the proposed approach.

A framework for diffusion of color images is presented. The method is based on the theory of thermodynamics of irreversible transformations which provides a suitable basis for designing correlations between the different color channels. More precisely, we derive an equation for color evolution which comprises a purely spatial diffusive term and a nonlinear term that depends on the interactions among color channels over space. We apply the proposed equation to images represented in several color spaces, such as RGB, CIELAB, Opponent colors, and IHS.

The evolution of information in images undergoing fine-to-coarse anisotropic transformations is analyzed by using an approach based on the theory of irreversible transformations. In particular, we show that, when an anisotropic diffusion model is used, local variation of entropy production over space and scale provides the basis for a general method to extract relevant image features.

Loss of information in images undergoing fine-to-coarse image transformations is analyzed by using an approach based on the theory of irreversible transformations. The method is applied to isotropic and anisotropic diffusion and an algorithm based on entropy variation is presented that extracts relevant features of the image and provides a method to discriminate between smooth, textured and edge-type regions.

We generalize to colour images a method to calculate the local information content in an image. The method is based on the theory of thermodynamics of irreversible transformations and relies on fine-to-coarse transformations, such as diffusion processes. To this end, we discuss an extension to multi-valued images of isotropic diffusion, where cross-interactions between the different colour channels are allowed. Results achieved in simulation are presented and discussed.

Employing a thermodynamical model applied to scale-space transformations, we derive a method to encode different types of image features, as regions of different information content. Information is locally defined in terms of activity of the density of entropy production along scales. An activity map is formed, which represents the conspicuity or saliency of each point of the image. According to the different levels of activity, labelled as high, medium or low, we can extract basic parts of the image by collecting points of similar saliency.

This paper deals with a novel way for representing and computing image features encapsulated within different regions of scale-space. Employing a thermodynamical model for scale-space generation, the method derives features as those corresponding to "entropy rich" image regions where, within a given range of spatial scales, the entropy gradient remains constant. Different types of image features, defining regions of different information content, are accordingly encoded by such regions within different bands of spatial scale.

We present a method for the detection of microcalcifications in digital mammographic images. Our approach is based on the wavelet transform, but differently from other techniques proposed in the literature, the detection is directly accomplished in the wavelet domain and no inverse transform is required. After a preliminary denoising pass, microcalcifications are separated from background tissue. This is performed by exploiting information gained through evaluation of Renyi's entropy at the different decomposition levels of the wavelet space. Experimental results achieved on the standard Nimegen data set are shown and discussed.

The notion of entropy production /spl Pscr/, as defined by the thermodynamics of irreversible transformations, is used to quantify information loss during a transformation from fine-to-coarse image representations. To this aim we first show the existence of a precise relation between /spl Pscr/ and the Kullback-Leibler distance. This relation allows one to gauge local and global information losses which are associated with the global and local information content information of the images. Connections between the proposed approach, segmentation issues, and classical information theory are discussed.

Presents a method for the detection of small objects embedded in a noisy background. The detection is performed on the wavelet transformed image. After a preliminary de-noising pass, the objects are separated from background by exploiting the evaluation of Renyi's information at the different decomposition levels of the wavelet transform. We apply the proposed technique to detect microcalcifications in digital mammographic images.

We present results obtained by different contrast enhancement methods applied to medical images. We take into account classical histogram specification, local and wavelet-based techniques and a novel approach for multiscale contrast enhancement. The latter, whose rationale grounds in theories of visual perception, exploits a local definition of the Fechner-Weber's contrast within the context of a non-linear scale-space representation generated by anisotropic diffusion. Our experimental fields concerns a difficult kind of medical images, namely digital mammographic images.

This paper contributes a novel approach to contrast enhancement. The proposed approach measures local contrast within the context of a nonlinear scale-space representation. The original image is locally probed at multiple resolutions generated through anisotropic diffusion. Once local contrast has been estimated across an optimal range of scales, its value is used to enhance the initial image. Due to the choice of the anisotropic scale-space, the method also accounts for nonlinear edge contributions. Properties of local contrast within scale-space are introduced and discussed.

For the purpose of enhancing mammographic images, the authors introduce filters based on anisotropic diffusion, and they investigate their relations with classical methods like median, weighted median and tree-structured filters. The performances of the different techniques are quantitatively evaluated and discussed.

Filters for mammography based on anisotropic diffusion are introduced and evaluated with respect to well known filters adopted within this field. Results reported concern with images including microcalcifications. The performances of anisotropic filters are assessed both on the basis of figures of merit, allowing a preliminary quantitative enhancement evaluation, and according to goal-directed evaluation relying upon a simple segmentation module. Results indicate that proposed filters exhibit an effective and appealing behavior but they are subject to a more complex tuning, if compared to traditional filters.

In todays hospitals the medical workstation is a basic component of any image management and communication system. The design of such component can be very complex, because of the challenging engineering requirements. In this work we present an architectural model of a flexible and portable software platform upon which medical workstations can be realized. The model is developed within the overall framework provided by the object oriented paradigm.>

An object-oriented approach to the development of an environment for document processing and analysis is presented. The use of object-oriented techniques may play an important role in the domain we are considering. Document processing systems built today are different from what they were a few years ago: they are larger and more complex. The need of achieving effective and stable systems has gained higher priority. Furthermore, recent systems are expected to devote a deeper attention to the user's perspective. A general model that can be suitably adopted for building such systems is outlined, and it is shown how the object-oriented paradigm can provide a unifying framework for developing the experimental environment. Methodological steps involved are illustrated and a preliminary version of the environment is described.>