Simulations are frequently used in first aid training to help first aid operators experience real-world scenarios in a controlled, safe environment. In these simulations, special effects can be used to enhance the realism of the scene, such as fake blood and makeup. Although these offer very high realism, they are also non-reusable and have a long setup time. In this paper, we propose an augmented reality (AR) system to provide a platform for designing and implementing special effects for first aid simulations. The AR system was implemented with a focus on the simulation of bleeding wounds, and later tested on 18 operators from the local Red Cross section. Data collected with the questionnaires show that all operators provided medical assistance during the simulation, and all rated the AR experience as useful. Furthermore, 13 did not find any difficulties in using the AR system, while 7 mentioned realism as a positive aspect. However, 3 described the video clarity as insufficient and 3 noted the absence of sounds. Overall, the collected feedback indicates that the AR system can be a useful tool for training Red Cross operators in first-aid scenarios, and participants’ opinions provided valuable suggestions for further development.

This study presents a Subject-Aware Transformer-based neural network designed for the Electroencephalogram (EEG) Emotion Recognition task (SATEER), which entails the analysis of EEG signals to classify and interpret human emotional states. SATEER processes the EEG waveforms by transforming them into Mel spectrograms, which can be seen as particular cases of images with the number of channels equal to the number of electrodes used during the recording process; this type of data can thus be processed using a Computer Vision pipeline. Distinct from preceding approaches, this model addresses the variability in individual responses to identical stimuli by incorporating a User Embedder module. This module enables the association of individual profiles with their EEGs, thereby enhancing classification accuracy. The efficacy of the model was rigorously evaluated using four publicly available datasets, demonstrating superior performance over existing methods in all conducted benchmarks. For instance, on the AMIGOS dataset (A dataset for Multimodal research of affect, personality traits, and mood on Individuals and GrOupS), SATEER's accuracy exceeds 99.8% accuracy across all labels and showcases an improvement of 0.47% over the state of the art. Furthermore, an exhaustive ablation study underscores the pivotal role of the User Embedder module and each other component of the presented model in achieving these advancements.

Abstract Nowadays, Virtual and Augmented Reality technologies play a supportive role in many research fields. In cultural heritage, various examples are available, including storytelling and narratives, where they can provide immersive and enhanced experiences to visitors and tourists, especially for entertainment and educational purposes. This review aims to investigate the opportunities that soundscape design and advanced sonic interactions in virtual and augmented environments can bring to cultural heritage sites and museums in terms of presence, emotional content, and cultural dissemination. Nineteen-two papers have been identified through the PRISMA methodology, and a promising positive effect of sonic interaction on user experience in a virtual environment can be observed in various studies, notwithstanding a general lack of specific contributions on the use of sound rendering and audio spatialisation for improving such experiences. Moreover, this work identifies the main involved research areas and discusses the state-of-the-art best practices and case studies where sonic interactions may assume a central role. The final part suggests possible future directions and applications for more engaging and immersive storytelling in the cultural heritage domain.

Wave Digital Filter (WDF) theory has been widely used to design nonlinear digital filters that behave like reference analog circuits, especially in the field of Virtual Analog modeling, i.e., the digital emulation of audio circuits. WDF principles allow us to implement circuits with one nonlinearity in an explicit fashion, by properly setting the free parameters that are introduced in the Wave Digital (WD) domain. Although this property does not extend to the WDF realization of circuits with multiple nonlinearities, which instead requires the use of iterative solvers, a proper setting of the free parameters is beneficial also in these cases, in terms of both robustness and efficiency of the implementation. In particular, recent research shows that a common optimal policy for the setting of free parameters can be adopted when either a WD fixed-point method or a WD Newton-Raphson (NR) method is used to solve the same nonlinear circuit. In this work, we present a class of WD extended fixed-point solvers with order from zero to infinity that generalizes the aforementioned iterative methods; the zero-order case corresponds to the existent WD fixed-point method, while the infinite-order case to WD NR. The proposed solvers do not require to compute inverse Jacobian matrices and are characterized by superlinear speed of convergence. Moreover, we show that the very same policy of free parameter setting can be applied independently of the order, causing, in any case, an increase of robustness and convergence speed.

A family of iterative root-finding methods for nonlinear discrete-time systems of equations is presented, with a formulation that puts it in between the Fixed-Point (FP) and Newton-Raphson (NR) methods. Applicability of this family is allowed, provided that the Jacobian matrix of the nonlinear system has a spectral radius less than one. By varying the order of a matrix geometric sum that approximates the inverse Jacobian matrix, root-finding at any iteration can be steered toward the FP or conversely toward the NR method, becoming identical to either of them if the order is equal to zero or infinitely large, respectively. Since the methods in this family do not need the solution of a linear system at each iteration as required by NR, their computational cost makes them palatable for the online digital implementation of nonlinear models. As an example of application, a Virtual Analog model of the voltage-controlled filter onboard a popular music synthesizer is tested, showing that for some orders of the aforementioned geometric sum the proposed methods perform better than FP and NR in terms of computational cost, while exhibiting the same accuracy.

Live music performances and music production often involve the manipulation of several parameters during sound generation, processing, and mixing. In hardware layouts, those parameters are usually controlled using knobs, sliders and buttons. When these layouts are virtualized, the use of physical (e.g. MIDI) controllers can make interaction easier and reduce the cognitive load associated to sound manipulation. The addition of haptic feedback can further improve such interaction by facilitating the detection of the nature (continuous / discrete) and value of a parameter. To this end, we have realized an endless-knob controller prototype with programmable resistance to rotation, able to render various haptic effects. Ten subjects assessed the effectiveness of the provided haptic feedback in a target-matching task where either visual-only or visual-haptic feedback was provided; the experiment reported significantly lower errors in presence of haptic feedback. Finally, the knob was configured as a multi-parametric controller for a real-time audio effect software written in Python, simulating the voltage-controlled filter aboard the EMS VCS3. The integration of the sound algorithm and the haptic knob is discussed, together with various haptic feedback effects in response to control actions.

The digital implementation of a nonlinear audio circuit often employs the Newton-Raphson (NR) method for solving the corresponding system of implicit ordinary differential equations in the discrete-time domain. Although its quadratic convergence speed makes NR attractive for real-time audio applications, quadratic convergence is not always guaranteed, since it depends on initial conditions, and also divergence might occur. For this reason, especially in the context of Virtual Analog modeling, techniques for increasing the robustness of NR are in order. Among the various approaches, the Wave Digital (WD) formalism recently showed potential to rethink traditional circuit simulation methods. In this manuscript, we discuss an original formulation of the NR method in the WD domain for the solution of audio circuits with multiple one-port nonlinearities. We provide an in-depth theoretical analysis of the proposed iterative method and we show how its quadratic convergence strongly depends on the free parameters (called port resistances) introduced when modeling the reference circuit in the WD domain. In particular, we demonstrate that the size of the basin where the WD NR solver can be initialized to converge on a solution with quadratic speed is a function of the free parameters. We also show that by setting each port resistance value as close as possible to the derivative w.r.t. current of the nonlinear element v-i characteristic we keep the basin size large. We finally implement an audio ring modulator circuit with four diodes in order to test the proposed iterative method.

Two experiments studied the role of impact sounds and vibrations in classification of materials. The task consisted of feeling on an actuated surface and listening through headphones to the recorded feedback of a ping-pong ball hitting three flat objects respectively made of wood, plastic, and metal, and then identifying their material. In Experiment 1, sounds and vibrations were recorded by keeping the objects in mechanical isolation. In Experiment 2, recordings were taken while the same objects stood on a table, causing their resonances to fade faster due to mechanical coupling with the support. A control experiment, where participants listened to and touched the real objects in mechanical isolation, showed high accuracy of classification from either sounds (90% correct) or vibrations (67% correct). Classification of reproduced bounces in Experiments 1 and 2 was less precise. In both experiments, the main effect of material was statistically significant; conversely, the main effect of modality (auditory or tactile) was significant only in the control. Identification of plastic and especially metal was less accurate in Experiment 2, suggesting that participants, when possible, classified materials by longer resonance tails. Audio-tactile summation of classification accuracy was found, suggesting that multisensory integration influences the perception of materials. Such results have prospective application to the nonvisual design of virtual buttons, which is the object of our current research.

The poster describes a multisensory simulation of plucking guitar strings in virtual reality and a user study evaluating the simulation. Auditory feedback is generated by a physics-based simulation of guitar strings, and haptic feedback is provided by a combination of high fidelity vibrotactile actuators and a Phantom Omni. The study compared four conditions: no haptic feedback, vibrotactile feedback, force feedback, and a combination of force and vibrotactile feedback. The results indicate that the combination of vibrotactile and force feedback elicits the most realistic experience, and during this condition, participants were less likely to inadvertently hit strings. Notably, no significant differences were found between the conditions involving either vibrotactile or force feedback.

For their numerical properties and speed of convergence, Newton-Raphson methods are frequently used to compute nonlinear audio electronic circuit models in the digital domain. These methods are traditionally employed regardless of preliminary considerations about their applicability, primarily because of a lack of flexible mathematical tools making the convergence analysis an easy task. We define the basin delimiter, a tool that can be applied to the case when the nonlinear circuit is modeled by a delay-free loop network. This tool is derived from a known convergence theorem providing a sufficient condition for quadratic speed of convergence of the method. After substituting the nonlinear characteristics with equivalent linear filters that compute Newton-Raphson on the existing network, through the basin delimiter, we figure out constraints guaranteeing quadratic convergence speed in the diode clipper. Further application to a ring modulator circuit does not lead to comparably useful constraints for quadratic convergence; however, also in this circuit, the basin delimiter has a magnitude roughly proportional to the number of iterations needed by the solver to find a solution. Together, such case studies foster refinement and generalization of this tool as a speed predictor, with potential application to the design of virtual analogue systems for real-time digital audio effects.

An iterative method is proposed for the explicit computation of discrete-time nonlinear filter networks containing delay-free loops. The method relies on a fixed-point search of the signal values at every temporal step. The formal as well as numerical properties of fixed-point solvers delimit its applicability: On the one hand, the method allows for a reliable prediction of the frequency rates where the simulation is stable, while, on the other hand, its straightforward applicability is counterbalanced by low speed of convergence. Especially in presence of specific nonlinear characteristics, the use of a fixed-point search is limited if the real-time constraint holds. For this reason, the method becomes useful especially during the digital model prototyping stage, as exemplified while revisiting a previous discrete-time realization of the voltage-controlled filter aboard the EMS VCS3 analog synthesizer. Further tests conducted on a digital ring modulator model support the above considerations.

A decimated version of a broad family of Finite-Difference Time-Domain schemes is derived by an algebraic rearrangement of their matrix formulation, allowing for computing the grid nodes at half the temporal steps compared to the original scheme. This rearrangement can ask for solving a generalized matrix inversion problem. The decimated scheme generates solutions having comparable accuracy to that exhibited by the original simulations. However, the broader applicability of the proposed technique requires to solve currently unanswered theoretical issues of spatial grid decimation, as well as to make extensive tests using large matrices.

Interactive audio processing systems sometimes include user-controlled digital filters, whose response is obtained by interpolating between different transfer functions. Audible artifacts produced by these filters are caused by migration across the unit circle of the zeros in their transfer function, depending on the interpolation parameter values. There are cases in which the Nyquist stability criterion for discrete-time systems can turn useful to inspect, and possibly modify the trajectories of such zeros. After a general discussion, two examples are illustrated in which Nyquist stability is used to optimize the response of filters realizing a linear interpolation between two transfer functions.

This paper presents a real-time piano synthesizer where both the transverse and longitudinal motion of the string is modeled by modal synthesis, resulting in a coherent and highly parallel model structure. The paper applies recent developments in piano modeling and focuses on the issues related to practical implementation (e.g., numerical stability, aliasing, and efficiency). A strong emphasis is given to modeling nonlinear string vibrations, and a new variation of earlier synthesis techniques is proposed which is particularly well suited for modal synthesis. For soundboard modeling, the possibilities of using fast Fourier transform-based fast convolution and parallel second-order filters are discussed. Additionally, the paper describes the details of the software implementation and discusses the computational complexity of each model block. The piano model runs on current computer hardware with full polyphony in real time.

A previously developed nonlinear differential equation system, representative of the diode-based voltage-controlled filter on board of the EMS VCS3 analog synthesizer, can be modeled in terms of a filter network. The properties of this network can be converted block-by-block into a discrete-time version of the same model, in which the relationships existing between the system variables take the form of delay-free loops. In the presence of such loops, the discrete-time filter network is solved by iterating over a fixed-point scheme. Simulations running at 176.4 kHz show reasonable fidelity of the model in the regimes of interest, as well as shorter computation time compared to previous implementations of the same system. While reducing aliasing, such a sampling rate optimizes also the computation time as it requires fewer fixed-point iterations. The new model represents an improved alternative to previously developed voltage-controlled filter designs in the field of virtual analog.

The 16 papers in this special issue focus on virtual audio effects and musical instruments.

Spatialization of sound sources in depth allows a hierarchical display of multiple audio streams and therefore may be an efficient tool for developing novel auditory interfaces. In this paper we present an audio-haptic interface for audio browsing based on rendering distance cues for ordering sound sources in depth. The haptic interface includes a linear position tactile sensor made by conductive material. The touch position on the ribbon is mapped onto the listening position on a rectangular virtual membrane, modeled by a bidimensional Digital Waveguide Mesh and providing distance cues of four equally spaced sound sources. Furthermore a knob of a MIDI controller controls the position of the mesh along the playlist, which allows to browse the whole set of files. Subjects involved in a user study found the interface intuitive and entertaining. In particular the interaction with the stripe was highly appreciated.

In a study of auditory distance perception, we investigated the effects of exaggeration the acoustic cue of reverberation where the intensity of sound did not vary noticeably. The set of stimuli was obtained by moving a sound source inside a 10.2-m long pipe having a 0.3-m diameter. Twelve subjects were asked to listen to a speech sound while keeping their head inside the pipe and then to estimate the egocentric distance from the sound source using a magnitude production procedure. The procedure was repeated eighteen times using six different positions of the sound source. Results show that the point at which perceived distance equals physical distance is located approximately 3.5 m away from the listening point, with an average range of distance estimates of approximately 3.3 m, i.e., 1.65 to 4.9 m. The absence of intensity cues makes the acoustic pipe a potentially interesting modeling paradigm for the design of auditory interfaces in which distance is rendered independently of loudness. The proposed acoustic environment also confirms the known unreliability of certain distance cues.

A discrete-time realization of first-order (shelving) and second-order equalization filters is developed, providing bandpass/bandstop magnitude-complementary transfer functions. The bandpass transfer function is turned into a complementary one, and vice versa, switching between two structures that share a common allpass section containing the state variables of the equalizer. The present realization is an alternative to existing solutions, particularly in applications where the equalization parameters are dynamically varied.

A method that computes linear digital filter networks containing delay-free loops is proposed. Compared to existing techniques the proposed method does not require a rearrangement of the network structure, conversely it makes use of matrices describing this structure and specifying the connections between the filter blocks forming the network. For this reason the efficiency of the method becomes interesting when the filter blocks are densely interconnected. The Triangular Waveguide Mesh is an example of "dense" filter network: Using the proposed method we can compute a transformed, delay-free version of this mesh, obtaining simulations that are significantly more accurate compared to those provided by the traditional, explicitly computable formulation of the triangular mesh.

A virtual resonating environment aiming at enhancing our perception of distance is proposed. This environment reproduces the acoustics inside a tube, thus conveying peculiar distance cues to the listener. The corresponding resonator has been prototyped using a wave-based numerical scheme called waveguide mesh, that gave the necessary versatility to the model during the design and parameterization of the listening environment. Psychophysical tests show that this virtual environment conveys robust distance cues.

This paper describes an electronic travel aid device, that may enable blind individuals to "see the world with their ears". A wearable prototype will be assembled using low-cost hardware: earphones, sunglasses fitted with two micro cameras, and a palmtop computer. The system, which currently runs on a desktop computer, is able to detect the light spot produced by a laser pointer, compute its angular position and depth, and generate a corresponding sound providing auditory cues for perception of the position and distance of the pointed surface patch. It permits different sonification modes that can be chosen by drawing, with the laser pointer, a predefined stroke which will be recognized by a hidden Markov model. In this way a blind person can use a common pointer as a replacement for the cane and will interact with the device using a flexible and natural sketch based interface.

Waveguide meshes are efficient and versatile models of wave propagation along a multidimensional ideal medium. The choice of the mesh geometry affects both the computational cost and the accuracy of simulations. In this paper, we focus on two-dimensional (2-D) geometries and use multidimensional sampling theory to compare the square, triangular, and hexagonal meshes in terms of sampling efficiency and dispersion error under conditions of critical sampling. The analysis shows that the triangular geometry exhibits the most desirable tradeoff between accuracy and computational cost.