Live music performances and music production often involve the manipulation of several parameters during sound generation, processing, and mixing. In hardware layouts, those parameters are usually controlled using knobs, sliders and buttons. When these layouts are virtualized, the use of physical (e.g. MIDI) controllers can make interaction easier and reduce the cognitive load associated to sound manipulation. The addition of haptic feedback can further improve such interaction by facilitating the detection of the nature (continuous / discrete) and value of a parameter. To this end, we have realized an endless-knob controller prototype with programmable resistance to rotation, able to render various haptic effects. Ten subjects assessed the effectiveness of the provided haptic feedback in a target-matching task where either visual-only or visual-haptic feedback was provided; the experiment reported significantly lower errors in presence of haptic feedback. Finally, the knob was configured as a multi-parametric controller for a real-time audio effect software written in Python, simulating the voltage-controlled filter aboard the EMS VCS3. The integration of the sound algorithm and the haptic knob is discussed, together with various haptic feedback effects in response to control actions.

Two experiments studied the role of impact sounds and vibrations in classification of materials. The task consisted of feeling on an actuated surface and listening through headphones to the recorded feedback of a ping-pong ball hitting three flat objects respectively made of wood, plastic, and metal, and then identifying their material. In Experiment 1, sounds and vibrations were recorded by keeping the objects in mechanical isolation. In Experiment 2, recordings were taken while the same objects stood on a table, causing their resonances to fade faster due to mechanical coupling with the support. A control experiment, where participants listened to and touched the real objects in mechanical isolation, showed high accuracy of classification from either sounds (90% correct) or vibrations (67% correct). Classification of reproduced bounces in Experiments 1 and 2 was less precise. In both experiments, the main effect of material was statistically significant; conversely, the main effect of modality (auditory or tactile) was significant only in the control. Identification of plastic and especially metal was less accurate in Experiment 2, suggesting that participants, when possible, classified materials by longer resonance tails. Audio-tactile summation of classification accuracy was found, suggesting that multisensory integration influences the perception of materials. Such results have prospective application to the nonvisual design of virtual buttons, which is the object of our current research.

A tool for the multisensory stylus-based exploration of virtual textures was used to investigate how different feedback modalities (static or dynamically deformed images, vibration, sound) affect exploratory gestures. To this end, we ran an experiment where participants had to steer a path with the stylus through a curved corridor on the surface of a graphic tablet/display, and we measured steering time, dispersion of trajectories, and applied force. Despite the variety of subjective impressions elicited by the different feedback conditions, we found that only nonvisual feedback induced significant variations in trajectories and an increase in movement time. In a post-experiment, using a paper-and-wood physical realization of the same texture, we recorded a variety of gestural behaviors markedly different from those found with the virtual texture. With the physical setup, movement time was shorter and texture-dependent lateral accelerations could be observed. This work highlights the limits of multisensory pseudo-haptic techniques in the exploration of surface textures.

An experiment was performed to study the effect of actively applied forces on vibrotactile thresholds. The task consisted in pressing the fingertip against a flat rigid surface which provided broadband vibration noise of varying amplitude. Three force levels were considered, ranging from light touch to hard press. Possibly due to the concurrent effect of large contact area, spectrally complex stimuli and active pressing force, the measured sensitivity thresholds are considerably lower than what is found in most of the previous literature. Moreover, significant differences in thresholds were found between the lowest and middle force level, and the highest and middle force level.

A physically based impact model-already known and exploited in the field of sound synthesis-is studied using both analytical tools and numerical simulations. It is shown that the Hamiltonian of a physical system composed of a mass impacting on a wall can be expressed analytically as a function of the mass velocity during contact. Moreover, an efficient and accurate approximation for the mass outbound velocity is presented, which allows to estimate the Hamiltonian at the end of the contact. Analytical results are then compared to numerical simulations obtained by discretizing the system with several numerical methods. It is shown that, for some regions of the parameter space, the trajectories of the discretized systems may significantly drift from the analytically derived curves. Two approaches, based on enforcing numerical energy consistency, are then proposed to improve the accuracy of numerical simulations.