In the context of efficient sound synthesis by digital waveguides, we present a novel methodology for joint modeling of string instrument body radiativity and driving-point bridge admittance functions, as obtained from experimental data. From our modeling framework, aimed at simulation of guitar and bowed string sound, here we focus on the body of the instrument and leave aside the strings. First, a modal decomposition of the measured bridge admittance is obtained by means of a novel frequency-domain algorithm for optimization of recursive digital filters in parallel form. Then, from extracted modal parameters, the radiativity, and admittance functions are modeled by projecting measurements over a common modal basis, enforcing passivity of the two-dimensional admittance model by means of semidefinite programming. We propose a formulation that enables the joint realization of bridge reflectance and sound radiativity as a lumped delay line termination in which a single bank of resonant filters is shared among all string reflection and body radiation outputs. Our approach provides efficient means to model two-dimensional (2-D) bridge reflectance, 2-D string-string coupling, sound radiation with an arbitrary number of outputs, and (implicitly) vibrational energy loss from the bridge transmittance to nonradiating modes and dissipation.

We present a technique to iteratively optimize poles of a recursive digital filter in parallel form. Only exposing the poles as the variables to optimize, we employ a linearly constrained gradient descent routine in which the numerical estimation of the error gradient involves first obtaining the zeros by projecting the target response over a basis of responses defined by the pole positions at a given step. Example fits are presented for exponentially decaying white noise and measured violin radiativity filters.

In this technical demo we present repoVizz (http://repovizz.upf.edu), an integrated online system capable of structural formatting and remote storage, browsing, exchange, annotation, and visualization of synchronous multi-modal, time-aligned data. Motivated by a growing need for data-driven collaborative research, repoVizz aims to resolve commonly encountered difficulties in sharing or browsing large collections of multi-modal data. At its current state, repoVizz is designed to hold time-aligned streams of heterogeneous data: audio, video, motion capture, physiological signals, extracted descriptors, annotations, et cetera. Most popular formats for audio and video are supported, while Broadcast WAVE or CSV formats are adopted for streams other than audio or video (e.g., motion capture or physiological signals). The data itself is structured via customized XML files, allowing the user to (re-) organize multi-modal data in any hierarchical manner, as the XML structure only holds metadata and pointers to data files. Datasets are stored in an online database, allowing the user to interact with the data remotely through a powerful HTML5 visual interface accessible from any standard web browser; this feature can be considered a key aspect of repoVizz since data can be explored, annotated, or visualized from any location or device. Data exchange and upload/download is made easy and secure via a number of data conversion tools and a user/permission management system.

The objective of this research is to model the relationship between actions performed by a violinist and the sound which these actions produce. Violinist actions and audio are captured during real performances by means of a newly developed sensing system from which bowing and audio descriptors are computed. A database is built with this data and used to train a generative model based on neural networks. The model is driven by a continuous sequence of bowing and fingering controls and is able to generate their corresponding sequence of spectral envelopes. The model is used for synthesis, either alone as a purely spectral model, by filling the predicted envelopes with harmonic and noisy components, or coupled with a concatenative synthesizer, where the predicted envelopes are used as time-varying filters to transform the concatenated samples. The combination of sample concatenation with the timbre model allows for the preservation of sound quality inherent in samples, while providing a high level of control. Additionally, we perform an analysis of the violinist control space and the influence of the controls on the timbre.

We describe an evolutionary approach to one of the most challenging problems in computer music: modeling how skilled musicians manipulate sound properties such as timing and amplitude in order to express their view of the emotional content of musical pieces. Starting with a collection of audio recordings of real performances, we apply a sequential-covering genetic algorithm in order to obtain computational models for different aspects of expressive performance. We use these models to automatically synthesize performances with the timing and energy expressiveness that characterizes the music generated by a professional musician. The reported results indicate that evolutionary computation is an appropriate technique for solving the problem considered. Specifically, our evolutionary algorithm provides a number of potential advantages over other supervised learning algorithms, such as a method for non-deterministically obtaining models capturing different possible interpretations of a musical piece.

We present a machine learning approach to modeling bowing control parameter contours in violin performance. Using accurate sensing techniques we obtain relevant timbre-related bowing control parameters such as bow transversal velocity, bow pressing force, and bow-bridge distance of each performed n ote. Each performed note is represented by a curve parameter vector and a number of note classes are defined. The principal components of the data represented by the set of curve parameter vectors are obtained for each class. Once curve parameter vectors are expressed in the new space defined by the principal components, we train a model based on inductive logic programming, able to predict curve parameter vectors used for rendering bowing controls. We evaluate the prediction results and show the potential of the model by predicting bowing control parameter contours from an annotated input score.

Excitation-continuous music instrument control patterns are often not explicitly represented in current sound synthesis techniques when applied to automatic performance. Both physical model-based and sample-based synthesis paradigms would benefit from a flexible and accurate instrument control model, enabling the improvement of naturalness and realism. We present a framework for modeling bowing control parameters in violin performance. Nearly non-intrusive sensing techniques allow for accurate acquisition of relevant timbre-related bowing control parameter signals. We model the temporal contour of bow velocity, bow pressing force, and bow-bridge distance as sequences of short Be¿zier cubic curve segments. Considering different articulations, dynamics, and performance contexts, a number of note classes are defined. Contours of bowing parameters in a performance database are analyzed at note-level by following a predefined grammar that dictates characteristics of curve segment sequences for each of the classes in consideration. As a result, contour analysis of bowing parameters of each note yields an optimal representation vector that is sufficient for reconstructing original contours with significant fidelity. From the resulting representation vectors, we construct a statistical model based on Gaussian mixtures suitable for both the analysis and synthesis of bowing parameter contours. By using the estimated models, synthetic contours can be generated through a bow planning algorithm able to reproduce possible constraints caused by the finite length of the bow. Rendered contours are successfully used in two preliminary synthesis frameworks: digital waveguide-based bowed string physical modeling and sample-based spectral-domain synthesis.

We propose a novel approach to the task of identifying performers from their playing styles. We investigate how skilled musicians (Jazz saxophone players in particular) express and communicate their view of the musical and emotional content of musical pieces and how to use this information in order to automatically identify performers. We study deviations of parameters such as pitch, timing, amplitude and timbre both at an inter-note level and at an intra-note level. Our approach to performer identification consists of establishing a performer dependent mapping of inter-note features (essentially a "score" whether or not the score physically exists) to a repertoire of inflections characterized by intra-note features. We present a successful performer identification case study