We present an end-to-end text-to-speech (TTS) synthesis system that generates audio and synchronized tongue motion directly from text. This is achieved by adapting a three-dimensional model of the tongue surface to an articulatory dataset and training a statistical parametric speech synthesis system directly on the tongue model parameters. We evaluate the model at every step by comparing the spatial coordinates of predicted articulatory movements against the reference data. The results indicate a global mean Euclidean distance of less than 2.8 mm, and our approach can be adapted to add an articulatory modality to conventional TTS applications without the need for extra data.

Vocal tract magnetic resonance imaging (MRI) has become one of the preferred imagingmodalities for the analysis of human speech production. However, the raw image data must be segmented before further analysis can take place. This paper describes a hybrid approach to extract a 3D tongue model from 3D or 2D MRI scans of the vocal tract during speech, which combines unsupervised image segmentation with a mesh deformation technique. An efficient, minimally supervised segmentation algorithm can also be used as an alternative to provide a robust fallback in certain isolated cases. Both image segmentation algorithms produce a point cloud, which is completed and registered by deforming a template mesh to the data. Since the mesh deformation can be applied even with a sparse point cloud, it is possible to extract realistic 3D tongue shapes even from the 2D video frames of real-time MRI. Our approach is applied to several sets of available MRI data and yields promising results.

The reliable estimation of the Lagrangian stress tensor from an image sequence is a challenging problem in mechanical engineering. Since this tensor involves first order motion derivatives, it appears tempting to estimate the optical flow field with a highly accurate variational model and compute its derivatives afterwards. In this paper we explain why this idea is inappropriate due to lower order smoothness assumptions and the ill-posedness of differentiation. As a remedy, we propose a variational framework that performs higher order regularisation of the optical flow field and directly computes the Lagrangian stress tensor from the image measurements. Due to its recursive structure, this framework is very generic. It can incorporate smoothness assumptions of arbitrary high order and allows to compute derivatives of any desired order in a stable way. With a biaxial tensile experiment with an elastomer we demonstrate that our novel approach gives substantially better results for the Lagrangian stress tensor than computing derivatives of the optical flow field. Moreover, it also outperforms a frequently used commercial software that marks the state-of-the-art for Lagrangian stress tensor computation.