Aurio is an open source software library written for audio-based processing, analysis, and retrieval of audio and video recordings. The novelty of this library is the implementation of a number of fingerprinting and time warping algorithms to retrieve, match and synchronize media streams, which no other library currently offers. It is designed with simplicity, performance and versatility in mind, can be easily integrated into .NET applications, and offers a collection of many basic signal processing methods. It can read many file formats, offers multiple export abilities for further processing, and contains various UI widgets for graphical applications. Built upon the Aurio library, AudioAlign is an additionally released open source application for the (semi-)automatic synchronization of media recordings.

Most multimedia synchronization methods developed in the past are unimodal and consider only the audio data or the video data. Just recently, methods started to emerge that embrace multimodality by utilizing both audio and video processing to improve synchronization results. Although promising, their results are still not sufficient for fully automatic synchronization of recordings from heterogeneous sources. Video processing is also often too expensive to be used on large corpora of recordings, e.g. as they are commonly produced by crowds at social events. In my doctoral thesis, I will try to develop synchronization methods further by (a) examining fundamental problems that are usually ignored by lab-developed methods and therefore compromising real-world applications, (b) creating a publicly available synchronization-method benchmarking dataset, and (c) developing a low-level video feature based synchronization method with a computational complexity not higher than current state of the art audio-based methods.

Parallel recordings made at the same event with different devices, e.g. by visitors of a concert, contain semantically the same content but do not run at the same speed when played back in parallel on a computer, which makes their synchronization difficult. This effect, time drift, concerns all current consumer multimedia recording devices and results from their internal clocks not running at the same speed, leading to deviations from their nominal sampling rates. We present a mobile application capable of conducting instant measurements of this time drift, thus helping in determining devices that go well together, or correcting the speed differences in post processing.

Manual synchronization of audio and video recordings is a very annoying and time consuming task, especially if the tracks are very long and/or of large quantity. If the tracks aren't just short clips (of a few seconds or minutes) and recorded from heterogeneous sources, an additional problem comes into play - time drift - which arises if different recording devices aren't synchronized. This demo paper presents the experimental software Audio Align, which aims to simplify the manual synchronization process with the ultimate goal to automate it altogether. It gives a short introduction to the topic, discusses the approach, method, implementation and preliminary results and gives an outlook at possible improvements.