State-of-the-art anomalous sound detection systems often utilize angular margin losses to learn suitable representations of acoustic data using an auxiliary task, which usually is a supervised or self-supervised classification task. The underlying idea is that, in order to solve this auxiliary task, specific information about normal data needs to be captured in the learned representations and that this information is also sufficient to differentiate between normal and anomalous samples. Especially in noisy conditions, discriminative models based on angular margin losses tend to significantly outperform systems based on generative or one-class models. The goal of this work is to investigate why using angular margin losses with auxiliary tasks works well for detecting anomalous sounds. To this end, it is shown, both theoretically and experimentally, that minimizing angular margin losses also minimizes compactness loss while inherently preventing learning trivial solutions. Furthermore, multiple experiments are conducted to show that using a related classification task as an auxiliary task teaches the model to learn representations suitable for detecting anomalous sounds in noisy conditions. Among these experiments are performance evaluations, visualizing the embedding space with t-SNE and visualizing the input representations with respect to the anomaly score using randomized input sampling for explanation.

This paper proposes a novel approach for robustly detecting multiply repeating audio events in monitoring recordings. We consider the practically important case that the sequence of inter onset intervals between subsequent events is not constant but differs by some jitter. In such cases classical approaches based on autocorrelation (ACF) are of limited use. To overcome this problem we propose to use ACF together with a variant of dynamic time warping. Combining both techniques in an iterative algorithm, we obtain a method for significantly improved detection of jittered multiply repeating events. In this paper we describe the new iterated time-warped ACF algorithm and evaluate its performance on the bioacoustic application of detecting repeating bird calls in monitoring recordings.

S.2631-2635

We propose a method for robustly detecting and extracting repeated signal components within a source signal. The method is based on the recently introduced shift autocorrelation (shift-ACF) which outperforms classical ACF in signal detection if a signal component is repeated more than once. In this paper, we extend shift-ACF to analyze the spectral structure of repeating signal components by using a subband decomposition. Subsequently, an algorithm for repeated event detection and extraction is proposed. An evaluation shows that the proposed subband shift-ACF outperforms detection based on classical cepstrum. We discuss several possible applications in the domain of sensor signal analysis, and particularly in audio monitoring.

We present a novel method for robustly extracting the fundamental frequency (F0) of noisy speech signals. Our method uses the recently proposed shift autocorrelation to locally emphasize harmonically distributed energy in the spectrogram. Subsequently, a trajectory extraction algorithm based on an optimization technique is used to determine local F0 contours of voiced segments. Our evaluation shows that the proposed method is capable of estimating F0 even in the presence of severe noises such as in radio communications.

The extraction of local tempo and beat information from audio recordings constitutes a challenging task, particularly for music that reveals significant tempo variations. Furthermore, the existence of various pulse levels such as measure, tactus, and tatum often makes the determination of absolute tempo problematic. In this paper, we present a robust mid-level representation that encodes local tempo information. Similar to the well-known concept of cyclic chroma features, where pitches differing by octaves are identified, we introduce the concept of cyclic tempograms, where tempi differing by a power of two are identified. Furthermore, we describe how to derive cyclic tempograms from music signals using two different methods for periodicity analysis and finally sketch some applications to tempo-based audio segmentation.

We propose a new type of audio feature (HFCC-ENS) as well as an unsupervised method for detecting short sequences of spoken words (key-phrases) within long speech recordings. Our technical contributions are threefold: Firstly, we propose to use bandwidth-adapted filterbanks instead of classical MFCC-style filters in the feature extraction step. Secondly, the time resolution of the resulting features is adapted to account for the temporal characteristics of the spoken phrases. Thirdly, the key-phrase detection step is performed by matching sequences of the resulting HFCC-ENS features with features extracted from a target speech recording. We evaluate the proposed method using the German Kiel Corpus and furthermore investigate speech-related properties of the proposed feature.

Recent digitization efforts have led to large music collections, which contain music documents of various modes comprising textual, visual and acoustic data. In this paper, we present a multimodal music player for presenting and browsing digitized music collections consisting of heterogeneous document types. In particular, we concentrate on music documents of two widely used types for representing a musical work, namely visual music representation (scanned images of sheet music) and associated interpretations (audio recordings). We introduce novel user interfaces for multimodal (audio-visual) music presentation as well as intuitive navigation and browsing. Our system offers high quality audio playback with time-synchronous display of the digitized sheet music associated to a musical work. Furthermore, our system enables a user to seamlessly crossfade between various interpretations belonging to the currently selected musical work.

Given a large audio database of music recordings, the goal of classical audio identification is to identify a particular audio recording by means of a short audio fragment. Even though recent identification algorithms show a significant degree of robustness towards noise, MP3 compression artifacts, and uniform temporal distortions, the notion of similarity is rather close to the identity. In this paper, we address a higher level retrieval problem, which we refer to as audio matching: given a short query audio clip, the goal is to automatically retrieve all excerpts from all recordings within the database that musically correspond to the query. In our matching scenario, opposed to classical audio identification, we allow semantically motivated variations as they typically occur in different interpretations of a piece of music. To this end, this paper presents an efficient and robust audio matching procedure that works even in the presence of significant variations, such as nonlinear temporal, dynamical, and spectral deviations, where existing algorithms for audio identification would fail. Furthermore, the combination of various deformation- and fault-tolerance mechanisms allows us to employ standard indexing techniques to obtain an efficient, index-based matching procedure, thus providing an important step towards semantically searching large-scale real-world music collections.

This tutorial describes techniques essential for searching the large multimedia databases that are now common on the Internet. There are up to 10 million songs in commercial music catalogues and over 300 million images stored in online photo services such as Flickr. How can we find the music, videos or images we want? How can we organize such large collections: find duplicates, create links between similar documents, extract and annotate semantic structures from complex audiovisual documents? Conventional methods for handling large data sets, such as hashing, get us part of the way, but those methods may not straightforwardly be used for similarity-based matching and retrieval in audiovisual document collections. On the other hand, several elaborate methods from multimedia retrieval are available for semantic document analysis. Unfortunately, those methods generally do not scale for large data sets. Instead, new classes of algorithms combining the best of the two worlds of large data methods and semantic analysis are needed to handle large multimedia databases. Innovative methods such as locality sensitive hashing, which are based on randomized probes, are the new workhorses. This tutorial covers methods for multimedia retrieval on large document collections. Starting with audio retrieval, we describe both the theory (i.e., randomized algorithms for hashing) and the implementation details (how do you store hash values for millions of songs?). A special focus is on how to combine large data methods with semantically meaningful descriptors in order to facilitate efficient similarity-based retrieval. Besides audio, the tutorial also covers image, 3d motion and video retrieval.

Similarity matrices have become an important tool in music audio analysis. However, the quadratic time and space complexity as well as the intricacy of extracting the desired structural information from these matrices are often prohibitive with regard to real-world applications. In this paper, we describe an approach for enhancing the structural properties of similarity matrices based on two concepts: first, we introduce a new class of robust and scalable audio features which absorb local temporal variations. As a second contribution, we then incorporate contextual information into the local similarity measure. The resulting enhancement leads to significant reduction in matrix size and also eases the structure extraction step. As an example, we sketch the application of our techniques to the problems of audio summarization and audio synchronization, obtaining effective and computationally feasible algorithms

In this paper, we propose a unified approach to fast index-based music recognition. As an important area within the field of music information retrieval (MIR), the goal of music recognition is, given a database of musical pieces and a query document, to locate all occurrences of that document within the database, up to certain possible errors. In particular, the identification of the query with regard to the database becomes possible. The approach presented in this paper is based on a general algorithmic framework for searching complex patterns of objects in large databases. We describe how this approach may be applied to two important music recognition tasks: The polyphonic (musical score-based) search in polyphonic score data and the identification of pulse-code modulation audio material from a given acoustic waveform. We give an overview on the various aspects of our technology including fault-tolerant search methods. Several areas of application are suggested. We describe several prototypic systems we have developed for those applications including the notify! and the audentify! systems for score- and waveform-based music recognition, respectively.