Most contemporary music tagging systems rely on large volumes of annotated data. As an alternative, we investigate the extent to which synthetically generated music excerpts can improve tagging systems when only small annotated collections are available. To this end, we release GTZAN-synth, a synthetic dataset that follows the taxonomy of the well-known GTZAN dataset while being ten times larger in data volume. We first observe that simply adding this synthetic dataset to the training split of GTZAN does not result into performance improvements. We then proceed to investigating domain adaptation, transfer learning and fine-tuning strategies for the task at hand and draw the conclusion that the last two options yield an increase in accuracy. Overall, the proposed approach can be considered as a first guide in a promising field for future research.

Abstract Predictive analytics over mobility data is of great importance since it can assist an analyst to predict events, such as collisions, encounters, traffic jams, etc. A typical example is anticipated location prediction, where the goal is to predict the future location of a moving object, given a look-ahead time. What is even more challenging is to be able to accurately predict collective behavioural patterns of movement, such as co-movement patterns as well as their course over time. In this paper, we address the problem of Online Prediction of Co-movement Patterns. Furthermore, in order to be able to calculate the accuracy of our solution, we propose a co-movement pattern similarity measure, which facilitates the comparison between the predicted clusters and the actual ones. Finally, we calculate the clusters’ evolution through time (survive, split, etc.) and compare the cluster evolution predicted by our framework with the actual one. Our experimental study uses two real-world mobility datasets from the maritime and urban domain, respectively, and demonstrates the effectiveness of the proposed framework.

Forecasting vessel locations is of major importance in the maritime domain, with applications in safety, logistics, etc. Nowadays, vessel tracking has become possible largely due to the increased GPS-based data availability. This paper introduces a novel Vessel Location Forecasting (VLF) framework, based on Long-Short Term Memory (LSTM) Neural Networks, aiming to perform effective location forecasting in time horizons up to 60 minutes, even for vessels not recorded in the past. The proposed VLF framework is specially designed for handling vessel data by addressing some major GPS-related obstacles including variable sampling rate, sparse trajectories, and noise contained in such data. Our framework also learns by incorporating a novel trajectory data augmentation method to improve its predictive power. We validate VLF framework using three real-word datasets of vessels moving in different sea areas, comparing with various methods, and examining several aspects. Results prove VLF framework’s generic nature, robustness regarding parameter changes, and superiority against state of the art in terms of prediction accuracy (higher than 30%) and computational effort.

In this work we use deep reinforcement learning to create an autonomous agent that can navigate in a two-dimensional space using only raw auditory sensory information from the environment, a problem that has received very little attention in the reinforcement learning literature. Our experiments show that the agent can successfully identify a particular target speaker among a set of N predefined speakers in a room and move itself towards that speaker, while avoiding collision with other speakers or going outside the room boundaries. The agent is shown to be robust to speaker pitch shifting and it can learn to navigate the environment, even when a limited number of training utterances are available for each speaker.

Recent advances in deep learning have enabled the extraction of high-level skeletal features from raw images and video sequences, paving the way for new possibilities in a variety of artificial intelligence tasks, including automatically synthesized human motion sequences. In this paper we present a system that combines 2D skeletal data and musical information to generate skeletal dancing sequences. The architecture is implemented solely with convolutional operations and trained by following a teacher-force supervised learning approach, while the synthesis of novel motion sequences follows an autoregressive process. Additionally, by employing an attention mechanism we fuse the latent representations of past music and motion information in order to condition the generation process. For assessing the system performance, we generated 900 sequences and evaluated the perceived realism, motion diversity and multimodality of the generated sequences based on various diversity metrics.

We propose a method for optimizing the random action selection of the ε-greedy policy to facilitate more efficient exploration of an environment by a reinforcement learning agent. Our directed ε-greedy policy selects actions with a biased probability where some actions are more likely to be selected than others. The probability distribution used for selecting random actions is the one that tends to lead to actions which increase the agent's uncertainty about its environment. The agent's uncertainty is measured by the error in self-supervised prediction of future environment states at the pixel level, given the previous states and the probabilities of next actions. By propagating the reverse gradient from the future state predictor model to a model generating probability distributions from random noise we create an end-to-end trainable model which learns to generate such action probability distributions for ε-greedy, so as to facilitate directed exploration of the environment. We evaluate our method in two environments: Minecraft and Super Mario Bros. The directed ε-greedy policy achieves an efficient curiosity-driven exploration without the use of any intrinsic reward function, outperforming vanilla ε-greedy exploration, softmax exploration and exploration using intrinsic rewards.

The i4sea research project provides effective and efficient big data integration, processing and analysis technologies to deliver both real-time and historical operational snapshots of fishing vessels activity in national sea areas. This paper presents the architecture of the i4sea big data platform for sea area monitoring and analysis of fishing vessels activity and demonstrates the operation of some use-case pilot scenarios.

In this work, we investigate the potential of deep feedforward neural networks for user position estimation for a multi-path directional channel model in presence as well as absence of the line of sight path. Furthermore, we take advantage of a triplet network architecture combined with a triplet loss function, which allows us to exploit intrinsic properties of channel state information. We show that despite the high-dimensional nature of CSI, the proposed network can be trained to learn from the low-dimensional space and with less amount of training samples compared to the long-established approaches in the literature for fingerprinting localization. Finally, in order to investigate the performance and emphasize the benefits of triplet loss, we compare it to another network based on classification.

In music, repetition is a fundamental concept to establish structure and create temporal relationships. Previous approaches to detecting repetition in music recordings have mainly focused on discovering repeated patterns of variable length and instrumentation at arbitrary locations. In this paper, we present a novel method for the discovery of repeated sung phrases in folk music recordings and, in particular, in oral music traditions, where written scores are usually unavailable. At a first stage, a segmentation algorithm partitions automatically generated note-level transcriptions of the singing melody into sections that correspond to the structural unit of a phrase. A clustering algorithm is then used to form clusters of phrases, where each cluster contains instances of the same melodic content. The clustering algorithm operates on the basis of a distance measure between melodic sequences and, to this end, various melodic distance measures are investigated. A detailed evaluation procedure is used to assess the performance of the algorithm on three different European music traditions and the influence of transcription and segmentation errors is investigated. The proposed system is shown to outperform the state-of-the-art in audio-based approaches to repeated phrase discovery for this task.

This paper presents pattern classification to a predefined set of classes as a missing data task. This is achieved by first augmenting the feature vector of each training pattern with the corresponding binary codeword representing its class. A Restricted Boltzmann Machine (RBM) or a Dictionary Learning (DL) algorithm is then trained on the augmented feature space. During the classification stage, the binary codeword of the unknown pattern is treated as missing data. In the case of the RBM, it is filled in by means of an alternating Gibbs sampling procedure. In the case of the DL method, the set of atoms in the dictionary is first learned from the training data, and the label of the unknown pattern is predicted based on those atoms that represent this pattern. Application of the method in an audio genre classification task verifies that the obtained results are highly competitive compared with state-of-the-art methods. Moreover, the DL approach lends itself readily for online implementations, in line with the current trend in big data applications.

In this paper we propose an FPGA-based hardware platform to accelerate an audio tracking method. Our tracking approach is inspired by the problem of molecular sequence alignment and adopts a well-known dynamic programming algorithm (Smith-Waterman algorithm) from the area of bioinformatics. However, the high computational complexity of such algorithms imposes a significant barrier to their adoption by audio tracking systems. To alleviate the time-consuming problem and achieve realistic response times, we propose the acceleration of computationally intensive parts of our tracking method using an FPGA-based platform. Our FPGA accelerator is actually based on the systolization of the Smith-Waterman algorithm proposed in previous approaches for the acceleration of bio-sequence scanning but the special requirements of the audio tracking method impose significant design challenges in the accelerator architecture. The accelerator has been implemented in a Xilinx Virtex-5 device and the experimental results show that it achieves significant speedup compared with the software implementation of the tracking method. The proposed approach has been tested in the context of detecting animal sounds in audio streams from movies, where a basic requirement is to reduce the noisiness of the detection results by means of exploiting the statistical nature of the scores that are generated by the dynamic programming algorithm.

This paper presents a method for detecting violent content in video sharing sites. The proposed approach operates on a fusion of three modalities: audio, moving image and text data, the latter being collected from the accompanying user comments. The problem is treated as a binary classification task (violent vs non-violent content) on a 9-dimensional feature space, where 7 out of 9 features are extracted from the audio stream. The proposed method has been evaluated on 210 YouTube videos and the overall accuracy has reached 82%.

In this paper we present a novel method for extracting affective information from movies, based on speech data. The method is based on a 2D representation of speech emotions (Emotion Wheel). The goal is twofold. First, to investigate whether the Emotion Wheel offers a good representation for emotions associated with speech signals. To this end, several humans have manually annotated speech data from movies using the Emotion Wheel and the level of disagreement has been computed as a measure of representation quality. The results indicate that the emotion wheel is a good representation of emotions in speech data. Second, a regression approach is adopted, in order to predict the location of an unknown speech segment in the Emotion Wheel. Each speech segment is represented by a vector of ten audio features. The results indicate that the resulting architecture can estimate emotion states of speech from movies, with sufficient accuracy.

This paper treats gunshot detection in audio streams from movies as a maximization task, where the solution is obtained by means of dynamic programming. The proposed method seeks the sequence of segments and respective class labels, i.e., gunshots vs. all other audio types, that maximize the product of posterior class label probabilities, given the segments' data. The required posterior probabilities are estimated by combining soft classification decisions from a set of Bayesian Network combiners. Tests that have been performed on a large set of audio streams indicate that the proposed method yields high performance in terms of both precision and recall of detected gunshot events.

In this paper, a novel approach to audio segmentation is presented. The problem of detecting audio segmentspsila limits is treated as a binary classification task. Frames are classified as ldquosegment limitsrdquo vs ldquononsegment limitsrdquo. For each audio frame a spectrogram is computed and eight feature values are extracted from respective frequency bands. Final decisions are taken based on a classifier combination scheme. The algorithm has very low complexity with almost real time performance. It achieves 86% accuracy rate on real audio streams extracted from movies. Moreover, it introduces a general framework to audio segmentation, which does not depend explicitly on the number of audio classes.

This paper presents a robust and computationally efficient method for tracking music in audio streams from movies. The audio stream is first mid-term processed with a fixed length moving window and four features are extracted per window. Each feature is fed as input to a simple classifier which produces a soft output for the binary problem of music vs. all other types of audio. The soft outputs are then combined to yield a measure of confidence quantifying whether the segment corresponds to music or not. At a final step, thresholding is applied to filter out segments where the confidence measure is low. The proposed approach has been tested with audio streams from various movies and its performance was measured both on a mid-term segment basis as well as on an event detection basis. Reported results demonstrate that the method exhibits high performance even when music is mixed with other types of audio in the stream.

This paper presents a multistage system for speech/music discrimination which is based on a three-step procedure. The first step is a computationally efficient scheme consisting of a region growing technique and operates on a 1-D feature sequence, which is extracted from the raw audio stream. This scheme is used as a preprocessing stage and yields segments with high music and speech precision at the expense of leaving certain parts of the audio recording unclassified. The unclassified parts of the audio stream are then fed as input to a more computationally demanding scheme. The latter treats speech/music discrimination of radio recordings as a probabilistic segmentation task, where the solution is obtained by means of dynamic programming. The proposed scheme seeks the sequence of segments and respective class labels (i.e., speech/music) that maximize the product of posterior class probabilities, given the data that form the segments. To this end, a Bayesian Network combiner is embedded as a posterior probability estimator. At a final stage, an algorithm that performs boundary correction is applied to remove possible errors at the boundaries of the segments (speech or music) that have been previously generated. The proposed system has been tested on radio recordings from various sources. The overall system accuracy is approximately 96%. Performance results are also reported on a musical genre basis and a comparison with existing methods is given.

This work addresses the problem of locating rhythmic patterns in music recordings. During the feature extraction stage, a short-term processing technique is applied, in order to detect significant changes in the spectral and energy evolution of the music signal. The detected changes are in turn treated as onsets of events and a sequence of inter-onset intervals is extracted. The resulting sequence is long-term segmented and is fed as input to a hidden Markov model (HMM) which models a predefined rhythmic pattern. An enhanced Viterbi algorithm is proposed, that extracts a best-state sequence, which determines the pattern location boundaries. Our method was tested on a set of music recordings of music meter 2/4, 3/4, 7/8 and 9/8 and steady tempo. The proposed method exhibits excellent precision (100%) over pattern locations and a recall ranging from ~ 34% up to ~ 74% depending on the music genre.

In this work, we present a multi-class classification algorithm for audio segments recorded from movies, focusing on the detection of violent content, for protecting sensitive social groups (e.g. children). Towards this end, we have used twelve audio features stemming from the nature of the signals under study. In order to classify the audio segments into six classes (three of them violent), Bayesian networks have been used in combination with the one versus all classification architecture. The overall system has been trained and tested on a large data set (5000 audio segments), recorded from more than 30 movies of several genres. Experiments showed, that the proposed method can be used as an accurate multi-class classification scheme, but also, as a binary classifier for the problem of violent -non violent audio content.

This paper presents a speech/music discriminator for radio recordings. The segmentation stage is based on the detection of changes in the energy distribution of the audio signal. For the classification stage, Bayesian networks have been adopted in order to combine the results of nine k-nearest neighbor classifiers trained on individual features. To this end, a comparison of the performance of three popular Bayesian network architectures is presented. Furthermore, in order to reduce the number of features used for classification, a new feature selection scheme is introduced, that is also based on the properties of Bayesian networks. The proposed system has been tested on real Internet broadcasts of BBC radio stations

This paper presents a new extension to the variable duration hidden Markov model (HMM), capable of classifying musical pattens that have been extracted from raw audio data into a set of predefined classes. Each musical pattern is converted into a sequence of music intervals by means of a fundamental frequency tracking procedure. This sequence is subsequently presented as input to a set of variable-duration HMMs. Each one of these models has been trained to recognize patterns of a corresponding predefined class. Classification is determined based on the highest recognition probability. The new type of variable-duration hidden Markov modeling proposed in this paper results in enhanced performance because 1) it deals effectively with errors that commonly originate during the feature extraction stage, and 2) it accounts for variations due to the individual expressive performance of different instrument players. To demonstrate its effectiveness, the novel classification scheme has been employed in the context of Greek traditional music, to monophonic musical patterns of a popular instrument, the Greek traditional clarinet. Although the method is also appropriate for western-style music, Greek traditional music poses extra difficulties and makes music pattern recognition a harder task. The classification results demonstrate that the new approach outperforms previous work based on conventional HMMs

Automatic recognition of musical patterns plays a crucial part in musicological and ethnomusicological research and can become an indispensable tool for the search and comparison of music extracts within a large multimedia database. This paper presents an efficient method for recognizing isolated musical patterns in a monophonic environment, using a novel extension of dynamic time warping, which we call context dependent dynamic time warping. Each pattern, to be recognized, is converted into a sequence of frequency jumps by means of a fundamental frequency tracking algorithm, followed by a quantizer. The resulting sequence of frequency jumps is presented to the input of the recognizer. The main characteristic of context dependent dynamic time warping is that it exploits the correlation exhibited among adjacent frequency jumps of the feature sequence. The methodology has been tested in the context of Greek traditional music, which exhibits certain characteristics that make the classification task harder, when compared with western musical tradition. A recognition rate higher than 95% was achieved.