In this paper, we describe the euTV system, which provides a flexible approach to collect, manage, annotate and publish collections of images, videos and textual documents. The system is based on a Service Oriented Architecture that allows to combine and orchestrate a large set of web services for automatic and manual annotation, retrieval, browsing, ingestion and authoring of multimedia sources. euTV tools have been used to create several publicly available vertical applications, addressing different use cases. Positive results of user evaluations have shown that the system can be effectively used to create different types of applications.

While global characteristics of the speaker’s source and spectral features have been successfully employed in pathological voice detection, the underlying text has largely been ignored. In this work, we focus on experiments that exploit the text stimulus that is read by the subject. Features derived from text include the mean cepstral distortion of the subject from an average intelligible speaker, and prosodic features include the speaking rate, statistics of phoneme durations, etc. The phonetic labeling information is also exploited to ignore all the unvoiced regions of the speech samples to improve the discriminability between intelligible and pathological voices. We also designed features that capture the speaker’s overall closeness to intelligible instances of the same text stimulus from other speakers. Our experiments show that the proposed text-derived features improve the detection of pathological voices by 20%. Index Terms: Pathological voices, example based detection, text-driven features, fusion of classification methods.

There are large amounts of information as video and audio not searchable. In a time where Business Intelligence is fundamental for all areas doing this kind of analysis only on text sources is a limiting factor. The use of large vocabulary speech recognition systems with increasing performance is giving rise to different applications. Despite the diversity, these applications share the extensive use of the contents of the transcription. In this paper we describe the results of a development project between a startup company and a research lab to build a full automatic system for monitoring TV and Radio channels. This system is composed by three main blocks: a recording block (records the selected channels in high and web streaming quality and broadcast to the next block), a processing block (generate metadata information) and a storage and accessing block (make available the metadata and videos). There is an optional block, the delivering, that could be customized according to the use and client needs. The processing block receives the video from the recording block time filtered by a scheduling interface. For that video the goal is to generate a full annotation with metadata describing the content and semantic information. This metadata make possible a filtering process for selective dissemination of information.

In this paper, a novel approach to video temporal decomposition into semantic units, termed scenes, is presented. In contrast to previous temporal segmentation approaches that employ mostly low-level visual or audiovisual features, we introduce a technique that jointly exploits low-level and high-level features automatically extracted from the visual and the auditory channel. This technique is built upon the well-known method of the scene transition graph (STG), first by introducing a new STG approximation that features reduced computational cost, and then by extending the unimodal STG-based temporal segmentation technique to a method for multimodal scene segmentation. The latter exploits, among others, the results of a large number of TRECVID-type trained visual concept detectors and audio event detectors, and is based on a probabilistic merging process that combines multiple individual STGs while at the same time diminishing the need for selecting and fine-tuning several STG construction parameters. The proposed approach is evaluated on three test datasets, comprising TRECVID documentary films, movies, and news-related videos, respectively. The experimental results demonstrate the improved performance of the proposed approach in comparison to other unimodal and multimodal techniques of the relevant literature and highlight the contribution of high-level audiovisual features toward improved video segmentation to scenes.

This paper describes our recent work on extending the punctuation module of automatic subtitles for Portuguese Broadcast News. The main improvement was achieved by the use of prosodic information. This enabled the extension of the previous module which covered only full stops and commas, to cover question marks as well. The approach uses lexical, acoustic and prosodic information. Our results show that the latter is relevant for all types of punctuation. An analysis of the results also shows what type of interrogative is better dealt with by our method, taking into account the specificities of Portuguese. This may lead to different results for different types of corpora, depending on the types of interrogatives that are more frequent.

This paper presents a description of the INESC-ID Spoken Language Systems Laboratory (L2F) Age and Gender classification system submitted to the INTERSPEECH 2010 Paralinguistic Challenge. The L2F Age classification system and the Gender classification system are composed respectively by the fusion of four and six individual sub-systems trained with short and long term acoustic and prosodic features, different classification strategies (GMM-UBM, MLP and SVM) and using four different speech corpora. The best results obtained by the calibration and linear logistic regression fusion back-end show an absolute improvement of 4.1% on the unweighted accuracy value for the Age and 5.8% for the Gender when compared to the competition baseline systems in the development set.

This paper describes the integration of multimedia documents in the Portuguese version of REAP, a tutoring system for vocabulary learning. The documents result from the pipeline processing of Broadcast News videos that automatically segments the audio files, transcribes them, adds punctuation and capitalization, and breaks them into stories classified by topics. The integration of these materials in REAP was done in a way that tries to decrease the impact of potential errors of the automatic chain in the learning process.

This paper summarizes the contributions to semantic video search that can be derived from the audio signal. Because of space restrictions, the emphasis will be on non-linguistic cues. The paper thus covers what is generally known as audio segmentation, as well as audio event detection. Using machine learning approaches, we have built detectors for over 50 semantic audio concepts.

In this work the problem of automatic decomposition of video into elementary semantic units, known in the literature as scenes, is addressed. Two multi-modal automatic scene segmentation techniques are proposed, both building upon the Scene Transition Graph (STG). In the first of the proposed approaches, speaker diarization results are used for introducing a post-processing step to the STG construction algorithm, with the objective of discarding scene boundaries erroneously identified according to visual-only dissimilarity. In the second approach, speaker diarization and additional audio analysis results are employed and a separate audio-based STG is constructed, in parallel to the original STG based on visual information. The two STGs are subsequently combined. Preliminary results from the application of the proposed techniques to broadcast videos reveal their improved performance over previous approaches.

The subtitling of broadcast news programs are starting to become a very interesting application due to the technological advances in automatic speech recognition and associated technologies. However, to build this kind of systems, several advances are necessary both in terms of the technological components and on main blocks integration. In this paper, we are presenting the overall architecture of a subtitling system running daily at RTP (the Portuguese public broadcast company). The goal is to integrate our components in a system for the subtitling of RTP programs. The global system includes the subtitling of recorded and direct programs.

Broadcast news play an important role in our lives provid-ing access to news, information and entertainment. The exis-tence of subtitles is an important medium for inclusion of peo-ple with special needs and also an advantage on noisy and pop-ulated environments. In this work we will describe and evalu-ate a system for subtitling live broadcast news for RTP (Rádio Televisão de Portugal) the Portuguese public broadcast com-pany. Developing a fully automatic subtitling system is a huge breakthrough which results from the convergence of different research models and software developments to create a work-ing system. Our online system has 12 % word error rate for the displayed subtitles working under real time with an average la-tency of just 6.5 seconds.

This paper describes our work on the development of a low latency stream-based audio pre-processing system for broadcast news using model-based techniques. It performs speech/nonspeech classification, speaker segmentation, speaker clustering, gender and background conditions classification. As a way to increase the modelling accuracy our algorithms make extensive use of Artificial Neural Networks (ANN) thus avoiding the rough assumptions normally made about the audio signal distribution. Experiments were conducted on the COST278 multilingual TV broadcast news database and compared with current state of the art algorithms using standard evaluation tools. Additionally we investigated the impact of automatic audio preprocessing system within the recognition using a large broadcast news test database for the European Portuguese. These testsshow a smalldegradation in recognition performance when compared with hand labelled audio segmentation. Our system is part of a prototype close-captioning system that is daily processing the main news show of two Portuguese Broadcasters.

This paper describes a large scale experiment in which eight research institutions have tested their audio partitioning and labeling algorithms on the same data, a multi-lingual database of news broadcasts, using the same evaluation tools and protocols. The experiments have provide more insight in the cross-lingual robustness of the methods and they have demonstrated that by further collaborating in thedomains of speaker change detection and speaker clustering it should be possible to achieve further technological progress in the near future.

The paper describes our work on the development of an audio segmentation, classification and clustering system applied to a broadcast news task for the European Portuguese language. We developed a new algorithm for audio segmentation that is both accurate and uses fewer computational resources than other approaches. Our speaker clustering module uses a modified BIC (Bayesian information criterion) algorithm which performs substantially better than the standard symmetric Kullback-Liebler, KL2, and is much faster than the full BIC. Finally, we developed a scheme for tagging certain speaker clusters (anchors) using trained cluster models. A series of tests were conducted showing the advantage of the new algorithms. This system is part of a prototype system that is daily processing the main news show of the national Portuguese broadcaster.

This paper describes the evaluation of the system for selective dissemination of Broadcast News that we developed in the context of the European project ALERT.Each component of the main processing block of our system was evaluated separately, using the ALERT corpus.Likewise, the user interface was also evaluated separately.Besides this modular evaluation which will be briefly mentioned here, as a reference, the system can also be evaluated as a whole, in a field trial from the point of view of a potential user.This is the main topic of this paper.The analysis of the main sources of problems hinted at a large number of issues that must be dealt with in order to improve the performance.In spite of these pending problems, we believe that having a fully operational system is a must for being able to address user needs in the future in this type of service. 10.

This paper summarizes the work that has been done concerning the Portuguese language in the scope of the ALERT project during its first year.The media watch system that is the goal of this project comprises many different modules, some of them common among the three languages of the project.This paper concentrates on the definition and collection of the necessary linguistic resources for Portuguese, and the development of the speech recognition, topic and jingle detection modules.The first version of the ALERT demo for European Portuguese is also described.

Recent works have showed that the use of syllables as the basic unit in a speech recognition system could be very useful.These works introduced methods exploiting syllable information as a mean to add robustness in "traditional" systems that use phonemes/phones as the basic unit.Being the Portuguese a highly syllabic language we expected that information from syllables would introduce potential benefits in speech recognition tasks.Following these ideas we started by developing different methods of automatic syllable segmentation.Next we applied the best segmentation method to our large vocabulary continuous speech corpus (BD-PUBLICO) achieving an accuracy of 72%.We developed a process to use the segmentation information in the acoustic models of our baseline speech recognisers for the Portuguese language.The results obtained by the modified recognition systems on 5k and 27k vocabulary tasks showed that the use of basic syllable segmentation information helps the systems to improve their overall performance by roughly 10%.

Recent developments have suggested that the use of syllables as the basic unit in a speech recognition system could be very usefull. Since syllable boundaries are more precise and well defined than phoneme ones there is a large scope for their application on the continuous speech recognition process. In this work we developed different methods of syllable segmentation in continuous speech. These methods are based on perceptually oriented feature extraction techniques. These features were post-processed through simple threshold mechanisms or by an artificial neural network based classifier in order to estimate the syllable boundaries. These systems were trained and evaluated using a Portuguese database with continuous speech. The results show that large context input windows (260ms) are the most appropriate, achieving results of 93% detection of onsets with insertion rates of only 15%. Keywords: Syllables, onset detection, artificial neural networks, continuous speech 1. INTRODUCTION ...

The last years show a great development of large vocabulary, speaker-independent continuous speech recognition systems and some research in multilingual aspects. To allow that development to also be extended to the European Portuguese language we decided to develop and collect a large database of continuous speech based on a large amount of text. In the development of this new Portuguese database our aim was to create a corpus equivalent in size to WSJ0. We selected the database texts from the P ' UBLICO newspaper, which is characterized by a broad coverage of matters and different writing styles. The recording population was selected from a large engineering school, assuring a large variability of speakers. The recordings are being done as we write this paper and we expect to release the database in CD format in September 1997. 1. INTRODUCTION The development of a large vocabulary, speakerindependent continuous speech recognition system needs an appropriate database of both text a...