Sound Event Detection (SED) requires models that can accurately localize and classify overlapping audio events within complex acoustic environments. Conformer-based architectures have demonstrated promising performance by leveraging self-attention to capture long-range dependencies. However, this global attention can be accumulated across layers, which can blur local temporal boundaries and reduce detection accuracy, especially for short or closely spaced events. While increasing the input sequence length can help recover temporal detail, the quadratic complexity of Conformers’ self-attention significantly increases computational costs. To address this, we propose integrating the Efficient Conformer architecture, which introduces subsampling along the input sequence length, effectively reducing the temporal dimension within blocks. This design enables processing longer input sequences at finer temporal resolution, enhancing localization accuracy without extending output length. Using the DCASE Challenge 2023 Task 4 benchmark, system performance is evaluated via the threshold-independent Polyphonic Sound Detection Score (PSDS), measuring both localization precision (PSDS1) and class robustness (PSDS2). Experiments on the DESED validation dataset demonstrate that the Efficient Conformer not only improves temporal resolution and long-range dependency modeling, but also outperforms standard Conformer and Convolutional Recurrent Neural Network (CRNN) baselines in PSDS2. Additionally, we explore lightweight attention mechanisms employing squeeze-and-excitation blocks to emulate frequency-axis translation invariance of Frequency Dynamic Convolutions (FDY). Our approach achieves performance comparable to heavier models like FDY+Conformer, while reducing computational cost by over 69%, showing promising results for Conformer-based systems in terms of precision and model efficiency.

Over the last few years, most of the tasks employing Deep Learning techniques for audio processing have achieved state-of-the-art results employing Conformer-based systems. However, when it comes to sound event detection (SED), it was scarcely used after it won the DCASE Challenge 2020 Task 4. In previous research, we found that Conformer-based systems achieved a higher performance in terms of sound events classification compared to other architectures frequently employed, such as Convolutional Recurrent Neural Networks (CRNNs). Given that the second scenario proposed for the Polyphonic Sound Detection Score (PSDS2) is focused on avoiding confusion between classes, in this paper we propose to optimize a Conformer-based system to maximize the performance on this scenario. For this purpose, we performed a hyperparameter tuning and incorporated recently proposed Frequency Dynamic Convolutions (FDY) to enhance its classification properties. Additionally, we employed our previously proposed multi-resolution approach not only to enhance the performance but also to gain a deeper understanding of the Conformer architecture for SED, analyzing its advantages and disadvantages, and finding possible solutions to them. Additionally, we explored the integration of embeddings from the pre-trained model BEATs, an iterative framework to learn Bidirectional Encoder representation from Audio Transformers. By concatenating these embeddings into the input of the Conformer blocks, results were further improved, achieving a PSDS2 value of 0.813 and considerably outperforming SED systems based on CRNNs.

In this work, we propose an end-to-end approach to the language identification (LID) problem based on Convolutional Deep Neural Networks (CDNNs). The use of CDNNs is mainly motivated by the ability they have shown when modeling speech signals, and their relatively low-cost with respect to other deep architectures in terms of number of free parameters. We evaluate different configurations in a subset of 8 languages within the NIST Language Recognition Evaluation 2009 Voice of America (VOA) dataset, for the task of short test durations (segments up to 3 seconds of speech). The proposed CDNN-based systems achieve comparable performances to our baseline i-vector system, while reducing drastically the number of parameters to tune (at least 100 times fewer parameters). Then, we combine these CDNN-based systems and the i-vector baseline with a simple fusion at score level. This combination outperforms our best standalone system (up to 11% of relative improvement in terms of EER).

Proceedings of Interspeech 2011, Florence (Italy)

A novel way of managing the compromise between noise reduction and speech distortion in Wiener filters is presented. It is based on adjusting the amount of noise reduced, and therefore the speech distortion introduced, on a phone-by-phone basis. We show empirically that optimal Wiener filters produce different amounts of speech distortion for different phones. Therefore we propose a phone-conditioned suboptimal Wiener filter that uses different amounts of noise reduction for each phone, based on a previous estimation of the amount of distortion introduced. Speech recognition results have shown that phone conditioning suboptimal Wiener filtering can provide almost a 5% additional relative improvement in word accuracy over comparable optimal Wiener filtering.

Discriminative confidence estimation along with confidence normalisation have been shown to construct robust decision maker modules in spoken term detection (STD) systems. Discriminative confidence estimation, making use of termdependent features, has been shown to improve the widely used lattice-based confidence estimation in STD. In this work, we augment the set of these term-dependent features and show a significant improvement in the STD performance both in terms of ATWV and DET curves in experiments conducted on a Spanish geographical corpus. This work also proposes a multiple linear regression analysis to carry out the feature selection. Next, the most informative features derived from it are used within the discriminative confidence on the STD system.

Band-limited speech (speech for which parts of the spectrum are completely lost) is a major cause for accuracy degradation of automatic speech recognition (ASR) systems particularly when acoustic models have been trained with data with a different spectral range. In this paper, we present an extensive study of the problem of ASR of band-limited speech with full-bandwidth acoustic models. Our focus is mainly on band-limited feature compensation, covering even the case of time-varying band-limiting distortions, but we also compare this approach to more common model-side techniques (adaptation and retraining) and explore the combination of feature-based and model-side approaches. The feature compensation algorithms proposed are organized in a unified framework supported by a novel mathematical model of the impact of such distortions on Mel-frequency cepstral coefficient (MFCC) features. A crucial and novel contribution is the analysis made of the relative correlation of different elements in the MFCC feature vector for the cases of full-bandwidth and limited-bandwidth speech, which justifies an important modification in the feature compensation scheme. Furthermore, an intensive experimental analysis is provided. Experiments are conducted on real telephone channels, as well as artificial low-pass and bandpass filters applied over TIMIT data, and results are given for different experimental constraints and variations of the feature compensation method. Results for other well-known robustness approaches, such as cepstral mean normalization (CMN), model retraining, and model adaptation are also given for comparison. ASR performance with our approach is similar or even better than model adaptation, and we argue that in particular cases such as rapidly varying distortions, or limited computational or memory resources, feature compensation is more convenient. Furthermore, we show that feature-side and model-side approaches may be combined, outperforming any of those approaches alone.

Proceedings of Interspeech 2008, Brisbane (Australia)

Proceedings of Interspeech 2008, Brisbane (Australia)

Proceedings of Interspeech 2007, Antwerp (Belgium)

Mismatch in speech bandwidth between training and real operation greatly affects automatic speech recognition. This letter extends previous work on feature compensation of band-limited speech to establish a framework for blind compensation of speech data of unknown bandwidth, valid even when the distortion (band-limiting channel) changes rapidly and continuously in time. The available bandwidth of the input speech signal is automatically detected, and the band-limited feature vectors are compensated prior to being compared against full-bandwidth acoustic models. For a fixed bandwidth limitation, phoneme recognition performance using the proposed method is similar to that achieved by models adapted to match the distortion. However, compared to model adaptation approaches, this new approach can seamlessly be extended to rapidly time-varying conditions while maintaining low computational and memory costs

Forensic DNA profiling is acknowledged as the model for a scientifically defensible approach in forensic identification science, as it meets the most stringent court admissibility requirements demanding transparency in scientific evaluation of evidence and testability of systems and protocols. In this paper, we propose a unified approach to forensic speaker recognition (FSR) oriented to fulfil these admissibility requirements within a framework which is transparent, testable, and understandable, both for scientists and fact-finders. We show how the evaluation of DNA evidence, which is based on a probabilistic similarity-typicality metric in the form of likelihood ratios (LR), can also be generalized to continuous LR estimation, thus providing a common framework for phonetic-linguistic methods and automatic systems. We highlight the importance of calibration, and we exemplify with LRs from diphthongal F-pattern, and LRs in NIST-SRE06 tasks. The application of the proposed approach in daily casework remains a sensitive issue, and special caution is enjoined. Our objective is to show how traditional and automatic FSR methodologies can be transparent and testable, but simultaneously remain conscious of the present limitations. We conclude with a discussion on the combined use of traditional and automatic approaches and current challenges for the admissibility of speech evidence.

This paper deals with the problem of speech recognition on band-limited speech. In our previous work we showed how a simple polynomial correction framework could be used for compensation of band-limited speech to minimize the mismatch using full-bandwidth acoustic models. This paper extends this approach to time-varying multiple-channel environments. The compensation framework is extended to perform automatic channel classification prior to compensation, thus allowing for unsupervised multi-channel compensation without the need for an explicit channel classifier. Performance is demonstrated on a wide range of channel bandwidth conditions. This extension makes our compensation approach potentially applicable in a much wider range of scenarios with only very limited performance degradation compared to the supervised approach

As a result of the evolution in the field of biometrics, a new breed of techniques and methods for user identity recognition and verification has appeared based on the recognition and verification of several biometric features considered unique to each individual. Signature and voice characteristics, facial features, and iris and fingerprint patterns have all been used to identify a person or just to verify that the person is who he/she claims to be. Although still relatively new, these new technologies have already reached a level of development that allows its commercialization. However, there is a lack of studies devoted to the evaluation of these technologies from a user-centered perspective. This paper is intended to promote user-centered design and evaluation of biometric technologies. Towards this end, we have developed a platform to perform empirical evaluations of commercial biometric identity verification systems, including fingerprint, voice and signature verification. In this article, we present an initial empirical study in which we evaluate, compare and try to get insights into the factors that are crucial for the usability of these systems.

This paper presents an algorithm for formant tracking using HMMs and analyzes the influence of HMM initialization, training and context-dependency on the accuracy of the formant tracks obtained with the HMMs. Formant trackers usually include two different phases: one in which the speech is analyzed and formant candidates are obtained, and another in which, by imposing different constraints, the most likely formants are chosen. While the first stage usually relies on standard spectrum estimation techniques, the second stage has evolved notably in the recent years. Traditionally the second phase tries to impose continuity constraints on the formant selection process. Lately there has been ongoing research to include phonemic knowledge in the second stage to make formant tracking more reliable. In order to incorporate phonemic knowledge newer approaches make use of the orthographic transcription of the speech utterance. From the orthographic transcription, the phonemic transcription is obtained, and from this and the speech itself a phonemic segmentation can be obtained. This phonemic segmentation, along with the phonemic transcription and some knowledge of the nominal formant positions for the different phonemes provides extra information that can be used to obtain more accurate formant tracks. This paper presents a complete HMM-based data-driven algorithm for formant tracking suitable to combine different levels of acoustic and phonemic information. A detailed analysis on the performance of this algorithm is discussed for: different initialization strategies using different levels of knowledge, different degrees of training, and context-independent and dependent HMMs. Experimental speaker-dependent results show that the efficient use of phonemic information in HMM training and context-dependent modeling significantly reduces the formant tracking error rate especially for formants$ F_2$and$ F_3$.

The paper addresses the problem of bandwidth expansion for the purpose of robust speech recognition. We show that an HMM-based ASR engine trained with full spectrum range data (0-8 kHz) can successfully perform speech recognition tasks over band-filtered test data compensated by means of a series of simple MFCC parameter corrector functions. The problem is important when ASR is employed for audio streams of unknown frequency bandwidth, common in spoken document retrieval. Evaluation is based on recognition rates. Accuracy varies depending on the width and spectral regions eliminated, but the system shows great advantages over the use of uncompensated filtered test data. The theoretical maximum recognition rates using corrector functions over filtered test data are very close to the base rate (unfiltered data) even when the greatest part of the spectrum of the original data is suppressed. These rates are even better than those obtained in the matched train/test HMMs with filtered data.

Proceedings of Interspeech-Eurospeech 2005, Lisbon (Portugal)

Proceedings of Interspeech-Eurospeech 2005, Lisbon (Portugal)

This paper presents the results and conclusions of a thorough study on automatic phonetic segmentation. It starts with a review of the state of the art in this field. Then, it analyzes the most frequently used approach-based on a modified Hidden Markov Model (HMM) phonetic recognizer. For this approach, a statistical correction procedure is proposed to compensate for the systematic errors produced by context-dependent HMMs, and the use of speaker adaptation techniques is considered to increase segmentation precision. Finally, this paper explores the possibility of locally refining the boundaries obtained with the former techniques. A general framework is proposed for the local refinement of boundaries, and the performance of several pattern classification approaches (fuzzy logic, neural networks and Gaussian mixture models) is compared within this framework. The resulting phonetic segmentation scheme was able to increase the performance of a baseline HMM segmentation tool from 27.12%, 79.27%, and 97.75% of automatic boundary marks with errors smaller than 5, 20, and 50 ms, respectively, to 65.86%, 96.01%, and 99.31% in speaker-dependent mode, which is a reasonably good approximation to manual segmentation.

This work is an extension of a previous work in which an automatic speech segmentation and labeling system was proposed based on a hidden Markov model (HMM) speech recognizer followed by a fuzzy-logic boundary correction system. In this paper we explore the possibility of substituting that difficult to design fuzzy-logic system by a neural network (NN) based system that can be automatically trained. First, the whole fuzzy-logic boundary correction system, which used different rule sets for each kind of phonetic transition, has been substituted by a single NN. Results show that this single NN outperforms the complete fuzzy-logic system. Then, the possibility of using different NNs specialized in each kind of phonetic transition has been explored. Results are again clearly better than the results obtained with the fuzzy-logic system, but not clearly better than the results obtained with just one NN.

In accordance with the new emerging voice response systems that use flexible vocabulary recognizers (FVRs), the prediction of word confusabilities have received increasing interest during the last few years. We present a new method for transcription confusabilities estimation based on a new statistical modelling criterion. We propose the use of the new transcription confusability measure in two different word error rate (WER) reduction procedures for FVRs: an automatic vocabulary selection procedure suitable for those applications where the set of vocabulary words is not totally defined by the application, and an automatic procedure for generation of alternative transcriptions. Experimental results using a telephonic database show a 20% WER relative reduction using the automatic alternative transcription generation procedure for a 37 word vocabulary, and over a 50% (20%) WER relative reduction using our unrestricted (restricted by groups of synonyms) vocabulary selection procedure instead of random word selection.