We present two multilingual LLMs, Teuken 7B-base and Teuken 7B-instruct, designed to embrace Europe’s linguistic diversity by supporting all 24 official languages of the European Union. Trained on a dataset comprising around 60% non-English data and utilizing a custom multilingual tokenizer, our models address the limitations of existing Large Language Models (LLMs) that predominantly focus on English or a few high-resource languages. We detail the models’ development principles, i.e., data composition, tokenizer optimization, and training methodologies. The models demonstrate strong performance across multilingual benchmarks, as evidenced by their performance on European versions of ARC, HellaSwag, and TruthfulQA.

Mehdi Ali, Manuel Brack, Max Lübbering, Elias Wendt, Abbas Goher Khan, Richard Rutmann, Alex Jude, Maurice Kraus, Alexander Arno Weber, Felix Stollenwerk, David Kaczér, Florian Mai, Lucie Flek, Rafet Sifa, Nicolas Flores-Herr, Joachim Koehler, Patrick Schramowski, Michael Fromm, Kristian Kersting. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.

For research in audiovisual interview archives often it is not only of interest what is said but also how. Sentiment analysis and emotion recognition can help capture, categorize and make these different facets searchable. In particular, for oral history archives, such indexing technologies can be of great interest. These technologies can help understand the role of emotions in historical remembering. However, humans often perceive sentiments and emotions ambiguously and subjectively. Moreover, oral history interviews have multi-layered levels of complex, sometimes contradictory, sometimes very subtle facets of emotions. Therefore, the question arises of the chance machines and humans have capturing and assigning these into predefined categories. This paper investigates the ambiguity in human perception of emotions and sentiment in German oral history interviews and the impact on machine learning systems. Our experiments reveal substantial differences in human perception for different emotions. Furthermore, we report from ongoing machine learning experiments with different modalities. We show that the human perceptual ambiguity and other challenges, such as class imbalance and lack of training data, currently limit the opportunities of these technologies for oral history archives. Nonetheless, our work uncovers promising observations and possibilities for further research.

Sequence labeling systems should perform reliably not only under ideal conditions but also with corrupted inputs-as these systems often process user-generated text or follow an errorprone upstream component.To this end, we formulate the noisy sequence labeling problem, where the input may undergo an unknown noising process and propose two Noise-Aware Training (NAT) objectives that improve robustness of sequence labeling performed on perturbed input: Our data augmentation method trains a neural model using a mixture of clean and noisy samples, whereas our stability training algorithm encourages the model to create a noise-invariant latent representation.We employ a vanilla noise model at training time.For evaluation, we use both the original data and its variants perturbed with real OCR errors and misspellings.Extensive experiments on English and German named entity recognition benchmarks confirmed that NAT consistently improved robustness of popular sequence labeling models, preserving accuracy on the original input.We make our code and data publicly available for the research community.

While recent automatic speech recognition systems achieve remarkable performance when large amounts of adequate, high quality annotated speech data is used for training, the same systems often only achieve an unsatisfactory result for tasks in domains that greatly deviate from the conditions represented by the training data. For many real-world applications, there is a lack of sufficient data that can be directly used for training robust speech recognition systems. To address this issue, we propose and investigate an approach that performs a robust acoustic model adaption to a target domain in a cross-lingual, multi-staged manner. Our approach enables the exploitation of large-scale training data from other domains in both the same and other languages. We evaluate our approach using the challenging task of German oral history interviews, where we achieve a relative reduction of the word error rate by more than 30% compared to a model trained from scratch only on the target domain, and 6-7% relative compared to a model trained robustly on 1000 hours of same-language out-of-domain training data.

Multilingualism is a cultural cornerstone of Europe and firmly anchored in the European treaties including full language equality. However, language barriers impacting business, cross-lingual and cross-cultural communication are still omnipresent. Language Technologies (LTs) are a powerful means to break down these barriers. While the last decade has seen various initiatives that created a multitude of approaches and technologies tailored to Europe’s specific needs, there is still an immense level of fragmentation. At the same time, AI has become an increasingly important concept in the European Information and Communication Technology area. For a few years now, AI – including many opportunities, synergies but also misconceptions – has been overshadowing every other topic. We present an overview of the European LT landscape, describing funding programmes, activities, actions and challenges in the different countries with regard to LT, including the current state of play in industry and the LT market. We present a brief overview of the main LT-related activities on the EU level in the last ten years and develop strategic guidance with regard to four key dimensions.

In automatic speech recognition, often little training data is available for specific challenging tasks, but training of state-of-the-art automatic speech recognition systems requires large amounts of annotated speech. To address this issue, we propose a two-staged approach to acoustic modeling that combines noise and reverberation data augmentation with transfer learning to robustly address challenges such as difficult acoustic recording conditions, spontaneous speech, and speech of elderly people. We evaluate our approach using the example of German oral history interviews, where a relative average reduction of the word error rate by 19.3% is achieved.

Recently, neural sequence-to-sequence (Seq2Seq) models have been applied to the problem of grapheme-to-phoneme (G2P) conversion. These models offer a straightforward way of modeling the conversion by jointly learning the alignment and translation of input to output tokens in an end-to-end fashion. However, until now this approach did not show improved error rates on its own compared to traditional joint-sequence based n-gram models for G2P. In this paper, we investigate how multitask learning can improve the performance of Seq2Seq G2P models. A single Seq2Seq model is trained on multiple phoneme lexicon datasets containing multiple languages and phonetic alphabets. Although multi-language learning does not show improved error rates, combining standard datasets and crawled data with different phonetic alphabets of the same language shows promising error reductions on English and German Seq2Seq G2P conversion. Finally, combining Seq2seq G2P models with standard n-grams based models yields significant improvements over using either model alone.

In spoken term detection, subword speech recognition is a viable means for addressing the out-of-vocabulary (OOV) problem at query time. Applying fuzzy error compensation techniques is needed for coping with inevitable recognition errors, but can lead to high false alarm rates especially for short queries. We propose two novel methods which reject false alarms based on the context of the hypothesized result and the distance to phonetically similar queries. Using the proposed methods, we obtain an increase in precision of 11% absolute at equal recall.

Abstract We describe how complementary search spaces, addressed bytwo different methods used in Spoken Term Detection (STD),can be merged for German subword STD. We propose fuzzy-search techniques on lattices to narrow the gap between sub-word and word retrieval. The first technique is based on anedit-distance, where no a priori knowledge about confusions isemployed. Additionally, we propose a weighting method whichexplicitly models pronunciation variation on a subword leveland thus improves robustness against false positives. Recall isimproved by 6% absolute when retrieving on the merged searchspace rather than using an exact lattice search. By modelingsubword pronunciation variation, we increase recall in a high-precision setting by 2% absolute compared to the edit-distancemethod.Index Terms: subword speech recognition, spoken term detec-tion, pronunciation variation 1. Introduction As vast amounts of media can be stored digitally, methods arerequired to make them searchable. For data containing speech,a challenging task is Spoken Term Detection (STD) in whichall occurrences of a word or phrase have to be located preciselyin a database.A straightforward approach to STD is to use a large vocabularycontinuous speech recognizer (LVCSR) to transcribe thespeech parts of the data. However, this implies that if a wordis not in the vocabulary it cannot be transcribed, and thuscannot be found upon search time. This phenomenon is knownas the out-of-vocabulary (OOV) problem and is especiallyprominent in languages that make use of compounding oremploy complex morphological patterns to create new words.Examples of such languages are Finnish, German and Turkish[1, 2, 3]. One way to alleviate this problem is to use subwordsas the recognition unit. In contrast to words, the inventory ofsubwords, e.g. syllables, is finite and hence known a priori.Their compositional nature allows for retrieval of compoundterms that otherwise could not be found on the word level.However, a decrease in recall compared to word STD has beenobserved [2, 4].The 1-best transcription given by the LVCSR will containother misrecognitions besides those caused by OOVs. This isespecially the case for spontaneous speech, which is often un-planned and may contain a high number of false starts. SeveralSTD techniques have emerged to cope with these recognitionerrors. One approach is retrieval on word or subword lattices(or lattice-like variants) instead of the 1-best transcription [5].In [4] we compared this approach to an edit-distance searchon the 1-best transcription [6], which matches the query withthe 1-best transcription and calculates a distance according toa similarity measure. Like other fuzzy approaches to subwordSTD [7], this method can tolerate a degree of mismatchbetween the recognition output and the query, and is thus ableto find keywords in erroneous transcriptions. On a GermanSTD task, we found that on a subword level, each approachcovers a different area of the search space, and comes withrelative advantages and disadvantages.In this contribution we present ways of merging the searchspaces covered by the two previously mentioned retrievalapproaches. The aim is to exploit the complementary nature ofsubword lattice and subword fuzzy-search by combining theminto a single, unified search method. Further, we investigatehow explicit modeling of pronunciation variation on a subwordlevel can improve robustness of the retrieval method.

No abstract available.

Paper 1451-Wed3CaP.2

In this paper we investigate on-line zero-crossing based audio stream segmentation and classification into speech and other segments. We consider such segments as applause, noise of the auditorium, and silence. We demonstrate that the features extracted from zero-crossing are stable and valid to be used for speech and other signal discrimination and classification and don't require large amount of data for the training. We describe the optimal segmentation of unlimited audio signals using results of the frames classification. We demonstrate that using optimal segmentation is better than using traditional sliding window technique.

This paper describes the MPEG-7 compliant indexing and retrieval system iFinder based on XML and open source database technology. The iFinder system automatically extracts metadata from A/V-content and allows access to the enriched content by means of a client/server-based retrieval engine. This multimedia retrieval system allows for search and retrieval of short video segments in huge multimedia archives. As a reference application, the iFinder system is used to index speeches from the German Parliament. The user can search for fragments of political speeches and receives matching video segments through a video streaming service.

S.945-948

An application-specific perceptual evaluation was carried out in order to compare six high-quality German text-to-speech systems. Subjects judged the systems ’ reading of an email message and a newspaper article according to four applicationspecific questions and six voice quality attributes. The results indicate significant differences between the systems. Possible applications of the systems were judged rather unfavourably. The main reasons for this proved to be the synthetic prosody and voice quality. Errors concerning text conversion were less important. 1.

This paper presents our new results on multilingual phone modeling and adaptation into a new target language which is not included in the trained multilingual models. The experiments were carried out with the SpeechDat(M) and MacroPhone databases including the languages of French, German, Italian, Portuguese, Spanish and American English. First, we constructed language-dependent and multilingual phone models. The recognition rate for an isolated word task decreased in average only by 3.2% using 95 multilingual instead of 232 language-dependent models. Second, we investigated adaptation techniques for cross-language transfer and showed that only 100 utterances from a new language were needed for adaptation. Using the MAP algorithm the recognition rate was improved from 79.9% to 84.3%. Finally, we defined a phonetic based dissimilarity measure between 2 languages and compared language-dependent and multilingual models for the purpose of cross-language transfer.

In this paper we report on advances regarding our approach to porting an automatic speech recognition system to a new target task. In cases where there is not enough acoustic data available to allow for thorough estimation of HMM parameters it is impossible to train an appropriate model. The basic idea to overcome this problem is to create a task independent seed model that can cope with all tasks equally well. However, the performance of such a generalist model is of course lower than the performance of task dependent models (if these were available). So, the seed model is gradually enhanced by using its own recognition results for incremental online task adaptation. Here, we use a multilingual romanic/germanic seed model for a slavic target task. In tests on Slovene digits multilingual modeling yields the best recognition accuracy compared to other language dependent models. Applying unsupervised online task adaptation we observe a remarkable boost of recognition performance.