Integrating a source of structured prior knowledge, such as a knowledge graph, into transformer-based language models is an increasingly popular method for increasing data efficiency and adapting them to a target domain. However, most methods for integrating structured knowledge into language models require additional training in order to adapt the model to the non-textual modality. This process typically leads to some amount of catastrophic forgetting on the general domain. KnowBert is one such knowledge integration method which can incorporate information from a variety of knowledge graphs to enhance the capabilities of transformer-based language models such as BERT. We conduct a qualitative analysis of the results of KnowBert-UMLS, a biomedically specialized KnowBert model, on a variety of linguistic tasks. Our results reveal that its increased understanding of biomedical concepts comes at the cost, specifically, of general common-sense knowledge and understanding of casual speech.

Domain adaptation in pretrained language models usually comes at some cost, most notably out-of-domain performance. This type of specialization typically relies on pre-training over a large in-domain corpus, which has the side effect of causing catastrophic forgetting on general text. We seek to specialize a language model by incorporating information from a knowledge base into its contextualized representations, thus reducing its reliance on specialized text. We achieve this by following the KnowBert method, applied to the UMLS biomedical knowledge base. We evaluate our model on in-domain and out-of-domain tasks, comparing against BERT and other specialized models. We find that our performance on biomedical tasks is competitive with the state-of-the-art with virtually no loss of generality. Our results demonstrate the applicability of this knowledge integration technique to the biomedical domain as well as its shortcomings. The reduced risk of catastrophic forgetting displayed by this approach to domain adaptation broadens the scope of applicability of specialized language models.

This paper presents two methods for building lightweight neural networks with similar accuracy than heavyweight ones with the advantage to be less greedy in memory and computing resources. So it can be implemented in edge and IoT devices. The presented distillation methods are respectively based on Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA). The two methods basically rely on the successive dimension reduction of a given dense neural network (teacher) hidden features, and the learning of a smaller neural network (student) which solves the initial learning problem along with a mapping problem to the reduced successive features spaces. The presented methods are compared to baselines -learning the student networks from scratch-, and we show that the additional mapping problem significantly improves the performance (accuracy, memory and computing resources) of the student networks.

In the scope of Financial Watch project, several targeted events have been required by contacted users in banking and investment domains. Financial news are classified with respect of the list of desired events. In this paper, a conceptual approach for indexing short English news in the financial domain is presented. By using a supervised original learning approach, a categorization method is proposed. Experimentation of the method on a sample of news showed that in almost all cases, occurrences of the right events in each document have been recognized, with respect to a corpus on the financial domain.

Translation memories play a vital role in Computer-Aided Translation (CAT). Approaches to align sentences from parallel corpora are generally used to construct translation memories. In this paper, we present, on the one hand, a hybrid approach to align sentences combining different information sources (bilingual lexicon, sentence length and sentence position, semantic similarity), and on the other hand, a Computer-Aided Translation prototype based on a cross-language information retrieval approach. This approach consists in building a database of source and target sentences augmented with grammatical information of words and considering the sentence to translate as a "query" to that database. The Computer-Aided Translation prototype has been experimented on a set of examples and the experimental results we obtained are encouraging and outperform the current CAT tools.

In this paper we evaluate the robustness of the basic descriptors for the indexing of the audio documents. These descriptors are employed in the production of document fingerprints used to allow access by content to a large database of audio documents. The realised tool is integrated in adWarning system developed by Advestigo and used successfully to detect illicit peer to peer exchanges

To overcome the main drawbacks of parallel SIMD image processing architectures, a new computer, called SYMPATIX, is proposed. The authors' goal is to implement low and intermediate levels of image processing on a SIMD computer. A study leads us to propose a network establishing asynchronous communications in a synchronous parallel structure. The performances of SYMPATIX have been evaluated on a VHDL model of the new computer. The machine supports image processing operations and image transforms. The design of the circuits is under progress using electronic synthesis tools.>