The training energy efficiency of deep neural networks became an extensively studied research topic in the last years. Some of the existing approaches seek to reduce the size of the architecture by either starting the training with a large network and pruning it, or by beginning with a seed architecture and then growing it. Instead of compressing the architecture, other approaches aim to reduce the number of training examples through data selection. While various approaches belonging to these two categories have been proposed, only a few works actually conduct energy measurements. Others merely mention potential gains in efficiency or rely on alternative evaluation metrics such as FLOPs. In this paper, we conduct a series of experiments both on a synthetic dataset and on image classification benchmarks in order to compare the impact of pruning, architecture growing and data selection on training energy consumption and prediction quality. Our results show that growing maintains a high prediction quality but brings limited energy gains when the size of the resulting architecture is large. Pruning can offer high gains, but also impacts accuracy, making it more suited for large models. Data selection provides energy gains correlated with the selectivity rate but causes an accuracy loss. We find that the effectiveness of every technique depends on its hyperparameters and on the architecture size.

Information Extraction (IE) pipelines aim to extract meaningful entities and relations from documents and structure them into a knowledge graph that can then be used in downstream applications. Training and evaluating such pipelines requires a dataset annotated with entities, coreferences, relations, and entity-linking. However, existing datasets either lack entity-linking labels, are too small, not diverse enough, or automatically annotated (that is, without a strong guarantee of the correction of annotations). Therefore, we propose Linked-DocRED, to the best of our knowledge, the first manually-annotated, large-scale, document-level IE dataset. We enhance the existing and widely-used DocRED dataset with entity-linking labels that are generated thanks to a semi-automatic process that guarantees high-quality annotations. In particular, we use hyperlinks in Wikipedia articles to provide disambiguation candidates. We also propose a complete framework of metrics to benchmark end-to-end IE pipelines, and we define an entity-centric metric to evaluate entity-linking. The evaluation of a baseline shows promising results while highlighting the challenges of an end-to-end IE pipeline. Linked-DocRED, the source code for the entity-linking, the baseline, and the metrics are distributed under an open-source license and can be downloaded from a public repository.

Unsupervised Relation Extraction (RE) aims to identify relations between entities in text, without having access to labeled data during training. This setting is particularly relevant for domain specific RE where no annotated dataset is available and for open-domain RE where the types of relations are a priori unknown. Although recent approaches achieve promising results, they heavily depend on hyperparameters whose tuning would most often require labeled data. To mitigate the reliance on hyperparameters, we propose PromptORE, a "Prompt-based Open Relation Extraction" model. We adapt the novel prompt-tuning paradigm to work in an unsupervised setting, and use it to embed sentences expressing a relation. We then cluster these embeddings to discover candidate relations, and we experiment different strategies to automatically estimate an adequate number of clusters. To the best of our knowledge, PromptORE is the first unsupervised RE model that does not need hyperparameter tuning. Results on three general and specific domain datasets show that PromptORE consistently outperforms state-of-the-art models with a relative gain of more than 40% in B3, V-measure and ARI. Qualitative analysis also indicates PromptORE's ability to identify semantically coherent clusters that are very close to true relations.

Worldwide, highway accidents have important social and financial impacts. Crash Predictions Models (CPM) are used to reduce their frequency and gravity. They belong to two main categories: generalized linear models (GLM) and nonparametric machine learning (ML) algorithms. Broadly speaking, the former offer better interpretability but tend to have worse predictive performances than the latter. However, for highway infrastructures managers, efficient predictions of accident count must come with explanations so as to give rise to efficient safety actions. Therefore, to balance predictive power and interpretability, we propose a methodology that combines Bayesian learning of hierarchical GLM with automatic detection of latent structures and interactions through methods borrowed from the field of explainable artificial intelligence (XAI). Promising results are obtained with experiments conducted on crash count data from 2008 to 2017 on a large part of the French highway network. Moreover, we tested our approach on three public datasets covering a broad variety of contexts in terms of volume, data types and tasks (viz. classification and regression). These experiments confirm that our framework outperforms traditional GLM models while getting close to the best ML models and remaining interpretable.

Multistructured (M-S) data models were introduced to allow the expression of multilevel, concurrent annotation. However, most models lack either a consistent or an efficient validation mechanism. In a former paper, we introduced extended Annotation Graphs (eAG), a cyclic-graph data model equipped with a novel schema mechanism that, by allowing validation "by construction", bypasses the typical algorithmic cost of traditional methods for the validation of graph-structured data. We introduce here LeAG, a markup syntax for eAG annotations over text data. LeAG takes the shape of a classic, inline markup model. A LeAG annotation can then be written, in a human-readable form, in any notepad application, and saved as a text file; the syntax is simple and familiar -- yet LeAG proposes a natural syntax for multilayer annotation with (self-) overlap and links. From a theoretical point of view, LeAG inaugurates a hybrid markup paradigm. Syntactically speaking, it is a full inline model, since the tags are all inserted along the annotated resources; still, we evidence that representing independent elements' co-occurring in an inline manner requires to make the annotation rest upon a notion of reference value, that is typical of stand-off markup. To our knowledge, LeAG is the first inline markup syntax to properly conceptualize the notion of elements' accidental co-occurring, that is yet fundamental in multilevel annotation.

The inferences of a machine learning algorithm are naturally limited by the available data. In many real-world applications, the provided internal data is domain-specific and we use external background knowledge to derive or add new features. Semantic networks, like linked open data, provide a largely unused treasure trove of background knowledge. This drives a recent surge of interest in unsupervised methods to automatically extract such semantic background knowledge and inject it into machine learning algorithms. In this work, we describe the general process of extracting knowledge from semantic networks through vector space embeddings. The locations in the vector space then reflect relations in the original semantic network. We perform this extraction for geographic background knowledge and inject it into a neural network for the complicated real-world task of credit-card fraud detection. This improves the performance by 11.2%.

Multistructured (M-S) documents were introduced as an answer to the need of ever more expressive data models for scholarly annotation, as experienced in the frame of Digital Humanities. Many proposals go beyond XML, that is the gold standard for annotation, and allow the expression of multilevel, concurrent annotation. However, most of them lack support for algorithmic tasks like validation and querying, despite those being central in most of their application contexts.

The main objective of this research was to identify co-referent entities located in several linked open data (LOD) sources that are described in various natural languages. The problem is approached from two perspectives. First, we do a multi-scale analysis of the RDF graph to discover structural similarities of entities. This was implemented as a tensor decomposition of the RDF graph with each predicate corresponding to a horizontal slide of the tensor. Hereafter, we used the term "structural evidence" to refer to the result of this analysis. Second, for each entity, we associated textual data coming from the Web of documents. Thus, after some preprocessing (viz. removing empty words, applying a weighting scheme such as tf-idf, ...), we represented each entity in a high dimensional space with each dimension corresponding to a term. Next, through a Singular Value Decomposition (SVD), we find a subspace such that the sum of squared distances from the original space to the sub space is minimized. This dimensionality reduction allows us to find language independent similarities between entities. Hereafter, we use the term "textual evidence" to refer to the result of this analysis.

We consider documents as the results of dynamic processes of documentary fragments' associations. We have experienced that once a substantial number of associations exist, users need some synoptic views. One possible way of providing such views relies in the organization of associations into relevant subsets that we call "dimensions". Thus, dimensions offer orders along which a documentary archive can be traversed. Many works have proposed efficient ways of presenting combinations of dimensions through graphical user interfaces. Moreover, there are studies on the structural properties of dimensional hypertexts. However, the problem of the origins and evolution of dimensions has not yet received a similar attention. Thus, we propose a mechanism based on a simple structural constraint for helping users in the construction of dimensions: if a cycle appears within a dimension while a user is creating a new dimension by the aggregation of existing ones, he will be encouraged (and assisted in his task) to restructure the dimensions in order to cut the cycle. This is a first step towards a rational control of the emergence and evolution of dimensions.

In this article, we introduce a new problem: the construction of multi-structured documents. We first offer an overview of existing solutions to the representation of such documents. We then notice that none of them consider the problem of their construction. In this context, we use our experience with philosophers who are building a digital edition of the work of Jean-Toussaint Desanti, in order to present a methodology for the construction of multi-structured documents. This methodology is based on the MSDM model in order to represent such documents. Moreover each step of the methodology has been implemented in the Haskell functional programming language.