Simplifying text while preserving its meaning is a complex yet essential task, especially in sensitive domain applications like legal texts.When applied to a specialized field, like the legal domain, preservation differs significantly from its role in regular texts.This paper introduces FrJUDGE, a new dataset to assess legal meaning preservation between two legal texts.It also introduces JUDGEBERT, a novel evaluation metric designed to assess legal meaning preservation in French legal text simplification.JUDGEBERT demonstrates a superior correlation with human judgment compared to existing metrics.It also passes two crucial sanity checks, while other metrics did not: For two identical sentences, it always returns a score of 100%; on the other hand, it returns 0% for two unrelated sentences.Our findings highlight its potential to transform legal NLP applications, ensuring accuracy and accessibility for text simplification for legal practitioners and lay users.Annotators.We selected five native Frenchspeaking law students at the Faculty of Law of University Laval as our annotators.A meeting was held with them to introduce the task, instructions, and annotation guide and interface.Instructions included that they must spend at most 5 minutes per sentence pair.Furthermore, 15 instances were annotated during a pilot phase to familiarize them with the task.Finally, during a second meeting after evaluating the practice instance, annotators received feedback and advice on what phenomena

Large language models (LLM) perform outstandingly in various downstream tasks.However, there is limited understanding regarding how these models internalize linguistic knowledge, so various linguistic benchmarks have recently been proposed to facilitate syntactic evaluation of language models (LM) across languages.This paper introduces QFrCoLA (Quebec-French Corpus of Linguistic Acceptability Judgments), a normative binary acceptability judgments dataset comprising 25,153 in-domain and 2,675 out-of-domain sentences.Our study leverages the QFrCoLA dataset and seven other linguistic binary acceptability judgments corpus to benchmark eight LM.The results demonstrate that, on average, finetuned Transformer-based LM are strong baselines for most languages and that zero-shot binary classification LLM perform worse than the naive baseline on the task.However, for the QFrCoLA benchmark, on average, a finetuned Transformer-based LM outperformed other methods tested.It also shows that pretrained cross-lingual LLMs selected for our experimentation do not seem to have acquired linguistic judgment capabilities during their pre-training for Quebec French.Finally, our experiment results on QFrCoLA show that our dataset, built from examples that illustrate linguistic norms rather than speakers' feelings, is similar to linguistic acceptability judgment; it is a challenging dataset that can benchmark LM on their linguistic judgment capabilities.

Plumitifs (dockets) were initially a tool for law clerks.Nowadays, they are used as summaries presenting all the steps of a judicial case.Information concerning parties' identity, jurisdiction in charge of administering the case, and some information relating to the nature and the course of the preceding are available through plumitifs.They are publicly accessible but barely understandable; they are written using abbreviations and referring to provisions from the Criminal Code of Canada, which makes them hard to reason about.In this paper, we propose a simple yet efficient multi-source language generation architecture that leverages both the plumitif and the Criminal Code's content to generate intelligible plumitifs descriptions.It goes without saying that ethical considerations rise with these sensitive documents made readable and available at scale, legitimate concerns that we address in this paper.This is, to the best of our knowledge, the first application of plumitifs descriptions generation made available for French speakers along with an ethical discussion about the topic.

In this paper, we reproduce the experiments of Artetxe et al. (2018b) regarding the robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings. We show that the reproduction of their method is indeed feasible with some minor assumptions. We further investigate the robustness of their model by introducing four new languages that are less similar to English than the ones proposed by the original paper. In order to assess the stability of their model, we also conduct a grid search over sensible hyperparameters. We then propose key recommendations that apply to any research project in order to deliver fully reproducible research.