Multi-party social dialogue remains underexplored in the literature, in part due to the difficulty and cost of evaluation.As a result, recent work on synthetic dialogue generation often relies on automated metrics and LLM-as-a-Judge frameworks, despite limited evidence that such judges reflect human preferences in social settings.In this work, we introduce a lightweight and controllable multi-party dialogue generation framework (MPOD) as an experimental instrument for studying generation and evaluation in social interaction.Using this framework, we conduct human evaluations of opendomain multi-party dialogue simulation and directly compare human judgments against stateof-the-art LLM judges.Across 319 pairwise comparisons, we observe near-random agreement between humans and automated judges (Cohen's κ ≈ 0.11), driven by systematic behaviors including extreme tie aversion and strong sensitivity to assistant-style verbosity.Crucially, human-human inter-annotator agreement (κ = 0.29) is substantially higher than human-LLM agreement.To isolate the mechanism underlying this misalignment, we introduce a controlled Transplant Ablation, showing that LLM judges consistently prefer conversations containing a single proprietary, assistantstyle agent.Additional stress tests show that judges prefer GPT-style conversations even when utterance order is randomly shuffled, indicating insensitivity to conversational structure and coherence.Our findings provide controlled evidence that current instruction-tuned LLM judges do not reliably reflect human preferences for naturalness, engagingness, and overall quality in multi-party social dialogue, calling into question their widespread use for validating synthetic conversational data.

Simplifying text while preserving its meaning is a complex yet essential task, especially in sensitive domain applications like legal texts.When applied to a specialized field, like the legal domain, preservation differs significantly from its role in regular texts.This paper introduces FrJUDGE, a new dataset to assess legal meaning preservation between two legal texts.It also introduces JUDGEBERT, a novel evaluation metric designed to assess legal meaning preservation in French legal text simplification.JUDGEBERT demonstrates a superior correlation with human judgment compared to existing metrics.It also passes two crucial sanity checks, while other metrics did not: For two identical sentences, it always returns a score of 100%; on the other hand, it returns 0% for two unrelated sentences.Our findings highlight its potential to transform legal NLP applications, ensuring accuracy and accessibility for text simplification for legal practitioners and lay users.Annotators.We selected five native Frenchspeaking law students at the Faculty of Law of University Laval as our annotators.A meeting was held with them to introduce the task, instructions, and annotation guide and interface.Instructions included that they must spend at most 5 minutes per sentence pair.Furthermore, 15 instances were annotated during a pilot phase to familiarize them with the task.Finally, during a second meeting after evaluating the practice instance, annotators received feedback and advice on what phenomena

Large language models (LLM) perform outstandingly in various downstream tasks.However, there is limited understanding regarding how these models internalize linguistic knowledge, so various linguistic benchmarks have recently been proposed to facilitate syntactic evaluation of language models (LM) across languages.This paper introduces QFrCoLA (Quebec-French Corpus of Linguistic Acceptability Judgments), a normative binary acceptability judgments dataset comprising 25,153 in-domain and 2,675 out-of-domain sentences.Our study leverages the QFrCoLA dataset and seven other linguistic binary acceptability judgments corpus to benchmark eight LM.The results demonstrate that, on average, finetuned Transformer-based LM are strong baselines for most languages and that zero-shot binary classification LLM perform worse than the naive baseline on the task.However, for the QFrCoLA benchmark, on average, a finetuned Transformer-based LM outperformed other methods tested.It also shows that pretrained cross-lingual LLMs selected for our experimentation do not seem to have acquired linguistic judgment capabilities during their pre-training for Quebec French.Finally, our experiment results on QFrCoLA show that our dataset, built from examples that illustrate linguistic norms rather than speakers' feelings, is similar to linguistic acceptability judgment; it is a challenging dataset that can benchmark LM on their linguistic judgment capabilities.

Security logs are widely used to monitor data, networks, and computer activities. By analyzing them, security experts can pick out anomalies that reveal the presence of cyber attacks or information leaks and stop them quickly before serious damage occurs. This paper presents a systematic literature review on the use of security logs for data leak detection. Our findings are fourfold: (i) we propose a new classification of information leaks, which uses the GDPR principles; (ii) we identify the twenty most widely used publicly available datasets in threat detection; (iii) we describe twenty types of attacks present in public datasets; and (iv) we describe thirty algorithms used for data leak detection. The selected papers point to many opportunities that can be investigated by researchers interested in contributing to this area of research.

Filtering out Cyberbullying of communities has proven to be a challenge, and efforts have led to the creation of many different datasets to train classifiers. Through these datasets, we will explore the variety of definitions of cyberbullying behaviors and the impact of these differences on the portability of one classifier to another community. We also gain insight on the generalization power of these classifiers. A study of ensemble models combining these classifiers will help us understand how they interact with each other.

Language Identification (LID) is a challenging task, especially when the input texts are short and noisy such as microblog posts on social media or chat logs on gaming forums. The task has been tackled by either designing a feature set for a traditional classifier (e.g. Naive Bayes) or applying a deep neural network classifier (e.g. Bi-directional GRU, Encoder-Decoder). These methods are usually trained and tested on a private corpus, then used as off-the-shelf packages by other researchers on their own datasets, and consequently the various results published are not directly comparable. In this paper, we first create a new massive labeled dataset based on one year of Twitter data. We use this dataset to test several existing LID systems, in order to obtain a set of coherent benchmarks, and we make our dataset publicly available so that others can add to this set of benchmarks. Finally, we propose a shallow but efficient neural LID system, which is a ngram-regional convolution neural network enhanced with an attention mechanism. Experimental results show that our architecture is able to predict tens of thousands of samples per second and surpasses all state-of-the-art systems in accuracy and F1 score, including outperforming the popular langid system by 5%.

Plumitifs (dockets) were initially a tool for law clerks.Nowadays, they are used as summaries presenting all the steps of a judicial case.Information concerning parties' identity, jurisdiction in charge of administering the case, and some information relating to the nature and the course of the preceding are available through plumitifs.They are publicly accessible but barely understandable; they are written using abbreviations and referring to provisions from the Criminal Code of Canada, which makes them hard to reason about.In this paper, we propose a simple yet efficient multi-source language generation architecture that leverages both the plumitif and the Criminal Code's content to generate intelligible plumitifs descriptions.It goes without saying that ethical considerations rise with these sensitive documents made readable and available at scale, legitimate concerns that we address in this paper.This is, to the best of our knowledge, the first application of plumitifs descriptions generation made available for French speakers along with an ethical discussion about the topic.

This paper presents a novel particle swarm optimization (PSO) algorithm that combines the strengths of several PSO variants into a single competitive algorithm. This novel algorithm, named Time Adaptive Dual Particle Swarm Optimization (TAD-PSO), is comprised of two specialized populations, with one focusing on exploration of the search space and the other on exploitation. The main population, specialized in exploration, uses orthogonal learning to create information-rich exemplars which intelligently guide particle movement throughout the search space. The auxiliary population uses a PSO variant known for its very fast convergence speed, and thus very high performance on unimodal problems. This population is specialized in exploitation of the interesting local minima. The main population size decays linearly, to foster exploration early and convergence in the later stages of the optimization procedure. Additionally, TAD-PSO does not have the topological structure of the swarm as an algorithm hyper-parameter, making it a fast and simple algorithm to apply to new problems. TAD-PSO was tested extensively and compared to 6 widely used PSO variants on 19 benchmark problems, for 10, 30 and 100 dimensions. TAD-PSO consistently ranked first in each dimensional space, making it a competitive optimization algorithm on both unimodal and multimodal problems.

Microtext normalization is the challenge of discovering the English words corresponding to the unusually-spelled words used in social-media messages and posts. In this paper, we propose a novel method for doing this by rendering both English and microtext words phonetically based on their spelling, and matching similar ones together. We present our algorithm to learn spelling-to-phonetic probabilities and to efficiently search the English language and match words together. Our results demonstrate that our system correctly handles many types of normalization problems.

Microtext normalization is the challenge of discovering the English words corresponding to the unusually-spelled words used in social-media messages and posts. In this paper, we propose a novel method for doing this by rendering both English and microtext words phonetically based on their spelling, and matching similar ones together. We present our algorithm to learn spelling-to-phonetic probabilities and to efficiently search the English language and match words together. Our results demonstrate that our system correctly handles many types of normalization problems.

This study explores a new domain representation method for natural language processing based on an application of possibility theory. In our method, domain-specific information is extracted from natural language documents using a mathematical process based on Rieger's notion of semantic distances, and represented in the form of possibility distributions. We implement the distributions in the context of a possibilistic domain classifier, which is trained using the SchoolNet corpus.

Modern statistical techniques used in the field of natural language processing are limited in their applications by the fact they suffer from the loss of most of the semantic information contained in text documents. Fuzzy techniques have been proposed as a way to correct this problem through the modelling of the relationships between words while accommodating the ambiguities of natural languages. However, these techniques are currently either restricted to modelling the effects of simple words or are specialized in a single domain. In this paper, we propose a novel statistical-fuzzy methodology to represent the actions described in a variety of text documents by modelling the relationships between subject-verb-object triplets. The research will focus in the first place on the technique used to accurately extract the triplets from the text, on the necessary equations to compute the statistics of the subject-verb and verb-object pairs, and on the formulas needed to interpolate the fuzzy membership functions from these statistics and on those needed to de fuzzify the membership value of unseen triplets. Taken together, these sets of equations constitute a comprehensive system that allows the quantification and evaluation of the meaning of text documents, while being general enough to be applied to any domain. In the second phase, this paper will proceed to experimentally demonstrate the validity of our new methodology by applying it to the implementation of a fuzzy classifier conceived especially for this research. This classifier is trained using a section of the Brown Corpus, and its efficiency is tested with a corpus of 20 unseen documents drawn from three different domains. The positive results obtained from these experimental tests confirm the soundness of our new approach and show that it is a promising avenue of research.

We propose here to develop a methodology to extract semantic knowledge from plain written English documents and represent it using a formal mathematical expression, in order to facilitate its use in practical applications. Our fundamental conjecture is that most of the semantic information of a sentence lies in the action described by that sentence. Consequently, we focus on extracting from the text the words whose relationships represent actions, and on modelling those relationships. We then demonstrate the practicality of our methodology by applying it to a domain classifier.