Multi-label text classification (MLTC) is an essential task in NLP applications.Traditional methods require extensive labeled data and are limited to fixed label sets.Extracting labels with large language models (LLMs) is more effective and universal, but incurs high computational costs.In this work, we introduce a distillation-based T5 generalist model for zero-shot MLTC and few-shot fine-tuning.Our model accommodates variable label sets with general domain-agnostic pretraining, while modeling dependency between labels.Experiments show that our approach outperforms baselines of similar size on three few-shot tasks.Our code is available at repository.

In this paper, we propose a novel method for joint entity and relation extraction from unstructured text by framing it as a conditional sequence generation problem. In contrast to conventional generative information extraction models that are left-to-right token-level generators, our approach is \textit{span-based}. It generates a linearized graph where nodes represent text spans and edges represent relation triplets. Our method employs a transformer encoder-decoder architecture with pointing mechanism on a dynamic vocabulary of spans and relation types. Our model can capture the structural characteristics and boundaries of entities and relations through span representations while simultaneously grounding the generated output in the original text thanks to the pointing mechanism. Evaluation on benchmark datasets validates the effectiveness of our approach, demonstrating competitive results. Code is available at https://github.com/urchade/ATG.

Information extraction (IE) from text documents is an important NLP task that includes entity, relation, and event extraction. These tasks are often addressed jointly as a graph generation problem, where entities and event triggers represent nodes and where relations and event arguments represent edges. Most existing systems use local classifiers for nodes and edges, trained using cross-entropy loss, and employ inference strategies such as beam search to approximate the optimal graph structure. These approaches typically suffer from exposure bias due to the discrepancy between training and decoding. In this paper, we tackle this problem by casting graph generation as auto-regressive sequence labeling and making its training aware of the decoding procedure by using a differentiable version of beam search. We evaluate the effectiveness of our approach through extensive experiments conducted on the ACE05 and ConLL04 datasets across diverse languages. Our experimental findings affirm that our model outperforms its non-decoding-aware version for all datasets employed. Furthermore, we conduct ablation studies that emphasize the effectiveness of aligning training and inference. Additionally, we introduce a novel quantification of exposure bias within this context, providing valuable insights into the functioning of our model.

As pretrained language model emerge and consistently develop, prompt-based training has become a well-studied paradigm to improve the exploitation of models for many natural language processing tasks. Furthermore, prompting demonstrates great performance compared to conventional fine-tuning in scenarios with limited annotated data, such as zero-shot or few-shot situations. Verbalizers are crucial in this context, as they help interpret masked word distributions generated by language models into output predictions. This study introduces a benchmarking approach to assess three common baselines of verbalizers for topic classification in few-shot learning scenarios. Additionally, we find that increasing the number of label words for automatic label word searching enhances model performance. Moreover, we investigate the effectiveness of template assembling with various aggregation strategies to develop stronger classifiers that outperform models trained with individual templates. Our approach achieves comparable results to prior research while using significantly fewer resources. Our code is available at https://github.com/quang-anh-nguyen/verbalizer_benchmark.git.

Urchade Zaratiana, Nadi Tomeh, Pierre Holat, Thierry Charnois. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.

Word embeddings are used with success for a variety of tasks involving lexical semantic similarities between individual words.Using unsupervised methods and just cosine similarity, encouraging results were obtained for analogical similarities.In this paper, we explore the potential of pre-trained word embeddings to identify generic types of semantic relations in an unsupervised experiment.We propose a new relational similarity measure based on the combination of word2vec's CBOW input and output vectors which outperforms alternative vector representations, when used for unsupervised clustering on SemEval 2010 Relation Classification data.

Sequential pattern mining is one of the most studied and challenging tasks in data mining. However, the extension of well-known methods from many other classical patterns to sequences is not a trivial task. In this paper we study the notion of δ-freeness for sequences. While this notion has extensively been discussed for itemsets, this work is the first to extend it to sequences. We define an efficient algorithm devoted to the extraction of δ-free sequential patterns. Furthermore, we show the advantage of the δ-free sequences and highlight their importance when building sequence classifiers, and we show how they can be used to address the feature selection problem in statistical classifiers, as well as to build symbolic classifiers which optimizes both accuracy and earliness of predictions.

Sequential pattern mining under various constraints is a challenging data mining task. The paper provides a generic framework based on constraint programming to discover sequence patterns defined by constraints on local patterns (e.g., Gap, regular expressions) or constraints on patterns involving combination of local patterns such as relevant subgroups and top-k patterns. This framework enables the user to mine in a declarative way both kinds of patterns. The solving step is done by exploiting the machinery of Constraint Programming. For complex patterns involving combination of local patterns, we improve the mining step by using dynamic CSP. Finally, we present two case studies in biomedical information extraction and stylistic analysis in linguistics.

Due to the considerable increase of freely available data, the discovery of relevant information from textual content is a critical challenge. The work presented here takes part in ongoing researches to develop a global knowledge gathering system. It aims at building knowledge sheets summarizing all the pieces of information we know about events extracted from text. For this sake, we define a global process bringing together different methods and components from multiple domains of research.

Orphanet provides an international web-based knowledge portal for rare diseases including a collection of review articles. However, reviews and literature monitoring are manual. Thus, new documentation about a rare disease is a time-consuming process and automatically discovering knowledge from a large collection of texts is a crucial issue. This context represents a strong motivation to address the problem of extracting gene-rare diseases relationships from texts. In this paper, we tackle this issue with a cross-fertilization of information extraction and data mining techniques (sequential pattern mining under constraints). Experiments show the interest of the method for the documentation of rare diseases.

This paper presents a modeling of time, aspect and verbal meanings in natural language processing within Simple Conceptual Graphs (SCG) by way of Semantico-Cognitive Schemes (SCS) and the aspectual-temporal theory. Our system translates a semantico-cognitive representation in terms of SCGs. The SCS allows us to build a representation of a text taking into account fine subtleties of natural language as the information about time and aspect. The Conceptual Graphs formalism provides a powerful inferential mechanism which makes it possible to reason from texts. Our work bears on French texts. A text is represented by two different structures both represented within the SCG model. The first structure models the semantico-cognitive representation while the second one is the temporal diagram representing the temporal constraints between the situations described in the text. Linking these structures entails an expansion of the original SCG model.