Evaluating and optimising authorial style in long-form story generation remains challenging because style is often assessed with ad hoc prompting and is frequently conflated with overall writing quality. We propose a two-stage pipeline. First, we train a dedicated style-similarity judge by fine-tuning a sentence-transformer with authorship-verification supervision, and calibrate its similarity outputs into a bounded $[0,1]$ reward. Second, we use this judge as the primary reward in Group Relative Policy Optimization (GRPO) to fine-tune an 8B story generator for style-conditioned writing, avoiding the accept/reject supervision required by Direct Preference Optimization (DPO). Across four target authors (Mark Twain, Jane Austen, Charles Dickens, Thomas Hardy), the GRPO-trained 8B model achieves higher style scores than open-weight baselines, with an average style score of 0.893 across authors. These results suggest that AV-calibrated reward modelling provides a practical mechanism for controllable style transfer in long-form generation under a moderate model size and training budget.

Pretrained multilingual models exhibit the same social bias as models processing English texts.This systematic review analyzes emerging research that extends bias evaluation and mitigation approaches into multilingual and non-English contexts.We examine these studies with respect to linguistic diversity, cultural awareness, and their choice of evaluation metrics and mitigation techniques.Our survey illuminates gaps in the field's dominant methodological design choices (e.g., preference for certain languages, scarcity of multilingual mitigation experiments) while cataloging common issues encountered and solutions implemented in adapting bias benchmarks across languages and cultures.Drawing from the implications of our findings, we chart directions for future research that can reinforce the multilingual bias literature's inclusivity, cross-cultural appropriateness, and alignment with state-of-the-art NLP advancements.

Mixture-of-Experts (MoE) architectures in large language models (LLMs) achieve exceptional performance, but face prohibitive storage and memory requirements. To address these challenges, we present $D^2$-MoE, a new delta decompression compressor for reducing the parameters of MoE LLMs. Based on observations of expert diversity, we decompose their weights into a shared base weight and unique delta weights. Specifically, our method first merges each expert's weight into the base weight using the Fisher information matrix to capture shared components. Then, we compress delta weights through Singular Value Decomposition (SVD) by exploiting their low-rank properties. Finally, we introduce a semi-dynamical structured pruning strategy for the base weights, combining static and dynamic redundancy analysis to achieve further parameter reduction while maintaining input adaptivity. In this way, our $D^2$-MoE successfully compacts MoE LLMs to high compression ratios without additional training. Extensive experiments highlight the superiority of our approach, with over 13\% performance gains than other compressors on Mixtral|Phi-3.5|DeepSeek|Qwen2 MoE LLMs at 40$\sim$60\% compression rates. Codes are available in https://github.com/lliai/D2MoE.

Mixture of Experts (MoE) architecture improves Large Language Models (LLMs) with better scaling, but its higher parameter counts and memory demands create challenges for deployment. In this paper, we present MoE-SVD, a new decomposition-based compression framework tailored for MoE LLMs without any extra training. By harnessing the power of Singular Value Decomposition (SVD), MoE-SVD addresses the critical issues of decomposition collapse and matrix redundancy in MoE architectures. Specifically, we first decompose experts into compact low-rank matrices, resulting in accelerated inference and memory optimization. In particular, we propose selective decomposition strategy by measuring sensitivity metrics based on weight singular values and activation statistics to automatically identify decomposable expert layers. Then, we share a single V-matrix across all experts and employ a top-k selection for U-matrices. This low-rank matrix sharing and trimming scheme allows for significant parameter reduction while preserving diversity among experts. Comprehensive experiments on Mixtral, Phi-3.5, DeepSeek, and Qwen2 MoE LLMs show MoE-SVD outperforms other compression methods, achieving a 60% compression ratio and 1.5$\times$ faster inference with minimal performance loss.

Code-switching is a prevalent linguistic phenomenon in which multilingual individuals seamlessly alternate between languages. Despite its widespread use online and recent research trends in this area, research in code-switching presents unique challenges, primarily stemming from the scarcity of labelled data and available resources. In this study we investigate how pre-trained Language Models handle code-switched text in three dimensions: a) the ability of PLMs to detect code-switched text, b) variations in the structural information that PLMs utilise to capture code-switched text, and c) the consistency of semantic information representation in code-switched text. To conduct a systematic and controlled evaluation of the language models in question, we create a novel dataset of well-formed naturalistic code-switched text along with parallel translations into the source languages. Our findings reveal that pre-trained language models are effective in generalising to code-switched text, shedding light on abilities of these models to generalise representations to CS corpora. We release all our code and data, including the novel corpus, at https://github.com/francesita/code-mixed-probes.

Large Language Models (LLMs) have revolutionized the field of natural language processing with their impressive capabilities. However, their enormous size presents challenges for deploying them in real-world applications. Traditional compression techniques, like pruning, often lead to suboptimal performance due to their uniform pruning ratios and lack of consideration for the varying importance of features across different layers. To address these limitations, we present a novel Adaptive Layer Sparsity (ALS) approach to optimize LLMs. Our approach consists of two key steps. Firstly, we estimate the correlation matrix between intermediate layers by leveraging the concept of information orthogonality. This novel perspective allows for a precise measurement of the importance of each layer across the model. Secondly, we employ a linear optimization algorithm to develop an adaptive sparse allocation strategy based on evaluating the correlation matrix. This strategy enables us to selectively prune features in intermediate layers, achieving fine-grained optimization of the LLM model. Considering the varying importance across different layers, we can significantly reduce the model size without sacrificing performance. We conduct extensive experiments on publicly available language processing datasets, including the LLaMA-V1|V2|V3 family and OPT, covering various benchmarks. Our experimental results validate the effectiveness of our ALS method, showcasing its superiority over previous approaches. The performance gains demonstrate its potential for enhancing LLMs' efficiency and resource utilization. Notably, our approach surpasses the state-of-the-art models Wanda and SparseGPT, showcasing its ability to excel even under high sparsity levels. Codes at: https://github.com/lliai/ALS.

The Urdu language is used by approximately 200 million people for spoken and written communications on a daily basis. There is a substantial amount of unstructured Urdu textual data that is available worldwide. Data mining techniques can be used to extract meaningful knowledge from such a large, potentially informative source of data. There are many text processing systems available to process unstructured textual data. However, these systems are mostly language specific and developed for a variety of languages such as English, Spanish, Chinese, and so on. Unfortunately, there are not as many language processing resources available for Urdu. Stemming is one of the most important preprocessing steps in the text mining process and its goal is to reduce grammatical words form, e.g., parts of speech, gender, tense, and so on, to their root form. In this work, we have extended the stemming capabilities of our existing pattern-based comprehensive stemming system for Urdu text. In addition to the existing stemming rules in previous work, we introduce novel stemming rules for prefix, and infix stemming. We also optimize the existing suffix removal rules and extend the add character lists for word normalization. These stemming rules are generic and have the ability to generate the stem of Urdu words as well as loan words (words belonging to other languages i.e., Arabic, Persian, Turkish). In the experimental evaluation, we have observed a significant improvement in the overall stemming accuracy of our proposed pattern-based Urud stemmer, which demonstrates the adoptability of the proposed stemming approach for a variety of text-processing applications.

Venelin Kovatchev, Phillip Smith, Mark Lee, Rory Devine. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

In this paper we present the first work on the automated scoring of mindreading ability in middle childhood and early adolescence.We create MIND-CA, a new corpus of 11,311 question-answer pairs in English from 1,066 children aged 7 to 14.We perform machine learning experiments and carry out extensive quantitative and qualitative evaluation.We obtain promising results, demonstrating the applicability of state-of-the-art NLP solutions to a new domain and task.

Fake news detection has recently gained much attention from the wider NLP community due to its importance for preventing the spread of misinformation and its negative impact through the social media. The goal of this task is to classify the veracity labels of a statement expressed by a politician into fine-grained classes (degrees of truth). Previous deep learning approaches have significantly improved the performance of Political Fake Statement Detection by modeling statement with the speaker's credit history. However, the credit history may not be available in reality and most approaches did not consider about the evidence that supporting or denying claims when detecting fake news. In addition, state-of-the-art models may struggle to detect fine-grained labels because the statement of the speaker expresses factual and incorrect instances at the same time. In this paper, we approach the Political Fake Statement Detection problem by proposing two multi-stage feature-assisted neural models that consider claims and justifications as an input in a stance detection manner. We explore five-stage and three-stage classification strategies to better discern between the fine-grained labels of fake news. The proposed model in each stage is built on the powerful combination between dual GRU layers and lexical features which we further optimise by using Gaussian Noise. An extensive experimental work on a real-world benchmark LIARPLUS (an extended version of LIAR) dataset shows that three-stage model achieves state-of-the-art Accuracy (46.13%) and F1-score (45.13%) without using metadata and the credit history of the speaker. We also experimentally show that modeling the credit history in conjunction with statement and justification gives more than 6% improvement (e.g. 52.23% and 52.26% respectively).

We present a system for Answer Selection that integrates fine-grained Question Classification with a Deep Learning model designed for Answer Selection. We detail the necessary changes to the Question Classification taxonomy and system, the creation of a new Entity Identification system and methods of highlighting entities to achieve this objective. Our experiments show that Question Classes are a strong signal to Deep Learning models for Answer Selection, and enable us to outperform the current state of the art in all variations of our experiments except one. In the best configuration, our MRR and MAP scores outperform the current state of the art by between 3 and 5 points on both versions of the TREC Answer Selection test set, a standard dataset for this task.

Abstract Clinical guidelines specify sequences of steps (care pathways) to treat patients with single conditions. Increasingly, many patients exhibit “multimorbidity,” several chronic conditions needing concurrent treatment. However, applying multiple guidelines in parallel can lead to conflicts, eg, between prescribed drugs, lifestyle intervention recommendations, or treatment schedules. In computer science, process languages used to design and reason about software development and business process management are similar to clinical pathways. Using formal model transformation, composition and analysis methods, models can be combined and conflicts detected and resolved. We propose BPMN+V, a data‐driven formal model for clinical care pathways, as an extension of Business Process Model and Notation. We describe a method for conflict detection using a transformation of BPMN+V to Coloured Petri Nets and a state‐space method for detection of conflict in composed models. We present results from a case study, showing that common conflicts are successfully detected, and propose extension to a complete framework for efficiently recommending resolutions to medical conflicts in composed care pathway models.

We present in this paper a purely rule-based system for Question Classification which we divide into two parts: The first is the extraction of relevant words from a question by use of its structure, and the second is the classification of questions based on rules that associate these words to Concepts. We achieve an accuracy of 97.2%, close to a 6 point improvement over the previous State of the Art of 91.6%. Additionally, we believe that machine learning algorithms can be applied on top of this method to further improve accuracy.

Object Constraint Language (OCL) plays a key role in Unified Modeling Language (UML). In the UML standards, OCL is used for expressing constraints such as well-definedness criteria. In addition OCL can be used for specifying constraints on the models and pre/post conditions on operations, improving the precision of the specification. As a result, OCL has received considerable attention from the research community. However, despite its key role, there is a common consensus that OCL is the least adopted among all languages in the UML. It is often argued that, software practitioners shy away from OCL due to its unfamiliar syntax. To ensure better adoption of OCL, the usability issues related to producing OCL statement must be addressed. To address this problem, this paper aims to preset a method involving using Natural Language expressions and Model Transformation technology. The aim of the method is to produce a framework so that the user of UML tool can write constraints and pre/post conditions in English and the framework converts such natural language expressions to the equivalent OCL statements. As a result, the approach aims at simplifying the process of generation of OCL statements, allowing the user to benefit form the advantages provided by UML tools that support OCL. The suggested approach relies on Semantic Business Vocabulary and Rules (SBVR) to support formulation of natural language expressions and their transformations to OCL. The paper also presents outline of a prototype tool that implements the method.