Lisa Alazraki, Lihu Chen, Ana Brassard, Joe Stacey, Hossein A. Rahmani, Marek Rei. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Large Language Models (LLMs) possess vast amounts of knowledge within their parameters, prompting research into methods for locating and editing this knowledge. Previous work has largely focused on locating entity-related (often single-token) facts in smaller models. However, several key questions remain unanswered: (1) How can we effectively locate query-relevant neurons in contemporary autoregressive LLMs, such as Llama and Mistral? (2) How can we address the challenge of long-form text generation? (3) Are there localized knowledge regions in LLMs? In this study, we introduce Query-Relevant Neuron Cluster Attribution (QRNCA), a novel architecture-agnostic framework capable of identifying query-relevant neurons in LLMs. QRNCA allows for the examination of long-form answers beyond triplet facts by employing the proxy task of multi-choice question answering. To evaluate the effectiveness of our detected neurons, we build two multi-choice QA datasets spanning diverse domains and languages. Empirical evaluations demonstrate that our method outperforms baseline methods significantly. Further, analysis of neuron distributions reveals the presence of visible localized regions, particularly within different domains. Finally, we show potential applications of our detected neurons in knowledge editing and neuron-based prediction.

To meet the access demands of massive terminal users, the space-based Internet of Things (IoT) requires sufficient frequency resources for allocation. However, the frequency resources that are currently available have already been allocated to a great extent. Furthermore, the utilization rate of the allocated frequency resources is low. To support massive user access under restricted frequency resources, this work proposes a scheme based on Doppler frequency offset (DFO) pre-compensation to enhance spectrum utilization efficiency. By calculating the relative motion between the satellite and the transmitting terminal, combined with the length and transmission rate of the message, the optimal compensation value of the Doppler frequency deviation is determined. The frequency-protection interval is reduced. Simulation results show that the pre-compensation method can expand the user access volume by 90–400 times. Properly selecting the number of message splits and transmission rate to perform DFO pre-compensation calculations can increase user access by an additional 45% or more. This method improves the spectrum utilization efficiency and provides a solution to the challenge of access by a large number of users.

International audience

The Satellite-based Internet of Things (S-IoT) system offers a promising solution for accessing IoT devices in areas where terrestrial networks are inaccessible. These devices have significant roles in various scenarios, including aeronautic and maritime surveillance, data collection from sensors, and distress signals, from sinking ships. However, covering numerous IoT targets with a minimal number of satellites poses a challenge. This article proposes a solution known as the 5S payload designed for S-IoT. The 5S payload integrates five subsystems into a single unit, comprising the following components: 1) automatic identification system (AIS) for ship surveillance; 2) very high-frequency data exchange system (VDES) enabling two-way communication for ships; 3) automatic dependent surveillance-broadcast (ADS-B) system for aircraft surveillance; 4) data collection system (DCS) for user-defined sensors; and 5) emergency search and rescue (ESR) system. The design of the 5S payload adheres to the CubeSat standard and occupies a compact 1.5U size, enabling rapid manufacturing and deployment. The feasibility of the 5S payload was demonstrated on the TianTuo-5 satellite launched on 23 August 2020. Additionally, this article presents in-orbit experimental results spanning three years to further illustrate the effectiveness of the 5S payload.

Acronym Disambiguation (AD) is crucial for natural language understanding on various sources, including biomedical reports, scientific papers, and search engine queries.However, existing acronym disambiguation benchmarks and tools are limited to specific domains, and the size of prior benchmarks is rather small.To accelerate the research on acronym disambiguation, we construct a new benchmark named GLADIS with three components: (1) a much larger acronym dictionary with 1.5M acronyms and 6.4M long forms;(2) a pre-training corpus with 160 million sentences; (3) three datasets that cover the general, scientific, and biomedical domains.We then pre-train a language model, AcroBERT, on our constructed corpus for general acronym disambiguation, and show the challenges and values of our new benchmark.

State-of-the-art NLP systems represent inputs with word embeddings, but these are brittle when faced with Out-of-Vocabulary (OOV) words.To address this issue, we follow the principle of mimick-like models to generate vectors for unseen words, by learning the behavior of pre-trained embeddings using only the surface form of words.We present a simple contrastive learning framework, LOVE, which extends the word representation of an existing pre-trained language model (such as BERT), and makes it robust to OOV with few additional parameters.Extensive evaluations demonstrate that our lightweight model achieves similar or even better performances than prior competitors, both on original datasets and on corrupted variants.Moreover, it can be used in a plug-and-play fashion with FastText and BERT, where it significantly improves their robustness.

Biomedical entity linking aims to map biomedical mentions, such as diseases and drugs, to standard entities in a given knowledge base. The specific challenge in this context is that the same biomedical entity can have a wide range of names, including synonyms, morphological variations, and names with different word orderings. Recently, BERT-based methods have advanced the state-of-the-art by allowing for rich representations of word sequences. However, they often have hundreds of millions of parameters and require heavy computing resources, which limits their applications in resource-limited scenarios. Here, we propose a lightweight neural method for biomedical entity linking, which needs just a fraction of the parameters of a BERT model and much less computing resources. Our method uses a simple alignment layer with attention mechanisms to capture the variations between mention and entity names. Yet, we show that our model is competitive with previous work on standard evaluation benchmarks.