Limited availability of multilingual text corpora for training language models often leads to poor performance on downstream tasks due to undertrained representation spaces for languages other than English.This 'under-representation' has motivated recent cross-lingual transfer methods to leverage the English representation space by e.g.mixing English and 'non-English' tokens at the input level or extending model parameters to accommodate new languages.However, these approaches often come at the cost of increased computational complexity.We propose Fusion for Language Representations (FLARE) in adapters, a novel method that enhances representation quality and downstream performance for languages other than English while maintaining parameter efficiency.FLARE integrates source and target language representations within low-rank (LoRA) adapters using lightweight linear transformations, maintaining parameter efficiency while improving transfer performance.A series of experiments across representative crosslingual natural language understanding tasks, including natural language inference, questionanswering and sentiment analysis, demonstrate FLARE's effectiveness.FLARE achieves performance improvements of 4.9% for Llama 3.1 and 2.2% for Gemma 2 compared to standard LoRA fine-tuning on question-answering tasks, as measured by the exact match metric. 1 method for cross-lingual language understanding.

We introduce CORE, a dataset for few-shot relation classification (RC) focused on company relations and business entities.CORE includes 4,708 instances of 12 relation types with corresponding textual evidence extracted from company Wikipedia pages.Company names and business entities pose a challenge for few-shot RC models due to the rich and diverse information associated with them.For example, a company name may represent the legal entity, products, people, or business divisions depending on the context.Therefore, deriving the relation type between entities is highly dependent on textual context.To evaluate the performance of state-of-the-art RC models on the CORE dataset, we conduct experiments in the few-shot domain adaptation setting.Our results reveal substantial performance gaps, confirming that models trained on different domains struggle to adapt to CORE.Interestingly, we find that models trained on CORE showcase improved out-of-domain performance, which highlights the importance of high-quality data for robust domain adaptation.Specifically, the information richness embedded in business entities allows models to focus on contextual nuances, reducing their reliance on superficial clues such as relation-specific verbs.In addition to the dataset, we provide relevant code snippets to facilitate reproducibility and encourage further research in the field. 1

This paper contains explicit statements that are potentially offensive.

Question answering over hybrid contexts is a complex task, which requires the combination of information extracted from unstructured texts and structured tables in various ways.Recently, In-Context Learning demonstrated significant performance advances for reasoning tasks.In this paradigm, a large language model performs predictions based on a small set of supporting exemplars.The performance of In-Context Learning depends heavily on the selection procedure of the supporting exemplars, particularly in the case of HybridQA, where considering the diversity of reasoning chains and the large size of the hybrid contexts becomes crucial.In this work, we present Selection of ExEmplars for hybrid Reasoning (SEER), a novel method for selecting a set of exemplars that is both representative and diverse.The key novelty of SEER is that it formulates exemplar selection as a Knapsack Integer Linear Program.The Knapsack framework provides the flexibility to incorporate diversity constraints that prioritize exemplars with desirable attributes, and capacity constraints that ensure that the prompt size respects the provided capacity budgets.The effectiveness of SEER is demonstrated on FinQA and TAT-QA, two real-world benchmarks for HybridQA, where it outperforms previous exemplar selection methods 1 .