Abstract In practical use, language models (LM) must efficiently adapt to new tasks and knowledge while avoiding catastrophic forgetting, a requirement that sparked research on continual learning (CL). Despite advances, current CL methods still lack a unified solution that delivers strong knowledge transfer and parameter-efficient capacity management while preventing catastrophic forgetting, which limits effective use of task synergies under tight training and memory budgets. We bridge this gap by introducing Gated Expandable Parameter-Efficient Fine-Tuning (GE-PEFT), a novel approach that shares knowledge of previous tasks through leveraging a single, dynamically expanding PEFT module within LMs while selectively gating irrelevant previous tasks. Our experiments across multiple task-incremental CL benchmarks show that GE-PEFT outperforms existing state-of-the-art CL approaches in both full CL and few-shot settings. Our ablation and parameter sensitivity studies highlight the benefit of each proposed component, demonstrating that GE-PEFT offers a more efficient and adaptive solution for CL in LMs.

Analyzing sequences of interactions between users and items, sequential recommendation models can learn user intent and make predictions about the next item. Next to item interactions, most systems also have interactions with what we call non-item pages: these pages are not related to specific items but still can provide insights into the user’s interests, as, for example, navigation pages. We therefore propose a general way to include these non-item pages in sequential recommendation models to enhance next-item prediction. First, we demonstrate the influence of non-item pages on following interactions using the hypotheses testing framework HypTrails and propose methods for representing non-item pages in sequential recommendation models. Subsequently, we adapt popular sequential recommender models to integrate non-item pages and investigate their performance with different item representation strategies as well as their ability to handle noisy data. To show the general capabilities of the models to integrate non-item pages, we create a synthetic dataset for a controlled setting and then evaluate the improvements from including non-item pages on two real-world datasets. Our results show that non-item pages are a valuable source of information, and incorporating them in sequential recommendation models increases the performance of next-item prediction across all analyzed model architectures.

Even though companies store large amounts of business data in enterprise resource planning (ERP) systems, obtaining data for financial fraud detection is prohibitively difficult due to privacy concerns and companies protecting trade secrets. One possible solution is game-based generation of synthetic ERP data, which poses the challenge of designing an environment that generates realistic ERP data and allows players to commit many different types of fraud. In this work, we design a multiplayer game that allows players to cooperatively run a fictional company, while simultaneously challenging them to maximize their personal gain. We introduce an approach for letting players explore fraud scenarios through emergent gameplay and present a prototype that may be primed with information from real world ERP systems to generate realistic data.