Model merging has emerged as a crucial technique in Deep Learning, enabling the integration of multiple models into a unified system while preserving performance and scalability. In this respect, the compositional properties of low-rank adaptation techniques (e.g., LoRA) have proven beneficial, as simple averaging LoRA modules yields a single model that mostly integrates the capabilities of all individual modules. Building on LoRA, we take a step further by imposing that the merged model matches the responses of all learned modules. Solving this ob- jective in closed form yields an indeterminate system with A and B as unknown variables, indicating the existence of infinitely many closed-form solutions. To address this challenge, we introduce LoRM, an alternating optimization strategy that trains one LoRA matrix at a time. This allows solving for each unknown variable individually, thus finding a unique solution. We apply our proposed methodology to Federated Class-Incremental Learning (FCIL), ensuring alignment of model responses both between clients and across tasks. Our method demonstrates state-of-the-art performance across a range of FCIL scenarios. The code to reproduce our experiments is available at github.com/aimagelab/fed-mammoth.

Cyber Situational Awareness (CSA) is an important element in both cyber security and cyber defence to inform processes and activities on strategic, tactical, and operational level. Furthermore, CSA enables informed decision making. The ongoing digitization and interconnection of previously unconnected components and sectors equally affects the civilian and military sector. In defence, this means that the cyber domain is both a separate military domain as well as a cross-domain and connecting element for the other military domains comprising land, air, sea, and space. Therefore, CSA must support perception, comprehension, and projection of events in the cyber space for persons with different roles and expertise. This paper introduces NEWSROOM, a research initiative to improve technologies, methods, and processes specifically related to CSA in cyber defence. For this purpose, NEWSROOM aims to improve methods for attacker behavior classification, cyber threat intelligence (CTI) collection and interaction, secure information access and sharing, as well as human computer interfaces (HCI) and visualizations to provide persons with different roles and expertise with accurate and easy to comprehend mission- and situation-specific CSA. Eventually, NEWSROOM’s core objective is to enable informed and fast decision-making in stressful situations of military operations. The paper outlines the concept of NEWSROOM and explains how its components can be applied in relevant application scenarios.

The recent surge of pervasive devices that generate dynamic data streams has underscored the necessity for learning systems to adapt continually to data distributional shifts. To tackle this challenge, the research community has put forth a spectrum of methodologies, including the demanding pursuit of class-incremental learning without replay data. In this study, we present MIND, a parameter isolation method that aims to significantly enhance the performance of replay-free solutions and achieve state-of-the-art results on several widely studied datasets. Our approach introduces two main contributions: two alternative distillation procedures that significantly improve the efficiency of MIND increasing the accumulated knowledge of each sub-network, and the optimization of the BachNorm layers across tasks inside the sub-networks. Overall, MIND outperforms all the state-of-the-art methods for rehearsal-free Class-Incremental learning (with an increment in classification accuracy of approx. +6% on CIFAR-100/10 and +10% on TinyImageNet/10) reaching up to approx. +40% accuracy in Domain-Incremental scenarios. Moreover, we ablated each contribution to demonstrate its impact on performance improvement. Our results showcase the superior performance of MIND indicating its potential for addressing the challenges posed by Class-incremental and Domain-Incremental learning in resource-constrained environments.