Large Language Models (LLMs) have demonstrated their ability to solve tasks across various domains, including software engineering. However, their extensive number of parameters makes full fine-tuning computationally prohibitive. While Parameter-Efficient Fine-Tuning (PEFT) methods, such as adapter fine-tuning, have been proposed to address this issue, yet they typically employ default configurations that use the same adapter settings across all layers. Concurrently, Automated Machine Learning (AutoML) has demonstrated success in hyperparameter optimization, while Neural Architecture Search (NAS) has proven effective in optimizing neural network architectures. Building on these successes, we introduce AutoAdapt, a novel approach that leverages NAS to automatically discover task-specific, layer-wide adapter configurations, allowing each layer to adopt distinct adapter parameters. AutoAdapt defines a search space tailored for adapter-based fine-tuning and employs an evolutionary algorithm to explore a diverse range of configurations, thereby evaluating the benefits of customizing each layer individually. We evaluate AutoAdapt on well-established software engineering tasks, including vulnerability detection, code clone detection, and code search. Our empirical results demonstrate that AutoAdapt outperforms manually engineered adapter configurations, achieving up to a 5% improvement in F1-score for clone detection and defect detection, and up to a 25% improvement in MRR for code search. Additionally, it surpasses other PEFT techniques, such as Prefix Tuning and LoRA. Furthermore, AutoAdapt is capable of identifying configurations that outperform even full fine-tuning, while training less than 2.5% of the model parameters. A comprehensive analysis reveals that factors such as selective layer adaptation, module selection (e.g., attention versus feed-forward layers), normalization, and dropout significantly influence performance across different tasks. Additionally, our findings suggest the possibility of transferring adapter configurations to similar datasets and tasks, thus simplifying the search for optimal PEFT settings. Our code and data are available for access at: https://github.com/serval-uni-lu/AutoAdapt .

Flaky tests are tests that yield different outcomes when run on the same version of a program. This non-deterministic behaviour plagues continuous integration with false signals, wasting developers’ time and reducing their trust in test suites. Studies highlighted the importance of keeping tests flakiness-free. Recently, the research community has been pushing towards the detection of flaky tests by suggesting many static and dynamic approaches. While promising, those approaches mainly focus on classifying tests as flaky or not and, even when high performances are reported, it remains challenging to understand the cause of flakiness. This part is crucial for researchers and developers that aim to fix it. To help with the comprehension of a given flaky test, we propose FlakyCat, the first approach to classify flaky tests based on their root cause category. FlakyCat relies on CodeBERT for code representation and leverages Siamese networks to train a multi-class classifier. We train and evaluate FlakyCat on a set of 451 flaky tests collected from open-source Java projects. Our evaluation shows that FlakyCat categorises flaky tests accurately, with an F1 score of 73%. Furthermore, we investigate the performance of our approach for each category, revealing that Async waits, Unordered collections and Time-related flaky tests are accurately classified, while Concurrency-related flaky tests are more challenging to predict. Finally, to facilitate the comprehension of FlakyCat’s predictions, we present a new technique for CodeBERT-based model interpretability that highlights code statements influencing the categorization.