The pretrain+fine-tune paradigm is foundational for deploying large language models (LLMs) across various downstream applications. Within this framework, Low-Rank Adaptation (LoRA) stands out for its parameter-efficient fine-tuning (PEFT), producing numerous reusable task-specific LoRA adapters. However, this approach requires explicit task intention selection, posing challenges for autonomous task sensing and switching during inference with multiple existing LoRA adapters embedded in a single LLM. In this work, we introduce MeteoRA (Multiple-Tasks embedded LoRA), a scalable and efficient framework that reuses multiple task-specific LoRA adapters into the base LLM via a full-mode Mixture-of-Experts (MoE) architecture. This framework also includes novel MoE forward acceleration strategies to address the efficiency challenges of traditional MoE implementations. Our evaluation, using the LlaMA2-13B and LlaMA3-8B base models equipped with 28 existing LoRA adapters through MeteoRA, demonstrates equivalent performance with the traditional PEFT method. Moreover, the LLM equipped with MeteoRA achieves superior performance in handling composite tasks, effectively solving ten sequential problems in a single inference pass, thereby demonstrating the framework's enhanced capability for timely adapter switching.

Autonomous Driving Systems (ADS) are advancing rapidly due to progress in deep learning, yet critical challenges remain, particularly in the realm of safety verification.As safety-critical systems, ADS must undergo rigorous testing across diverse scenarios.Realworld data, while valuable, are inherently inflexible for interaction and scenario customization.In contrast, simulator-generated synthetic scenarios provide a platform that enables interaction, control, editability, and adaptability to specific needs.However, current simulation approaches are limited-either relying on costly, manually crafted, overly templated scenarios or generating unconditioned trivial behaviors based on learned distributions.In this work, we introduce LASER, an innovative framework that leverages large language models (LLMs) to conduct traffic simulations based on natural language inputs.The framework operates in two phases.First, it generates scripts from user-provided descriptions.Second, it executes these scripts by guiding autonomous agents within the CARLA simulator to perform tasks in real-time.This method effectively decomposes tasks, allocates controls, and integrates interactive elements to create dynamic and scalable simulations that align with user requirements.By using LASER, we overcome the rigid constraints of traditional simulation methods, enabling the creation of complex, diverse, flexible and on-demand driving scenarios.The approach significantly enhances the process of generating ADS training and testing data, addressing the scalability and diversity issues associated with previous simulation models.The code and all demos are available anonymously at https://njudeepengine.

Symbolic execution is a powerful program analysis technique. External environment construction and internal path explosion are two long-standing problems which may affect the effectiveness and performance of symbolic execution on complex programs. The intrinsic challenge is to achieve a sufficient understanding of the program context to construct a set of execution environments which can guide the selection of symbolic states. In this paper, we propose a novel program-context-guided symbolic execution framework LangSym based on program’s instruction/user manual. Leveraging the capabilities of natural language understanding and code generation in large language models (LLMs), LangSym can automatically extract the knowledge related to the functionality of the program, and generate adequate test cases and the corresponding environments as the prior knowledge for symbolic execution. We instantiate LangSym in KLEE, a widely adopted symbolic execution engine, to build a pipeline that could automatically leverage LLMs to boost the symbolic execution. We evaluate LangSym on almost all GNU Coreutils programs and considerable large-scale programs, showing that LangSym outperforms the existing strategies in KLEE with at least a 10% increase for line coverage.

Regression errors of Deep Neural Network (DNN) models refer to the case that predictions were correct by the old-version model but wrong by the new-version model. They frequently occur when upgrading DNN models in production systems, causing disproportionate user experience degradation. In this paper, we propose a lightweight regression error reduction approach with two goals: 1) requiring no model retraining and even data, and 2) not sacrificing the accuracy. The proposed approach is built upon the key insight rooted in the unmanaged model uncertainty, which is intrinsic to DNN models, but has not been thoroughly explored especially in the context of quality assurance of DNN models. Specifically, we propose a simple yet effective ensemble strategy that estimates and aligns the two models' uncertainty. We show that a Pareto improvement that reduces the regression errors without compromising the overall accuracy can be guaranteed in theory and largely achieved in practice. Comprehensive experiments with various representative models and datasets confirm that our approaches significantly outperform the state-of-the-art alternatives.

Machine learning methods have achieved great success in many software engineering tasks. However, as a data-driven paradigm, how would the data quality impact the effectiveness of these methods remains largely unexplored. In this paper, we explore this problem under the context of just-in-time obsolete comment detection. Specifically, we first conduct data cleaning on the existing benchmark dataset, and empirically observe that with only 0.22% label corrections and even 15.0% fewer data, the existing obsolete comment detection approaches can achieve up to 10.7% relative accuracy improvement. To further mitigate the data quality issues, we propose an adversarial learning framework to simultaneously estimate the data quality and make the final predictions. Experimental evaluations show that this adversarial learning framework can further improve the relative accuracy by up to 18.1% compared to the state-of-the-art method. Although our current results are from the obsolete comment detection problem, we believe that the proposed two-phase solution, which handles the data quality issues through both the data aspect and the algorithm aspect, is also generalizable and applicable to other machine learning based software engineering tasks.

Neuro-symbolic systems combine the abilities of neural perception and logical reasoning. However, end-to-end learning of neuro-symbolic systems is still an unsolved challenge. This paper proposes a natural framework that fuses neural network training, symbol grounding, and logical constraint synthesis into a coherent and efficient end-to-end learning process. The capability of this framework comes from the improved interactions between the neural and the symbolic parts of the system in both the training and inference stages. Technically, to bridge the gap between the continuous neural network and the discrete logical constraint, we introduce a difference-of-convex programming technique to relax the logical constraints while maintaining their precision. We also employ cardinality constraints as the language for logical constraint learning and incorporate a trust region method to avoid the degeneracy of logical constraint in learning. Both theoretical analyses and empirical evaluations substantiate the effectiveness of the proposed framework.

Effective quality assurance methods for autonomous driving systems ADS have attracted growing interests recently. In this paper, we report a new testing platform ADEPT, aiming to provide practically realistic and comprehensive testing facilities for DNN-based ADS. ADEPT is based on the virtual simulator CARLA and provides numerous testing facilities such as scene construction, ADS importation, test execution and recording, etc. In particular, ADEPT features two distinguished test scenario generation strategies designed for autonomous driving. First, we make use of real-life accident reports from which we leverage natural language processing to fabricate abundant driving scenarios. Second, we synthesize physically-robust adversarial attacks by taking the feedback of ADS into consideration and thus are able to generate closed-loop test scenarios. The experiments confirm the efficacy of the platform.

Executing a family of Deep Neural Networks (DNNs) training jobs on the same or similar datasets in parallel is typical in current deep learning scenarios. It is time-consuming and resource-intensive because each job repetitively prepares (i.e., loads and preprocesses) the data independently, causing redundant consumption of I/O and computations. Although the page cache or a centralized cache component can alleviate the redundancies by reusing the data prep work, each job's data sampled uniformly at random presents a low sampling locality in the shared dataset that causes the heavy cache thrashing. Prior work tries to solve the problem by enforcing all training jobs iterating over the dataset in the same order and requesting each data in lockstep, leading to strong constraints: all jobs must have the same dataset and run simultaneously. In this paper, we propose a dependent sampling algorithm (DSA) and domain-specific cache policy to relax the constraints. Besides, a novel tree data structure is designed to efficiently implement DSA. Based on the proposed technologies, we implemented a prototype system, named Joader, which can share data prep work as long as the datasets share partially. We evaluate the proposed Joader in practical scenarios, showing a greater versatility and superiority over training speed improvement (up to 500% in ResNet18).

Deep learning (DL) applications are becoming increasingly popular. Their reliabilities largely depend on the performance of DL models integrated in these applications as a central classifying module. Traditional techniques need to retrain the models or rebuild and redeploy the applications for coping with unexpected conditions beyond the models' handling capabilities. In this paper, we take a fault tolerance approach, Dissector, to distinguishing those inputs that represent unexpected conditions (beyond-inputs) from normal inputs that are still within the models' handling capabilities (within-inputs), thus keeping the applications still function with expected reliabilities. The key insight of Dissector is that a DL model should interpret a within-input with increasing confidence, while a beyond-input would probably cause confused guesses in the prediction process. Dissector works in an application-specific way, adaptive to DL models used in applications, and extremely efficiently, scalable to large-size datasets from complex scenarios. The experimental evaluation shows that Dissector outperformed state-of-the-art techniques in the effectiveness (AUC: avg. 0.8935 and up to 0.9894) and efficiency (runtime overhead: only 3.3--5.8 milliseconds). Besides, it also exhibited encouraging usefulness in defensing against adversarial inputs (AUC: avg. 0.9983) and improving a DL model's actual accuracy in use (up to 16% for CIFAR-100 and 20% for ImageNet).

Deep Neural Network models are integrated as parts of many real-world software applications. Due to the huge model size and complex computation, distributed deep learning (DDL) framework aims to provide a high-quality cluster scheduler to manage DDL training jobs from both resource allocation and job scheduling. However, existing schedulers either allocate a fixed amount of resources, or lack the control over task placement, which lead less efficient training. In this paper, we propose DeepSys, a GPU cluster scheduler tailored for DDL jobs. For single model, DeepSys builds a speed model to predict accurate training speed, and a memory model for high-quality resource utilization. For job scheduling, DeepSys considers resource allocation and task placement to provide efficient job scheduling in cluster. Experiments implemented on Kubernetes in two clusters show the advantage to the compared methods by 20% - 25% and 10% - 15% on average job completion time and makespan, respectively.

Trained DNN models are increasingly adopted as integral parts of software systems, but they often perform deficiently in the field. A particularly damaging problem is that DNN models often give false predictions with high confidence, due to the unavoidable slight divergences between operation data and training data. To minimize the loss caused by inaccurate confidence, operational calibration, i.e., calibrating the confidence function of a DNN classifier against its operation domain, becomes a necessary debugging step in the engineering of the whole system.

Deep neural networks (DNNs) have achieved impressive performance in many difficult tasks. However, DNN models are essentially uninterpretable to humans, and unfortunately prone to adversarial attacks, which hinders their adoption in security and safety-critical scenarios. The robustness of a DNN model, which measures its stableness against adversarial attacks, becomes an important topic in both the machine learning and the software engineering communities. Analytical evaluation of DNN robustness is difficult due to the high-dimensionality of inputs, the huge amount of parameters, and the nonlinear network structure. In practice, the degree of robustness of DNNs is empirically approximated with adversarial searching, which is computationally expensive and cannot be applied in resource constrained settings such as embedded computing. In this paper, we propose to predict the robustness of a DNN model for each input with another DNN model, which takes the output of neurons of the former model as input. We train a regression model to encode the connections between output of the penultimate layer of a DNN model and its robustness. With this trained model, the robustness for an input can be predicted instantaneously. Experiments with MNIST and CIFAR10 datasets and LeNet, VGG and ResNet DNN models were conducted to evaluate the efficacy of the proposed approach. The results indicated that our approach achieved 0.05-0.21 mean absolute errors and significantly outperformed confidence and surprise adequacy-based approaches.

With the increasing adoption of Deep Neural Network (DNN) models as integral parts of software systems, efficient operational testing of DNNs is much in demand to ensure these models' actual performance in field conditions. A challenge is that the testing often needs to produce precise results with a very limited budget for labeling data collected in field.

Understanding the relevance between the Free/Libra Open Source Software projects is important for developers to perform code and design reuse, discover and develop new features, keep their projects up-to-date, and etc. However, it is challenging to perform relevance ratings between the FLOSS projects mainly because: 1) beyond simple code similarity, there are complex aspects considered when measuring the relevance; and 2) the prohibitive large amount of FLOSS projects available. To address the problem, in this paper, we propose a method to measure and further predict the relevance ratings between FLOSS projects. Our method uses topic features extracted by the LDA topic model to describe the characteristics of a project. By using the topic features, multiple aspects of FLOSS projects such as the application domain, technology used, and programming language are extracted and further used to measure and predict their relevance ratings. Based on the topic features, our method uses matrix factorization to leverage the partially known relevance ratings between the projects to learn the mapping between different topic features to the relevance ratings. Finally, our method combines the topic modeling and matrix factorization technologies to predict the relevance ratings between software projects without human intervention, which is scalable to a large amount of projects. We evaluate the performance of the proposed method by applying our topic extraction and relevance modeling methods using 300 projects from GitHub. The result of topic extraction experiment shows that, for topic modeling, our LDA-based approach achieves the highest hit rate of 98.3% and the highest average accuracy of 29.8%. And the relevance modeling experiment shows that our relevance modeling approach achieves the minimum average predict error of 0.093, suggesting the effectiveness of applying the proposed method on real-world data sets.

Latent factor models have become a prevalent method in recommender systems, to predict users' preference on items based on the historical user feedback. Most of the existing methods, explicitly or implicitly, are built upon the first-order rating distance principle, which aims to minimize the difference between the estimated and real ratings. In this paper, we generalize such first-order rating distance principle and propose a new latent factor model (HoORaYs) for recommender systems. The core idea of the proposed method is to explore high-order rating distance, which aims to minimize not only (i) the difference between the estimated and real ratings of the same (user, item) pair (i.e., the first-order rating distance), but also (ii) the difference between the estimated and real rating difference of the same user across different items (i.e., the second-order rating distance). We formulate it as a regularized optimization problem, and propose an effective and scalable algorithm to solve it. Our analysis from the geometry and Bayesian perspectives indicate that by exploring the high-order rating distance, it helps to reduce the variance of the estimator, which in turns leads to better generalization performance (e.g., smaller prediction error). We evaluate the proposed method on four real-world data sets, two with explicit user feedback and the other two with implicit user feedback. Experimental results show that the proposed method consistently outperforms the state-of-the-art methods in terms of the prediction accuracy.

In recent years, recommender system is one of indispensable components in many e-commerce websites. One of the major challenges that largely remains open is the cold-start problem, which can be viewed as a barrier that keeps the cold-start users/items away from the existing ones. In this paper, we aim to break through this barrier for cold-start users/items by the assistance of existing ones. In particular, inspired by the classic Elo Rating System, which has been widely adopted in chess tournaments, we propose a novel rating comparison strategy (RAPARE) to learn the latent profiles of cold-start users/items. The centerpiece of our RAPARE is to provide a fine-grained calibration on the latent profiles of cold-start users/items by exploring the differences between cold-start and existing users/items. As a generic strategy, our proposed strategy can be instantiated into existing methods in recommender systems. To reveal the capability of RAPARE strategy, we instantiate our strategy on two prevalent methods in recommender systems, i.e., the matrix factorization based and neighborhood based collaborative filtering. Experimental evaluations on five real data sets validate the superiority of our approach over the existing methods in cold-start scenario.

With the rapid advances of wearable sensors and wireless networks. There is a growing research interest in building Body Area Sensor Network (BSN) systems to support applications such as human activity recognition and daily health monitoring. The main challenges of building a BSN system include: First, the wearable sensor nodes are highly constrained in resources; Second, the sensor nodes are prong to failures; Third, the applications requires the sensors to work with high sampling rate which results in the heavy load on each sensor node. Beside the technical challenges, the usability of the system is also a critical issue when deployed into the the real-world environment.