Neural networks have become integral to many fields due to their exceptional performance. The open-source community has witnessed a rapid influx of neural network (NN) repositories with fast-paced iterations, making it crucial for practitioners to analyze their evolution to guide development and stay ahead of trends. While extensive research has explored traditional software evolution using Software Bill of Materials (SBOMs), these are ill-suited for NN software, which relies on pre-defined modules and pre-trained models (PTMs) with distinct component structures and reuse patterns. Conceptual AI Bills of Materials (AIBOMs) also lack practical implementations for large-scale evolutionary analysis. To fill this gap, we introduce the Neural Network Bill of Material (NNBOM), a comprehensive dataset construct tailored for NN software. We create a large-scale NNBOM database from 55,997 curated PyTorch GitHub repositories, cataloging their TPLs, PTMs, and modules. Leveraging this database, we conduct a comprehensive empirical study of neural network software evolution across software scale, component reuse, and inter-domain dependency, providing maintainers and developers with a holistic view of its long-term trends. Building on these findings, we develop two prototype applications, Multi repository Evolution Analyzer and Single repository Component Assessor and Recommender, to demonstrate the practical value of our analysis.

Neural networks are an emerging data-driven programming paradigm widely used in many areas. Unlike traditional software systems consisting of decomposable modules, a neural network is usually delivered as a monolithic package, raising challenges for some maintenance tasks such as model restructure and re-adaption. In this work, we propose DeepArc, a novel modularization method for neural networks, to reduce the cost of model maintenance tasks. Specifically, DeepArc decomposes a neural network into several consecutive modules, each of which encapsulates consecutive layers with similar semantics. The network modularization facilitates practical tasks such as refactoring the model to preserve existing features (e.g., model compression) and enhancing the model with new features (e.g., fitting new samples). The modularization and encapsulation allow us to restructure or retrain the model by only pruning and tuning a few localized neurons and layers. Our experiments show that (1) DeepArc can boost the runtime efficiency of the state-of-the-art model compression techniques by 14.8%; (2) compared to the traditional model retraining, DeepArc only needs to train less than 20% of the neurons on average to fit adversarial samples and repair under-performing models, leading to 32.85% faster training performance while achieving similar model prediction performance.

Identifying which vulnerabilities need to be prioritized is a long-term challenge in IT security, especially as the number of vulnerabilities grows. Faced with a large number of vulnerability reports, there is an urgent need for automated tools or models to assess the potential severity and exploitability of vulnerabilities. This will help security experts screen vulnerabilities that should be focused on. In this study, we aim to predict vulnerability severity and exploitability characteristics using only vulnerability descriptions. Some previous studies are based on traditional deep learning models, and their performance is relatively backward in the current era of pre-trained language models (PLMs). Therefore, we introduce a prompt learning method based on PLMs to predict vulnerability characteristics. The conventional fine-tuning PLMs method is difficult to make full use of the domain knowledge in PLMs and performs poorly with less training data. Unlike the fine-tuning paradigm, prompt learning imitates the pre-training process of PLM by reconstructing the task input and adding prompts, and uses the output of PLM itself as the prediction output. Combined with prompt ensembling and transfer learning, the performance of prompt learning in the above tasks is further improved. Our experiments show that prompt learning can make more effective use of the knowledge in PLMs. Compared with fine-tuning PLMs and other deep learning models, prompt learning based on BERT or RoBERTa achieves better performance in the above tasks. This advantage is more significant in predicting exploitability with few samples, which proves the ability of prompt learning in few-sample scenarios. In addition, prompt learning also shows the transferability between different tasks in the domain.