Abstract Many test coverage metrics have been proposed to measure the deep neural network (DNN) testing effectiveness, including structural coverage and nonstructural coverage. These test coverage metrics are proposed based on the fundamental assumption: They are correlated with test effectiveness. However, the fundamental assumption is still not validated sufficiently and reasonably, which brings question on the usefulness of DNN test coverage. This paper conducted a revisiting study on the existing DNN test coverage from the test effectiveness perspective, to effectively validate the fundamental assumption. Here, we carefully considered the diversity of subjects, three test effectiveness criteria, and both typical and state‐of‐the‐art test coverage metrics. Different from all the existing studies that deliver negative conclusions on the usefulness of existing DNN test coverage, we identified some positive conclusions on their usefulness from the test effectiveness perspective. In particular, we found the complementary relationship between structural and nonstructural coverage and identified the practical usage scenarios and promising research directions for these existing test coverage metrics.

Test case prioritization (TCP) aims to schedule the execution order of test cases for earlier fault detection. A recent study has demonstrated that the information-retrieval-based (IR-based) TCP approaches achieve the state-of-the-art effectiveness. The current IR-based TCP approaches leverage lexical similarity between test cases and code changes to guide TCP while ignoring rich code semantics, which may limit their effectiveness to some degree. In this paper, we conduct the first study to explore whether code semantic information can further boost IR-based TCP. Here, we studied two types of code representation methods (i.e., general-purpose and task-associated models) and explored two modes of utilizing the code representation embeddings (i.e., unsupervised and supervised modes) for IR-based TCP. Our results demonstrate that incorporating code semantics through the supervised mode of code representation can achieve a 16.96% improvement in the efficiency of fault detection over the state-of-the-art IR-based TCP approach (which is based on lexical similarity).

For programmers, learning the usage of APIs (Application Programming Interfaces) of a software library is important yet difficult.API recommendation tools can help developers use APIs by recommending which APIs to be used next given the APIs that have been written.Traditionally, language models such as N-gram are applied to API recommendation.However, because the software libraries keep changing and new libraries keep emerging, new APIs are common.These new APIs can be seen as OOV (out of vocabulary) words and cannot be handled well by existing API recommendation approaches due to the lack of training data.In this paper, we propose APIRecX, the first cross-library API recommendation approach, which uses BPE to split each API call in each API sequence and pre-trains a GPTbased language model.It then recommends APIs by fine-tuning the pre-trained model.APIRecX can migrate the knowledge of existing libraries to a new library, and can recommend APIs that are previously regarded as OOV.We evaluate APIRecX on six libraries and the results confirm its effectiveness by comparing with two typical API recommendation approaches.