Virtual Reality (VR) is an emerging technique that provides a unique real-time experience for users. VR technologies have provided revolutionary user experiences in various scenarios (e.g., training, education, gaming, etc.). However, testing VR applications is challenging due to their nature which necessitates physical interactivity, and their reliance on specific hardware systems. Despite the recent advancements in VR technology and its usage scenarios, we still know little about VR application testing. To fill up this knowledge gap, we performed an empirical study on 314 open-source VR applications. Our analysis identified that 79% of the VR projects evaluated did not have any automatic tests, and for the VR projects that did, the median functional-method to test-method ratios were lower than those of other project types. Moreover, we uncovered tool support issues concerning the measurement of VR code coverage, and the assertion density results we were able to generate were relatively low, with an average of 17.63%. Finally, through a manual analysis of 370 test cases, we identified the different categories of test cases being used to validate VR application quality attributes. Furthermore, we extracted which of these categories are VR-attention, meaning that test writers need to pay special attention to VR characteristics when writing tests of these categories. We believe that our findings constitute a call to action for the VR development community to improve their automatic testing practices and provide directions for software engineering researchers to develop advanced techniques for automatic test case generation and test quality analysis for VR applications. Our replication package containing the dataset we used, software tools we developed, and the results we found, is accessible at ‍https://doi.org/10.6084/m9.figshare.19678938.

Performance efficiency and scalability are the major design goals for high performance computing (HPC) applications. However, it is challenging to achieve high efficiency and scalability for such applications due to complex underlying hardware architecture, inefficient algorithm implementation, suboptimal code generation by the compilers, inefficient parallelization, and so on. As a result, the HPC community spends a significant effort detecting and fixing the performance bugs frequently appearing in scientific applications. However, it is important to accumulate the experience to guide the scientific software engineering community to write performance-efficient code.In this paper, we investigate open-source HPC applications to categorize the performance bugs and their fixes and measure the programmer’s effort and experience to fix them. For this purpose, we first perform a large-scale empirical analysis on 1729 HPC performance commits collected from 23 real-world projects. Through our manual analysis, we identify 186 performance issues from these projects. Furthermore, we study the root cause of these performance issues and generate a performance bug taxonomy for HPC applications. Our analysis identifies that inefficient algorithm implementation (39.3%), inefficient code for target micro-architecture (31.2%), and missing parallelism and inefficient parallelization (14.5%) are the top three most prevalent categories of performance issues for HPC applications. Additionally, to understand how the performance bugs are fixed, we analyze the performance fix commits and categorize them into eight performance fix types. We further measure the developer’s efforts and expertise required to fix performance bugs. The analysis identified that performance bug fixes are complicated with a median patch size (LOC) of 35 lines and are mostly fixed by experienced developers.