Augmented Reality (AR) technology opens up exciting possibilities in various fields, such as education, work guidance, shopping, communication, and gaming. However, users often encounter usability and user experience issues in current AR apps, often due to the imprecise placement of virtual objects. Detecting these inaccuracies is crucial for AR app testing, but automating the process is challenging due to its reliance on human perception and validation. This paper introduces VOPA (Virtual Object Placement Assessment), a novel approach that automatically identifies imprecise virtual object placements in real-world AR apps. VOPA involves instrumenting real-world AR apps to collect screenshots representing various object placement scenarios and their corresponding metadata under real-world scenes. The collected data are then labeled through crowdsourcing and used to train a hybrid neural network that identifies object placement errors. VOPA aims to enhance AR app testing by automating the assessment of virtual object placement quality and detecting imprecise instances. In our evaluation of a test set of 304 screenshots, VOPA achieved an accuracy of 99.34%, precision of 96.92% and recall of 100%. Furthermore, VOPA successfully identified 38 real-world object placement errors, including instances where objects were hovering between two surfaces or appearing embedded in the wall.

Virtual Reality (VR) is an emerging technique that has been applied to more and more areas such as gaming, remote conference, and education. Since VR user interface has very different characteristics compared with traditional graphic user interface (GUI), VR applications also require new testing techniques for quality assurance. Recently, some frameworks (e.g., VRTest) have been proposed to automate VR user interface testing by automatically controlling the player camera. However, their testing strategies are not able to address VR-specific testing challenges such as object occlusion and movement. In this paper, we propose a novel testing technique called VRGuide to explore VR scenes more efficiently. In particular, VRGuide adapts a computer geometry technique called Cut Extension to optimize the camera routes for covering all interact-able objects. We compared the testing strategy with VRTest on eight top VR software projects with scenes. The results show that VRGuide is able to achieve higher test coverage upon testing timeout in two of the projects, and achieve saturation coverage with averagely 31% less testing time than VRTest on the remaining six projects. Furthermore, VRGuide detected and reported four unknown bugs confirmed by developers, only one of which is also detected by VRTest.

While the emerging Augmented Reality (AR) technique allows a lot of new application opportunities, from education and communication to gaming, current augmented apps often have complaints about their usability and/or user experience due to placement errors of virtual objects. Therefore, identifying noticeable placement errors is an important goal in the testing of AR apps. However, placement errors can only be perceived by human beings and may need to be confirmed by multiple users, making automatic testing very challenging. In this paper, we propose PredART, a novel approach to predict human ratings of virtual object placements that can be used as test oracles in automated AR testing. PredART is based on automatic screenshot sampling, crowd sourcing, and a hybrid neural network for image regression. The evaluation on a test set of 480 screenshots shows that our approach can achieve an accuracy of 85.0% and a mean absolute error, mean squared error, and root mean squared error of 0.047, 0.008, and 0.091, respectively.