Android applications bring many challenges for testing due to the notorious fragmentation issues and their diverse usage environments. Even though classic crowdsourced testing can improve the usability and reliability of Android applications, it often requires many efforts and resources. Meanwhile, on the other hand, automated analysis techniques provide highly efficient testing solutions for Android applications, which can use rich test resources and computing power to save manual test costs. Therefore, to leverage the advantages of both manual and automated testing, in this paper, we propose an approach that combines classic static program analysis and crowdsourced testing to implement human-machine collaborative testing for Android applications. We first employ the static analysis technique to model the possible GUI window sequences into window transition graphs (WTG). Then, we use a depth-first search algorithm to traverse the WTG and generate the testing task lists. In the testing process, we recommend these tasks for testers and adjust the task priority based on user feedback to optimize collaborative testing efficiency and effectiveness. To validate our approach, we experiment it with 9 open-source Android applications. The experimental results show that the path coverage rate of human-machine collaborative testing is 18.2% higher than that of classic crowdsourced testing and can reduce duplicate bug reports by 17.0 percent and improve testing efficiency.

Abstract Recent research works have proven that duplicate bug reports can provide helpful information to assist developers in software tasks such as fault localization and program fixing, while thoroughly reading duplicate bug reports is time‐consuming and inefficient. Summarization is a possible solution for gaining essential information quickly. However, there are many challenges when applying existing summarizing techniques on duplicate bug reports. Duplicate bug reports describe the same problem from different views and vary in quality, content, and writing style. Moreover, the code snippet understanding and the semantic gap between natural and programming languages make the summary generation even more difficult. Thus, in this paper, we want to investigate whether the state‐of‐the‐art summarization approaches can overcome the resistance and generate an effective summary for duplicate bug reports. We collected more than 8,000 groups of duplicate reports from GitHub and labeled 60 groups with 149 reports manually for the evaluation. Results showed that although the existing summarization approaches can work on duplicate bug reports, there are significant differences between them when it comes to code snippet summarization. Moreover, several methods can be very sluggish for summarizing long bug reports. Our study provides insights and guidelines for choosing proper summarization approaches in different scenarios.

Crowdsourced testing has been widely adopted to improve the quality of various software products. Crowdsourced workers typically perform testing tasks and report their experiences through test reports. While the crowdsourced test reports provide feedbacks from real usage scenarios, inspecting such a large number of reports becomes a time-consuming yet inevitable task. To improve the efficiency of this task, existing widely used issue-tracking systems, such as JIRA, Bugzilla, and Mantis, have provided keyword-search-based methods to assist users in identifying duplicate test reports. However, on mobile devices (such as mobile phones), where the crowdsourced test reports often contain insufficient text descriptions but instead rich screenshots, these text-analysis-based methods become less effective because the data has fundamentally changed. In this paper, instead of focusing on only detecting duplicates based on textual descriptions, we present CTRAS: a novel approach to leveraging duplicates to enrich the content of bug descriptions and improve the efficiency of inspecting these reports. CTRAS is capable of automatically aggregating duplicates based on both textual information and screenshots, and further summarizes the duplicate test reports into a comprehensive and comprehensible report. To validate CTRAS, we conducted quantitative studies using more than 5000 test reports, collected from 12 industrial crowdsourced projects. The experimental results reveal that CTRAS can reach an accuracy of 0.87, on average, regarding automatically detecting duplicate reports, and it outperforms the classic Max-Coverage-based and MMR summarization methods under Jensen Shannon divergence metric. Moreover, we conducted a task-based user study with 30 participants, whose result indicates that CTRAS can save nearly 30% time cost on average without loss of correctness.

In this paper, we present CTRAS, a tool for automatically aggregating and summarizing duplicate crowdsourced test reports on the fly. CTRAS can automatically detect duplicates based on both textual information and the screenshots, and further aggregates and summarizes the duplicate test reports. CTRAS provides end users with a comprehensive and comprehensible understanding of all duplicates by identifying the main topics across the group of aggregated test reports and highlighting supplementary topics that are mentioned in subgroups of test reports. Also, it provides the classic tool of issue tracking systems, such as the project-report dashboard and keyword searching, and automates their classic functionalities, such as bug triaging and best fixer recommendation, to assist end users in managing and diagnosing test reports. Video: https://youtu.be/PNP10gKIPFs