When Computer Science (CS) students try to use or extend open-source software (OSS) projects, they often encounter the common challenge of OSS failing to build on their local machines. Even though OSS often provides ready-to-build packages, subtle differences in local environment setups can lead to build issues, costing students tremendous time and effort in debugging. Despite the prevalence of build issues faced by CS students, there is a lack of studies exploring this topic. To investigate the build issues frequently encountered by CS students and explore methods to help them resolve these issues, we conducted a novel dual-phase study involving 330 build tasks among 55 CS students. Phase I characterized the build issues students faced, their resolution attempts, and the effectiveness of those attempts. Based on these findings, Phase II introduced an intervention method that emphasized key information (e.g., recommended programming language versions) to students. The study demonstrated the effectiveness of our intervention in improving build success rates. Our research will shed light on future directions in related areas, such as CS education on best practices for software builds and enhanced tool support to simplify the build process.

Open-source software (OSS) often needs to be built by roles who are not contributors. Despite the prevalence of build issues experienced by non-contributors, there is a lack of studies on this topic. This paper presents a study aimed at understanding the symptoms and causes of build issues experienced by non-contributors. The findings highlight certain build issues that are challenging to resolve and underscore the importance of understanding non-contributors' behavior. This work lays the foundation for further research aimed at enhancing the non-contributors' experience in dealing with build issues.

Modern software development extensively depends on existing libraries written by other developer teams from the same or a different organization. When a developer executes the software, the execution trace may go across the boundaries of multiple software products and create cross-project failures (CPFs). Existing studies show that a stand-alone executable failure report may enable the most effective communication, but creating such a report is often challenging due to the complicated files and dependencies interactions in the software ecosystems. In this paper, to solve the CPF report trilemma, we developed PExReport, which automatically creates stand-alone executable CPF reports. PExReport leverages build tools to prune source code and dependencies, and further analyzes the build process to create a pruned build environment for reproducing the CPF. We performed an evaluation on 74 software project issues with 198 CPFs, and the evaluation results show that PExReport can create executable CPF reports for 184 out of 198 test failures in our dataset, with an average reduction of 72.97% on source classes and the classes in internal JARs.

Modern Java software development extensively depends on existing libraries written by other developer teams from the same or a different organization. When a developer executes the test, the execution trace may go across the boundaries of multiple dependencies and create cross-project failures (CPFs). A readable, executable, and concise CPF report may enable the most effective communication, but creating such a report is often challenging in Java ecosystems. We developed PExReport-Maven to automatically create the ideal CPF reports in the Maven build system. PExReport-Maven leverages the Maven build system to prune source code, dependencies, and the build environment to create a concise stand-alone executable CPF reproduction package from the original CPF project. The reproduction package includes the source code, dependencies, and build environment necessary to reproduce the CPF, making it an ideal CPF report. We performed an evaluation on 74 software project issues with 198 cross-project failures, and the evaluation results show that PExReport can create pruned reproduction packages for 184 out of the 198 test failures in our dataset, with an average reduction of 72.97% in Java classes. A future study will be conducted based on user feedback from using this tool to report real-world CPFs. PExReport-Maven is publicly available at https://github.com/wereHuang/PExReport-Maven. The tool demo is available on the PExReport website: https://sites.google.com/view/pexreport/home.

With the growing popularity of cloud gaming and cloud virtual reality (VR), interactive 3D applications have become a major class of workloads for the cloud. However, despite their growing importance, there is limited public research on how to design cloud systems to efficiently support these applications due to the lack of an open and reliable research infrastructure, including benchmarks and performance analysis tools. The challenges of generating human-like inputs under various system/application nondeterminism and dissecting the performance of complex graphics systems make it very difficult to design such an infrastructure. In this paper, we present the design of a novel research infrastructure, Pictor, for cloud 3D applications and systems. Pictor employs AI to mimic human interactions with complex 3D applications. It can also track the processing of user inputs to provide in-depth performance measurements for the complex software and hardware stack used for cloud 3D-graphics rendering. With Pictor, we designed a benchmark suite with six interactive 3D applications. Performance analyses were conducted with these benchmarks, which show that cloud system designs, including both system software and hardware designs, are crucial to the performance of cloud 3D applications. The analyses also show that energy consumption can be reduced by at least 37% when two 3D applications share a could server. To demonstrate the effectiveness of Pictor, we also implemented two optimizations to address two performance bottlenecks discovered in a state-of-the-art cloud 3D-graphics rendering system. These two optimizations improved the frame rate by 57.7% on average.

The paradigm shift of deploying applications to the cloud has introduced both opportunities and challenges. Although clouds use elasticity to scale resource usage at runtime to help meet an application's performance requirements, developers are still challenged by unpredictable performance, little control of execution environment, and differences among cloud service providers, all while being charged for their cloud usages. Application performance stability is particularly affected by multi-tenancy in which the hardware is shared among varying applications and virtual machines. Developers porting their applications need to meet performance requirements, but testing on the cloud under the effects of performance uncertainty is difficult and expensive, due to high cloud usage costs. This paper presents a first approach to testing an application with typical inputs for how its performance will be affected by performance uncertainty, without incurring undue costs of brute force testing in the cloud. We specify cloud uncertainty testing criteria, design a test-based strategy to characterize the black box cloud's performance distributions using these testing criteria, and support execution of tests to characterize the resource usage and cloud baseline performance of the application to be deployed. Importantly, we developed a smart test oracle that estimates the application's performance with certain confidence levels using the above characterization test results and determines whether it will meet its performance requirements. We evaluated our testing approach on both the Chameleon cloud and Amazon web services; results indicate that this testing strategy shows promise as a cost-effective approach to test for performance effects of cloud uncertainty when porting an application to the cloud.