In the context of testing, descriptive test names are desirable because they document the purpose of tests and facilitate comprehension tasks during maintenance. Unfortunately, prior work has shown that tests often do not have descriptive names. To address this limitation, techniques have been developed to automatically generate descriptive names. However, they often generated names that are invalid or do not meet developer approval. To help address these limitations, we present a novel approach to extract the attributes of a given test that make it unique among its siblings. Because such attributes often serve as the basis for descriptive names, identifying them is an important first step towards improving test name generation approaches. To evaluate the approach, we created a prototype implementation for JUnit tests and compared its output with human judgment. The results of the evaluation demonstrate that the attributes identified by the approach are consistent with human judgment and are likely to be useful for future name generation techniques.

Work in Mobile App Energy Consumption (MAEC) has drawn participants from a broad range of communities such as systems, networking, hardware, testing, analysis and design. This diversity has enriched and informed the research from many different angles. For example, knowledge about the physical properties of batteries (e.g., that their performance is temperature dependent) is necessary to control for con-founding variables during experiments. However, it has also led to a confusing and conflicting mix of terms, names, and expressions as researchers from different domains each attempt to apply their existing terminology or invent new terms to describe various kinds of energy related issues.

Because tests are important to the development process, developers need to know when a test suite is missing tests. Missing tests - tests that should be included in a test suite but are not - reduce the utility that developers can derive from a test suite. Currently, developers find missing tests by using coverage information such as line coverage or mutation coverage. However, coverage metrics are limited in their ability to reveal missing tests and show only what code needs to be tested, not how to test it. We present a method for finding missing tests that addresses the shortcomings of coverage metrics based on the fact that similar code entities are often tested in the same way. We are able to find what code is missing tests by identifying code entities which are not tested in the same way as other similar entities. We then show how a code entity with a missing test should be tested by leveraging the tests written for those similar entities. Our results show that our approach offers several benefits over a coverage-based approach and is able to find missing tests in a range of software projects while generating few erroneous identifications of missing tests.

Abstract With the increasing use of battery‐powered devices comes the need to test mobile applications for energy consumption and energy issues. Unfortunately, energy testing is expensive because it is a manual, labor‐intensive process that often requires multiple, separate, energy‐measuring devices to collect energy usage data. The high costs of energy testing can negatively affect the planning process of application evolution. For example, developers might be limited in the number of changes they can include in a release because they must conservatively plan to conduct energy testing after each change. In this paper, we present a new approach to provide developers with feedback on executing/skipping energy tests for proposed code changes. Our technique leverages change impact analysis and precomputed API energy usage information. More specifically, for a proposed change, the technique predicts whether energy testing will be required, and if so, which energy tests will need to be run. Such information may allow developers to avoid spending unnecessary time for energy testing and develop an effective application evolution timeline. To investigate the feasibility of our technique, we implemented a prototype for Android applications and conducted three case studies at different granularity levels on 10 Android applications.

Energy consumption have become an important non-functional requirement for applications running on battery powered devices through data centers. Despite the increased interest on detecting and understanding what causes an application to be energy inefficient, few works focus on helping developers to automatically make their applications more energy efficient based on developers’ design and implementation decisions. This paper explores how search strategies based on genetic algorithms can help developers automatically find an energy efficient version of an application based on transformations corresponding to developers’ high level decisions (e.g., selecting API implementations). Our results show how different search strategies can help to improve the energy efficiency for nine Java applications. These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

The energy consumption of software is an increasing concern as the use of mobile applications, embedded systems, and data center-based services expands. While research in green software engineering is correspondingly increasing, little is known about the current practices and perspectives of software engineers in the field. This paper describes the first empirical study of how practitioners think about energy when they write requirements, design, construct, test, and maintain their software. We report findings from a quantitative, targeted survey of 464 practitioners from ABB, Google, IBM, and Microsoft, which was motivated by and supported with qualitative data from 18 in-depth interviews with Microsoft employees. The major findings and implications from the collected data contextualize existing green software engineering research and suggest directions for researchers aiming to develop strategies and tools to help practitioners improve the energy usage of their applications.

Because of the numerous benefits of tests, developers often wish their applications had more tests. Unfortunately, it is challenging to determine what new tests to add in order to improve the quality of the test suite. A number of approaches, including numerous coverage criteria, have been proposed by the research community to help developers focus their limited testing resources. However, coverage criteria often fall short of this goal because achieving 100% coverage is often infeasible, necessitating the difficult process of determining if a piece of uncovered code is actually executable, and the criteria do not take into account how the code is covered. In this paper, we propose a new approach for interpreting coverage information, based on the concepts of direct coverage and indirect coverage, that address these limitations. We also presents the results of an empirical study of 17 applications that demonstrate that indirectly covered code is common in real world software, faults in indirectly covered code are significantly less likely to be detected than faults located in directly covered code, and indirectly covered code typically clusters at the method level. This means that identifying indirectly covered methods can be effective at helping testers improve the quality of their test suites by directing them to insufficiently tested code.

During maintenance, developers often need to understand the purpose of a test. One of the most potentially useful sources of information for understanding a test is its name. Ideally, test names are descriptive in that they accurately summarize both the scenario and the expected outcome of the test. Despite the benefits of being descriptive, test names often fall short of this goal. In this paper we present a new approach for automatically generating descriptive names for existing test bodies. Using a combination of natural-language program analysis and text generation, the technique creates names that summarize the test's scenario and the expected outcome. The results of our evaluation show that, (1) compared to alternative approaches, the names generated by our technique are significantly more similar to human-generated names and are nearly always preferred by developers, (2) the names generated by our technique are preferred over or are equivalent to the original test names in 83% of cases, and (3) our technique is several orders of magnitude faster than manually writing test names.

Abstract Software piracy is an important concern for application developers. Such concerns are especially relevant in mobile application development, where piracy rates can be greater than 90%. The most common approach used by mobile developers to prevent piracy is code obfuscation. However, the decision to apply such transformations is currently made without regard to the impacts of obfuscations on another area of increasing concern for mobile application developers, energy usage. Because both software piracy and battery life are important concerns, mobile application developers must strike a balance between protecting their applications and preserving the battery lives of their users' devices. To help them make such choices, we conducted an empirical study of the effects of 18 code obfuscations on the amount of energy consumed by executing a total of 21 usage scenarios spread across 11 Android applications on four different mobile phone platforms. The results of the study indicate that, while obfuscations can have a statistically significant impact on energy usage and are more likely to increase energy usage than to decrease energy usage, the magnitudes of such impacts are unlikely to be meaningful to mobile application users. Copyright © 2016 John Wiley & Sons, Ltd.

Existing specification-based testing techniques require specifications that either do not exist or are too difficult to create. As a result, they often fall short of their goal of helping developers test expected behaviors. In this paper we present a novel, natural language-based approach that exploits the descriptive nature of test names to generate test templates. Similar to how modern IDEs simplify development by providing templates for common constructs such as loops, test templates can save time and lower the cognitive barrier for writing tests. The results of our evaluation show that the approach is feasible: despite the difficulty of the task, when test names contain a sufficient amount of information, the approach's accuracy is over 80% when parsing the relevant information from the test name and generating the template.

Context: Code refactoring's benefits to understandability, maintainability and extensibility are well known enough that automated support for refactoring is now common in IDEs. However, the decision to apply such transformations is currently performed without regard to the impacts of the refactorings on energy consumption. This is primarily due to a lack of information and tools to provide such relevant information to developers. Unfortunately, concerns about energy efficiency are rapidly becoming a high priority concern in many environments, including embedded systems, laptops, mobile devices, and data centers.

Reducing the energy usage of software is becoming more important in many environments, in particular, battery-powered mobile devices, embedded systems and data centers. Recent empirical studies indicate that software engineers can support the goal of reducing energy usage by making design and implementation decisions in ways that take into consideration how such decisions impact the energy usage of an application. However, the large number of possible choices and the lack of feedback and information available to software engineers necessitates some form of automated decision-making support. This paper describes the first known automated support for systematically optimizing the energy usage of applications by making code-level changes. It is effective at reducing energy usage while freeing developers from needing to deal with the low-level, tedious tasks of applying changes and monitoring the resulting impacts to the energy usage of their application. We present a general framework, SEEDS, as well as an instantiation of the framework that automatically optimizes Java applications by selecting the most energy-efficient library implementations for Java's Collections API. Our empirical evaluation of the framework and instantiation show that it is possible to improve the energy usage of an application in a fully automated manner for a reasonable cost.

Software piracy is an important concern for application developers. Such concerns are especially relevant in mobile application development, where piracy rates can approach 90%. The most commonly used approach by mobile developers for preventing piracy is code obfuscation. However, the decision to apply such transformations is currently made without regard to the impacts of obfuscations on another area of increasing concern for mobile application developers: energy consumption. Because both software piracy and battery life are important concerns, mobile application developers must strike a balance between protecting their applications and preserving the battery lives of their users' devices. To help them make such choices, we conducted an empirical study of the effects of 18 code obfuscations on the amount of energy consumed by executing a total of 15 usage scenarios spread across 11 Android applications. The results of the study indicate that, while obfuscations can have a statistically significant impact on energy usage and are significantly more likely to increase energy usage than to decrease energy usage, the magnitudes of such impacts are unlikely to impact mobile application users.

In situ testing techniques have become an important means of ensuring the reliability of embedded systems after they are deployed in the field. However, these techniques do not help testers optimize the energy consumption of their in situ test suites, which can needlessly waste the limited battery power of these systems. In this work, we extend prior techniques for test suite minimization in such a way as to allow testers to generate energy-efficient, minimized test suites with only minimal modifications to their existing work flow. We perform an extensive empirical evaluation of our approach using the test suites provided for real world applications. The results of the evaluation show that our technique is effective at generating, in less than one second, test suites that consume up to 95% less energy while maintaining coverage of the testing requirements.

Unintended information leakage is one of the most common and severe problems facing modern applications. To help developers detect information leaks before they can be leveraged by attackers, we present a new static analysis-based technique for detecting a specific type of information leak: information leaks via exceptions. Because it focuses on a specific type of leak, the technique is able to be efficient, effective, and easy to use, qualities that are often lacking in more general techniques. We implemented our technique in a prototype tool, UDLD, and performed an extensive empirical evaluation using 19 real web applications. The results of the evaluation show that UDLD is both efficient and effective at detecting unsafe information leaks via exceptions; for the subjects that we considered, UDLD is the fastest among several alternative tools. Moreover, it reported more true leaks than existing state-of-the-art tools with no known false negatives and no false positives.

Writing oracles is challenging. As a result, developers often create oracles that check too little, resulting in tests that are unable to detect failures, or check too much, resulting in tests that are brittle and difficult to maintain. In this paper we present a new technique for automatically analyzing test oracles. The technique is based on dynamic tainting and detects both brittle assertions—assertions that depend on values that are derived from uncontrolled inputs—and unused inputs—inputs provided by the test that are not checked by an assertion. We also presented OraclePolish, an implementation of the technique that can analyze tests that are written in Java and use the JUnit testing framework. Using OraclePolish, we conducted an empirical evaluation of more than 4000 real test cases. The results of the evaluation show that OraclePolish is effective; it detected 164 tests that contain brittle assertions and 1618 tests that have unused inputs. In addition, the results also demonstrate that the costs associated with using the technique are reasonable.

Programs written in languages allowing direct access to memory through pointers often contain memory-related faults, which cause nondeterministic failures and security vulnerabilities. We present a new dynamic tainting technique to detect illegal memory accesses. When memory is allocated, at runtime, we taint both the memory and the corresponding pointer using the same taint mark. Taint marks are then propagated and checked every time a memory address m is accessed through a pointer p; if the associated taint marks differ, an illegal access is reported. To allow always-on checking using a low overhead, hardware-assisted implementation, we make several key technical decisions. We use a configurable, low number of reusable taint marks instead of a unique mark for each allocated area of memory, reducing the performance overhead without losing the ability to target most memory-related faults. We also define the technique at the binary level, which helps handle applications using third-party libraries whose source code is unavailable. We created a software-only prototype of our technique and simulated a hardware-assisted implementation. Our results show that 1) it identifies a large class of memory-related faults, even when using only two unique taint marks, and 2) a hardware-assisted implementation can achieve performance overheads in single-digit percentages.

Privacy and security concerns have adversely affected the usefulness of many types of techniques that leverage information gathered from deployed applications. To address this issue, we present an approach for automatically anonymizing failure-inducing inputs that builds on a previously developed technique. Given an input I that causes a failure f, our approach generates an anonymized input I′ that is different from I but still causes f. I′ can thus be sent to developers to enable them to debug f without having to know I. We implemented our approach in a prototype tool, camouflage, and performed an extensive empirical evaluation where we applied camouflage to a large set of failure-inducing inputs for several real applications. The results of the evaluation are promising, as they show that camouflage is both practical and effective at generating anonymized inputs; for the inputs that we considered, I and I′ shared no sensitive information. The results also show that our approach can outperform the general technique it extends.

Most existing leak detection techniques for C and C++ applications only detect the existence of memory leaks. They do not provide any help for fixing the underlying memory management errors. In this paper, we present a new technique that not only detects leaks, but also points developers to the locations where the underlying errors may be fixed. Our technique tracks pointers to dynamically-allocated areas of memory and, for each memory area, records several pieces of relevant information. This information is used to identify the locations in an execution where memory leaks occur. To investigate our technique's feasibility and usefulness, we developed a prototype tool called LEAKPOINT and used it to perform an empirical evaluation. The results of this evaluation show that LEAKPOINT detects at least as many leaks as existing tools, reports zero false positives, and, most importantly, can be effective at helping developers fix the underlying memory management errors.

Most existing automated debugging techniques focus on reducing the amount of code to be inspected and tend to ignore an important component of software failures: the inputs that cause the failure to manifest. In this paper, we present a new technique based on dynamic tainting for automatically identifying subsets of a program's inputs that are relevant to a failure. The technique (1) marks program inputs when they enter the application, (2) tracks them as they propagate during execution, and (3) identifies, for an observed failure, the subset of inputs that are potentially relevant for debugging that failure. To investigate feasibility and usefulness of our technique, we created a prototype tool, PENUMBRA, and used it to evaluate our technique on several failures in real programs. Our results are promising, as they show that PENUMBRA can point developers to inputs that are actually relevant for investigating a failure and can be more practical than existing alternative approaches.

It is difficult to fully assess the quality of software in- house, outside the actual time and context in which it will execute after deployment. As a result, it is common for software to manifest field failures, failures that occur on user machines due to untested behavior. Field failures are typically difficult to recreate and investigate on developer platforms, and existing techniques based on crash reporting provide only limited support for this task. In this paper, we present a technique for recording, reproducing, and minimizing failing executions that enables and supports in- house debugging of field failures. We also present a tool that implements our technique and an empirical study that evaluates the technique on a widely used e-mail client.

Dynamic taint analysis is gaining momentum. Techniques based on dynamic tainting have been successfully used in the context of application security, and now their use is also being explored in different areas, such as program understanding, software testing, and debugging. Unfortunately, most existing approaches for dynamic tainting are defined in an ad-hoc manner, which makes it difficult to extend them, experiment with them, and adapt them to new contexts. Moreover, most existing approaches are focused on data-flow based tainting only and do not consider tainting due to control flow, which limits their applicability outside the security domain. To address these limitations and foster experimentation with dynamic tainting techniques, we defined and developed a general framework for dynamic tainting that (1) is highly flexible and customizable, (2) allows for performing both data-flow and control-flow based tainting conservatively, and (3) does not rely on any customized run-time system. We also present DYTAN, an implementation of our framework that works on x86 executables, and a set of preliminary studies that show how DYTAN can be used to implement different tainting-based approaches with limited effort. In the studies, we also show that DYTAN can be used on real software, by using FIREFOX as one of our subjects, and illustrate how the specific characteristics of the tainting approach used can affect efficiency and accuracy of the taint analysis, which further justifies the use of our framework to experiment with different variants of an approach.

Programs written in languages that provide direct access tomemory through pointers often contain memory-related faults, which may cause non-deterministic failures and even security vulnerabilities. In this paper, we present a new technique based on dynamic tainting for protecting programs from illegal memory accesses. When memory is allocated, at runtime, our technique taints both the memory and the corresponding pointer using the same taint mark. Taint marks are then suitably propagated while the program executes and are checked every time a memory address m is accessed through a pointer p; if the taint marks associated with mand p differ, the execution is stopped and the illegalaccess is reported. To allow for a low-overhead, hardware-assisted implementation of the approach, we make several key technical and engineering decisions in the definition of our technique. In particular, we use a configurable, low number of reusable taint marks instead of a unique mark for each area of memory allocated, which reduces the overhead of the approach without limiting its flexibility and ability to target most memory-related faults and attacks known to date. We also define the technique at the binary level, which lets us handle the (very) common case of applications that use third-party libraries whose source code is unavailable. To investigate the effectiveness and practicality of our approach, we implemented it for heap-allocated memory and performed a preliminary empirical study on a set of programs. Our results show that (1) our technique can identify a large class of memory-related faults, even when using only two unique taint marks, and (2)a hardware-assisted implementation of the technique could achieve overhead in the single digits

Producing reliable and robust software has become one of the most important software development concerns in recent years. Testing is a process by which software quality can be assured through the collection of information. While testing can improve software reliability, current tools typically are inflexible and have high over-heads, making it challenging to test large software projects. In this paper, we describe a new scalable and flexible framework for testing programs with a novel demand-driven approach based on execution paths to implement test coverage. This technique uses dynamic instrumentation on the binary code that can be inserted and removed on-the-fly to keep performance and memory overheads low. We describe and evaluate implementations of the framework for branch, node and defuse testing of Java programs. Experimental results for branch testing show that our approach has, on average, a 1.6 speed up over static instrumentation and also uses less memory.