Automatic machine learning, or AutoML, holds the promise of truly democratizing the use of machine learning (ML), by substantially automating the work of data scientists. However, the huge combinatorial search space of candidate pipelines means that current AutoML techniques, generate sub-optimal pipelines, or none at all, especially on large, complex datasets. In this work we propose an AutoML technique SapientML, that can learn from a corpus of existing datasets and their human-written pipelines, and efficiently generate a high-quality pipeline for a predictive task on a new dataset. To combat the search space explosion of AutoML, SapientML employs a novel divide-and-conquer strategy realized as a three-stage program synthesis approach, that reasons on successively smaller search spaces. The first stage uses meta-learning to predict a set of plausible ML components to constitute a pipeline. In the second stage, this is then refined into a small pool of viable concrete pipelines using a pipeline dataflow model derived from the corpus. Dynamically evaluating these few pipelines, in the third stage, provides the best solution. We instantiate SapientML as part of a fully automated tool-chain that creates a cleaned, labeled learning corpus by mining Kaggle, learns from it, and uses the learned models to then synthesize pipelines for new predictive tasks. We have created a training corpus of 1,094 pipelines spanning 170 datasets, and evaluated SapientML on a set of 41 benchmark datasets, including 10 new, large, real-world datasets from Kaggle, and against 3 state-of-the-art AutoML tools and 4 baselines. Our evaluation shows that SapientML produces the best or comparable accuracy on 27 of the benchmarks while the second best tool fails to even produce a pipeline on 9 of the instances. This difference is amplified on the 10 most challenging benchmarks, where SapientML wins on 9 instances with the other tools failing to produce pipelines on 4 or more benchmarks.

Runtime Exceptions (REs) are an important class of bugs that occur frequently during code development. Traditional Automatic Program Repair (APR) tools are of limited use in this “in-development” use case, since they require a test-suite to be available as a patching oracle. Thus, developers typically tend to manually resolve their in-development REs, often by referring to technical forums, such as Stack Overflow (SO). To automate this manual process we extend our previous work, MaesTro, to provide real-time assistance to developers for repairing Java REs by recommending a relevant patch-suggesting SO post and synthesizing a repair patch from this post to fix the RE in the developer's code. Maestro exploits a library of Runtime Exception Patterns (REPs) semi-automatically mined from SO posts, through a relatively inexpensive, one-time, incremental process. An REP is an abstracted sequence of statements that triggers a given RE. REPs are used to index SO posts, retrieve a post most relevant to the RE instance exhibited by a developer's code and then mediate the process of extracting a concrete repair from the SO post, abstracting out post-specific details, and concretizing the repair to the developer's buggy code. We evaluate MaesTro on a published RE benchmark comprised of 78 instances. Maestro is able to generate a correct repair patch at the top position in 27% of the cases, within the top-3 in 40% of the cases and overall return a useful artifact in 81% of the cases. Further, the use of REPs proves instrumental to all aspects of Maestro's performance, from ranking and searching of SO posts to synthesizing patches from a given post. In particular, 45% of correct patches generated by MaesTro could not be produced by a baseline technique not using REPs, even when provided with Maestro's SO-post ranking. Maestro is also fast, needing around 1 second, on average, to generate its output. Overall, these results indicate that Maestro can provide effective real-time assistance to developers in repairing REs.

Programmers often use Q&A sites (e.g., Stack Overflow) to understand a root cause of program bugs. Runtime exceptions is one of such important class of bugs that is actively discussed on Stack Overflow. However, it may be difficult for beginner programmers to come up with appropriate keywords for search. Moreover, they need to switch their attentions between IDE and browser, and it is time-consuming. To overcome these difficulties, we proposed a method, "Q&A MAESTRO", to find suitable Q&A posts automatically for Java runtime exception by utilizing structure information of codes described in programming Q&A website. In this paper, we describe a usage scenario of IDE-plugin, the architecture and user interface of the implementation, and results of user studies. A video is available at https://youtu.be/4X24jJrMUVw. A demo software is available at https://github.com/FujitsuLaboratories/Q-A-MAESTRO.

Summary Companies often employ (i18n) frameworks to provide translated text and localized media content on their websites in order to effectively communicate with a global audience. However, the varying lengths of text from different languages can cause undesired distortions in the layout of a web page. Such distortions, called Internationalization Presentation Failures (IPFs), can negatively affect the aesthetics or usability of the website. Most of the existing automated techniques developed for assisting repair of IPFs either produce fixes that are likely to significantly reduce the legibility and attractiveness of the pages or are limited to only detecting IPFs, with the actual repair itself remaining a labour intensive manual task. To address this problem, we propose a search‐based technique for automatically repairing IPFs in web applications, while ensuring a legible and attractive page. The empirical evaluation of our approach reported that our approach was able to successfully resolve 94% of the detected IPFs for 46 real‐world web pages. In a user study, participants rated the visual quality of our fixes significantly higher than the unfixed versions and also considered the repairs generated by our approach to be notably more legible and visually appealing than the repairs generated by existing techniques.

Using online Q&A forums, such as Stack Overflow (SO), for guidance to resolve program bugs, among other development issues, is commonplace in modern software development practice. Runtime exceptions (RE) is one such important class of bugs that is actively discussed on SO. In this work we present a technique and prototype tool called MAESTRO that can automatically recommend an SO post that is most relevant to a given Java RE in a developer's code. MAESTRO compares the exception-generating program scenario in the developer's code with that discussed in an SO post and returns the post with the closest match. To extract and compare the exception scenario effectively, MAESTRO first uses the answer code snippets in a post to implicate a subset of lines in the post's question code snippet as responsible for the exception and then compares these lines with the developer's code in terms of their respective Abstract Program Graph (APG) representations. The APG is a simplified and abstracted derivative of an abstract syntax tree, proposed in this work, that allows an effective comparison of the functionality embodied in the high-level program structure, while discarding many of the low-level syntactic or semantic differences. We evaluate MAESTRO on a benchmark of 78 instances of Java REs extracted from the top 500 Java projects on GitHub and show that MAESTRO can return either a highly relevant or somewhat relevant SO post corresponding to the exception instance in 71% of the cases, compared to relevant posts returned in only 8% - 44% instances, by four competitor tools based on state-of-the-art techniques. We also conduct a user experience study of MAESTRO with 10 Java developers, where the participants judge MAESTRO reporting a highly relevant or somewhat relevant post in 80% of the instances. In some cases the post is judged to be even better than the one manually found by the participant.

Mobile devices have become a primary means of accessing the Internet. Unfortunately, many websites are not designed to be mobile friendly. This results in problems such as unreadable text, cluttered navigation, and content overflowing a device's viewport; all of which can lead to a frustrating and poor user experience. Existing techniques are limited in helping developers repair these mobile friendly problems. To address this limitation of prior work, we designed a novel automated approach for repairing mobile friendly problems in web pages. Our empirical evaluation showed that our approach was able to successfully resolve mobile friendly problems in 95% of the evaluation subjects. In a user study, participants preferred our repaired versions of the subjects and also considered the repaired pages to be more readable than the originals.

A consistent cross-browser user experience is crucial for the success of a website. Layout Cross Browser Issues (XBIs) can severely undermine a website’s success by causing web pages to render incorrectly in certain browsers, thereby negatively impacting users’ impression of the quality and services that the web page delivers. Existing Cross Browser Testing (XBT) techniques can only detect XBIs in websites. Repairing them is, hitherto, a manual task that is labor intensive and requires significant expertise. Addressing this concern, our paper proposes a technique for automatically repairing layout XBIs in websites using guided search-based techniques. Our empirical evaluation showed that our approach was able to successfully fix 86% of layout XBIs reported for 15 different web pages studied, thereby improving their cross-browser consistency.

Differences in the rendering of a website across different browsers can cause inconsistencies in its appearance and usability, resulting in Layout Cross Browser Issues (XBIs). Such XBIs can negatively impact the functionality of a website as well as users’ impressions of its trustworthiness and reliability. Existing techniques can only detect XBIs, and therefore require developers to manually perform the labor intensive task of repair. In this demo paper we introduce our tool, XFix, that automatically repairs layout XBIs in web applications. To the best of our knowledge, XFix is the first automated technique for generating XBI repairs.

Summary The energy consumption of mobile apps has become an important consideration for developers as the underlying mobile devices are constrained by battery capacity. Display represents a significant portion of an app's energy consumption—up to 60% of an app's total energy consumption. However, developers lack techniques to identify the user interfaces in their apps for which energy needs to be improved. This paper presents a technique for detecting display energy hotspots—user interfaces of a mobile app whose energy consumption is greater than optimal. The technique leverages display power modeling and automated display transformation techniques to detect these hotspots and prioritize them for developers. The evaluation of the technique shows that it can predict display energy consumption to within 14% of the ground truth and accurately rank display energy hotspots. Furthermore, the approach found 398 display energy hotspots in a set of 962 popular Android apps, showing the pervasiveness of this problem. For these detected hotspots, the average power savings that could be realized through better user interface design was 30%. Taken together, these results indicate that the approach represents a potentially impactful technique for helping developers to detect energy related problems and reduce the energy consumption of their mobile apps.

Failures in the presentation layer of a web application can negatively impact its usability and end users' perception of the application's quality. The problem of verifying the consistency of a web application's user interface across its different pages is one of the many challenges that software development teams face in testing the presentation layer. In this paper we propose a novel automated approach to detect and localize visual inconsistencies in web applications. To detect visual inconsistencies, our approach uses computer vision techniques to compare a test web page with its reference. Then to localize, our approach analyzes the structure and style of the underlying HTML elements to find the faulty elements responsible for the observed inconsistencies.

Web applications can be easily made available to an international audience by leveraging frameworks and tools for automatic translation and localization. However, these automated changes can distort the appearance of web applications since it is challenging for developers to design their websites to accommodate the expansion and contraction of text after it is translated to another language. Existing web testing techniques do not support developers in checking for these types of problems and manually checking every page in every language can be a labor intensive and error prone task. To address this problem, we introduce an automated technique for detecting when a web page's appearance has been distorted due to internationalization efforts and identifying the HTML elements or text responsible for the observed problem. In evaluation, our approach was able to detect internationalization problems in a set of 54 web applications with high precision and recall and was able to accurately identify the underlying elements in the web pages that led to the observed problem.

Presentation failures in a website can undermine its success by giving users a negative perception of the trustworthiness of the site and the quality of the services it delivers. Unfortunately, existing techniques for debugging presentation failures do not provide developers with automated and broadly applicable solutions for finding the site's faulty HTML elements and CSS properties. To address this limitation, we propose a novel automated approach for debugging web sites that is based on image processing and probabilistic techniques. Our approach first builds a model that links observable changes in the web site's appearance to faulty elements and styling properties. Then using this model, our approach predicts the elements and styling properties most likely to cause the observed failure for the page under test and reports these to the developer. In evaluation, our approach was more accurate and faster than prior techniques for identifying faulty elements in a website.

An attractive and visually appealing appearance is important for the success of a website. Presentation failures in a site''s web pages can negatively impact end users'' perception of the quality of the site and the services it delivers. Debugging such failures is challenging because testers must visually inspect large web pages and analyze complex interactions among the HTML elements of a page. In this paper we propose a novel automated approach for debugging web page user interfaces. Our approach uses computer vision techniques to detect failures and can then identify HTML elements that are likely to be responsible for the failure. We evaluated our approach on a set of real-world web applications and found that the approach was able to accurately and quickly identify faulty HTML elements.

Presentation failures in a website can negatively impact end users' perception of the quality of the website, the services it delivers, and the branding a company is trying to achieve. Presentation failures can occur easily in modern web applications because of the highly complex and dynamic nature of the HTML, CSS, and JavaScript that define a web page's visual appearance. Debugging such failures manually is time consuming and error-prone, and existing techniques do not provide an automated debugging solution. In this paper, we present our tool, WebSee, that provides a fully automated debugging solution for presentation failures in web applications. When run on real-world web applications, WebSee was able to accurately and quickly identify faulty HTML elements.

Presentation failures in web applications can negatively affect an application's usability and user experience. To find such failures, testers must visually inspect the output of a web application or exhaustively specify invariants to automatically check a page's correctness. This makes finding presentation failures labor intensive and error prone. In this paper, we present a new automated approach for detecting and localizing presentation failures in web pages. To detect presentation failures, our approach uses image processing techniques to compare a web page and its oracle. Then, to localize the failures, our approach analyzes the page with respect to its visual layout and identifies the HTML elements likely to be responsible for the failure. We evaluated our approach on a set of real-world web applications and found that the approach was able to accurately detect failures and identify the faulty HTML elements.