Knowledge-intensive programming Q&A is an active research area in industry. Its application boosts developer productivity by aiding developers in quickly finding programming answers from the vast amount of information on the Internet. In this study, we propose ProQANS and its variants ReProQANS and ReAugProQANS to tackle programming Q&A. ProQANS is a neural search approach that leverages unlabeled data on the Internet (such as StackOverflow) to mitigate the cold-start problem. ReProQANS extends ProQANS by utilizing reformulated queries with a novel triplet loss. We further use an auxiliary generative model to augment the training queries, and design a novel dual triplet loss function to adapt these generated queries, to build another variant of ReProQANS termed as ReAugProQANS. In our empirical experiments, we show ReProQANS has the best performance when evaluated on the in-domain test set, while ReAugProQANS has the superior performance on the out-of-domain real programming questions, by outperforming the state-of-the-art model by up to 477% lift on the MRR metric respectively. The results suggest their robustness to previously unseen questions and its wide application to real programming questions.

The number of mobile apps and the number of mobile devices have increased considerably in the past few years. To succeed in the competitive market of mobile apps, such as Google Play Store, developers should improve the user-perceived quality of their apps. In this paper, we investigate the relationship between mobile device attributes and the user-perceived quality of Android apps. We observe that the user-perceived quality of apps varies across devices. Device attributes, such as the CPU and the screen resolution, share a significant relationship with the user-perceived quality. However, having a better characteristic of an attribute, such as a higher display resolution, does not necessarily share a positive relationship with the user-perceived quality. App developers should not only consider the app attributes but also consider the device attributes of the available devices to deliver high-quality apps. The original paper is published in the Empirical Software Engineering journal communicated by Lin Tan.

When encountering unfamiliar programming tasks (e.g., connecting to a database), there is a need to seek potential working code examples. Instead of using code search engines, software developers usually post related programming questions on online Q&A forums (e.g., Stack Overflow). One possible reason is that existing code search engines would return effective code examples only if a query contains identifiers (e.g., class or method names). In other words, existing code search engines do not handle natural-language queries well (e.g., a description of a programming task). However, developers may not know the appropriate identifiers at the time of the search. As the demand of searching code examples is increasing, it is of significant interest to enhance code search engines. We conjecture that expanding natural-language queries with their semantically related identifiers has a great potential to enhance code search engines. In this paper, we propose an automated approach to find identifiers (in particular API class-names) that are semantically related to a given natural-language query. We evaluate the effectiveness of our approach using 74 queries on a corpus of 23,677,216 code snippets that are extracted from 24,666 open source Java projects. The results show that our approach can effectively recommend semantically related API class-names to expand the original natural-language queries. For instance, our approach successfully retrieves relevant code examples in the top 10 retrieved results for 76 percent of 74 queries, while it is 36 percent when using the original natural-language query; and the median rank of the first relevant code example is increased from 22 to 7.

Recently, automatic code comment generation is proposed to facilitate program comprehension. Existing code comment generation techniques focus on describing the functionality of the source code. However, there are other aspects such as insights about quality or issues of the code, which are overlooked by earlier approaches. In this paper, we describe a mining approach that recommends insightful comments about the quality, deficiencies or scopes for further improvement of the source code. First, we conduct an exploratory study that motivates crowdsourced knowledge from Stack Overflow discussions as a potential resource for source code comment recommendation. Second, based on the findings from the exploratory study, we propose a heuristic-based technique for mining insightful comments from Stack Overflow Q & A site for source code comment recommendation. Experiments with 292 Stack Overflow code segments and 5,039 discussion comments show that our approach has a promising recall of 85.42%. We also conducted a complementary user study which confirms the accuracy and usefulness of the recommended comments.

Code clones are unavoidable entities in software ecosystems. A variety of clone-detection algorithms are available for finding code clones. For Type-3 clone detection at method granularity (i.e., similar methods with changes in statements), dissimilarity threshold is one of the possible configuration parameters. Existing approaches use a single threshold to detect Type-3 clones across a repository. However, our study shows that to detect Type-3 clones at method granularity on a large-scale heterogeneous repository, multiple thresholds are often required. We find that the performance of clone detection improves if selecting different thresholds for various groups of clones in a heterogeneous repository (i.e., various applications). In this paper, we propose a threshold-free approach to detect Type-3 clones at method granularity across a large number of applications. Our approach uses an unsupervised learning algorithm, i.e., k-means, to determine true and false clones. We use a clone benchmark with 330,840 tagged clones from 24,824 open source Java projects for our study. We observe that our approach improves the performance significantly by 12% in terms of F-measure. Furthermore, our threshold-free approach eliminates the concern of practitioners about possible misconfiguration of Type-3 clone detection tools.

Abstract Big data analysis is an emerging research topic in various domains, and clone detection is no exception. The goal is to create big data inter‐project clone corpora across open‐source or corporate‐source code repositories. Such corpora can be used to study developer behavior and to reduce engineering costs by extracting globally duplicated efforts into new APIs and as a basis for code completion and API usage support. However, building scalable clone detection tools is challenging. It is often impractical to use existing state‐of‐the‐art tools to analyze big data because the memory and execution time required exceed the average user's resources. Some tools have inherent limitations in their data structures and algorithms that prevent the analysis of big data even when extraordinary resources are available. These limitations are impossible to overcome if the source code of the tool is unavailable or if the user lacks the time or expertise to modify the tool without harming its performance or accuracy. In this research, we have investigated the use of our shuffling framework for scaling classical clone detection tools to big data. The framework achieves scalability on commodity hardware by partitioning the input dataset into subsets manageable by the tool and computing resources. A non‐deterministic process is used to randomly ‘shuffle’ the contents of the dataset into a series of subsets. The tool is executed for each subset, and its output for each is merged into a single report. This approach does not require modification to the subject tools, allowing their individual strengths and precision to be captured at an acceptable loss of recall. In our study, we explored the performance and applicability of the framework for the big data dataset, IJaDataset 2.0, which consists of 356 million lines of code from 25,000 open‐source Java projects. We begin with a computationally inexpensive version of our framework based on pure random shuffling. This version was successful at scaling the tools to IJaDataset but required many subsets to achieve a desirable recall. Using our findings, we incrementally improved the framework to achieve a satisfactory recall using fewer resources. We investigated the use of efficient file tracking and file‐similarity heuristics to bias the shuffling algorithm toward subsets of the dataset that contain undetected clone pairs. These changes were successful in improving the recall performance of the framework. Our study shows that the framework is able to achieve up to 90–95% of a tool's native recall using standard hardware. Copyright © 2014 John Wiley & Sons, Ltd.

Web service composition enables seamless and dynamic integration of Web services. The behavior of participant Web services determines the overall performance of a composition. Therefore, it is important to choose high quality services for service composition. Existing Web service selection and discovery approaches rely on non-functional aspects (also known as quality of service or QoS), e.g., response time and availability. Though these parameters are crucial for selecting Web services, they may not reflect the user's perspective of quality. In this paper, we explore the feasibility of incorporating perceived quality from user's perspective for service selection and composition. We name such quality attributes as quality of experience (QoE). First, we propose a solution that automatically mines and identifies QoE attributes from the Web. Second, we study the application of such dynamically extracted QoE attributes for service selection. For the evaluation purpose, we collected more than 34,000 reviews from 58 different services in six domains. Our findings show that it is possible to automatically identify QoE attributes with an average precision and recall of 92 and 80 percent respectively. Our study shows that there is a strong positive correlation between QoS and QoE. Hence QoE can be used during service selection especially when QoS data are not available. Furthermore, we found 70 percent of service discovery queries indeed contain QoE attributes showing the importance of QoE attributes during the service discovery phase.

End-users conduct various on-line activities. Quite often, they re-visit websites and use services to perform re-occurring activities, such as on-line shopping. The end-users are required to enter the same information into various web services to accomplish such re-occurring tasks. It can negatively impact user experience when a user needs to type the re-occurring information repetitively into such web services. In this paper, we propose an approach to prevent end-users from performing such repetitive tasks. Our approach propagates user inputs across services by linking similar input and output parameters. Our approach pre-fills values to the input parameters for an end-user using his or her previous inputs. To increase the chance of identifying a proper value for an input parameter performed by one end-user, our approach also leverages the inputs from other end-users. We identify and link similar end-users to enable the propagation of user inputs among end-users. We have designed and developed a prototype. We also conduct an empirical study to evaluate our approach using the real world services. The empirical results show that our approach using an end-user's previous inputs can reduce on average 41 percent of repetitive typing for the execution of composed services. Furthermore, the previous inputs from the similar end-users can improve our approach in reducing the repetitive typing for an end-user.

Working code examples are useful resources for pragmatic reuse in software development. A working code example provides a solution to a specific programming problem. Earlier studies have shown that existing code search engines are not successful in finding working code examples. They fail in ranking high quality code examples at the top of the result set. To address this shortcoming, a variety of pattern-based solutions are proposed in the literature. However, these solutions cannot be integrated seamlessly in Internet-scale source code engines due to their high time complexity or query language restrictions. In this paper, we propose an approach for spotting working code examples which can be adopted by Internet-scale source code search engines. The time complexity of our approach is as low as the complexity of existing code search engines on the Internet and considerably lower than the pattern-based approaches supporting free-form queries. We study the performance of our approach using a representative corpus of 25,000 open source Java projects. Our findings support the feasibility of our approach for Internet-scale code search. We also found that our approach outperforms Ohloh Code search engine, previously known as Koders, in spotting working code examples.

Recently, new applications of code clone detection and search have emerged that rely upon clones detected across thousands of software systems. Big data clone detection and search algorithms have been proposed as an embedded part of these new applications. However, there exists no previous benchmark data for evaluating the recall and precision of these emerging techniques. In this paper, we present a Big Data clone detection benchmark that consists of known true and false positive clones in a Big Data inter-project Java repository. The benchmark was built by mining and then manually checking clones of ten common functionalities. The benchmark contains six million true positive clones of different clone types: Type-1, Type-2, Type-3 and Type-4, including various strengths of Type-3 similarity (strong, moderate, weak). These clones were found by three judges over 216 hours of manual validation efforts. We show how the benchmark can be used to measure the recall and precision of clone detection techniques.

Web service composition enables seamless and dynamic integration of web services. The behavior of participant web services determines the overall performance of a composition. Therefore, it is important to choose the high quality participants for service composition. The state of the art in service discovery and selection rely on non-functional aspects also known as quality of service (QoS) e.g., response time and availability. Though these parameters are crucial for selecting web services, they do not reflect the end-user's perspective on quality. In this paper, we explore the feasibility of adopting the perceived quality from end-user's perspective for service selection and composition. We name such quality parameters as quality of experience (QoE). First, we propose a solution that automatically mines and identifies QoE parameters from the web. Second, we study the application of such dynamically extracted QoE attributes for service selection. For the evaluation purpose, we collected more than 24,000 reviews from 22 different services from four domains. Our result shows the automated approach identifies QoE attributes with an average precision and recall 90% and 79% respectively. Our study shows that there is a strong positive correlation between QoS and QoE. Hence QoE can be used during service selection especially when QoS data are not available.

End-users conduct various on-line activities. Quite often they re-visit websites and use services to perform repeated activities, such as on-line shopping. The end-users are required to enter the same information into various web services to accomplish such repeated tasks. Typing redundant information repetitively into such services negatively impacts the user experience. In this study, we propose an approach to prevent end-users from such unnecessary interruption. Our approach propagates user inputs across services by linking similar input and output parameters. Our approach also pre-fills values to the input parameters which could not be filled by the values from other input or output parameters. We propose a meta-data model for storing user inputs and an Input Parameter Context Model for identifying similar input or output parameters. We have implemented our approach and evaluated it on the real world services through an empirical study. Our overall approach can reduce on average 37% of input parameters through the execution of composed services.

To predict files with defects, a suitable prediction model must be built for a software project from either itself (within-project) or other projects (cross-project). A universal defect prediction model that is built from the entire set of diverse projects would relieve the need for building models for an individual project. A universal model could also be interpreted as a basic relationship between software metrics and defects. However, the variations in the distribution of predictors pose a formidable obstacle to build a universal model. Such variations exist among projects with different context factors (e.g., size and programming language). To overcome this challenge, we propose context-aware rank transformations for predictors. We cluster projects based on the similarity of the distribution of 26 predictors, and derive the rank transformations using quantiles of predictors for a cluster. We then fit the universal model on the transformed data of 1,398 open source projects hosted on SourceForge and GoogleCode. Adding context factors to the universal model improves the predictive power. The universal model obtains prediction performance comparable to the within-project models and yields similar results when applied on five external projects (one Apache and four Eclipse projects). These results suggest that a universal defect prediction model may be an achievable goal.

We introduce semantic-enabled clone detection, as an approach that emphasizes on importance of the token semantics during the pattern matching for clone detection. This approach can be realized using Semantic Web and its support for knowledge modeling. While the Semantic Web has found wide acceptance in various application and research domains, it still lacks the same acceptance in the source code analysis domain. In this paper we focus on how the Semantic Web and its support for semantic modeling and knowledge retrieval can be applied towards source code clone detection and search. We discuss both, open challenges in the source code clone detection domain, as well as both theoretical and practical aspects on how the Semantic Web can address some of these challenges.

In our research, we have focused on mining software repositories to support software engineering development and code clone search. We developed Doppel-Code, a clone visualization tool to aid searching for code clones across 18 000 open source projects. For clone visualization, we take into account clone impact - both globally and locally - to further aid developers' software maintenance activities.

There has been an ongoing trend towards open and shared source code that is published on the Internet in large software repositories to support collaborative development processes. While traditional source code analysis techniques perform well in single project contexts, new types of global source code analysis techniques are slowly introduced to address the analysis of global distributed and often incomplete source code. In this article, we discuss how the Semantic Web, an enabling technology for these emerging source code analysis domains, can support a standardized, formal, and semantic rich representation to model these corpora. We also illustrate how inference services can be used to provide support for emerging source code analysis approaches on this data, such as search, call graph construction, and clone detection.

In this poster, we review the limitations open source licences introduce to the application of Linked Data in Software Engineering. We investigate whether open source licences support special requirements to publish source code as Linked Data on the Internet.

The mining of software repository involves the extraction of both basic and value-added information from existing software repositories. Depending on stakeholders (e.g., researchers, management), these repositories are mined several times for different application purposes. To avoid unnecessary pre-processing steps and improve productivity, sharing, and integration of extracted facts and results are needed. The motivation of this research is to introduce a novel collaborative sharing platform for software datasets that supports on-the-fly inter-datasets integration. We want to facilitate and promote a paradigm shift in the source code analysis domain, similar to the one by Wikipedia in the knowledge-sharing domain. In this paper, we present the SeCold project, which is the first online, publicly available software ecosystem Linked Data dataset. As part of this research, not only theoretical background on how to publish such datasets is provided, but also the actual dataset. SeCold contains about two billion facts, such as source code statements, software licenses, and code clones from over 18.000 software projects. SeCold is also an official member of the Linked Data cloud and one of the eight largest online Linked Data datasets available on the cloud.

For decades stock market traders make financially critical buy/sell decisions depending on external and internal factors affecting the price of individual stocks. Moving Averages combined with technical analysis patterns are some of the most basic and widely used indicators to support buy/sell decisions. In this research, we present a novel cross-disciplinary approach that uses these technical stock market indicators for analyzing the community evolvability of open source software systems.

Different approaches to search for source code on the Internet exist. In this paper, we propose techniques that support a special type of Internet-scale code search using two novel syntactical and semantic clone search and detection methods. The syntactical search focuses on providing a scalable real-time engine, while the semantic clone detection is being used to enrich our knowledge base during the offline processing step.

SeByte is a semantic clone detection tool which accepts Java bytecode (binary) as input. SeByte provides a complementary approach to traditional pattern-based source code level clone detection. It is capable of detecting clones missed by existing clone detection tools since it exploits both pattern and content similarity at binary level.

The mining of software repositories involves the extraction of both basic and value-added information from existing software repositories. The repositories will be mined to extract facts by different stakeholders (e.g. researchers, managers) and for various purposes. To avoid unnecessary pre-processing and analysis steps, sharing and integration of both basic and value-added facts are needed. In this research, we introduce SeCold, an open and collaborative platform for sharing software datasets. SeCold provides the first online software ecosystem Linked Data platform that supports data extraction and on-the-fly inter-dataset integration from major version control, issue tracking, and quality evaluation systems. In its first release, the dataset contains about two billion facts, such as source code statements, software licenses, and code clones from 18 000 software projects. In its second release the SeCold project will contain additional facts mined from issue trackers and versioning systems. Our approach is based on the same fundamental principle as Wikipedia: researchers and tool developers share analysis results obtained from their tools by publishing them as part of the SeCold portal and therefore make them an integrated part of the global knowledge domain. The SeCold project is an official member of the Linked Data dataset cloud and is currently the eighth largest online dataset available on the Web.

Real-time code clone search is an emerging family of clone detection research that aims at finding clone pairs matching an input code fragment in fractions of a second. For these techniques to meet actual real world requirements, they have to be scalable and provide a short response time. Our research presents a hybrid clone search approach using source code pattern indexing, information retrieval clustering, and Semantic Web reasoning to respectively achieve short response time, handle false positives, and support automated grouping/querying.

Available code search engines provide typically coarse-grained lexical search. To address this limitation we present SE-CodeSearch, a Semantic Web-based approach for Internet-scale source code search. It uses an ontological representation of source code facts and analysis knowledge to complete missing information using inference engine. This approach allows us to reason and search across project boundaries containing often incomplete code fragments extracted in a one-pass and no-order manner. The infrastructure provides a scalable approach to process and query across large code bases mined from software repositories and code fragments found online. We have implemented our SE-CodeSearch as part of SE-Advisor framework to demonstrate the scalability and applicability of our Internet-scale code search in a software maintenance context.

Context awareness in Web services is gaining momentum. Since it is not a trivial task, it suffers from lack of a general solution. In this paper, we introduce a novel approach for context-aware semantic Web services which is applicable for any environment. It is established based on composition of context provider Web services and other context-aware semantic Web services. In addition, an extended version of the semantic Web service ontology language for semantic Web services is introduced, in order to make it possible to find appropriate context-aware semantic Web services based on available context information. So as to make it applicable for any environment, the solution does not hold any assumption about the user's context source. Rather, the property of context level is defined.