Developers use tools such as GitHub pull requests to review code, discuss proposed changes, and request modifications. While changed files are commonly presented in alphabetical order, this does not necessarily coincide with the reviewer’s preferred navigation sequence. This study investigates the different navigation orders developers follow while commenting on changes submitted in pull requests. We mined code review comments from 23,241 pull requests in 100 popular Java and Python repositories on GitHub to analyze the order in which the reviewers commented on the submitted changes. Our analysis shows that for 44.6% of pull requests, the reviewers comment in a non-alphabetical order. Among these pull requests, we identified traces of alternative meaningful orders: 20.6% (2,134) followed a largest-diff first order, 17.6% (1,827) were commented in the order of the files’ similarity to the pull request’s title and description, and 29% (1,188) of pull requests containing changes to both production and test files adhered to a test-first order. We also observed that the proportion of reviewed files to total submitted files was significantly higher in non-alphabetically ordered reviews, which also received slightly fewer approvals from reviewers, on average. Our findings highlight the need for additional support during code reviews, particularly for larger pull requests, where reviewers are more likely to adopt complex strategies rather than following a single predefined order.

Ensuring usability is crucial for the success of mobile apps. Usability issues can compromise user experience and negatively impact the perceived app quality. This paper presents UX-LLM, a novel tool powered by a Large Vision-Language Model that predicts usability issues in iOS apps. To evaluate the performance of UX-LLM, we predicted usability issues in two open-source apps of a medium complexity and asked two usability experts to assess the predictions. We also performed traditional usability testing and expert review for both apps and compared the results to those of UX-LLM. UX-LLM demonstrated precision ranging from 0.61 and 0.66 and recall between 0.35 and 0.38, indicating its ability to identify valid usability issues, yet failing to capture the majority of issues. Finally, we conducted a focus group with an app development team of a capstone project developing a transit app for visually impaired persons. The focus group expressed positive perceptions of UX-LLM as it identified unknown usability issues in their app. However, they also raised concerns about its integration into the development workflow, suggesting potential improvements. Our results show that UX-LLM cannot fully replace traditional usability evaluation methods but serves as a valuable supplement particularly for small teams with limited resources, to identify issues in less common user paths, due to its ability to inspect the source code.

Model cards are the primary documentation framework for developers of artificial intelligence (AI) models to communicate critical information to their users. Those users are often developers themselves looking for relevant documentation to ensure that their AI systems comply with the ethical requirements of existing laws, guidelines, and standards. Recent studies indicate inadequate model documentation practices, suggesting a gap between AI requirements and current practices in model documentation. To understand this gap and provide actionable guidance to bridge it, we conducted a thematic analysis of 26 guidelines on ethics and AI, three AI documentation frameworks, three quantitative studies of model cards, and ten actual model cards. We identified a total of 43 ethical requirements relevant to model documentation and organized them into a taxonomy featuring four themes and twelve sub-themes representing ethical principles. Our findings indicate that model developers predominantly emphasize model capabilities and reliability in the documentation while overlooking other ethical aspects, such as explainability, user autonomy, and fairness. This underscores the need for enhanced support in documenting ethical AI considerations. Our taxonomy serves as a foundation for a revised model card framework that holistically addresses ethical AI requirements.

Graphical User Interfaces (GUIs) are central to app development projects. App developers may use the GUIs of other apps as a means of requirements refinement and rapid prototyping or as a source of inspiration for designing and improving their own apps. Recent research has thus suggested retrieving relevant GUI designs that match a certain text query from screenshot datasets acquired through crowdsourced or automated exploration of GUIs. However, such text-to-GUI retrieval approaches only leverage the textual information of the GUI elements, neglecting visual information such as icons or background images. In addition, retrieved screenshots are not steered by app developers and lack app features that require particular input data. To overcome these limitations, this article proposes GUing, a GUI search engine based on a vision-language model called GUIClip, which we trained specifically for the problem of designing app GUIs. For this, we first collected from Google Play app introduction images which display the most representative screenshots and are often captioned (i.e., labeled) by app vendors. Then, we developed an automated pipeline to classify, crop, and extract the captions from these images. This resulted in a large dataset which we share with this article: including 303k app screenshots, out of which 135k have captions. We used this dataset to train a novel vision-language model, which is, to the best of our knowledge, the first of its kind for GUI retrieval. We evaluated our approach on various datasets from related work and in a manual experiment. The results demonstrate that our model outperforms previous approaches in text-to-GUI retrieval achieving a Recall@10 of up to 0.69 and a HIT@10 of 0.91. We also explored the performance of GUIClip for other GUI tasks, including GUI classification and sketch-to-GUI retrieval with encouraging results.

Code reviews are central for software quality assurance. Ideally, reviewers should explain their feedback to enable authors of code changes to understand the feedback and act accordingly. Different developers might need different explanations in different contexts. Therefore, assisting this process first requires understanding the types of explanations reviewers usually provide. The goal of this article is to study the types of explanations used in code reviews and explore the potential of Large Language Models (LLMs), specifically ChatGPT, in generating these specific types. We extracted 793 code review comments from Gerrit and manually labeled them based on whether they contained a suggestion, an explanation, or both. Our analysis shows that 42% of comments only include suggestions without explanations. We categorized the explanations into seven distinct types including rule or principle, similar examples, and future implications. When measuring their prevalence, we observed that some explanations are used differently by novice and experienced reviewers. Our manual evaluation shows that, when the explanation type is specified, ChatGPT can correctly generate the explanation in 88 out of 90 cases. This foundational work highlights the potential for future automation in code reviews, which can assist developers in sharing and obtaining different types of explanations as needed, thereby reducing back-and-forth communication.

Large language models (LLMs) bear great potential for automating tedious development tasks such as creating and maintaining code documentation. However, it is unclear to what extent developers can effectively prompt LLMs to create concise and useful documentation. We report on a controlled experiment with 20 professionals and 30 computer science students tasked with code documentation generation for two Python functions. The experimental group freely entered ad-hoc prompts in a Chat- GPT-like extension of Visual Studio Code, while the control group executed a predefined few-shot prompt. Our results reveal that professionals and students were unaware of or unable to apply prompt engineering techniques. Especially students perceived the documentation produced from ad-hoc prompts as significantly less readable, less concise, and less helpful than documentation from prepared prompts. Some professionals produced higher quality documentation by just including the keyword Docstring in their ad-hoc prompts. While students desired more support in formulating prompts, professionals appreciated the flexibility of ad-hoc prompting. Participants in both groups rarely assessed the output as perfect. Instead, they understood the tools as support to iteratively refine the documentation. Further research is needed to understand which prompting skills and preferences developers have and which support they need for certain tasks.

Over the past decade, app store (AppStore)-inspired requirements elicitation has proven to be highly beneficial. Developers often explore competitors' apps to gather inspiration for new features. With the advance of Generative AI, recent studies have demonstrated the potential of large language model (LLM)-inspired requirements elicitation. LLMs can assist in this process by providing inspiration for new feature ideas. While both approaches are gaining popularity in practice, there is a lack of insight into their differences. We report on a comparative study between AppStore- and LLM-based approaches for refining features into sub-features. By manually analyzing 1,200 sub-features recommended from both approaches, we identified their benefits, challenges, and key differences. While both approaches recommend highly relevant sub-features with clear descriptions, LLMs seem more powerful particularly concerning novel unseen app scopes. Moreover, some recommended features are imaginary with unclear feasibility, which suggests the importance of a human-analyst in the elicitation loop.

Previous works from research and industry have proposed a spatial representation of code in a canvas, arguing that a navigational code space confers developers the freedom to organise elements according to their understanding. By allowing developers to translate logical relatedness into spatial proximity, this code representation could aid in code navigation and comprehension. However, the association between developers' code comprehension and their visuo-spatial mental model of the code is not yet well understood. This mental model is affected on the one hand by the spatial code representation and on the other by the visuo-spatial working memory of developers. We address this knowledge gap by conducting an online experiment with 20 developers following a between-subject design. The control group used a conventional tab-based code visualization, while the experimental group used a code canvas to complete three code comprehension tasks. Furthermore, we measure the participants' visuo-spatial working memory using a Corsi Block test at the end of the tasks. Our results suggest that, overall, neither the spatial representation of code nor the visuo-spatial working memory of developers has a significant impact on comprehension performance. However, we identified significant differences in the time dedicated to different comprehension activities such as navigation, annotation, and UI interactions.

To create unit tests, it may be necessary to refactor the production code, e.g. by widening access to specific methods or by decomposing classes into smaller units that are easier to test independently. We report on an extensive study to understand such composite refactoring procedures for the purpose of improving testability. We collected and studied 346,841 java pull requests from 621 GitHub projects. First, we compared the atomic refactorings in two populations: pull requests with changed test-pairs (i.e. with co-changes in production and test code and thus potentially including testability refactoring) and pull requests without test-pairs. We found significantly more atomic refactorings in test-pairs pull requests, such as Change Variable Type Operation or Change Parameter Type. Second, we manually analyzed the code changes of 200 pull requests, where developers explicitly mention the terms “testability” or “refactor + test”. We identified ten composite refactoring procedures for the purpose of testability, which we call testability refactoring patterns. Third, we manually analyzed additional 524 test-pairs pull requests: both randomly selected and where we assumed to find testability refactorings, e.g. in pull requests about dependency or concurrency issues. About 25% of all analyzed pull requests actually included testability refactoring patterns. The most frequent were extract a method for override or for invocation, widen access to a method for invocation, and extract a class for invocation. We also report on frequent atomic refactorings which co-occur with the patterns and discuss the implications of our findings for research, practice, and education.

Abstract Stakeholders in software projects use issue trackers like JIRA or Bugzilla to capture and manage issues, including requirements, feature requests, and bugs. To ease issue navigation and structure project knowledge, stakeholders manually connect issues via links of certain types that reflect different dependencies, such as Epic-, Block-, Duplicate-, or Relate- links. Based on a large dataset of 16 JIRA repositories, we study the commonalities and differences in linking practices and link types across the repositories. We then investigate how state-of-the-art machine learning models can predict common link types. We observed significant differences across the repositories and link types, depending on how they are used and by whom. Additionally, we observed several inconsistencies, e.g., in how Duplicate links are used. We found that a transformer model trained on titles and descriptions of linked issues significantly outperforms other optimized models, achieving an encouraging average macro F1-score of 0.64 for predicting nine popular link types across all repositories (weighted F1-score of 0.73). For the specific Subtask- and Epic- links, the model achieves top F1-scores of 0.89 and 0.97, respectively. If we restrict the task to predict the mere existence of links, the average macro F1-score goes up to 0.95. In general, the shorter issue text, possibly indicating precise issues, seems to improve the prediction accuracy with a strong negative correlation of $$-$$ - 0.73. We found that Relate-links often get confused with the other links, which suggests that they are likely used as default links in unclear cases. Our findings particularly on the quality and heterogeinity of issue link data have implications for researching and applying issue link prediction in practice.

[Background] In large open-source software projects, development knowledge is often fragmented across multiple artefacts and contributors such that individual stakeholders are generally unaware of the full breadth of the product features. However, users want to know what the software is capable of, while contributors need to know where to fix, update, and add features. [Objective] This work aims at understanding how feature knowledge is documented in GitHub projects and how it is linked (if at all) to the source code. [Method] We conducted an in-depth qualitative exploratory content analysis of 25 popular GitHub repositories that provided the documentation artefacts recommended by GitHub’s Community Standards indicator. We first extracted strategies used to document software features in textual artefacts and then strategies used to link the feature documentation with source code. [Results] We observed feature documentation in all studied projects in artefacts such as READMEs, wikis, and website resource files. However, the features were often described in an unstructured way. Additionally, tracing techniques to connect feature documentation and source code were rarely used. [Conclusions] Our results suggest a lacking (or a low-prioritised) feature documentation in open-source projects, little use of normalised structures, and a rare explicit referencing to source code. As a result, product feature traceability is likely to be very limited, and maintainability to suffer over time.

Software projects use Issue Tracking Systems (ITS) like JIRA to track issues and organize the workflows around them. Issues are often inter-connected via different links such as the default JIRA link types Duplicate, Relate, Block, or Subtask. While previous research has mostly focused on analyzing and predicting duplication links, this work aims at understanding the various other link types, their prevalence, and characteristics towards a more reliable link type prediction. For this, we studied 607,208 links connecting 698,790 issues in 15 public JIRA repositories. Besides the default types, the custom types Depend, Incorporate, Split, and Cause were also common. We manually grouped all 75 link types used in the repositories into five general categories: General Relation, Duplication, Composition, Temporal / Causal, and Workflow. Comparing the structures of the corresponding graphs, we observed several trends. For instance, Duplication links tend to represent simpler issue graphs often with two components and Composition links present the highest amount of hierarchical tree structures (97.7%). Surprisingly, General Relation links have a significantly higher transitivity score than Duplication and Temporal / Causal links.

Organisations use issue tracking systems (ITSs) to track and document their projects' work in units called issues. This style of documentation encourages evolutionary refinement, as each issue can be independently improved, commented on, linked to other issues, and progressed through the organisational workflow. Commonly studied ITSs so far include GitHub, GitLab, and Bugzilla, while Jira, one of the most popular ITS in practice with a wealth of additional information, has yet to receive similar attention. Unfortunately, diverse public Jira datasets are rare, likely due to the difficulty in finding and accessing these repositories. With this paper, we release a dataset of 16 public Jiras with 1822 projects, spanning 2.7 million issues with a combined total of 32 million changes, 9 million comments, and 1 million issue links. We believe this Jira dataset will lead to many fruitful research projects investigating issue evolution, issue linking, cross-project analysis, as well as cross-tool analysis when combined with existing well-studied ITS datasets.

Stakeholders in software projects use issue trackers like JIRA to capture and manage issues, including requirements and bugs. To ease issue navigation and structure project knowledge, stakeholders manually connect issues via links of certain types that reflect different dependencies, such as Epic-, Block-, Duplicate-, or Relate-links. Based on a large dataset of 15 JIRA repositories, we study how well state-of-the-art machine learning models can automatically detect common link types. We found that a pure BERT model trained on titles and descriptions of linked issues significantly outperforms other optimized deep learning models, achieving an encouraging average macro F1-score of 0.64 for detecting 9 popular link types across all repositories (weighted F1-score of 0.73). For the specific Subtask- and Epic-links, the model achieved top F1-scores of 0.89 and 0.97, respectively. Our model does not simply learn the textual similarity of the issues. In general, shorter issue text seems to improve the prediction accuracy with a strong negative correlation of −0.70. We found that Relate-links often get confused with the other links, which suggests that they are likely used as default links in unclear cases. We also observed significant differences across the repositories, depending on how they are used and by whom.

Over the last years, user feedback has become a valuable source for requirements elicitation. Software vendors increasingly rely on user feedback to collect product issues and feature requests, discover requirements and monitor the overall sentiment of the users about a product. While the analysis of user feedback for requirements elicitation has revealed that feedback can contain helpful information for the product team, collecting valuable, informative, and actionable feedback is still challenging: User feedback is often vague, emotional, or missing important information, such as contextual information, to actually support a product team. Information describing the context of the reported feedback, such as the device model and software version, plays an essential role in increasing its value [1], [2]. Without a given context, reported issues can be complex to understand, reproduce, and address.

Research has repeatedly shown that high-quality requirements are essential for the success of development projects. While the term "quality" is pervasive in the field of requirements engineering and while the body of research on requirements quality is large, there is no meta-study of the field that overviews and compares the concrete quality attributes addressed by the community. To fill this knowledge gap, we conducted a systematic mapping study of the scientific literature. We retrieved 6905 articles from six academic databases, which we filtered down to 105 relevant primary studies. The primary studies use empirical research to explicitly define, improve, or evaluate requirements quality. We found that empirical research on requirements quality focuses on improvement techniques, with very few primary studies addressing evidence-based definitions and evaluations of quality attributes. Among the 12 quality attributes identified, the most prominent in the field are ambiguity, completeness, consistency, and correctness. We identified 111 sub-types of quality attributes such as "template conformance" for consistency or "passive voice" for ambiguity. Ambiguity has the largest share of these sub-types. The artefacts being studied are mostly referred to in the broadest sense as "requirements", while little research targets quality attributes in specific types of requirements such as use cases or user stories. Our findings highlight the need to conduct more empirically grounded research defining requirements quality, using more varied research methods, and addressing a more diverse set of requirements types.

App stores allow users to give valuable feedback on apps, and developers to find this feedback and use it for the software evolution. However, finding user feedback that matches existing bug reports in issue trackers is challenging as users and developers often use a different language. In this work, we introduce DeepMatcher, an automatic approach using state-of-the-art deep learning methods to match problem reports in app reviews to bug reports in issue trackers. We evaluated DeepMatcher with four open-source apps quantitatively and qualitatively. On average, DeepMatcher achieved a hit ratio of 0.71 and a Mean Average Precision of 0.55. For 91 problem reports, DeepMatcher did not find any matching bug report. When manually analyzing these 91 problem reports and the issue trackers of the studied apps, we found that in 47 cases, users actually described a problem before developers discovered and documented it in the issue tracker. We discuss our findings and different use cases for DeepMatcher.

Linguistic-Relativity-Theory states that language and its structure influence people’s world view and cognition. We investigate how this theory impacts the identification of requirements in practice. To this end, we conducted two controlled experiments with 101 participants. We randomly showed participants a set of requirements dimensions (i.e. a language structure) either with a focus on software quality or on sustainability and asked them to identify the requirements for a grocery shopping app according to these dimensions. Participants of the control group were not given any dimensions. The results show that the use of requirements dimensions significantly increases the number of identified requirements in comparison to the control group. Furthermore, participants who were given the sustainability dimensions identified more sustainability requirements. In follow up interviews with 16 practitioners, the interviewees reported benefits of the dimensions such as a holistic guidance but were also concerned about the customers acceptance. Furthermore, they stated challenges of implementing sustainability dimensions in the daily business but also suggested solutions like establishing sustainability as a common standard. Our study indicates that carefully structuring requirements engineering along sustainability dimensions can guide development teams towards considering and ensuring software sustainability.

Preclinical patient care is both mentally and physically challenging and exhausting for emergency teams. The teams intensively use medical technology to help the patient on site. However, they must carry and handle multiple heavy medical devices such as a monitor for the patient's vital signs, a ventilator to support an unconscious patient, and a resuscitation device. In an industry project, we aim at developing a combined device that lowers the emergency teams' mental and physical load caused by multiple screens, devices, and their high weight. The focus of this paper is to describe our ideation and requirements elicitation process regarding the user interface design of the combined device. For one year, we applied a fully digital customized version of the Applied Cognitive Task Analysis (ACTA) method to systematically elicit the requirements. Domain and requirements engineering experts created a detailed hierarchical task diagram of an extensive emergency scenario, conducted eleven interviews with subject matter experts (SMEs), and executed two design workshops, which led to 34 sketches and three mockups of the combined device's user interface. Cross-functional teams accompanied the entire process and brought together expertise in preclinical patient care, requirements engineering, and medical product development. We report on the lessons learned for each of the four consecutive stages of our customized ACTA process.

Estimating uncertainties of Neural Network predictions paves the way towards more reliable and trustful text classifications.However, common uncertainty estimation approaches remain as black-boxes without explaining which features have led to the uncertainty of a prediction.This hinders users from understanding the cause of unreliable model behaviour.We introduce an approach to decompose and visualize the uncertainty of text classifiers at the level of words.Our approach builds on top of Recurrent Neural Networks and Bayesian modelling in order to provide detailed explanations of uncertainties, enabling a deeper reasoning about unreliable model behaviours.We conduct a preliminary experiment to check the impact and correctness of our approach.By explaining and investigating the predictive uncertainties of a sentiment analysis task, we argue that our approach is able to provide a more profound understanding of artificial decision making.

In this paper we introduce the preliminary design of a new framework named ShapeRE, which will address two problems potentially impeding the development of sustainable software. The first problem is the common differentiation between “functional” and “non-functional” requirements, which hides relevant sustainability requirements and hampers their identification. The second problem is the lack of developer-oriented representation approaches to ensure that all requirements relevant to sustainable software will be finally implemented. To address these two issues ShapeRE provides an alternative multi-dimensional approach for identifying requirements and a developer-oriented representation guideline. We suggest nine dimensions which are social, economic, environmental, technical, individual, purpose, design-aesthetics, integrative and legal. The design of our frame-work relies on insights from the fields of linguistics and empirical aesthetics, requirements engineering, and building architecture. We introduce the framework and discuss our evaluation strategy consisting of an online-experiment, a case study, and a survey.

In the dynamic and fast-growing app market, monitoring and understanding how past releases are actually being used is indispensable for successful app maintenance and evolution. Current app usage analytics tools either log execution events, e.g., in stack traces, or general usage information such as the app activation time, location, and device. In this paper, we focus on analyzing the usages of the single app features as described in release notes and app pages. We suggest monitoring nine app-independent, privacy-friendly interaction events for training a machine learning model to learn app feature usages. We conducted a crowdsourcing study with 55 participants who labeled 5,815 feature usages of 170 unique apps for 18 days. Our within-apps evaluation shows that we could achieve encouraging precision and recall values already with ten labeled feature usages. For certain popular features such as browse newsfeed or send an email, we achieved F1 values above 88%. Betweenapps feature learning seems feasible with F1 values of up to 86%.

Managing software projects gets more and more complicated with an increasing project and product size. To cope with this complexity, many organizations use issue tracking systems, where tasks, bugs, and requirements are stored as issues. Unfortunately, managing software projects might remain chaotic even when using issue trackers. Particularly for long lasting projects with a large number of issues and links between them, it is often hard to maintain an overview of the dependencies, especially when dozens of new issues get reported every day. We present a Jira plug-in that supports developers, project managers, and product owners in managing and overviewing issues and their dependencies. Our tool visualizes the issue links, helps to find missing or unknown links between issues, and detects inconsistencies.

While many apps include built-in options to report bugs or request features, users still provide an increasing amount of feedback via social media, like Twitter. Compared to traditional issue trackers, the reporting process in social media is unstructured and the feedback often lacks basic context information, such as the app version or the device concerned when experiencing the issue. To make this feedback actionable to developers, support teams engage in recurring, effortful conversations with app users to clarify missing context items. This paper introduces a simple approach that accurately extracts basic context information from unstructured, informal user feedback on mobile apps, including the platform, device, app version, and system version. Evaluated against a truthset of 3014 tweets from official Twitter support accounts of the 3 popular apps Netflix, Snapchat, and Spotify, our approach achieved precisions from 81% to 99% and recalls from 86% to 98% for the different context item types. Combined with a chatbot that automatically requests missing context items from reporting users, our approach aims at auto-populating issue trackers with structured bug reports.

With the rise of social media like Twitter and distribution platforms like app stores, users have various ways to express their opinions about software products. Popular software vendors get user feedback thousandfold per day. Research has shown that such feedback contains valuable information for software development teams. However, a manual analysis of user feedback is cumbersome and hard to manage. We present OpenReq Analytics, a software requirements intelligence service, that collects, processes, analyzes, and visualizes user feedback.

Using API reference documentation like JavaDoc is an integral part of software development. Previous research introduced a grounded taxonomy that organizes API documentation knowledge in 12 types, including knowledge about the Functionality, Structure, and Quality of an API. We study how well modern text classification approaches can automatically identify documentation containing specific knowledge types. We compared conventional machine learning (k-NN and SVM) with deep learning approaches trained on manually-annotated Java and .NET API documentation (n = 5,574). When classifying the knowledge types individually (i.e., multiple binary classifiers) the best AUPRC was up to 87

App stores include an increasing amount of user feedback in form of app ratings and reviews. Research and recently also tool vendors have proposed analytics and data mining solutions to leverage this feedback to developers and analysts, e.g., for supporting release decisions. Research also showed that positive feedback improves apps’ downloads and sales figures and thus their success. As a side effect, a market for fake, incentivized app reviews emerged with yet unclear consequences for developers, app users, and app store operators. This paper studies fake reviews, their providers, characteristics, and how well they can be automatically detected. We conducted disguised questionnaires with 43 fake review providers and studied their review policies to understand their strategies and offers. By comparing 60,000 fake reviews with 62 million reviews from the Apple App Store we found significant differences, e.g., between the corresponding apps, reviewers, rating distribution, and frequency. This inspired the development of a simple classifier to automatically detect fake reviews in app stores. On a labelled and imbalanced dataset including one-tenth of fake reviews, as reported in other domains, our classifier achieved a recall of 91% and an AUC/ROC value of 98%. We discuss our findings and their impact on software engineering, app users, and app store operators.

Background: Requirement engineering is often considered a critical activity in system development projects. The increasing complexity of software as well as number and heterogeneity of stakeholders motivate the development of methods and tools for improving large-scale requirement engineering. Aims: The empirical study presented in this paper aim to identify and understand the characteristics and challenges of a platform, as desired by experts, to support requirement engineering for individual stakeholders, based on the current pain-points of their organizations when dealing with a large number requirements. Method: We conducted a multiple case study with three companies in different domains. We collected data through ten semi-structured interviews with experts from these companies. Results: The main pain-point for stakeholders is handling the vast amount of data from different sources. The foreseen platform should leverage such data to manage changes in requirements according to customers' and users' preferences. It should also offer stakeholders an estimation of how long a requirements engineering task will take to complete, along with an easier requirements dependency identification and requirements reuse strategy. Conclusions: The findings provide empirical evidence about how practitioners wish to improve their requirement engineering processes and tools. The insights are a starting point for in-depth investigations into the problems and solutions presented. Practitioners can use the results to improve existing or design new practices and tools.

Successful open source communities are constantly looking for new members and helping them become active developers. A common approach for developer onboarding in open source projects is to let newcomers focus on relevant yet easy-to-solve issues to familiarize themselves with the code and the community. The goal of this research is twofold. First, we aim at automatically identifying issues that newcomers can resolve by analyzing the history of resolved issues by simply using the title and description of issues. Second, we aim at automatically identifying issues, that can be resolved by newcomers who later become active developers. We mined the issue trackers of three large open source projects and extracted natural language features from the title and description of resolved issues. In a series of experiments, we optimized and compared the accuracy of four supervised classifiers to address our research goals. Random Forest, achieved up to 91% precision (F1-score 72%) towards the first goal while for the second goal, Decision Tree achieved a precision of 92% (F1-score 91%). A qualitative evaluation gave insights on what information in the issue description is helpful for newcomers. Our approach can be used to automatically identify, label, and recommend issues for newcomers in open source software projects based only on the text of the issues.

This document includes the welcome message from the chairs of the 26th IEEE International Requirements Engineering Conference, including an introduction to the program and main contributors.

User comments have become an essential part of online journalism. However, newsrooms are often overwhelmed by the vast number of diverse comments, for which a manual analysis is barely feasible. Identifying meta-comments that address or mention newsrooms, individual journalists, or moderators and that may call for reactions is particularly critical. In this paper, we present an automated approach to identify and classify meta-comments. We compare comment classification based on manually extracted features with an end-to-end learning approach. We develop, optimize, and evaluate multiple classifiers on a comment dataset of the large German online newsroom SPIEGEL Online and the "One Million Posts" corpus of DER STANDARD, an Austrian newspaper. Both optimized classification approaches achieved encouraging F 0.5 values between 76% and 91%. We report on the most significant classification features with the results of a qualitative analysis and discuss how our work contributes to making participation in online journalism more constructive.

Context and Motivation Natural language processing has been used since the 1980s to construct tools for performing natural language (NL) requirements engineering (RE) tasks. The RE field has often adopted information retrieval (IR) algorithms for use in implementing these NL RE tools. Problem Traditionally, the methods for evaluating an NL RE tool have been inherited from the IR field without adapting them to the requirements of the RE context in which the NL RE tool is used. Principal Ideas This panel discusses the problem and considers the evaluation of tools for a number of NL RE tasks in a number of contexts. Contribution The discussion is aimed at helping the RE field begin to consistently evaluate each of its tools according to the requirements of the tool's task.

A main advantage of app stores is that they aggregate important information created by both developers and users. In the app store product pages, developers usually describe and maintain the features of their apps. In the app reviews, users comment these features. Recent studies focused on mining app features either as described by developers or as reviewed by users. However, extracting and matching the features from the app descriptions and the reviews is essential to bear the app store advantages, e.g. allowing analysts to identify which app features are actually being reviewed and which are not. In this paper, we propose SAFE, a novel uniform approach to extract app features from the single app pages, the single reviews and to match them. We manually build 18 part-of-speech patterns and 5 sentence patterns that are frequently used in text referring to app features. We then apply these patterns with several text pre-and post-processing steps. A major advantage of our approach is that it does not require large training and configuration data. To evaluate its accuracy, we manually extracted the features mentioned in the pages and reviews of 10 apps. The extraction precision and recall outperformed two state-of-the-art approaches. For well-maintained app pages such as for Google Drive our approach has a precision of 87% and on average 56% for 10 evaluated apps. SAFE also matches 87% of the features extracted from user reviews to those extracted from the app descriptions.

Rationale refers to the reasoning and justification behind human decisions, opinions, and beliefs. In software engineering, rationale management focuses on capturing design and requirements decisions and on organizing and reusing project knowledge. This paper takes a different view on rationale written by users in online reviews. We studied 32,414 reviews for 52 software applications in the Amazon Store. Through a grounded theory approach and peer content analysis, we investigated how users argue and justify their decisions, e.g. about upgrading, installing, or switching software applications. We also studied the occurrence frequency of rationale concepts such as issues encountered or alternatives considered in the reviews and found that assessment criteria like performance, compatibility, and usability represent the most pervasive concept. We then used the truth set of manually labeled review sentences to explore how accurately we can mine rationale concepts from the reviews. Support Vector Classifier, Naive Bayes, and Logistic Regression, trained on the review metadata, syntax tree of the review text, and influential terms, achieved a precision around 80% for predicting sentences with alternatives and decisions, with top recall values of 98%. On the review level, precision was up to 13% higher with recall values reaching 99%. We discuss the findings and the rationale importance for supporting deliberation in user communities and synthesizing the reviews for developers.

In this paper, we take up the second RE17 data challenge: the identification of requirements types using the "Quality attributes (NFR)" dataset provided. We studied how accurately we can automatically classify requirements as functional (FR) and non-functional (NFR) in the dataset with supervised machine learning. Furthermore, we assessed how accurately we can identify various types of NFRs, in particular usability, security, operational, and performance requirements. We developed and evaluated a supervised machine learning approach employing meta-data, lexical, and syntactical features. We employed under-and over-sampling strategies to handle the imbalanced classes in the dataset and cross-validated the classifiers using precision, recall, and F1 metrics in a series of experiments based on the Support Vector Machine classifier algorithm. We achieve a precision and recall up to ~92% for automatically identifying FRs and NFRs. For the identification of specific NFRs, we achieve the highest precision and recall for security and performance NFRs with ~92% precision and ~90% recall. We discuss the most discriminating features of FRs and NFRs as well as the sampling strategies used with an additional dataset and their impact on the classification accuracy.

Developers’ tasks are often interrelated. A task might succeed, precede, block, or depend on another task. Or, two tasks might simply have a similar aim or require similar expertise. When working on tasks, developers interact with artifacts and tools, which constitute the contexts of the tasks. This work investigates the extent to which the similarity of the contexts predicts whether and how the respective tasks are related. The underlying assumption is simple: if during two tasks the same artifacts are touched or similar interactions are observed, the tasks might be interrelated. We define a task context as the set of all developer’s interactions with the artifacts during the task. We then apply Jaccard index, a popular similarity measure to compare two contexts. Instead of only counting the artifacts in the intersection and union of the contexts as Jaccard does, we scale the artifacts with their relevance to the task. For this, we suggest a simple heuristic based on the Frequency, Duration, and Age of the interactions with the artifacts (FDA). Alternatively, artifact relevance can be estimated by the Degree-of-Interest (DOI) used in task-focused programming. To compare the accuracy of the context similarity models for predicting task relationships, we conducted a field study with professionals, analyzed data from the open source task repository Bugzilla, and ran an experiment with students. We studied two types of relationships useful for work coordination (dependsOn and blocks) and two types useful for personal work management (isNextTo and isSimilarTo). We found that context similarity models clearly outperform a random prediction for all studied task relationships. We also found evidence that, the more interrelated the tasks are, the more accurate the context similarity predictions are. Our results show that context similarity is roughly as accurate to predict task relationships as comparing the textual content of the task descriptions. Context and content similarity models might thus be complementary in practice, depending on the availability of text descriptions or context data. We discuss several use cases for this research, e.g. to assist developers choose the next task or to recommend other tasks they should be aware of.

Nowadays, mobile apps are used for nearly every situation: for planning the day, communicating with colleagues, ordering goods, or entertaining and socializing. To understand users expectations in each situation and to provide context-aware services, researchers and app vendors started to capture users' interaction with the smartphone and to model user's behavior. This paper reports on a behavioral study based on app usage data logged over one year and the corresponding apps descriptions from the app store. Using Topic Modeling and clustering techniques, we segmented the usage data into meaningful clusters that correspond to different "states", in which users normally use their smartphone, e.g. socializing or consuming media. Researchers and app-vendors can use the insights from our work to improve their contextual recommendation techniques and the overall usage experience.

A large part of Requirements Engineering is concerned with involving system users, capturing their needs, and getting their feedback. As users are becoming more and more demanding, markets and technologies are evolving fast, and systems are getting more and more individual, a broad and systematic user involvement in Requirements Engineering is becoming more important than ever. This paper presents the idea of pushing user involvement in Requirements Engineering to its extreme by systematically delegating the responsibility for developing the requirements and deciding about future releases to the crowd of users. We summarize the pros and cons of this vision, its main challenges, and sketch promising solution concepts, which have been proposed and used in E-Participation and E-Democracy. We discussed our vision with ten experts from the fields of Requirements Engineering, politics, psychology, and market research, who were partly supportive partly skeptical.

App stores like Google Play and Apple AppStore have over 3 Million apps covering nearly every kind of software and service. Billions of users regularly download, use, and review these apps. Recent studies have shown that reviews written by the users represent a rich source of information for the app vendors and the developers, as they include information about bugs, ideas for new features, or documentation of released features. This paper introduces several probabilistic techniques to classify app reviews into four types: bug reports, feature requests, user experiences, and ratings. For this we use review metadata such as the star rating and the tense, as well as, text classification, natural language processing, and sentiment analysis techniques. We conducted a series of experiments to compare the accuracy of the techniques and compared them with simple string matching. We found that metadata alone results in a poor classification accuracy. When combined with natural language processing, the classification precision got between 70-95% while the recall between 80-90%. Multiple binary classifiers outperformed single multiclass classifiers. Our results impact the design of review analytics tools which help app vendors, developers, and users to deal with the large amount of reviews, filter critical reviews, and assign them to the appropriate stakeholders.

Data privacy when using online systems like Facebook and Amazon has become an increasingly popular topic in the last few years. However, only a little is known about how users and developers perceive privacy and which concrete measures would mitigate their privacy concerns. To investigate privacy requirements, we conducted an online survey with closed and open questions and collected 408 valid responses. Our results show that users often reduce privacy to security, with data sharing and data breaches being their biggest concerns. Users are more concerned about the content of their documents and their personal data such as location than about their interaction data. Unlike users, developers clearly prefer technical measures like data anonymization and think that privacy laws and policies are less effective. We also observed interesting differences between people from different geographies. For example, people from Europe are more concerned about data breaches than people from North America. People from Asia/Pacific and Europe believe that content and metadata are more critical for privacy than people from North America. Our results contribute to developing a user-driven privacy framework that is based on empirical evidence in addition to the legal, technical, and commercial perspectives.

We report on an exploratory study, which aims at understanding how development tutorials are structured, what types of tutorials exist, and how official tutorials differ from tutorials written by development communities. We analyzed over 1.200 tutorials for mobile application development provided by six different sources for the three major platforms: Android, Apple iOS, and Windows Phone. We found that a typical tutorial contains around 2700 words distributed over 4 pages and including a list of instructions with 18 items. Overall, 70% of the tutorials contain source code examples and a similar fraction contain images. On average, one tutorial has 6 images. When analyzing the images, we found that the studied iOS community posted the largest number of images, 14 images per tutorial, on average, from which 74% are plain images, i.e., mainly screenshots without stencils, diagrams, or highlights. In contrast, 36% of the images included in the official tutorials by Apple were diagrams or images with stencils. Community sites seem to follow a similar structure to the official sites but include items and images which are rather underrepresented in the official tutorials. From the analysis of the tutorials content by means of natural language processing combined with manual content analysis, we derived four categories for mobile development tutorials: infrastructure and design, application and services, distribution and maintenance, and development platform. Our categorization can help tutorial writers to better organize and evaluate the content of their tutorials and identify missing tutorials.

App stores allow users to submit feedback for downloaded apps in form of star ratings and text reviews. Recent studies analyzed this feedback and found that it includes information useful for app developers, such as user requirements, ideas for improvements, user sentiments about specific features, and descriptions of experiences with these features. However, for many apps, the amount of reviews is too large to be processed manually and their quality varies largely. The star ratings are given to the whole app and developers do not have a mean to analyze the feedback for the single features. In this paper we propose an automated approach that helps developers filter, aggregate, and analyze user reviews. We use natural language processing techniques to identify fine-grained app features in the reviews. We then extract the user sentiments about the identified features and give them a general score across all reviews. Finally, we use topic modeling techniques to group fine-grained features into more meaningful high-level features. We evaluated our approach with 7 apps from the Apple App Store and Google Play Store and compared its results with a manually, peer-conducted analysis of the reviews. On average, our approach has a precision of 0.59 and a recall of 0.51. The extracted features were coherent and relevant to requirements evolution tasks. Our approach can help app developers to systematically analyze user opinions about single features and filter irrelevant reviews.

Business rules represent constraints in a domain, which need to be taken into account either during the development or the usage of a system. Motivated by the knowledge reuse potentials when developing systems within the same domain, we studied business rules in a large software company. We interviewed 11 experienced practitioners on how they understand, capture, and use business rules. We also studied the role of business rules in requirements engineering in the host organization. We found that practitioners have a very broad perception for this term, ranging from flows of business processes to directives for calling external system interfaces. We identified 27 types of rules, which are typically captured as a free text in requirements documents and other project documentation. Practitioners stated the need to capture this tacit form of domain knowledge and to trace it to other artifacts as it impacts all activities in a software engineering project. We distill our results in 17 findings and discuss the implications for researchers and practitioners.

Research in program comprehension has evolved considerably over the past decades. However, only little is known about how developers practice program comprehension in their daily work. This article reports on qualitative and quantitative research to comprehend the strategies, tools, and knowledge used for program comprehension. We observed 28 professional developers, focusing on their comprehension behavior, strategies followed, and tools used. In an online survey with 1,477 respondents, we analyzed the importance of certain types of knowledge for comprehension and where developers typically access and share this knowledge. We found that developers follow pragmatic comprehension strategies depending on context. They try to avoid comprehension whenever possible and often put themselves in the role of users by inspecting graphical interfaces. Participants confirmed that standards, experience, and personal communication facilitate comprehension. The team size, its distribution, and open-source experience influence their knowledge sharing and access behavior. While face-to-face communication is preferred for accessing knowledge, knowledge is frequently shared in informal comments. Our results reveal a gap between research and practice, as we did not observe any use of comprehension tools and developers seem to be unaware of them. Overall, our findings call for reconsidering the research agendas towards context-aware tool support.

The first step to comprehend and fix a software bug is usually to reproduce the corresponding failure. Reproducing a failure requires information about steps to reproduce, i.e. the steps necessary to make a failure occur in the development environment. In case of an application with a user interface, steps to reproduce consist of the interactions between a user and the application that precede the failure. Unfortunately, bug reports typically lack this information. Users are either unaware of its importance to developers, are unable to describe it, or simply do not have time to report it. In this paper, we present a simple but effective and resource efficient approach to monitor interactions between users and their applications selectively at a high level of abstraction, e.g. editing operations and commands. This minimizes the monitoring overhead and enables developers to analyze user interaction traces. We map monitored interactions to a taxonomy of user interactions to help developers comprehend user behavior. Further, we present the Timeline Tool that visualizes monitored interaction traces preceding failures. To evaluate our approach we conducted an experiment with 12 participants and asked them to reproduce bug reports from an open-source project. We found that developers are able to derive steps to reproduce from monitored interaction traces. In particular, inexperienced developers profit from the Timeline Tool, as they are able to reproduce failures that they cannot reproduce without it. The monitoring overhead is rather small (approx. 5% CPU and 2-5% memory) and users feel it does not influence their work in a negative way.

Application distribution platforms - or app stores - such as Google Play or Apple AppStore allow users to submit feedback in form of ratings and reviews to downloaded applications. In the last few years, these platforms have become very popular to both application developers and users. However, their real potential for and impact on requirements engineering processes are not yet well understood. This paper reports on an exploratory study, which analyzes over one million reviews from the Apple AppStore. We investigated how and when users provide feedback, inspected the feedback content, and analyzed its impact on the user community. We found that most of the feedback is provided shortly after new releases, with a quickly decreasing frequency over time. Reviews typically contain multiple topics, such as user experience, bug reports, and feature requests. The quality and constructiveness vary widely, from helpful advices and innovative ideas to insulting offenses. Feedback content has an impact on download numbers: positive messages usually lead to better ratings and vice versa. Negative feedback such as shortcomings is typically destructive and misses context details and user experience. We discuss our findings and their impact on software and requirements engineering teams.

Reading reference documentation is an important part of programming with application programming interfaces (APIs). Reference documentation complements the API by providing information not obvious from the API syntax. To improve the quality of reference documentation and the efficiency with which the relevant information it contains can be accessed, we must first understand its content. We report on a study of the nature and organization of knowledge contained in the reference documentation of the hundreds of APIs provided as a part of two major technology platforms: Java SDK 6 and .NET 4.0. Our study involved the development of a taxonomy of knowledge types based on grounded methods and independent empirical validation. Seventeen trained coders used the taxonomy to rate a total of 5,574 randomly sampled documentation units to assess the knowledge they contain. Our results provide a comprehensive perspective on the patterns of knowledge in API documentation: observations about the types of knowledge it contains and how this knowledge is distributed throughout the documentation. The taxonomy and patterns of knowledge we present in this paper can be used to help practitioners evaluate the content of their API documentation, better organize their documentation, and limit the amount of low-value content. They also provide a vocabulary that can help structure and facilitate discussions about the content of APIs.

Software maintenance and support services are key factors to the customer perception of software product quality. The overall goal of FastFix is to provide developers with a real-time maintenance environment that increases efficiency and reduces costs, improving accuracy in identification of failure causes and facilitating their resolution. To achieve this goal, FastFix observes application execution and user interaction at runtime. We give an overview of the functionality of FastFix and present one of its main application scenarios.

Detecting the current activity of developers and problems they are facing is a prerequisite for a context-aware assistance and for capturing developers' experiences during their work. We present an approach to detect the current activity of software developers and if they are facing a problem. By observing developer actions like changing code or searching the web, we detect whether developers are locating the cause of a problem, searching for a solution, or applying a solution. We model development work as recurring problem solution cycle, detect developer's actions by instrumenting the IDE, translate developer actions to observations using ontologies, and infer developer activities by using Hidden Markov Models. In a preliminary evaluation, our approach was able to correctly detect 72% of all activities. However, a broader more reliable evaluation is still needed.

Research in program comprehension has considerably evolved over the past two decades. However, only little is known about how developers practice program comprehension under time and project pressure, and which methods and tools proposed by researchers are used in industry. This paper reports on an observational study of 28 professional developers from seven companies, investigating how developers comprehend software. In particular we focus on the strategies followed, information needed, and tools used. We found that developers put themselves in the role of end users by inspecting user interfaces. They try to avoid program comprehension, and employ recurring, structured comprehension strategies depending on work context. Further, we found that standards and experience facilitate comprehension. Program comprehension was considered a subtask of other maintenance tasks rather than a task by itself. We also found that face-to-face communication is preferred to documentation. Overall, our results show a gap between program comprehension research and practice as we did not observe any use of state of the art comprehension tools and developers seem to be unaware of them. Our findings call for further careful analysis and for reconsidering research agendas.

Conventional software engineering processes are rather transactional and lack a common theory for the involvement of users and their communities. Users are regarded as pure consumers, who are, at most, able to report issues. In the age of easy knowledge access and social media, discounting the users of software might threaten its success. Potentially valuable experiences and volunteered resources get lost. Frustrated users might even meet in social communities to argue against the software and harm its reputation. The goal of this research is to revolutionize the role of users, dissolving the boundaries to software engineers. We propose a novel framework for increasing the software social ness, being the degree of user and community involvement in the software lifecycle. Our framework consists of a benchmark, a process, and a reference architecture. The benchmark includes metrics for assessing and monitoring software social ness. The process enables engineering teams to systematically gather and exploit user feedback in the software lifecycle. The context aware reference architecture integrates social media into software systems and the engineering infrastructure. It observes users' interactions while they use the software and proactively collects in situ feedback.

We report on an exploratory study, which aims at understanding how software developers use social media compared to conventional development infrastructures. We analyzed the blogging and the committing behavior of 1,100 developers in four large open source communities. We observed that these communities intensively use blogs with one new entry about every 8 hours. A blog entry includes 14 times more words than a commit message. When analyzing the content of the blogs, we found that most popular topics represent high-level concepts such as functional requirements and domain concepts. Source code related topics are covered in less than 15% of the posts. Our results also show that developers are more likely to blog after corrective engineering and management activities than after forward engineering and re-engineering activities. Our findings call for a hypothesis-driven research to further understand the role of social media in software engineering and integrate it into development processes and tools.

Adaptation requires a system to monitor its operational context to ensure that when changes occur, a suitable adaptation action is planned and taken at runtime. The ultimate goal of adaptation is that users get their dynamic requirements met efficiently and correctly. Context changes and users' judgment of the role of the system in meeting their requirements are drivers for adaptation. In many cases, these drivers are hard to identify by designers at design time and hard to monitor by the use of exclusively technological means by the system at runtime. In this paper, we propose Social Sensing as the activity performed by users who act as monitors and provide information needed for adaptation at runtime. Such information helps the system cope with technology limitations and designers' uncertainty. We discuss the motivation and foundations of Social Sensing and outline a set of research challenges to address in future work.

Software is created by people and for people. People are heterogeneous in their beliefs, backgrounds, and preferences. Accommodating and exploiting the social variety is crucial for successful engineering and usage of software. On the one hand, software engineering is a social activity, performed by different individuals and teams. This necessitates methodologies and tools to deal with issues such as communication, coordination, knowledge sharing, compensation, and reconciliation. On the other hand, Social Software (Internet Forums, Wikis, Social Networks, Blogs, etc.) is an expanding computing paradigm, which inherently incorporates intensive social interactions and implications. Engineering Social Software magnifies a spectrum of challenges like group requirements engineering, social-awareness, privacy, security, and trust.

Work descriptions are informal notes taken by developers to summarize work achieved in a particular session. Existing studies indicate that maintaining them is a distracting task, which costs a developer more than 30 min. a day. The goal of this research is to analyze the purposes of work descriptions, and find out if automated tools can assist developers in efficiently creating them. For this, we mine a large dataset of heterogeneous work descriptions from open source and commercial projects. We analyze the semantics of these documents and identify common information entities and granularity levels. Information on performed actions, concerned artifacts, references and new work, shows the work management purpose of work descriptions. Information on problems, rationale and experience shows their knowledge sharing purpose. We discuss how work description information, in particular information used for work management, can be generated by observing developers' interactions. Our findings have many implications for next generation software engineering tools.

If software engineering tools are not "properly integrated", they can reduce engineers' productivity. Associating and retrieving information scattered across the tools become unsystematic and inefficient. Our work provides empirical evidence on what is a "poor" and a "proper" tool integration, focusing on practitioners' perspectives. We interviewed 62 engineers and analyzed the content of their project artifacts. We identified problem situations and practices related to tool integration. Engineers agreed that tool integration approaches must support change, heterogeneity and automatic linking of change to context. To quantify our results, we conducted a field experiment with 27 and a survey with 782 subjects. We found a strong correlation between change frequency and preferred integration approaches. Particularly in projects with short release cycles, tasks should be used to link information handled by different tools. We also found that half of engineers' work is not defined as tasks. Therefore, a context-based tool integration approach is more effective than a task-based one.

Developers take notes about their work sessions, either to remember the work status and share it with collaborators, or because employers explicitly require this for project management matters. We report on an exploratory study which aims at understanding how software developers describe their work. We analyzed more than 750,000 work descriptions of about 2,000 professionals taken over 8 years in three settings. We observed several similarities in the content and time meta-data of work descriptions. Most frequent terms, such as top-30 performed activities, are used consistently. Particular templates such as ldquoACTION concerning ARTIFACT because of CAUSErdquo occur frequently. Developers described sessions that last 30-120 min. 4-16 times a day. Maintaining diaries seems to consume between 3-6% of the total work time, and in 10% of the sessions, developers did not describe their work in sufficient detail. We argue that our results make the first step towards automatically generating work diaries for software developers.