Abstract Context Continuous experimentation (CE) is used by many companies with internet-facing products to improve their business models and software solutions based on user data. Some companies deliberately adopt a systematic experiment-driven approach to software development while some companies use CE in a more ad-hoc fashion. Objective The goal of this study is to identify factors for success in CE that explain the variations in the utility and efficacy of CE between different companies. Method We conducted a multi-case study of 12 companies involved with CE and performed 27 interviews with practitioners at these companies. Based on that empirical data, we then built a theory of factors at play in CE. Results We introduce a theory of Factors Affecting Continuous Experimentation (FACE). The theory includes three factors, namely 1) processes and infrastructure for CE, 2) the user problem complexity of the product offering, and 3) incentive structures for CE. The theory explains how these factors affect the effectiveness of CE and its ability to achieve problem-solution and product-market fit. Conclusions Our theory may inspire practitioners to assess an organisation’s potential for adopting CE and to identify factors that pose challenges in gaining value from CE practices. Our results also provide a basis for defining practitioner guidelines and a starting point for further research on how contextual factors affect CE and how these may be mitigated.

Context: Software selection in large-scale software development continues to be ad hoc and ill-structured. Previous proposals for software component selection tend to be technology-specific and/or do not consider business or ecosystem concerns. Objective: Our main aim is to develop an industrially relevant technology-agnostic method that can support practitioners in making informed decisions when selecting software components for use in tools or in products based on a holistic perspective of the overall environment. Method: We used method engineering to iteratively develop a software selection method for Ericsson AB based on a combination of published research and practitioner insights. We used interactive rapid reviews to systematically identify and analyse scientific literature and to support close cooperation and co-design with practitioners from Ericsson. The model has been validated through a focus group and by practical use at the case company. Results: The model consists of a high-level selection process and a wide range of criteria for assessing and for evaluating software to include in business products and tools. Conclusions: We have developed an industrially relevant model for component selection through active engagement from a company. Co-designing the model based on previous knowledge demonstrates a viable approach to industry-academia collaboration and provides a practical solution that can support practitioners in making informed decisions based on a holistic analysis of business, organisation and technical factors.

Abstract Context Prototyping is an established practice within product and user interface design that is also used as a requirements engineering (RE) practice within agile development. Even so, there is a lack of theory on prototyping. Aims Our main research objective is to support practitioners in improving on their prototyping practices. Method We have designed a model that describes key aspects of the practice of prototyping. The model is based on a systematic mapping study consisting of thirty-three primary studies and on empirical data from twelve case companies. We validate and demonstrate the applicability of our model through a focus group at one company and through semi-structured interviews at eleven (other) startup companies. Results Our prototyping aspects model (PAM) consists of five aspects of prototyping, namely purpose, prototype scope, prototype media, prototype use, and exploration strategy. This model has enabled practitioners to discuss their prototyping practices in terms of the concepts provided by our model. Conclusions The model can be used to categorise prototyping instances and can thereby support practitioners in reflecting and improving on their prototyping practices.

Abstract Large-scale software engineering is a collaborative effort where teams need to communicate to develop software products. Managers face the challenge of how to organise work to facilitate necessary communication between teams and individuals. This includes a range of decisions from distributing work over teams located in multiple buildings and sites, through work processes and tools for coordinating work, to softer issues including ensuring well-functioning teams. In this case study, we focus on inter-team communication by considering geographical, cognitive and psychological distances between teams, and factors and strategies that can affect this communication. Data was collected for ten test teams within a large development organisation, in two main phases: (1) measuring cognitive and psychological distance between teams using interactive posters, and (2) five focus group sessions where the obtained distance measurements were discussed. We present ten factors and five strategies, and how these relate to inter-team communication. We see three types of arenas that facilitate inter-team communication, namely physical, virtual and organisational arenas. Our findings can support managers in assessing and improving communication within large development organisations. In addition, the findings can provide insights into factors that may explain the challenges of scaling development organisations, in particular agile organisations that place a large emphasis on direct communication over written documentation.

Background: Prototyping is an established practice for user interface design and for requirements engineering within agile software development, even so there is a lack of theory on prototyping. Aims: The main research objective is to provide a means to categorise prototyping instances, in order to enable comparison and reflection of prototyping practices. Method: We have performed a systematic mapping study of methodological aspects of prototyping consisting of thirty-three primary studies upon which we designed a model of prototyping that was validated through a focus group at a case company. Results: Our model consists of four aspects of prototyping, namely purpose, prototype scope, prototype use, and exploration strategy. This model supported the focus group participants in discussing prototyping practices by considering concrete prototyping instances in terms of the concepts provided by our model. Conclusions: The model can be used to categorise prototyping instances and can support practitioners in reflecting on their prototyping practices. Our study provides a starting point for further research on prototyping and into how the practice can be applied more cost-effectively to elicit, validate, and communicate requirements.

Abstract Empirical software engineering research relies on good communication with industrial partners. Conducting joint research both requires and contributes to bridging the communication gap between industry and academia (IA) in software engineering. This study aims to explore communication between the two parties in such a setting. To better understand what facilitates good IA communication and what project outcomes such communication promotes, we performed a case study, in the context of a long‐term IA joint project, followed by a validating survey among practitioners and researchers with experience of working in similar settings. We identified five facilitators of IA communication and nine project outcomes related to this communication. The facilitators concern the relevance of the research, practitioners' attitude and involvement in research, frequency of communication and longevity of the collaboration. The project outcomes promoted by this communication include, for researchers, changes in teaching and new scientific venues, and for practitioners, increased awareness, changes to practice, and new tools and source code. Besides, both parties gain new knowledge and develop social‐networks through IA communication. Our study presents empirically based insights that can provide advise on how to improve communication in IA research projects and thus the co‐creation of software engineering knowledge that is anchored in both practice and research.

The communication of requirements within software development is vital for project success. Requirements engineering and testing are two processes that when aligned can enable the discovery of issues and misunderstandings earlier, rather than later, and avoid costly and time-consuming rework and delays. There are a number of practices that support requirements-test alignment. However, each organisation and project is different and there is no one-fits-all set of practices. The software process improvement method called Gap Finder is designed to increase requirements-test alignment. The method contains two parts: an assessment part and a prescriptive part. It detects potential communication gaps between people and between artefacts (the assessment part), and identifies practices for mitigating these gaps (the prescriptive part). This paper presents the design and formative evaluation of the prescriptive part; an evaluation of the assessment part was published previously. The Gap Finder method was constructed using a design science research approach and is built on the Theory of Distances for Software Engineering, which in turn is grounded in empirical evidence from five case companies. The formative evaluation was performed through a case study in which Gap Finder was applied to an on-going development project. A qualitative and mixed-method approach was taken in the evaluation, including ethnographically-informed observations. The results show that Gap Finder can detect relevant communication gaps and seven of the nine prescribed practices were deemed practically relevant for mitigating these gaps. The project team found the method to be useful and supported joint reflection and improvement of their requirements communication. Our findings demonstrate that an empirically-based theory can be used to improve software development practices and provide a foundation for further research on factors that affect requirements communication.

Controlled experiments on software variants enable e-commerce companies to increase sales by providing user-adapted functionality. Our goal is to understand how the context of experimentation influences tool support. We performed a case study at Apptus that develops algorithms for e-commerce. We investigated how the case company uses experiments through five semi-structured interviews. We identified four main scenarios of experimentation and found that there are stark differences in tool support for them. The scenarios illustrate that the aptness of tool support for experiments depend on four characteristics: (1) what the goal of the experiment is;validateoroptimize, (2) whether the experiment is performedinternallyin the organisation orexternally, (3) whether decisions are takenautomaticallyormanually, and finally (4) whether the experiment should be repeated or is asingleton. These insight can be used by practitioners with an interest in efficient experimentation and to form a basis for further research into a taxonomy of experiments for software.

Abundant data is the key to successful machine learning. However, supervised learning requires annotated data that are often hard to obtain. In a classification task with limited resources, Active Learning (AL) promises to guide annotators to examples that bring the most value for a classifier. AL can be successfully combined with self-training, i.e., extending a training set with the unlabelled examples for which a classifier is the most certain. We report our experiences on using AL in a systematic manner to train an SVM classifier for Stack Overflow posts discussing performance of software components. We show that the training examples deemed as the most valuable to the classifier are also the most difficult for humans to annotate. Despite carefully evolved annotation criteria, we report low inter-rater agreement, but we also propose mitigation strategies. Finally, based on one annotator's work, we show that self-training can improve the classification accuracy. We conclude the paper by discussing implication for future text miners aspiring to use AL and self-training.

Background. Search and selection of primary studies in Systematic Literature Reviews (SLR) is labour intensive, and hard to replicate and update. Aims. We explore a machine learning approach to support semi-automated search and selection in SLRs to address these weaknesses. Method. We 1) train a classifier on an initial set of papers, 2) extend this set of papers by automated search and snowballing, 3) have the researcher validate the top paper, selected by the classifier, and 4) update the set of papers and iterate the process until a stopping criterion is met. Results. We demonstrate with a proof-of-concept tool that the proposed automated search and selection approach generates valid search strings and that the performance for subsets of primary studies can reduce the manual work by half. Conclusions. The approach is promising and the demonstrated advantages include cost savings and replicability. The next steps include further tool development and evaluate the approach on a complete SLR.

Empirical software engineering research aims to generate prescriptive knowledge that can help software engineers improve their work and overcome their challenges, but deriving these insights from real-world problems can be challenging. In this paper, we promote design science as an effective way to produce and communicate prescriptive knowledge. We propose using a visual abstract template to communicate design science contributions and highlight the main problem/solution constructs of this area of research, as well as to present the validity aspects of design knowledge. Our conceptualization of design science is derived from existing literature and we illustrate its use by applying the visual abstract to an example use case. This is work in progress and further evaluation by practitioners and researchers will be forthcoming.

The RET (Requirements Engineering and Testing) workshop provides a meeting point for researchers and practitioners from the two separate fields of Requirements Engineering (RE) and Testing. The goal is to improve the connection and alignment of these two areas through an exchange of ideas, challenges, practices, experiences and results. The long term aim is to build a community and a body of knowledge within the intersection of RE and Testing. One of the main outputs of the 1st workshop was a collaboratively constructed map of the area of RET showing the topics relevant to RET for these. The 2nd workshop will continue in the same interactive vein and include a keynote, paper presentations with ample time for discussions, and a group exercise. For true impact and relevance this cross-cutting area requires contribution from both RE and Testing, and from both researchers and practitioners. For that reason we welcome a range of paper contributions from short experience papers to full research papers that both clearly cover connections between the two fields.

Communication is essential for software development as its efficiency throughout the entire project life-cycle is a key factor in developing and releasing successful software products to the market. This paper reports on findings from an explanatory case study aiming at a deeper understanding of the causes and effects of communication gaps in a large-scale industrial set up. Based on an assumption of what causes gaps in communication of requirements and what effects such gaps have, a semi-structured interview study was performed with nine practitioners at a large market-driven software company. We found four main factors that affect the requirements communication, namely scale, temporal aspects, common views and decision structures. The results also show that communication gaps lead to failure to meet the customers' expectations, quality issues and wasted effort. An increased awareness of these factors is a help in identifying what to address to achieve a more efficient requirements management, and ultimately more efficient and successful software development. By closing the communication gaps the requirements may continue all the way through the project life-cycle and be more likely to result in software that meets the customers' expectations.