Abstract Context Reflection is a fundamental skill in software engineering education, particularly in project-based courses where students learn through extended group work and need to develop their ability to reflect iteratively throughout their work. For students to benefit from reflection, their written reflections need to be assessed so that feedback can guide and improve their reflective practice. However, manually assessing written reflections to guide reflections is time-consuming. Furthermore, assessment of reflections often results in broad, non-specific feedback for a student to improve. Objective This study builds on reflective writing frameworks to produce an eight-indicator scheme for assessing student reflections in software engineering. Furthermore, this study validates an automated classifier for assessing reflections against the framework, enabling scalable and structured feedback whilst reducing instructor workload. Method We adapted existing reflection frameworks through iterative refinement to create our eight-indicator framework. Three annotators labelled student reflection texts, establishing moderate to reliable inter-rater agreement. We then trained and evaluated multiple encoder-only transformer models and compared them with decoder-only large language models using zero-shot prompting. Results The fine-tuned RoBERTa model achieved the strongest performance, substantially outperforming decoder-only models in both accuracy and speed. The classifier demonstrated human-level agreement on most indicators whilst enabling near-instantaneous classification. We provide two model variants optimised for different assessment priorities. Conclusions Our fine-tuned encoder-only models enable efficient automated assessment of reflective writing. The framework and automated classifier offer a means to provide timely, structured feedback on student reflections in software engineering.

GenAI-based coding assistants have disrupted software development. The next generation of these tools is agent-based, operating with more autonomy and potentially without human oversight. Like human developers, AI agents require contextual information to develop solutions that are in line with the standards, policies, and workflows of the software projects they operate in. Vendors of popular agentic tools (e.g., Claude Code) recommend maintaining version-controlled Markdown files that describe aspects such as the project structure, code style, or building and testing. The content of these files is then automatically added to each prompt. Recently, AGENTS.md has emerged as a potential standard that consolidates existing tool-specific formats. However, little is known about whether and how developers adopt this format. Therefore, in this paper, we present the results of a preliminary study investigating the adoption of AI context files in 466 open-source software projects. We analyze the information that developers provide in AGENTS.md files, how they present that information, and how the files evolve over time. Our findings indicate that there is no established content structure yet and that there is a lot of variation in terms of how context is provided (descriptive, prescriptive, prohibitive, explanatory, conditional). Our commit-level analysis provides first insights into the evolution of the provided context. AI context files provide a unique opportunity to study real-world context engineering. In particular, we see great potential in studying which structural or presentational modifications can positively affect the quality of the generated content.

Empathy, i.e., the ability to understand and feel what others are going through, is essential for value-based and user-centered software development. Empathy helps software engineers fully understand client needs, but also impacts how software engineers work with each other (e.g., within their team). However, junior and less experienced software engineers may not always understand what empathy means and why it matters in a technical domain like software development (and therefore do not pursue opportunities to develop it). We present a video-based training technique for empathy of software engineers. We also show preliminary findings of using the technique in a software engineering project course for second-year software engineering students. We report on student learning, engagement, as well as the perceptions of students on the training technique.

Abstract Context: Managing data related to a software product and its development poses significant challenges for software projects and agile development teams. These include integrating data from diverse sources and ensuring data quality amidst continuous change and adaptation. Objective: The paper systematically explores data management challenges and potential solutions in agile projects, aiming to provide insights into data management challenges and solutions for both researchers and practitioners. Method: We employed a mixed-methods approach, including a systematic literature review (SLR) to understand the state-of-research followed by a survey with practitioners to reflect on the state-of-practice. The SLR reviewed 45 studies, identifying and categorizing data management aspects along with their associated challenges and solutions. The practitioner survey captured practical experiences and solutions from 32 industry practitioners who were significantly involved in data management to complement the findings from the SLR. Results: Our findings identified major data management challenges in practice, such as managing data integration processes, capturing diverse data, automating data collection, and meeting real-time analysis requirements. To address these challenges, solutions such as automation tools, decentralized data management practices, and ontology-based approaches have been identified. These solutions enhance data integration, improve data quality, and enable real-time decision-making by providing flexible frameworks tailored to agile project needs. Conclusion: The study pinpointed significant challenges and actionable solutions in data management for agile software development. Our findings provide practical implications for practitioners and researchers, emphasizing the development of effective data management practices and tools to address those challenges and improve project success.

Project-based software engineering courses often incorporate industry practices, e.g., the “Daily Scrum” (i.e., daily stand-ups). While the main purpose of the Daily Scrum is to discuss progress and blockers, we can also utilize them as an educational tool to facilitate student engagement and reflection since reflection is a crucial aspect of experience-based learning. This paper seeks to enhance reflective practices in software engineering project courses through a pre-meeting check-in add-on to the Daily Scrum. We also aim to assess if this increases self-reflection and consequently impacts student engagement and performance. To this end, we adapted the Daily Scrum in a software engineering project course with a pre-meeting semi-guided self-reflection on a student's work. This was supported by a bespoke online tool. Effects of the reflection were analyzed from the student perspective using surveys, application usage data, formal assessment data, and observations from teaching staff. From our experiences, we observed the pre-meeting check-in was beneficial in enhancing student self-reflectivity. Quan-titative data, supported by qualitative evidence from student feedback, also suggested positive effects on student engagement and performance. Students who used pre-meeting check-ins reported increased self-reflection and showed greater levels of improvement in assessed performance. Based on our experiences we recommend incorporating regular individual self-reflection sessions in software engineering project courses.

This paper recounts our experiences with the ‘ScrumBoard’, a custom-built open-source digital project man-agement tool specifically designed for software engineering project courses. We discuss how it has supported experiential learning pedagogy and assessment over multiple iterations of a two-semester, 3rdyear project course, having now been used extensively by more than 200 students. While offering a similar experience to commercial software project management tools, we reflect on how the ScrumBoard has supported our ability to assess students more accurately and efficiently, and how the tool has enabled us to identify problematic work practices in real-time, allowing for earlier interventions from teaching staff and an improved learning experience for students.

Background: Soft skills, such as the ability to communicate effectively and efficiently or work in a team, are important for software engineering practitioners. Understanding which soft skills are necessary for software professionals can assist in staff recruitment, training, and curriculum development. Various previous works have explored soft skills in job adverts in different contexts (i.e. based on what employers ask for). Aims: In this work, we rely on a study performed in New Zealand, and conduct a similar analysis in Cyprus to explore soft skills for the software industry in a different country in a different continent. Method: A manual analysis of 689 job adverts has been used to analyze job adverts from 2023 and 2024. Qualitative analysis and descriptive statistics were mainly employed for analysis purposes. Results: We have found 36 soft skills, whereas between 2023 and 2024 there are slight differences in the number of skills present per job advert. We also encountered differences in soft skills presence based on the job position category and the company size. Conclusion: The results confirm the existing findings concerning the most in demand soft skills, showing that the software industry needs are global. Nevertheless, there are differences in the soft skills popularity. The results can be used for improving job adverts concerning listing soft skills and performing curricula updates.

In the context of video-based learning, particularly active video watching, social learning is facilitated by allowing students to review comments written by their peers. In previous studies with AVW-Space, all students had access to the same comments, despite differences in their knowledge. This paper presents a study in which each student received a personalized list of comments to review, based on their student model. The results show that our intervention encouraged learners to think deeper about the comments during reviewing comments.

To learn effectively by watching videos, learners need to engage actively with video content. Writing comments on videos (also known as video annotation) is a common way of engagement in Active Video Watching (AVW). Reviewing comments on videos written by peers, as a form of social learning, has also been shown to increase learning. In this paper, we extended social learning in AVW by enabling learners to respond to comments written by their peers (rather than just reviewing peer comments) and replacing a categorical comment rating design with a numerable binary one. We investigated the impact of using such a form of comment reviewing on learning, engagement, and students' perceptions. The findings show that our intervention results in increased learning, engagement and student satisfaction compared to the situation when students could not respond to comments.

Active Video Watching supports engagement through scalable interventions, such as notetaking in the form of comments. Machine Learning is used to categorize comments based on their quality to provide personalized feedback to students. In previous work on AVW-Space, an online portal for active video watching, a machine learning model was trained using data from several studies on presentation skills. In this paper, we explore the effectiveness in assessing the comment quality of this model in Face-to-Face Meeting Communication skills in comparison to a model trained specifically for this soft skill. We used Explainable Artificial Intelligence to identify and compare the important features of the models. Results show the need for comment quality assessment models to be specific to the soft skill in question and show major differences between their important features, highlighting the necessity to create a model specific to a particular soft skill.

Recent studies on Explainable Artificial Intelligence (XAI) in education show benefits for student learning. However, integrating XAI in AI-based education (AIED) systems requires understanding students' explanation needs. Some approaches to adding XAI to AIED systems include participatory design and co-design involving learners. This study presents a participatory approach to implement explanations in Active Video Watching (AVW). We designed explanations based on the requirements on timing and presentation of explanations and additional feedback from learners during the participatory activity. The implemented explanations support students who made low to medium-quality comments on video content by explaining how comment quality was determined. Furthermore, explanations included recommendations to improve future comments. We present the results of a pilot study on explanations in an AVW platform.

Identifying the root cause of a software bug and fixing it is challenging. One reason for this is that many bugs are not reproducible during bug fixing. We aim to provide an overview of existing works on bug reproduction and localization. We ask four research questions: RQ1: What types of problems have been studied in the area of bug reproduction and localization? RQ2: How are problems studied in previous research? RQ3: What are the main findings and outcomes of previous studies? RQ4: What are the gaps and challenges identified in previous studies? We conducted a systematic mapping study analyzing research literature published between 2011 and 2021. The search for primary studies involved four major computer science digital libraries and resulted in 134 studies for analysis. Regarding RQ1 we found that many studies focus on information retrieval-based approaches to support bug reproduction and localization. Regarding RQ2 we found that bug reports and source code are the typical data sources of bug reproduction and localization. Also, most studies include experiments with historical data but do not investigate ongoing projects. Regarding RQ3 we found that many studies adapt or combine existing approaches for bug reproduction and localization to improve their accuracy or applicability (e.g., combine requirements-related information and bug reports to increase information-retrieval-based techniques). Regarding RQ4 we found that existing solutions for bug reproduction and localization have rarely been integrated into the workflow of developers. Although bug reproduction and localization have been studied in quite some detail, new challenges and gaps emerge due to the evolution of software technologies and practices and the practical needs of software developers. For example, bug reproduction approaches for traditional web applications do not work well with modern “Single Page Web Applications” (SPA) and related technologies, e.g., Angular or React.

A/B testing, also referred to as online controlled experimentation or continuous experimentation, is a form of hypothesis testing where two variants of a piece of software are compared in the field from an end user’s point of view. A/B testing is widely used in practice to enable data-driven decision making for software development. While a few studies have explored different facets of research on A/B testing, no comprehensive study has been conducted on the state-of-the-art in A/B testing. Such a study is crucial to provide a systematic overview of the field of A/B testing driving future research forward. To address this gap and provide an overview of the state-of-the-art in A/B testing, this paper reports the results of a systematic literature review that analyzed primary studies. The research questions focused on the subject of A/B testing, how A/B tests are designed and executed, what roles stakeholders have in this process, and the open challenges in the area. Analysis of the extracted data shows that the main targets of A/B testing are algorithms, visual elements, and workflow and processes. Single classic A/B tests are the dominating type of tests, primarily based in hypothesis tests. Stakeholders have three main roles in the design of A/B tests: concept designer, experiment architect, and setup technician. The primary types of data collected during the execution of A/B tests are product/system data, user-centric data, and spatio-temporal data. The dominating use of the test results are feature selection, feature rollout, continued feature development, and subsequent A/B test design. Stakeholders have two main roles during A/B test execution: experiment coordinator and experiment assessor. The main reported open problems are related to the enhancement of proposed approaches and their usability. From our study we derived three interesting lines for future research: strengthen the adoption of statistical methods in A/B testing, improving the process of A/B testing, and enhancing the automation of A/B testing.

When modeling topics from chat messages of developer instant messaging communication, individual chat messages are short text documents. Our study aims at understanding how short text topic models perform with conversations from developer instant messaging. We applied four models to nine Gitter chat rooms (with sizes ranging from ≈100 to ≈160,000 messages). To assess the quality of topics and identify the best performing models, we compared topics based on four metrics for topic coherence. Furthermore, for a subset of Gitter chat rooms we used two human-based assessments: intrusion tasks with 18 experts analyzing 40 topics each, and topic naming (assigning a name to a topic that summarizes its main concept) with eight additional experts naming 60 topics each. Models performed differently in terms of coherence metrics and human assessment depending on the corpus (small, medium or large chat room). Our findings offer recommendations for the selection and use of short text topic models with developer chat messages based on characteristics of models and their performance with different sizes of corpora, and based on different strategies to assess topic quality.

Web applications are often built as Single Page Applications (SPA), for example applications offered by Google, Facebook, Twitter or Netflix. Users interact with SPAs through a single HTML page that is dynamically rewritten with new data from the web server (instead of a web browser that loads entire new HTML pages). Just like with any type of software system, debugging is a common activity during the development and maintenance of SPAs. In order to fix bugs observed during runtime, developers often try to reproduce the bug first to better understand it. However, research has shown that reproducing bugs is not always possible. In this paper we (i) develop a technique for Application Monitoring (AM) to collect data to support bug reproduction; and (ii) apply the monitoring technique in a SPA test bed as well as a real-world SPA application to show its feasibility. As part of our research we developed an initial version of the AM technique and implemented it in a prototype. Our evaluation using this prototype showed that it not only improves the efficiency of the bug reproduction process but also reduces information gaps caused by incomplete bug reports submitted by users. Additionally, compared to the information provided by users, data provided by AM is more accurate and detailed and covers a wider range of data. Future work includes deploying the AM framework in more SPAs and investigating how AM can be integrated into software developer workflows.

Poor feedback practices for professional development can stall, or even reverse, the professional growth of software engineering professionals. Yet, when setting up feedback systems, organizations often rely on experience or “anecdotal evidence” due to a lack of empirically-grounded guidelines that support those who give and those who receive feedback. Based on responses from a web-based survey, we describe initial challenges and strategies for professional feedback specifically in software engineering. We then explore how software engineering practice can progress towards more evidence-based feedback systems to ensure the continuous growth of professionals. We also present new directions for future research.

Social inclusion is a fundamental feature of thriving societies. This paper first investigates barriers for social inclusion in online Software Engineering (SE) communities, by identifying a set of 11 attributes and organising them as a taxonomy. Second, by applying the taxonomy and analysing language used in the comments posted by members in 189 Gitter projects (with > 3 million comments), it presents the evidence for the social exclusion problem. It employs a keyword-based search approach for this purpose. Third, it presents a framework for improving social inclusion in SE communities.

Presentation skills are crucial for tertiary students and graduates but are difficult to teach. We augmented active video watching (AVW) approach with the possibility of interactions among students, and conducted an experiment with AVW- Space, an online platform which supports video-based learning. The participants watched and commented on videos first. In the second phase, the participants reviewed, rated, and responded to their peers’ comments. We found that students who interacted with other students and responded to their comments increased their conceptual understanding of presentation skills.

Supporting student engagement remains one of the key challenges in video-based learning. This challenge is addressed by active video watching (AVW), a learning approach that supports engagement through different interventions, such as note-taking in the form of comments that learners submit while watching videos. One platform to support AVW is AVW-Space. Previous studies on AVW-Space detail improvements in the system, such as the integration of Artificial Intelligence and Machine Learning (ML) models in the comment feature of the system. This study investigates two machine learning models used to automatically assess the quality of comments when learning communication skills via AVW. One model is generated based on a large set of comments created by students when engaging with videos about presentation skills. For this study, a new model is developed from comments that students submitted when engaging with videos about communication skills. Results show that the new model, which was created from data on communication skills, performed better when assessing comments for communication skills compared to the model generated from comments for another skill. This has been demonstrated by the higher value of inter-rater agreement with the comment quality assessment made by human coders.

Designing explanations for Artificial Intelligence (AI) systems continues to be a challenge due to AI's black-box nature. Among the solutions developed to help in designing explanations in AI technologies is the Question-Driven Design Process for Explainable Artificial Intelligence (XAI) User Experience. In this paper, we report on our experiences using the question-driven design process for XAI in active video watching. We used Active Video Watching (AVW)-Space, an AVW platform developed at the University of Canterbury, as the context for AVW. In the question analysis process, we elicited questions from users on the AI features of the system. We conducted a survey to elicit questions from users on the AI features of the system. We conducted a survey to elicit three human raters categorized the user questions into the different XAI bank categories. Results show that most users tend to ask "how" and "why" questions about the AI-enabled features in the platform. The results of the question analysis will be used in mapping the determined question categories to potential XAI techniques. This can help in deciding the types of explanations to provide to users of AVW in future works on XAI in active video watching.

Video-based learning (VBL) provides self-paced and flexible learning. However, VBL is often a passive learning method. Active video watching (AVW) has been proposed as an approach to increase learner engagement. We investigate the motivation and perception of learners towards gamification to further increase engagement in AVW. Results from a survey in New Zealand and the Philippines show a positive perception towards integrating gamification into AVW, with learners preferring a combination of game elements rather than individual elements. Our findings provide foundations for a gamification intervention in AVW.

Soft skills (e.g., communication) significantly contribute to software project success. We aim to understand (a) what are relevant soft skills in software engineering, (b) how soft skills relate to characteristics of hiring organizations, and (c) how reliably we can automatically identify soft skills in job adverts to support their continuous analysis. We focus on soft skills required by organizations in New Zealand, a country with a small but growing software sector characterized by a skills shortage, reliance on offshoring, and embedded in a bi-cultural context. We manually analyzed 530 job adverts from New Zealand’s largest portal for technology-related positions. We identified soft skills following an inductive approach, i.e., without pre-defined soft skills. We complemented the manual analysis with an automated analysis using Flexiterm (an approach for term recognition). We found explicit references to soft skills in 82% of adverts. Adverts from recruitment agencies (compared to hiring companies) included fewer soft skills. We identified 17 soft skills and proposed a contextualized software engineering description. Communication-related skills are most in demand. Soft skills related to broader human or societal values (e.g., empathy, cultural awareness) or distributed development are not common. Soft skills do not depend on company size or core business and domain of companies, or whether a company operates globally. Automatically identifying soft skills in adverts is error-prone. Employers explicitly ask for soft skills. Our findings support previous studies that highlight the importance of communication. On the other hand, identified soft skills only partially overlap with those reported in other skills classifications. Characteristics specific to New Zealand do not impact the demand for soft skills. Our findings benefit researchers in human aspects of software engineering and to those responsible for staff, curricula and professional development.

Objectives. Communication skills are crucial for effective software development teams, but those skills are difficult to teach. The goal of our project is to evaluate the effectiveness of teaching face-to-face communication skills using AVW-Space, a platform for video-based learning that provides personalized nudges to support student's engagement during video watching. Participants. The participants in our study are second-year software engineering students. The study was conducted over three years, with students enrolled in a semester-long project course. Study Method. We performed a quasi-experimental study over three years to teach face-to-face communication using AVW-Space, a video-based learning platform. We present the instance of AVW-Space we developed to teach face-to-face communication. Participants watched and commented on 10 videos and later commented on the recording of their own team meeting. In 2020, the participants ( n = 50) did not receive nudges, and we use the data collected that year as control. In 2021 ( n = 49) and 2022 ( n = 48), nudges were provided adaptively to encourage students to write more and higher-quality comments. Findings. The findings from the study show the effectiveness of nudges. We found significant differences in engagement when nudges were provided. Furthermore, there is a causal effect of nudges on the interaction time, the total number of comments written, and the number of high-quality comments, as well as on learning. Finally, participants exposed to nudges reported higher perceived learning. Conclusions. Our research shows the effect of nudges on student engagement and learning while using the instance of AVW-Space for teaching face-to-face communication skills. Future work will explore other soft skills, as well as providing explanations for the decisions made by AVW-Space.

Previous research highlighted concerns about empirical research in software engineering (e.g., reproducibility, applicability of findings). It is unclear how these concerns reflect views of those who conduct and evaluate research. Focusing on software architecture, one subfield of software engineering, we study perceptions of the research community on (1) how empirical research is applied, (2) human participants, (3) internal and external validity, and (4) replications. We collected responses from 105 key players in architecture research via a survey; we analyzed data quantitatively and qualitatively. Although respondents do generally not prefer either quantitative or qualitative research, around 40% express a preference for various reasons. Professionals are the preferred participants; there is no consensus on the value of student participants. Also, there is no consensus on when to focus on internal or external validity. Most respondents value replications, but acknowledge difficulties. A comparison with published research shows differences between how the community thinks research should be done. We provide evidence that consensus about empirical research is limited. Findings have implications for conducting and reviewing empirical research (e.g., training researchers and reviewers), and call for reflection on empirical research (e.g., to resolve conflicts). We outline actions for the future.

Computing systems form the backbone of many areas in our society, from manufacturing to traffic control, healthcare, and financial systems. When software plays a vital role in the design, construction, and operation, these systems are referred to as software-intensive systems. Self-adaptation equips a software-intensive system with a feedback loop that either automates tasks that otherwise need to be performed by human operators or deals with uncertain conditions. Such feedback loops have found their way to a variety of practical applications; typical examples are an elastic cloud to adapt computing resources and automated server management to respond quickly to business needs. To gain insight into the motivations for applying self-adaptation in practice, the problems solved using self-adaptation and how these problems are solved, and the difficulties and risks that industry faces in adopting self-adaptation, we performed a large-scale survey. We received 184 valid responses from practitioners spread over 21 countries. Based on the analysis of the survey data, we provide an empirically grounded overview the of state of the practice in the application of self-adaptation. From that, we derive insights for researchers to check their current research with industrial needs, and for practitioners to compare their current practice in applying self-adaptation. These insights also provide opportunities for applying self-adaptation in practice and pave the way for future industry-research collaborations.

Unit and automated acceptance testing have different objectives (e.g., testing units of code versus testing complete features). Testing practices (e.g., test-first, model-based) used for one “level” of testing (for either unit or acceptance testing) may require knowledge and skills that are not applicable to the other. This makes it difficult for practitioners to gain the skills required to effectively test at all levels and form a cohesive testing strategy. The aim of this systematic literature review is to understand whether there are any automated unit testing practices that have similarities with automated acceptance testing practices (and vice versa). Understanding these similarities can enable skill transfer across testing activities at different levels. This systematic literature review focuses on empirical research with an industry focus. We found that test-driven development (TDD) and model-based test generation (MBTG) are two practices widely researched for both unit testing and acceptance testing. For TDD we found that a design- and test-first mindset is required and helpful at both the unit and acceptance levels, but practitioners struggle with that practice. For MBTG we found that, despite its ability to increase code coverage, the additional manual effort to enable automated test generation may outweigh its benefits.

Background: Soft skills of software professionals (e.g., communication, interpersonal skills) significantly contribute to project and product success. Aims: We aim to understand (a) what are relevant soft skills in software engineering, (b) how soft skills relate to types of software engineering positions, and (c) how soft skills relate to characteristics of hiring organizations. We focus on organizations in New Zealand, a country with a relatively small but growing software sector characterized by a skills shortage and embedded in a bi-cultural context. Method: We used a qualitative research method and manually analyzed 530 job adverts from New Zealand’s largest job portal for technology-related positions. We identified soft skills following an inductive approach, i.e., without a pre-defined set of soft skills. Results: We found explicit references to soft skills in 82% of adverts. We identified 17 soft skills and proposed a contextualized software engineering description. Communication-related soft skills are most in demand, regardless of the type of position. Soft skills related to broader human or societal values (e.g., empathy or cultural awareness) or distributed development are not frequently requested. Soft skills do not depend on company size or core business. Conclusions: Employers explicitly ask for soft skills. Our findings support previous studies that highlight the importance of communication. Characteristics specific to New Zealand do not impact the demand for soft skills. Our findings benefit researchers in human aspects of software engineering and to those responsible for staff, curricula and professional development.

Software developers use instant messaging (e.g., Slack, Gitter) to collaboratively discuss software engineering problems and solutions. This communication takes place in chat rooms that generally contain a description of the main topic of discussion and the messages exchanged. To analyze whether and how the knowledge accumulated in these chat rooms is relevant to other developers, we first need to understand the themes discussed in these chat rooms. In this paper, we used thematic analysis to manually identify software engineering themes in the description of 87 chat rooms of Gitter, an instant messaging tool for software developers. Then, we checked whether these themes also occur in 184 public chat rooms of Slack, another instant messaging tool. We identified 47 themes in Gitter chat rooms, and regarding the applicability of themes, we could relate 36 of our themes to 173 Slack chat rooms. Our results indicate that, in the context of our study, chat rooms in developer instant messaging communication are mostly about software development technologies and practices rather than development processes. Furthermore, most chat rooms are topic- rather than project-related (e.g., a chat room used by developers of a particular software development project).

Although communi cation skills are widely recognized as crucial for effective software development teams, many graduates lack such skills, which are difficult to teach. We adopt the active video watching (AVW) approach to teach face -to-face communication skills to second -year software engineering project course. We conducted an experiment with AVW -Space, an online platform which supports video- based learning . The participants watched and commented on tutorial videos first, and later on provided videos of meetings. In the last phase, the participants commented on the recording of their own team meeting. We found that students who commented and rated others’ comments increase d their conceptual understanding of face -to-face communication skills.

Abstract Topic modeling using models such as Latent Dirichlet Allocation (LDA) is a text mining technique to extract human-readable semantic “topics” (i.e., word clusters) from a corpus of textual documents. In software engineering, topic modeling has been used to analyze textual data in empirical studies (e.g., to find out what developers talk about online), but also to build new techniques to support software engineering tasks (e.g., to support source code comprehension). Topic modeling needs to be applied carefully (e.g., depending on the type of textual data analyzed and modeling parameters). Our study aims at describing how topic modeling has been applied in software engineering research with a focus on four aspects: (1) which topic models and modeling techniques have been applied, (2) which textual inputs have been used for topic modeling, (3) how textual data was “prepared” (i.e., pre-processed) for topic modeling, and (4) how generated topics (i.e., word clusters) were named to give them a human-understandable meaning. We analyzed topic modeling as applied in 111 papers from ten highly-ranked software engineering venues (five journals and five conferences) published between 2009 and 2020. We found that (1) LDA and LDA-based techniques are the most frequent topic modeling techniques, (2) developer communication and bug reports have been modelled most, (3) data pre-processing and modeling parameters vary quite a bit and are often vaguely reported, and (4) manual topic naming (such as deducting names based on frequent words in a topic) is common.

Textual user stories capture interactions of users with the system as high-level requirements. However, user stories are typically rather short and backlogs can include many stories. This makes it hard to (a) maintain user stories and backlogs, (b) fully understand the scope of a software project without a detailed analysis of the backlog, and (c) analyse how user stories impact design decisions during sprint planning and implementation. This paper proposes a technique to automatically transform textual user stories into visual use case scenarios in the form of robustness diagrams (a semi-formal scenario-based visualisation of workflows). In addition to creating diagrams for individual stories, the technique allows combining diagrams of multiple stories into one diagram to visualise workflows within sets of stories (e.g., a backlog). Moreover, the technique supports "viewpoint-based" diagrams, i.e., diagrams that show relationships between actors, domain entities and user interfaces starting from a diagram element (e.g., an actor) selected by the analyst. The technique utilises natural language processing and rule-based transformations. We evaluated the technique with more than 1,400 user stories from 22 backlogs and show that (a) the technique generates syntactically valid robustness diagrams, and (b) the quality of automatically generated robustness diagrams compares to the quality of diagrams created by human experts, but depends on the quality of the textual user stories.

There are two kinds of network data: Network telemetry (e.g. packet counters) and business data (e.g. user roles). Existing approaches to querying network data keep these separate, increasing the number and complexity of queries users must write to answer questions about networks. We present Scout, a framework for creating tools which combine these two types of data. It is comprised of: An information model which can represent both network telemetry and business-domain data in use-case-specific schemas; a nascent query language for this information model; and an algorithm for executing queries on schemas. A preliminary evaluation showed that a Scout-based tool can answer questions pertaining to both network telemetry and business data, and reduces the knowledge and number of queries needed to answer realistic questions about networks.

Context: Systematic literature reviews and mapping studies usually rely on automated searches of digital libraries to identify primary studies. Defining proper search strings, executing semantically similar searches on different libraries, and reporting limitations of searches increase the reliability of secondary studies. Objective: We aim to survey the current state of using automated searches in secondary software engineering studies. In particular, we aim at analyzing how automated searches are reported and at understanding the reproducibility of secondary studies. Method: We perform a preliminary tertiary study that covers 50 recently published representative secondary studies from different software engineering venues and subfields. Results: We found that most secondary studies complement an automated search with a manual search and use four or more digital libraries. Also, we found that the quality of reporting search strings is rather poor. Finally, we found that most secondary studies do not acknowledge limitations of automated searches and implications of limitations on study findings. Conclusions: Our findings highlight implications for researchers (e.g., to properly report the search process) and for reviewers (e.g., to execute search strings reported in papers). Also, our findings indicate that secondary studies are difficult to replicate.

Background: Development processes and practices depend on the context in which software is developed (e.g., locations, organizations, projects, developers). Aim: We aim at understanding development processes and practices in New Zealand, a country with a relatively small but growing software sector. We are particularly interested in methods and practices used in such environment, the implementation technologies software development professionals use, how professionals ensure software quality, and how they manage software release processes. Method: We conducted a descriptive survey targeting individual software development professionals working in New Zealand software companies. Results: New Zealand professionals use similar methodologies as professionals in other countries. Popular programming languages differ somewhat to popular languages in other rankings. Quality assurance is rather ad-hoc and the release process is inspired by agile software development principles. Conclusions: Our findings highlight some differences of the New Zealand software industry to other countries. Furthermore, we identified some strengths and weaknesses related to processes and practices. Our findings can help software professionals and organizations reflect on (and potentially adjust) the way they work.

To make good architecture design decisions, developers need to know suitable architecture solution alternatives. However, with the rapid and continuous increase of solution alternatives (e.g. technologies, patterns, tactics), it is challenging to acquire architecture knowledge and to ensure that this knowledge is up to date. Our goal is to improve how architects search for architecturally relevant information in online developer communities. We developed a new search approach for architecturally relevant information using Stack Overflow as an example of an online developer community. Our search approach differs from a conventional keyword-based search in that it considers semantic information of architecturally relevant concepts in Stack Overflow. We also implemented the search approach as a web-based search engine. To show the effectiveness of the search approach compared to a conventional keyword-based search, we conducted an experiment with 16 practitioners. To ensure realism of the experiment, tasks given to practitioners are based on real scenarios identified in interviews with a different set of practitioners. The experiment showed that the new search approach outperforms a conventional keyword based search.

The study presented in this paper aims to provide evidence for the hypothesis that software product line research has been changing and that the works in industry and academia have diverged over time. We analysed a subset (140) of all (593) papers published at the Software Product Line Conference (SPLC) until 2017. The subset was randomly selected to cover all years as well as types of papers. We assessed the research type of the papers (academic or industry), the kind of evaluation (application example, empirical, etc.), and the application domain. Also, we assessed which product line life-cycle phases, development practices, and topics the papers address. We present an analysis of the topics covered by academic vs. industry research and discuss the evolution of these topics and their relation over the years. We also discuss implications for researchers and practitioners. We conclude that even though several topics have received more attention than others, academic and industry research on software product lines are actually rather in line with each other.

Monitoring software quality is a non-trivial task, since it requires the selection of: (a) quality attributes, based on application domain and development phase, and (b) appropriate metrics to quantify them. We aim to aid this process by reviewing the state-of-research on design-time qualities and metrics.

Context: The validity of survey-based research depends on, amongst other considerations, the number and validity of obtained data points. As with any empirical study that involves practitioners, collecting data via surveys is difficult. Objectives: We report our experiences derived while conducting an industry survey on the impact of agile practices on software process quality. Method: After unsuccessfully trying to collect data with an online questionnaire, we used an interactive approach with posters at practitioners-focussed software engineering community events to aid data collection. Results: We present a list of lessons learnt. In particular, the poster-based data collection approach provided utility, for both gathering a large amount of responses and facilitating follow-up interactions with study participants. Conclusion: Our experiences in this work may help those facing challenges associated with obtaining responses from practitioners through the use of potentially complex questionnaires.

To satisfy security requirements, software architects often adopt security tactics. These architectural tactics provide mechanisms for resisting, detecting, reacting to and recovering from attacks. Consequently, flaws in the implementation of security tactics or their deterioration during software evolution and maintenance can introduce severe vulnerabilities that could be exploited by attackers. However, we currently lack an in-depth understanding of the types and impact of vulnerabilities related to security tactics. Therefore, in this paper, we conduct a first-of-its-kind in-depth case study involving three large-scale open-source systems. We investigate the most common types of vulnerabilities associated with security tactics, how frequently they may occur over time, and how fixing them differs from fixing vulnerabilities that are not related to security tactics. Key findings are (i) most tactic-related vulnerabilities were related to the tactics "Validate Inputs" and "Authorize Actors", (ii) vulnerabilities related to tactics have a similar distribution over time and software releases as vulnerabilities that are not related to tactics, (iii) fixing tactic-related vulnerabilities is not necessarily more complex than fixing vulnerabilities that are not related to security tactics. This study highlights the importance of ensuring an appropriate implementation of security-related design decisions in code to avoid vulnerabilities rooted in the architecture.

Software architecting is a knowledge-intensive activity. However, obtaining and evaluating the quality of relevant and reusable knowledge (and ensuring that this knowledge is up-to-date) requires significant effort. In this paper, we explore how online developer communities (e.g., Stack Overflow), traditionally used by developers to solve coding problems, can help solve architectural problems. We develop an ontology that covers architectural knowledge concepts in Stack Overflow. The ontology provides a description of architecture-relevant information to represent and structure architectural knowledge in Stack Overflow. The ontology is empirically grounded through qualitative analyses of different Stack Overflow posts, as well as inter-coder reliability tests. Our results show that the architecture knowledge ontology in Stack Overflow captures architecture-relevant information and supports achieving practitioners' requirements and concerns.

Architectural tactics such as heartbeat, resource pooling, and scheduling provide solutions to satisfy reliability, security, performance, and other critical characteristics of a software system. Current design practices advocate rigorous up-front analysis of the system's quality concerns to identify tactics and where in the code they should be used. In this paper, we explore a bottom-up approach to recommend architectural tactics based on latent topics discovered in the source code of projects. We present a recommender system developed by building predictor models which capture relationships between topical concepts in source code and the use of specific architectural tactics in that code. Based on an extensive analysis of over 116,000 open source systems, we identify significant correlations between latent topics in source code and the usage of architectural tactics. We use this information to construct a predictor for generating tactic recommendations. Our approach is validated through a series of experiments which demonstrate the ability to generate package-level tactic recommendations. We provide further validation via two large-scale studies of Apache Hive and Hadoop to illustrate that our recommender system predicts tactics that are actually implemented by developers in later releases.

Software reference architectures provide templates and guidelines for designing systems in a particular domain. Companies use them to achieve interoperability of (parts of) their software, standardization, and faster development. In contrast to system-specific software architectures that "emerge" during development, reference architectures dictate significant parts of the software design early on. Agile software development frameworks (such as Scrum) acknowledge changing software requirements and the need to adapt the software design accordingly. In this paper, we present lessons learned about how reference architectures interact with Scrum (the most frequently used agile process framework). These lessons are based on observing software development projects in five companies. We found that reference architectures can support good practice in Scrum: They provide enough design upfront without too much effort, reduce documentation activities, facilitate knowledge sharing, and contribute to "architectural thinking" of developers. However, reference architectures can impose risks or even threats to the success of Scrum (e.g., to self-organizing and motivated teams).

Context: Empirical research helps gain well-founded insights about phenomena. Furthermore, empirical research creates evidence for the validity of research results. Objective: We aim at assessing the state-of-practice of empirical research in software architecture. Method: We conducted a comprehensive survey based on the systematic mapping method. We included all full technical research papers published at major software architecture conferences between 1999 and 2015. Results: 17% of papers report empirical work. The number of empirical studies in software architecture has started to increase in 2005. Looking at the number of papers, empirical studies are about equally frequently used to a) evaluate newly proposed approaches and b) to explore and describe phenomena to better understand software architecture practice. Case studies and experiments are the most frequently used empirical methods. Almost half of empirical studies involve human participants. The majority of these studies involve professionals rather than students. Conclusions: Our findings are meant to stimulate researchers in the community to think about their expectations and standards of empirical research. Our results indicate that software architecture has become a more mature domain with regards to applying empirical research. However, we also found issues in research practices that could be improved (e.g., when describing study objectives and acknowledging limitations).

Architectural decisions have a big influence on basic properties of a software system, and are difficult to change once implemented. Technology decisions deserve special attention because they are one of the most frequently occurring types of architectural decisions. Architectural knowledge is crucial for good decisions. Current architecture knowledge management approaches try to support architects by offering a rich base of architectural solutions and design decision rules. However, they mostly depend on manually capturing and maintaining the architectural knowledge. In this paper, we utilize the most popular online software development community (StackOverflow) as a source of knowledge for technology decisions to support architecture knowledge management approaches with a more efficient methods for knowledge capturing. We conducted an exploratory study, and followed a qualitative and quantitative content analysis approach. We analysed the posts in this community to identify architecture-relevant and technology-related knowledge, and to classify the posts into different types for the purpose of knowledge structuring. In addition, we evaluated our findings through feedback from practitioners.

The relationships and interdependencies between software requirements and the architectures of software-intensive systems are described in the Twin Peaks model. The fundamental idea of the Twin Peaks model is that Requirements Engineering and Software Architecture should not be treated in isolation. Instead, we need to progressively discover and specify requirements while concurrently exploring alternative architectural solutions. However, bridging the gap between Requirements Engineering and Software Architecture has mainly been discussed independently in the respective communities. Therefore, this ICSE workshop aims at bringing together researchers, practitioners and educators from the Requirements Engineering and Software Architecture fields to jointly explore the strong interdependencies between requirements and architecture. Based on the results from previous editions of the workshop, this edition focuses on agile software development contexts and on exploring lightweight techniques for integrating requirements and architectural thinking.

Most current literature on Software Architecture Knowledge Management (AKM) reports research aimed at developing AKM approaches or identifying challenges in and potential solutions to AKM. However, there has been little effort aimed at empirically studying and describing how AKM is practiced in real-world projects. This paper reports a study aimed at empirically understanding how AKM is currently practiced in industry. Our findings are based on a systematic and rigorous analysis of reports of AKM practices in 29 industrial projects.

Software architects make architectural decisions such as choosing architecture patterns or frameworks. Capturing architectural decisions reduces evolution costs for software systems. Existing approaches overlook the challenge of capturing tacit knowledge about architectural decisions. Previously, we proposed the REGAIN approach to capture tacit knowledge about architectural decisions. REGAIN is based on the Repertory Grid technique, a powerful knowledge acquisition approach from knowledge engineering. However, REGAIN lacked industrial validation. Therefore, in this paper, we initially present a study to ensure that REGAIN meets the needs of industrial architects. We interviewed 16 architects who indicated REGAIN advantages such as systematic decision-making support. Also, architects indicated improvement opportunities, in particular tool support and the possibility to prioritize concerns that are used to evaluate decision alternatives. Therefore, we conducted an additional study to evaluate two approaches for prioritizing concerns: pairwise-comparisons and the hundred-dollar approach. We conducted an experiment with 30 graduate students to compare the two prioritization approaches. Based on the results of the experiment, we added the hundred-dollar approach to REGAIN. Moreover, based on the study with architects and the experiment with students, we implemented open source tool support for REGAIN, which architects can use to capture architectural decisions.

Context: Variability (i.e., the ability of software systems or artifacts to be adjusted for different contexts) became a key property of many systems. Objective: We analyze existing research on variability in software systems. We investigate variability handling in major software engineering phases (e.g., requirements engineering, architecting). Method: We performed a systematic literature review. A manual search covered 13 premium software engineering journals and 18 premium conferences, resulting in 15,430 papers searched and 196 papers considered for analysis. To improve reliability and to increase reproducibility, we complemented the manual search with a targeted automated search. Results: Software quality attributes have not received much attention in the context of variability. Variability is studied in all software engineering phases, but testing is underrepresented. Data to motivate the applicability of current approaches are often insufficient; research designs are vaguely described. Conclusions: Based on our findings we propose dimensions of variability in software engineering. This empirically grounded classification provides a step towards a unifying, integrated perspective of variability in software systems, spanning across disparate or loosely coupled research themes in the software engineering community. Finally, we provide recommendations to bridge the gap between research and practice and point to opportunities for future research.

The disciplines of requirements engineering (RE) and software architecture (SA) are fundamental to the success of software projects. Even though RE and SA are often considered separately, it has been argued that drawing a line between RE and SA is neither feasible nor reasonable as requirements and architectural design processes impact each other. Requirements are constrained by what is feasible technically and also by time and budget restrictions. On the other hand, feedback from the architecture leads to renegotiating architecturally significant requirements with stakeholders. The topic of bridging RE and SA has been discussed in both the RE and SA communities, but mostly independently. Therefore, the motivation for this ICSE workshop is to bring both communities together in order to identify key issues, explore the state of the art in research and practice, identify emerging trends, and define challenges related to the transition and the relationship between RE and SA.

Background: Many empirical software engineering studies use students as subj ects and are conducted as part of university courses.Aim: We aim at reporting our experiences with using guidelines for integrating empirical studies with our research and teaching goals.Method: We document our experience from conducting three studies with graduate students in two software architecture cou rses.Results: Our results show some problems that we faced when following the guidelines and deviations we made from the original guidelines.Conclusions: Based on our results we propose recommendations for empirical software engineering studies that are integrated in university courses.

The growing complexity of software has posed challenges for the design, understanding and maintenance of software. Software developers often spend a lot of effort on understanding how software components are connected. Through analysis of the data obtained from a real software application, this paper demonstrates that network analysis approaches can help understand software component interconnections and their impact on software quality. Additionally, this paper shows that there is a correlation between the number of bugs found in an application and a set of measures used in component (interconnection) networks. Furthermore, a case study showed that the software performance improved over two releases as a result of improving the component network.

Abstract — Background: Case studies and experiments are research methods frequently applied in empirical software engineering. Experiments are well-understood and their value as an empirical method is recognized. On the other hand, there seem to be different opinions on what constitutes a case study, and about the value of case studies as a thorough research method. Aim: We aim at exploring the attitudes of software engineering researchers on case studies and experiments. Furthermore, we investigate how the perceptions of researchers vary along their views on what constitutes a case study. Method: We performed an exploratory survey involving 26 software engineering researchers. We collected data using a paper-based questionnaire. Results: We found that participants slightly prefer experiments over case studies. Moreover, participants believe there is more useful literature on experiments, than on case studies. By analyzing two different views on the nature of case studies, we found differences in the perceived validity of case studies. Conclusions: The survey provided insights into the perceptions of researchers on case studies and experiments. Moreover, the results help reconcile different views on case studies. Keywords- empirical software engineering; case studies; experiments; survey I.

Knowledge about the architecture of a software-intensive system tends to vaporize easily. This leads to increased maintenance costs. We explore a new idea: utilizing the repertory grid technique to capture tacit architectural knowledge. Particularly, we investigate the elicitation of design decision alternatives and their characteristics. To study the applicability of this idea, we performed an exploratory study. Seven independent subjects applied the repertory grid technique to document a design decision they had to take in previous projects. Then, we interviewed each subject to understand their perception about the technique. We identified advantages and disadvantages of using the technique. The main advantage is the reasoning support it provides; the main disadvantage is the additional effort it requires. Also, applying the technique depends on the context of the project. Using the repertory grid technique is a promising approach for fighting architectural knowledge vaporization.

Variability helps manage differences and commonalities within and across software systems. As variability is reflected in and facilitated through the software architecture, it is important to understand the problems that architects face when carrying out their tasks. This would help us improve methods for architecting variability-intensive software systems. In this paper, we therefore present an exploratory study to identify problems that occur when performing variability-related tasks during software architecting. Our subjects were 27 graduate students. We identified eleven problems as experienced by the subjects of our study. The paper also presents implications of the findings for the software architecture field. In particular, we discuss implications for architecture description, methods and tools, and the training of architects.

Variability is the ability of a software artifact to be changed for a specific context. Mechanisms to accommodate variability include software product lines, configuration wizards and tools in commercial software, configuration interfaces of software components, or the dynamic runtime composition of web services. Variability is primarily reflected in and facilitated through the software architecture. Also, the software architecture is the centerpiece of software systems and acts as reference point for many development activities, and many of today's software systems are built to accommodate variability. Thus, variability in software architecture should be well-understood and be treated as a first-class concern. The software architecture community acknowledges that variability is a concern of different stakeholders, and in turn affects other concerns. Nevertheless, treating variability related to the architecture and all architecture aspects, as a cross-cutting concern, is currently not well understood. Therefore, VARSA 2011 aims at identifying critical challenges and progressing the state-of-the-art on variability in software architecture.

Planning the next release in software release planning addresses the problem of assigning features to the next release such that technical, resource, risk, and budget constraints are met. This paper studies the planning for the next release of an evolving system from a bi-criteria perspective. We introduce a method called NRP-trade-off to adjust baseline release plans for more modifiability by replacing lower value features with features having a higher modifiability. For that purpose, we include a new approach for feature modeling and assessing modifiability by applying object-oriented design metrics to the feature domain. We also briefly introduce a case study.

The transition from software requirements to software architectures has consistently been one of the main challenges during software development. Various methodologies that aim at helping with this transition have been proposed. However, no systematic approach for assessing such methodologies exists. Also, there is little consensus on the technical and non-technical issues that a transition methodology should address. Hence, we present a method for assessing and comparing methodologies for the transition from requirements to architectures. This method also helps validate newly proposed transition methodologies. The objective of such validations is to assess whether or not a methodology has the potential to lead to better architectures. For that reason, this paper discusses a set of commonly known but previously only informally described criteria for transition methodologies and organizes them into a schema. In the paper we also use our method to characterize a set of 14 current transition methodologies. This is done to illustrate the usefulness of our approach for comparing transition methodologies as well as for validating newly proposed methodologies. Characterizing these 14 methodologies also gives an overview of current transition methodologies and research.

This paper presents an approach for the ad-hoc assessment of software architecture qualities. The method allows evaluating architectures at early development stages and focuses on development qualities (e.g., coupling), rather than on runtime qualities (e.g., performance). Early feedback in the development process facilitates immediate improvement of low quality architecture artifacts. The proposed method defines three levels of architecture quality attributes, some of which are expressed as computable metrics while the other quality attributes are derived from these metrics. Throughout the paper, the example of a cruise-control system illustrates the approach.

Elicitation, modeling and analysis of requirements are main challenges during software development. In parallel, software architectures have become a well established area in software engineering research and practice. However, there is still a lack of fundamental process models and guidelines for the transition between the two important phases of requirements engineering and software architectures within the software development cycle. Substantial progress has been separately made in both areas but very little research achievements can be found that address the relation between requirements and architectures. When doing a review on existing methods that try to overcome this gap we realized an urgent need for fundamental research in this area. The paper first presents, compares, classifies and evaluates the suitability of current approaches. Then, evaluation criteria and requirements for a transition methodology are defined. Finally, directions for further research are presented