Abstract Context Reflection is a fundamental skill in software engineering education, particularly in project-based courses where students learn through extended group work and need to develop their ability to reflect iteratively throughout their work. For students to benefit from reflection, their written reflections need to be assessed so that feedback can guide and improve their reflective practice. However, manually assessing written reflections to guide reflections is time-consuming. Furthermore, assessment of reflections often results in broad, non-specific feedback for a student to improve. Objective This study builds on reflective writing frameworks to produce an eight-indicator scheme for assessing student reflections in software engineering. Furthermore, this study validates an automated classifier for assessing reflections against the framework, enabling scalable and structured feedback whilst reducing instructor workload. Method We adapted existing reflection frameworks through iterative refinement to create our eight-indicator framework. Three annotators labelled student reflection texts, establishing moderate to reliable inter-rater agreement. We then trained and evaluated multiple encoder-only transformer models and compared them with decoder-only large language models using zero-shot prompting. Results The fine-tuned RoBERTa model achieved the strongest performance, substantially outperforming decoder-only models in both accuracy and speed. The classifier demonstrated human-level agreement on most indicators whilst enabling near-instantaneous classification. We provide two model variants optimised for different assessment priorities. Conclusions Our fine-tuned encoder-only models enable efficient automated assessment of reflective writing. The framework and automated classifier offer a means to provide timely, structured feedback on student reflections in software engineering.

Students working in teams in software engineering group project often communicate ineffectively, which reduces the quality of deliverables, and is therefore detrimental for project success. An important step towards addressing areas of improvement is identifying which changes to communication will improve team performance the most. We applied two different communication analysis techniques, triad census and socio-technical congruence, to data gathered from a twosemester software engineering group project. Triad census uses the presence of edges between groups of three nodes as a measure of network structure, while socio-technical congruence compares the fit of a team's communication to their technical dependencies. Our findings suggest that each team's triad census for a given sprint is promising as a predictor of the percentage of story points they pass, which is closely linked to project success. Meanwhile, socio-technical congruence is inadequate as the sole metric for predicting project success in this context. We discuss these findings, and their potential applications improve communication in a software engineering group project.

In this paper, we present FETT, a fault injection tool for educational and training purposes addressed to educators and students in cybersecurity. Our tool aims to analyze and inject vulnerabilities into existing Django web applications for educational purposes. Indeed, security education often relies on either abstract theoretical instruction or overly simplistic examples. This tool bridges the gap between theory and practice by modifying real web applications in a targeted, reproducible way. With its user-friendly interface and modular vulnerability injection, instructors can create challenges tailored to specific learning goals, while students engage directly with code that simulates production-level vulnerabilities. We evaluated FETT based on five publicly available GitHub projects and six student projects from the last three academic years (2022-2024). We successfully managed to efficiently inject vulnerabilities inspired by the OWASP top 10:2021 while keeping the core functionalities of the target application operational. The project is publicly available at https://gitlab.com/fabgilson/django-vulnerability-injector. A demonstration is available at https://youtu.be/ZYQs2vLzbyE.

In agile software development, user stories (US) and acceptance criteria (AC) are popular ways of recording requirements. While guidelines have been proposed in the literature to assess the quality of US and AC, their correct application remains a manual task. In this work, we designed both a machine learning (ML) and a natural language processing (NLP) classifier for automatically assessing agile software requirements following quality indicators found in the literature. We evaluated their performance to improve the quality of requirements in a user study, as well as the users’ perceptions on the usage of such tools as writing aids. While improvements were notable in the quality of requirements written by participants, the improvements were more marginal when using the NLP classifier compared to the ML one. However, participants reported more satisfaction towards the NLP classifier for its “explainability” compared to the ML one.

Project-based software engineering courses often incorporate industry practices, e.g., the “Daily Scrum” (i.e., daily stand-ups). While the main purpose of the Daily Scrum is to discuss progress and blockers, we can also utilize them as an educational tool to facilitate student engagement and reflection since reflection is a crucial aspect of experience-based learning. This paper seeks to enhance reflective practices in software engineering project courses through a pre-meeting check-in add-on to the Daily Scrum. We also aim to assess if this increases self-reflection and consequently impacts student engagement and performance. To this end, we adapted the Daily Scrum in a software engineering project course with a pre-meeting semi-guided self-reflection on a student's work. This was supported by a bespoke online tool. Effects of the reflection were analyzed from the student perspective using surveys, application usage data, formal assessment data, and observations from teaching staff. From our experiences, we observed the pre-meeting check-in was beneficial in enhancing student self-reflectivity. Quan-titative data, supported by qualitative evidence from student feedback, also suggested positive effects on student engagement and performance. Students who used pre-meeting check-ins reported increased self-reflection and showed greater levels of improvement in assessed performance. Based on our experiences we recommend incorporating regular individual self-reflection sessions in software engineering project courses.

This paper recounts our experiences with the ‘ScrumBoard’, a custom-built open-source digital project man-agement tool specifically designed for software engineering project courses. We discuss how it has supported experiential learning pedagogy and assessment over multiple iterations of a two-semester, 3rdyear project course, having now been used extensively by more than 200 students. While offering a similar experience to commercial software project management tools, we reflect on how the ScrumBoard has supported our ability to assess students more accurately and efficiently, and how the tool has enabled us to identify problematic work practices in real-time, allowing for earlier interventions from teaching staff and an improved learning experience for students.

When modeling topics from chat messages of developer instant messaging communication, individual chat messages are short text documents. Our study aims at understanding how short text topic models perform with conversations from developer instant messaging. We applied four models to nine Gitter chat rooms (with sizes ranging from ≈100 to ≈160,000 messages). To assess the quality of topics and identify the best performing models, we compared topics based on four metrics for topic coherence. Furthermore, for a subset of Gitter chat rooms we used two human-based assessments: intrusion tasks with 18 experts analyzing 40 topics each, and topic naming (assigning a name to a topic that summarizes its main concept) with eight additional experts naming 60 topics each. Models performed differently in terms of coherence metrics and human assessment depending on the corpus (small, medium or large chat room). Our findings offer recommendations for the selection and use of short text topic models with developer chat messages based on characteristics of models and their performance with different sizes of corpora, and based on different strategies to assess topic quality.

Unit and automated acceptance testing have different objectives (e.g., testing units of code versus testing complete features). Testing practices (e.g., test-first, model-based) used for one “level” of testing (for either unit or acceptance testing) may require knowledge and skills that are not applicable to the other. This makes it difficult for practitioners to gain the skills required to effectively test at all levels and form a cohesive testing strategy. The aim of this systematic literature review is to understand whether there are any automated unit testing practices that have similarities with automated acceptance testing practices (and vice versa). Understanding these similarities can enable skill transfer across testing activities at different levels. This systematic literature review focuses on empirical research with an industry focus. We found that test-driven development (TDD) and model-based test generation (MBTG) are two practices widely researched for both unit testing and acceptance testing. For TDD we found that a design- and test-first mindset is required and helpful at both the unit and acceptance levels, but practitioners struggle with that practice. For MBTG we found that, despite its ability to increase code coverage, the additional manual effort to enable automated test generation may outweigh its benefits.

Software developers use instant messaging (e.g., Slack, Gitter) to collaboratively discuss software engineering problems and solutions. This communication takes place in chat rooms that generally contain a description of the main topic of discussion and the messages exchanged. To analyze whether and how the knowledge accumulated in these chat rooms is relevant to other developers, we first need to understand the themes discussed in these chat rooms. In this paper, we used thematic analysis to manually identify software engineering themes in the description of 87 chat rooms of Gitter, an instant messaging tool for software developers. Then, we checked whether these themes also occur in 184 public chat rooms of Slack, another instant messaging tool. We identified 47 themes in Gitter chat rooms, and regarding the applicability of themes, we could relate 36 of our themes to 173 Slack chat rooms. Our results indicate that, in the context of our study, chat rooms in developer instant messaging communication are mostly about software development technologies and practices rather than development processes. Furthermore, most chat rooms are topic- rather than project-related (e.g., a chat room used by developers of a particular software development project).

Abstract Topic modeling using models such as Latent Dirichlet Allocation (LDA) is a text mining technique to extract human-readable semantic “topics” (i.e., word clusters) from a corpus of textual documents. In software engineering, topic modeling has been used to analyze textual data in empirical studies (e.g., to find out what developers talk about online), but also to build new techniques to support software engineering tasks (e.g., to support source code comprehension). Topic modeling needs to be applied carefully (e.g., depending on the type of textual data analyzed and modeling parameters). Our study aims at describing how topic modeling has been applied in software engineering research with a focus on four aspects: (1) which topic models and modeling techniques have been applied, (2) which textual inputs have been used for topic modeling, (3) how textual data was “prepared” (i.e., pre-processed) for topic modeling, and (4) how generated topics (i.e., word clusters) were named to give them a human-understandable meaning. We analyzed topic modeling as applied in 111 papers from ten highly-ranked software engineering venues (five journals and five conferences) published between 2009 and 2020. We found that (1) LDA and LDA-based techniques are the most frequent topic modeling techniques, (2) developer communication and bug reports have been modelled most, (3) data pre-processing and modeling parameters vary quite a bit and are often vaguely reported, and (4) manual topic naming (such as deducting names based on frequent words in a topic) is common.

Technical debt is a metaphor that measures the additional effort needed to continue to add more features in a software due to its inherent decrease in code quality. Most software systems suffer from technical debt at some point so that dedicated tools and metrics have been developed to monitor such debt. Alongside tools, appropriate engineering practices must be put in place by the development team to keep that debt at an acceptable level. In this empirical study, we observed and surveyed Scrum development teams composed of experienced students in order to understand their quality-related processes on a year-long academic project. We found that (1) students do use static analysis tools of many forms, but their actual usage is limited due to time pressure; (2) retrospective and non-constraining feedback on code quality has little to no effect, even when given regularly during the course of the project; and (3) junior developers value composite quality indicators (e.g., maintainability, reliability in SonarQube), even if they do not fully understand their meaning. From our findings, we propose a series of recommendations, both technical and methodological, on how to train junior developers to understand and manage technical debt.

Textual user stories capture interactions of users with the system as high-level requirements. However, user stories are typically rather short and backlogs can include many stories. This makes it hard to (a) maintain user stories and backlogs, (b) fully understand the scope of a software project without a detailed analysis of the backlog, and (c) analyse how user stories impact design decisions during sprint planning and implementation. This paper proposes a technique to automatically transform textual user stories into visual use case scenarios in the form of robustness diagrams (a semi-formal scenario-based visualisation of workflows). In addition to creating diagrams for individual stories, the technique allows combining diagrams of multiple stories into one diagram to visualise workflows within sets of stories (e.g., a backlog). Moreover, the technique supports "viewpoint-based" diagrams, i.e., diagrams that show relationships between actors, domain entities and user interfaces starting from a diagram element (e.g., an actor) selected by the analyst. The technique utilises natural language processing and rule-based transformations. We evaluated the technique with more than 1,400 user stories from 22 backlogs and show that (a) the technique generates syntactically valid robustness diagrams, and (b) the quality of automatically generated robustness diagrams compares to the quality of diagrams created by human experts, but depends on the quality of the textual user stories.

Hidden behind the Internet of Things (IoT), many actors are activelly filling the market with devices and services. From this profusion of actors, a large amount of technologies and APIs, sometimes proprietary, are available, making difficult the interoperability and configuration of systems for IoT technicians. In order to define and manipulate devices deployed in domestic environments, we propose IoTDSL, a Domain-Specific Language meant to specify, assemble and describe the behaviour of interconnected devices. Relying on a high-level rule-based language, users in charge of the deployment of IoT infrastructures are able to describe and combine in a declarative manner structural configurations as well as event-based semantics for devices. This way, language users are freed from technical aspects, playing with high-level representations of devices, while the complexity of the concrete implementation is handled in a dedicated layer where high-level rules are mapped to vendor’s API.

Software architectures are often abstracted as a combination of reusable components connected to each other by various means. Specifications of components' semantics have been widely studied and many modeling languages have been proposed from coarse-grained loosely-defined elements to operational objects with behavioral semantics that may be generated and executed in a dedicated framework. All these modeling facilities have proven their advantages in many domains through either case studies or real-world applications. However, most of those approaches either consider a subset of composition facilities, i.e. The available types of bindings between components, or do not even consider communication properties at all, staying at behavioral-related compatibility between components. Verifications of communication-related properties are then postponed to the hand of software developers and finally considered at deployment-time only. Part of a general architecture framework, we propose an abstraction formalism to specify communication paths between components. This modeling facility relies on a taxonomy of types of links and the specifications of communication protocols. This protocol serves as a reification element between abstract component compositions, architecture instances and deployment infrastructure, making explicit communication-related constraints and properties.

Software architecture design is a critical task as lots of requirements can be taken into account on which many decisions can be made. The maintenance and evolution of resulting models often become tricky, even impracticable when their rationale is lost. In a previous work, we introduced a set of languages used in a transformation-centric design method meant to tackle this scattering of requirements and to facilitate further model evolutions. But, we did not provided a formal validation of our proposal yet. The present work depicts a comparative case study we conducted on a group of students. The participants were asked to develop an online book store in two phases, the second one simulating an evolution of the system. We evaluated the functional completeness of the created software as well as the traceability of design decisions and rationale. The participants were also asked to criticize the design method and language they used in a textual report and through a questionnaire. Even if the size of the case study is rather limited, it clearly highlighs the advantages of our approach regarding, among others, its expressiveness and decisions traceability.

Stakeholders have to face requirements in increasing number and complexity. Their translations to system functionalities are often diluted into the overall architecture so that it becomes tricky to undertake future changes. Since information systems are intended to evolve in terms of functionalities and underlying technologies, the link between requirements and design artifacts is primordial. Agile design methods and documentation techniques have emerged in the past years in order to deal with the amount of requirements and to trace the decision process and the rationale sustaining a software model. Also, it is not unusual that numerous technologies with similar purpose are confronted to each other during the design phase. In the present work, we propose an integrated framework combining system requirement definitions, a component-based modeling language and model transformations. Architecturally-significant requirements are explicitly linked to software architecture elements and iteratively refined or implemented by model transformations. Any transformation must be documented, even briefly, and the framework retains the transformations tree. This way, the iterative decision and design processes are completely documented for future reference or modification, i.e, designers can (i) see the mapping between a system requirement and its implementation in the architecture model, (ii) explore design alternatives or apply structural modifications without losing previous versions of the model, and finally (iii), depending on the level of documentation, at least understand partially the reasons why the model is how it is.