Abstract Cloud services operate in highly dynamic and heterogeneous environments, requiring continuous and accurate assessment of service quality. While Quality of Service (QoS) models are widely used to monitor performance, deep learning (DL) architectures–such as Long Short-Term Memory (LSTM) and Bidirectional Gated Recurrent Units (BI-GRU)–offer enhanced capabilities for forecasting potential Service Level Agreement (SLA) violations. However, many existing experiments in this domain suffer from methodological shortcomings, including the use of outdated or proprietary datasets, a narrow set of QoS metrics, incomplete documentation of model architectures and training procedures, and a lack of statistical rigor, which undermines reproducibility and applicability in industrial contexts. This study empirically compares the performance of BI-GRU, LSTM, and AutoRegressive Integrated Moving Average (ARIMA) models for QoS forecasting using a rigorously designed experimental protocol that addresses these limitations. We build a multi-metric QoS dataset covering five months of operational data from a cloud service in an IT company, comprising 16 QoS metrics. Forecasting models were trained and evaluated using Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), and Mean Absolute Percentage Error (MAPE), with training time considered as an efficiency indicator. BI-GRU outperformed ARIMA across all QoS metrics and achieved statistically significant improvements over LSTM in 9 out of 16 metrics. In contrast, LSTM only significantly outperformed BI-GRU in one metric. Our findings demonstrate that most BI-GRU models provide superior accuracy and efficiency. Furthermore, the methodological rigor of the experimental design supports their applicability for proactive QoS management and informed decision-making in industrial cloud service environments.

Adaptive user interfaces dynamically change their content, presentation, and behavior to optimize the user experience, which has been primarily evaluated using classic usability measures but to a lesser extent by using neurological measures. While the perceived preference of specific user interface elements, such as graphical adaptive menus, has already been studied, no consensus exists regarding their performance and how to substitute a static menu with an adaptive one. To gain insights into how graphical adaptive menus could influence the user experience and to identify any correlation between users’ performance and their preferences, we conducted an experiment in which forty participants used twenty graphical adaptive menus while their brain activity was captured by employing electroencephalography to derive four measures ( i.e. , cognitive load, engagement, attraction, and memorization). User performance was measured using task completion time, specifically the time to select menu items. Statistical analysis suggested which graphical adaptive menus were significantly better or worse than the static menu, our baseline. These results are used as the basis to suggest implications for software developers and researchers to design more effective adaptive user interfaces.

Adaptive User Interfaces (AUI) play a crucial role in modern software applications by dynamically adjusting interface elements to accommodate users’ diverse and evolving needs. However, existing adaptation strategies often lack real-time responsiveness. Reinforcement Learning (RL) has emerged as a promising approach for addressing complex, sequential adaptation challenges, enabling adaptive systems to learn optimal policies based on previous adaptation experiences. Although RL has been applied to AUIs,integrating RL agents effectively within user interactions remains a challenge.

Abstract Context Adapting the User Interface (UI) of software systems to users’ requirements and their context of use is a challenging task. It involves determining the right adaptation, at the right time and place, to make it valuable for end-users. We believe that recent progress in Machine Learning (ML) techniques could provide useful ways in which to support adaptation more effectively. In particular, Reinforcement Learning (RL) has proven to be effective in planning a sequence of UI adaptations over a long time horizon. However, RL requires either manually specifying a reward function or learning a reward model. Currently there is no empirical evidence supporting the usefulness of reward models for UI adaptation. Objective This paper presents a confirmatory empirical study aimed at investigating the effectiveness of two different approaches to generating reward models in the context of UI adaptation using reinforcement learning: (1) a reward model derived exclusively from predictive Human-Computer Interaction (HCI) models (AUI-HCI), and (2) a reward model derived from predictive HCI models augmented by human feedback (AUI-HCI-HF), compared to non-adaptive (NA) interfaces. Method A controlled experiment with an AB/BA crossover design was conducted to evaluate the impact of these reward models on user experience, measured through objective and subjective engagement, as well as user satisfaction. Our study contributes to the understanding of how reward modeling can facilitate UI adaptation through RL. Results The results showed a significant improvement in objective engagement for AUI-HCI-HF compared to non-adaptive interfaces. However, no significant differences were found between AUI-HCI and non-adaptive interfaces for any of the other measurements, across any conditions. Conclusion Integrating human feedback into RL reward models enhances objective engagement, but its impact on subjective engagement and user satisfaction remains limited. While AUI-HCI-HF shows promise for improving interaction metrics, further research is needed to better align reward models with broader user perceptions and preferences, particularly compared to non-adaptive interfaces.

The landscape of software engineering is undergoing a transformative shift driven by advancements in machine learning, Artificial Intelligence (AI), and autonomous systems. This roadmap article explores how these technologies are reshaping the field, positioning humans not only as end users but also as critical components within expansive software ecosystems. We examine the challenges and opportunities arising from this human-centered paradigm, including ethical considerations, fairness, and the intricate interplay between technical and human factors. By recognizing humans at the heart of the software lifecycle—spanning professional engineers, end users, and end user developers—we emphasize the importance of inclusivity, human-aligned workflows, and the seamless integration of AI-augmented socio-technical systems. As software systems evolve to become more intelligent and human-centric, software engineering practices must adapt to this new reality. This article provides a comprehensive examination of this transformation, outlining current trends, key challenges, and opportunities that define the emerging research and practice landscape, and envisioning a future where software engineering and AI work synergistically to place humans at the core of the ecosystem.

The landscape of software engineering has dramatically changed in recent years. The impressive advances of artificial intelligence are just the latest and most disruptive innovation that has remarkably changed the software engineering research and practice. This special issue shares a roadmap to guide the software engineering community in this confused era. This roadmap is the outcome of a 2-day intensive discussion at the 2030 Software Engineering workshop. The roadmap spotlights and discusses seven main landmarks in the new software engineering landscape: artificial intelligence for software engineering, human aspects of software engineering, software security, verification and validation, sustainable software engineering, automatic programming, and quantum software engineering. This editorial summarizes the core aspects discussed in the 37 papers that comprise the seven sections of the special issue and guides the interested readers throughout the issue. This roadmap is a living body that we will refine with follow-up workshops that will update the roadmap for a series of forthcoming ACM TOSEM special issues.

Abstract Modeling is often associated with complex and heavy tooling, leading to a negative perception among practitioners. However, alternative paradigms, such as everything-as-code or low-code, are gaining acceptance due to their perceived ease of use. This paper explores the dichotomy between these perceptions through the lens of “modeler experience” (MX). MX includes factors such as user experience, motivation, integration, collaboration and versioning, and language complexity. We examine the relationships between these factors and their impact on different modeling usage scenarios. Our findings highlight the importance of considering MX when understanding how developers interact with modeling tools and the complexities of modeling and associated tooling.

Abstract Modelling is an important activity in software development and it is essential that students learn the relevant skills. Modelling relies on dedicated tools and these can be complex to install, configure, and use—distracting students from learning key modelling concepts and creating accidental complexity for teachers. To address these challenges, we believe that modelling tools specifically aimed at use in teaching are required. Based on discussions at a working session organised at MODELS 2023 and the results from an internationally shared questionnaire, we report on requirements for such modelling tools for teaching. We also present examples of existing modelling tools for teaching and how they address some of the requirements identified.

Background: Adaptive user interfaces have the advantage of being able to dynamically change their aspect and/or behaviour depending on the characteristics of the context of use, i.e. to improve user experience. User experience is an important quality factor that has been primarily evaluated with classical measures (e.g. effectiveness, efficiency, satisfaction), but to a lesser extent with physiological measures, such as emotion recognition, skin response, or brain activity. Aim: In a previous exploratory experiment involving users with different profiles and a wide range of ages, we analysed user experience in terms of cognitive load, engagement, attraction and memorisation when employing twenty graphical adaptive menus through the use of an Electroencephalogram (EEG) device. The results indicated that there were statistically significant differences for these four variables. However, we considered that it was necessary to confirm or reject these findings using a more homogeneous group of users. Method: We conducted an operational internal replication study with 40 participants. We also investigated the potential correlation between EEG signals and the participants’ user experience ratings, such as their preferences. Results: The results of this experiment confirm that there are statistically significant differences between the EEG variables when the participants interact with the different adaptive menus. Moreover, there is a high correlation among the participants’ user experience ratings and the EEG signals, and a trend regarding performance has emerged from our analysis. Conclusions: These findings suggest that EEG signals could be used to evaluate user experience. With regard to the menus studied, our results suggest that graphical menus with different structures and font types produce more differences in users’ brain responses, while menus which use colours produce more similarities in users’ brain responses. Several insights with which to improve users’ experience of graphical adaptive menus are outlined.

Abstract Adapting the user interface of a software system to the requirements of the context of use continues to be a major challenge, particularly when users become more demanding in terms of adaptation quality. A considerable number of methods have, over the past three decades, provided some form of modelling with which to support user interface adaptation. There is, however, a crucial issue as regards in analysing the concepts, the underlying knowledge, and the user experience afforded by these methods as regards comparing their benefits and shortcomings. These methods are so numerous that positioning a new method in the state of the art is challenging. This paper, therefore, defines a conceptual reference framework for intelligent user interface adaptation containing a set of conceptual adaptation properties that are useful for model-based user interface adaptation. The objective of this set of properties is to understand any method, to compare various methods and to generate new ideas for adaptation. We also analyse the opportunities that machine learning techniques could provide for data processing and analysis in this context, and identify some open challenges in order to guarantee an appropriate user experience for end-users. The relevant literature and our experience in research and industrial collaboration have been used as the basis on which to propose future directions in which these challenges can be addressed.

Context. Several early requirements approaches focus on modeling objectives, interest or benefits of related stakeholders. However, as they can be used for different purposes as identifying problems, exploring system solutions, evaluating alternatives, etc., there are no clear guidelines on how to build these models, which constructs of the language must be used in each case, and most importantly, how to use these models downstream to the software requirements and design artifacts. Background. In a previous work, we proposed a specialization of the GRL language ([email protected]) to specify stakeholders' goals when dealing with early requirements in the context of incremental software development. Goal/Method. This paper reports on a controlled experiment aimed at comparing the goal model quality and the productivity, perceived ease of use, and perceived usefulness of participants when using [email protected] and i* languages. Results. The results showed that [email protected] obtained better results than i* as a goal modeling language indicating that it can be considered as a promising emerging approach in this area. Conclusions. [email protected] allows obtaining goal models with good quality that may be later used downstream software development activities.

Summary Service‐based cloud applications are software systems that continuously evolve to satisfy new user requirements and technological changes. This kind of applications also require elasticity, scalability, and high availability, which means that deployment of new functionalities or architectural adaptations to fulfill service level agreements (SLAs) should be performed while the application is in execution. Dynamic architectural reconfiguration is essential to minimize system disruptions while new or modified services are being integrated into existing cloud applications. Thus, cloud applications should be developed following principles that support dynamic reconfiguration of services, and also tools to automate these reconfigurations at runtime are needed. This paper presents an extension of a model‐driven method for dynamic and incremental architecture reconfiguration of cloud services that allows developers to specify new services as software increments, and the tool to generate the implementation code for the services integration logic and the deployment and architectural reconfiguration scripts specific to the cloud environment in which the service will be deployed (e.g., Microsoft Azure). We also report the results of a quasi‐experiment that empirically validate our method. It was conducted to evaluate their perceived ease of use, perceived usefulness, and perceived intention to use. The results show that the participants perceive the method to be useful, and they also expressed their intention to use the method in the future. Although further experiments must be carried out to corroborate these results, the method has proven to be a promising architectural reconfiguration process for cloud applications in the context of agile and incremental development processes. Copyright © 2016 John Wiley & Sons, Ltd.

Context: The size and complexity of software systems along with the demand for ensuring quality requirements have fostered the interest in software architecture evaluation methods. Although several empirical studies have been reported, the actual body of knowledge is still insufficient. To address this concern, we presented a family of four controlled experiments that compares a recently proposed method, the Quality-Driven Architecture Derivation and Improvement (QuaDAI) method against the well-known Architecture Tradeoff Analysis Method (ATAM).

Since its infancy, Model Driven Engineering (MDE) research has primarily focused on technical issues. Although it is becoming increasingly common for MDE research papers to evaluate their theoretical and practical solutions, extensive usability studies are still uncommon. We observe a scarcity of User eXperience (UX)-related research in the MDE community, and posit that many existing tools and languages have room for improvement with respect to UX [26], [44], [37], where UX is a key focus area in the software development industry. We consider this gap a fundamental problem that needs to be addressed by the community if MDE is to gain widespread use. In this vision paper, we explore how and where UX fits into MDE by considering motivating use cases that revolve around different dimensions of integration: model integration, tool integration, and integration between process and tool support. Based on the literature and our collective experience in research and industrial collaborations, we propose future directions for addressing these challenges.

A number of monitoring approaches for cloud services have been proposed in the last years. However, they suffer from several limitations mainly due to changes in the monitoring requirements or because of the complexity in dealing with raw data from services at runtime. To address these problems, in a previous work, we proposed a platform-independent monitoring middleware for cloud services using models@runtime. This middleware was implemented in Microsoft Azure to monitor the quality of cloud services. To provide further evidence about the generalizability of our approach, in this work, we introduce the implementation of the monitoring middleware in the Google App Engine. The middleware gather raw data from the running cloud services to perform measurements that assess the compliance to the quality requirements established in a SLA. The use of models@runtime allowed us to provide a high-level of flexibility to change quality requirements at runtime.

Modeling is a fundamental activity within the requirements engineering process and concerns the construction of abstract descriptions of requirements that are amenable to interpretation and validation. The choice of a modeling technique is critical whenever it is necessary to discuss the interpretation and validation of requirements. This is particularly true in the case of functional requirements and stakeholders with divergent goals and different backgrounds and experience. This paper presents the results of a family of experiments conducted with students and professionals to investigate whether the comprehension of functional requirements is influenced by the use of dynamic models that are represented by means of the UML sequence diagrams. The family contains five experiments performed in different locations and with 112 participants of different abilities and levels of experience with UML. The results show that sequence diagrams improve the comprehension of the modeled functional requirements in the case of high ability and more experienced participants.

Usability evaluation methods have become critical in the Web domain to ensure the success of Web applications. Aim: Since a large number of proposals have been presented during the last few years, a question arises: Which usability evaluation methods have proven to be the most effective in the Web domain? Method: This paper presents a systematic review that was motivated by previous results obtained from a systematic mapping study in the Web usability evaluation field. Results: A total of 18 studies were selected from an initial set of 206 in order to extract, code, and synthesize empirical data concerning the effectiveness of usability evaluation methods for the Web. Conclusions: We detected a need of more empirical studies and more standardized effectiveness measures for comparing usability evaluation methods. Our results suggest several evaluation methods which may be useful in allowing researchers and practitioners to perform effective Web usability evaluations.

Currently, there is a lack of empirically validated usability evaluation methods that can properly be integrated during the early stages of Web development processes. This has motivated us to propose a usability inspection method called WUEP that can be integrated into different model-driven Web development processes. In previous work, we presented the operationalization and validation of WUEP in a specific process based on the Object-Oriented Hypermedia (OO-H) method. In this paper, we present further analysis of the empirical validation of the operationalization of WUEP into WebML, which is one of the most well-known industrial model-driven Web development process. The effectiveness, efficiency, perceived ease of use, and satisfaction of WUEP was evaluated in comparison to Heuristic Evaluation. The results show that WUEP is more effective and efficient than heuristic evaluation in the detection of usability problems. The inspectors were also satisfied when applying WUEP, and found it easier to use than heuristic evaluation.

Some verification and validation techniques have been evaluated both theoretically and empirically. Most empirical studies have been conducted without subjects, passing over any effect testers have when they apply the techniques. We have run an experiment with students to evaluate the effectiveness of three verification and validation techniques (equivalence partitioning, branch testing and code reading by stepwise abstraction). We have studied how well able the techniques are to reveal defects in three programs. We have replicated the experiment eight times at different sites. Our results show that equivalence partitioning and branch testing are equally effective and better than code reading by stepwise abstraction. The effectiveness of code reading by stepwise abstraction varies significantly from program to program. Finally, we have identified project contextual variables that should be considered when applying any verification and validation technique or to choose one particular technique.

In the development of complex systems the requirements for the system will almost always remain uncertain late into the software development. In gas turbine engine control systems at Rolls-Royce, typically 50% of requirements will change between Critical Design Review and Entry into Service. Ignoring or not planning for requirements uncertainty will cause scrap and rework that will manifest later in the project. This paper evaluates the impact of not managing these uncertainties and describes how Rolls-Royce uses Requirements Uncertainty Analysis to reduce this impact. The paper summarises the findings from an extensive Six Sigma study into requirements uncertainty and provides an overview of the technique now used to identify and monitor uncertainty through a project life. The return on investment of this technique has been between 100:1 and 500:1.

The objective of this workshop is to bring together researchers and practitioners to report and discuss the challenges and opportunities for integrating quantitative methods in product line engineering with the objective of achieving both technical and business goals. In particular, we are seeking contributions that, on the one hand, deal with product line estimation and metrics for the effective management of product line projects, and on the other hand, provide some insight into new trends in value-based product line engineering.

A good estimation tool offers a "model" of a project and is usually used to estimate cost and schedule, but it can also be used to help make trade decisions that affect cost and schedule as well as to estimate risks and opportunities. It was evident that Rolls-Royce needed a cost model to underpin decisions when they launched a Software Product Line initiative. The first generation cost model was based on COCOMO II, which represents the software product as a single size measure (Source Lines of Code) but makes limited use of the architecture or any characteristics of the product being developed. The next generation of the cost model, currently under development, is intended to account for the quality attributes of the core assets and the resulting products in order to estimate their impact on cost and net-benefit to the business. The objective of this paper is to describe our current efforts to integrate key quality attributes into the SPL cost model. We describe the quality attributes selected, the reason for their selection and the benefits we expect to obtain after integrating them into the model.

A complex system's requirements almost always remain uncertain late into its software development. In gas turbine engine control systems at Rolls-Royce, for a traditional project (non-product line) typically 50% of requirements will change between Critical Design Review and Entry into Service. Requirements uncertainty is particularly relevant when defining the scope of a Software Product Line. If the core asset team fails to recognise or accommodate requirements uncertainty, changes will manifest later in the product line. If the core asset team over-compensates by adding too much functionality or variability to account for a wide range of uncertainty, they will invest effort that may never be required. The optimal balance can be found through an application of requirements uncertainty analysis and understanding the balance between the impact of risk and mitigation effort. This paper first describes the use of the requirements uncertainty analysis technique at Rolls-Royce for traditional (non-product line) software development and then explains how this technique works in the context of a software product line.

The challenge of developing more usable Web applications has promoted the emergence of several usability evaluation methods. However, there is a lack of empirically validated methods that can properly be integrated during the early stages of Web development processes. This has motivated us to propose a Web Usability Evaluation Method (WUEP) which can be integrated into model-driven Web development processes. This paper presents the first steps in the empirical validation of WUEP through a controlled experiment. This experiment was designed in order to evaluate the effectiveness, efficiency, perceived ease of use, and satisfaction with WUEP in comparison to a widely-used inspection method: Heuristic evaluation (HE). Results show that WUEP is more effective and efficient than HE in the detection of usability problems in artifacts obtained from a model-driven Web development process. The evaluators were also satisfied when applying WUEP, and found it easier to use than HE.

This paper describes the results of an external replication of an experiment for assessing whether the use of dynamic modeling influences the comprehension of software requirements. The results of the original experiment conducted in Italy did not confirm that there was a significant difference in the comprehension of software requirements when dynamic modeling is used. The goal of the replication was therefore to verify these findings with a group of more experienced students at the Universidad Politeacutecnica de Valencia (UPV) in Spain. The results shows that the use of dynamic modeling does significantly improve the comprehension of software requirements, thus providing evidence that dynamic modeling facilitates the interpretation and comprehension of requirements.

This paper reports on a controlled experiment that investigates the influence of stereotypes in UML sequence diagrams. The comprehension of UML sequence diagrams with and without stereotypes is analyzed from three different perspectives: semantic comprehension, retention and transfer. The experiment was carried out with 77 undergraduate students of Computer Science from the University of Bari in Italy. The results obtained show a slight tendency in favor of the use of stereotypes in facilitating the comprehension of UML sequence diagrams. Further replications are needed to obtain more conclusive results.

This paper presents a replication study that investigates the efficacy and likely adoption of a measurement procedure for sizing Web applications from conceptual models (OOmFPWeb). The goal of the replication was to provide evidence for the generalization of the results by repeating the experiment in a different environment, using different subjects. The results of the replica carried out in Austria have confirmed the results of the original experiment, which was carried out in Spain. OOmFPWeb is efficient when compared to current industry practices. It provides reproducible functional size measurements and is perceived as easy to use and useful by its users, who also expressed their intention to use OOmFPWeb in the future. The analysis further supports the validity and reliability of Moody’s Method Evaluation Model for evaluating functional size measurement methods.

In this paper, we briefly introduce a controlled experiment to investigate the selection of alternative transformation rules through which to obtain UML class diagrams from a Requirements Model. The main goal of this experiment was to determine which of the transformation rules for structural relationships between classes (association (Al), aggregation (A2) and association class (A3)) produces the UML class diagram that is easiest to understand. More details about the transformations and about the experiment are provided. We focus upon the understandability of UML class diagrams because it is well recognized that if a model is easier to understand it will be easier to maintain, reuse, etc.