Abstract Cloud services operate in highly dynamic and heterogeneous environments, requiring continuous and accurate assessment of service quality. While Quality of Service (QoS) models are widely used to monitor performance, deep learning (DL) architectures–such as Long Short-Term Memory (LSTM) and Bidirectional Gated Recurrent Units (BI-GRU)–offer enhanced capabilities for forecasting potential Service Level Agreement (SLA) violations. However, many existing experiments in this domain suffer from methodological shortcomings, including the use of outdated or proprietary datasets, a narrow set of QoS metrics, incomplete documentation of model architectures and training procedures, and a lack of statistical rigor, which undermines reproducibility and applicability in industrial contexts. This study empirically compares the performance of BI-GRU, LSTM, and AutoRegressive Integrated Moving Average (ARIMA) models for QoS forecasting using a rigorously designed experimental protocol that addresses these limitations. We build a multi-metric QoS dataset covering five months of operational data from a cloud service in an IT company, comprising 16 QoS metrics. Forecasting models were trained and evaluated using Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), and Mean Absolute Percentage Error (MAPE), with training time considered as an efficiency indicator. BI-GRU outperformed ARIMA across all QoS metrics and achieved statistically significant improvements over LSTM in 9 out of 16 metrics. In contrast, LSTM only significantly outperformed BI-GRU in one metric. Our findings demonstrate that most BI-GRU models provide superior accuracy and efficiency. Furthermore, the methodological rigor of the experimental design supports their applicability for proactive QoS management and informed decision-making in industrial cloud service environments.

Adaptive user interfaces dynamically change their content, presentation, and behavior to optimize the user experience, which has been primarily evaluated using classic usability measures but to a lesser extent by using neurological measures. While the perceived preference of specific user interface elements, such as graphical adaptive menus, has already been studied, no consensus exists regarding their performance and how to substitute a static menu with an adaptive one. To gain insights into how graphical adaptive menus could influence the user experience and to identify any correlation between users’ performance and their preferences, we conducted an experiment in which forty participants used twenty graphical adaptive menus while their brain activity was captured by employing electroencephalography to derive four measures ( i.e. , cognitive load, engagement, attraction, and memorization). User performance was measured using task completion time, specifically the time to select menu items. Statistical analysis suggested which graphical adaptive menus were significantly better or worse than the static menu, our baseline. These results are used as the basis to suggest implications for software developers and researchers to design more effective adaptive user interfaces.

User interface requirements elicitation is an early and critical stage in the software development life cycle: requirements are elicited from stakeholders, and user interface elements that should satisfy these requirements are also created, shared, and edited. To effectively capture the diverse perspectives of stakeholders on these requirements, crowd elicitation is a promising approach to leverage the collective knowledge of a time- and space-distributed group of stakeholders to manage user interface requirements and their associated user interface elements. Annotating these elements manually in freeform digital ink leads to results that are difficult to interpret and time-consuming to exploit by designers and developers. Instead, we propose that stakeholders annotate user interface elements by commonly-agreed gestures that will be automatically recognized to populate 10 annotation metrics. To this end, we present Crowditivity , a publicly available web application that effectively harnesses crowd elicitation in three related activities: (1) collecting and managing user interface functional and non-functional requirements according to an adapted Volere Shell template; (2) assigning user interface elements of a mockup or prototype to requirements to satisfy them; and (3) enabling stakeholders to annotate these elements by ink-based gestures. A 2D multi-stroke gesture classifier is specifically tailored to recognize gesture classes corresponding to 18 annotation types, that are empirically validated through a gesture elicitation study ( n 1 = 26) confirmed by an identification study ( n 2 = 59). Through an evaluation of a case study for a web application, we investigated how eight representative designers valued the Crowditivity platform. We finally define and illustrate a set of ten metrics to quantify the output of Crowditivity in terms of annotations and annotators resulting from the gesture recognition.

Adaptive User Interfaces (AUI) play a crucial role in modern software applications by dynamically adjusting interface elements to accommodate users’ diverse and evolving needs. However, existing adaptation strategies often lack real-time responsiveness. Reinforcement Learning (RL) has emerged as a promising approach for addressing complex, sequential adaptation challenges, enabling adaptive systems to learn optimal policies based on previous adaptation experiences. Although RL has been applied to AUIs,integrating RL agents effectively within user interactions remains a challenge.

Abstract Context Adapting the User Interface (UI) of software systems to users’ requirements and their context of use is a challenging task. It involves determining the right adaptation, at the right time and place, to make it valuable for end-users. We believe that recent progress in Machine Learning (ML) techniques could provide useful ways in which to support adaptation more effectively. In particular, Reinforcement Learning (RL) has proven to be effective in planning a sequence of UI adaptations over a long time horizon. However, RL requires either manually specifying a reward function or learning a reward model. Currently there is no empirical evidence supporting the usefulness of reward models for UI adaptation. Objective This paper presents a confirmatory empirical study aimed at investigating the effectiveness of two different approaches to generating reward models in the context of UI adaptation using reinforcement learning: (1) a reward model derived exclusively from predictive Human-Computer Interaction (HCI) models (AUI-HCI), and (2) a reward model derived from predictive HCI models augmented by human feedback (AUI-HCI-HF), compared to non-adaptive (NA) interfaces. Method A controlled experiment with an AB/BA crossover design was conducted to evaluate the impact of these reward models on user experience, measured through objective and subjective engagement, as well as user satisfaction. Our study contributes to the understanding of how reward modeling can facilitate UI adaptation through RL. Results The results showed a significant improvement in objective engagement for AUI-HCI-HF compared to non-adaptive interfaces. However, no significant differences were found between AUI-HCI and non-adaptive interfaces for any of the other measurements, across any conditions. Conclusion Integrating human feedback into RL reward models enhances objective engagement, but its impact on subjective engagement and user satisfaction remains limited. While AUI-HCI-HF shows promise for improving interaction metrics, further research is needed to better align reward models with broader user perceptions and preferences, particularly compared to non-adaptive interfaces.

Background: Adaptive user interfaces have the advantage of being able to dynamically change their aspect and/or behaviour depending on the characteristics of the context of use, i.e. to improve user experience. User experience is an important quality factor that has been primarily evaluated with classical measures (e.g. effectiveness, efficiency, satisfaction), but to a lesser extent with physiological measures, such as emotion recognition, skin response, or brain activity. Aim: In a previous exploratory experiment involving users with different profiles and a wide range of ages, we analysed user experience in terms of cognitive load, engagement, attraction and memorisation when employing twenty graphical adaptive menus through the use of an Electroencephalogram (EEG) device. The results indicated that there were statistically significant differences for these four variables. However, we considered that it was necessary to confirm or reject these findings using a more homogeneous group of users. Method: We conducted an operational internal replication study with 40 participants. We also investigated the potential correlation between EEG signals and the participants’ user experience ratings, such as their preferences. Results: The results of this experiment confirm that there are statistically significant differences between the EEG variables when the participants interact with the different adaptive menus. Moreover, there is a high correlation among the participants’ user experience ratings and the EEG signals, and a trend regarding performance has emerged from our analysis. Conclusions: These findings suggest that EEG signals could be used to evaluate user experience. With regard to the menus studied, our results suggest that graphical menus with different structures and font types produce more differences in users’ brain responses, while menus which use colours produce more similarities in users’ brain responses. Several insights with which to improve users’ experience of graphical adaptive menus are outlined.

Abstract Adapting the user interface of a software system to the requirements of the context of use continues to be a major challenge, particularly when users become more demanding in terms of adaptation quality. A considerable number of methods have, over the past three decades, provided some form of modelling with which to support user interface adaptation. There is, however, a crucial issue as regards in analysing the concepts, the underlying knowledge, and the user experience afforded by these methods as regards comparing their benefits and shortcomings. These methods are so numerous that positioning a new method in the state of the art is challenging. This paper, therefore, defines a conceptual reference framework for intelligent user interface adaptation containing a set of conceptual adaptation properties that are useful for model-based user interface adaptation. The objective of this set of properties is to understand any method, to compare various methods and to generate new ideas for adaptation. We also analyse the opportunities that machine learning techniques could provide for data processing and analysis in this context, and identify some open challenges in order to guarantee an appropriate user experience for end-users. The relevant literature and our experience in research and industrial collaboration have been used as the basis on which to propose future directions in which these challenges can be addressed.

Context. Several early requirements approaches focus on modeling objectives, interest or benefits of related stakeholders. However, as they can be used for different purposes as identifying problems, exploring system solutions, evaluating alternatives, etc., there are no clear guidelines on how to build these models, which constructs of the language must be used in each case, and most importantly, how to use these models downstream to the software requirements and design artifacts. Background. In a previous work, we proposed a specialization of the GRL language ([email protected]) to specify stakeholders' goals when dealing with early requirements in the context of incremental software development. Goal/Method. This paper reports on a controlled experiment aimed at comparing the goal model quality and the productivity, perceived ease of use, and perceived usefulness of participants when using [email protected] and i* languages. Results. The results showed that [email protected] obtained better results than i* as a goal modeling language indicating that it can be considered as a promising emerging approach in this area. Conclusions. [email protected] allows obtaining goal models with good quality that may be later used downstream software development activities.

Summary Service‐based cloud applications are software systems that continuously evolve to satisfy new user requirements and technological changes. This kind of applications also require elasticity, scalability, and high availability, which means that deployment of new functionalities or architectural adaptations to fulfill service level agreements (SLAs) should be performed while the application is in execution. Dynamic architectural reconfiguration is essential to minimize system disruptions while new or modified services are being integrated into existing cloud applications. Thus, cloud applications should be developed following principles that support dynamic reconfiguration of services, and also tools to automate these reconfigurations at runtime are needed. This paper presents an extension of a model‐driven method for dynamic and incremental architecture reconfiguration of cloud services that allows developers to specify new services as software increments, and the tool to generate the implementation code for the services integration logic and the deployment and architectural reconfiguration scripts specific to the cloud environment in which the service will be deployed (e.g., Microsoft Azure). We also report the results of a quasi‐experiment that empirically validate our method. It was conducted to evaluate their perceived ease of use, perceived usefulness, and perceived intention to use. The results show that the participants perceive the method to be useful, and they also expressed their intention to use the method in the future. Although further experiments must be carried out to corroborate these results, the method has proven to be a promising architectural reconfiguration process for cloud applications in the context of agile and incremental development processes. Copyright © 2016 John Wiley & Sons, Ltd.

Context: The size and complexity of software systems along with the demand for ensuring quality requirements have fostered the interest in software architecture evaluation methods. Although several empirical studies have been reported, the actual body of knowledge is still insufficient. To address this concern, we presented a family of four controlled experiments that compares a recently proposed method, the Quality-Driven Architecture Derivation and Improvement (QuaDAI) method against the well-known Architecture Tradeoff Analysis Method (ATAM).

A number of monitoring approaches for cloud services have been proposed in the last years. However, they suffer from several limitations mainly due to changes in the monitoring requirements or because of the complexity in dealing with raw data from services at runtime. To address these problems, in a previous work, we proposed a platform-independent monitoring middleware for cloud services using models@runtime. This middleware was implemented in Microsoft Azure to monitor the quality of cloud services. To provide further evidence about the generalizability of our approach, in this work, we introduce the implementation of the monitoring middleware in the Google App Engine. The middleware gather raw data from the running cloud services to perform measurements that assess the compliance to the quality requirements established in a SLA. The use of models@runtime allowed us to provide a high-level of flexibility to change quality requirements at runtime.

Modeling is a fundamental activity within the requirements engineering process and concerns the construction of abstract descriptions of requirements that are amenable to interpretation and validation. The choice of a modeling technique is critical whenever it is necessary to discuss the interpretation and validation of requirements. This is particularly true in the case of functional requirements and stakeholders with divergent goals and different backgrounds and experience. This paper presents the results of a family of experiments conducted with students and professionals to investigate whether the comprehension of functional requirements is influenced by the use of dynamic models that are represented by means of the UML sequence diagrams. The family contains five experiments performed in different locations and with 112 participants of different abilities and levels of experience with UML. The results show that sequence diagrams improve the comprehension of the modeled functional requirements in the case of high ability and more experienced participants.

Usability evaluation methods have become critical in the Web domain to ensure the success of Web applications. Aim: Since a large number of proposals have been presented during the last few years, a question arises: Which usability evaluation methods have proven to be the most effective in the Web domain? Method: This paper presents a systematic review that was motivated by previous results obtained from a systematic mapping study in the Web usability evaluation field. Results: A total of 18 studies were selected from an initial set of 206 in order to extract, code, and synthesize empirical data concerning the effectiveness of usability evaluation methods for the Web. Conclusions: We detected a need of more empirical studies and more standardized effectiveness measures for comparing usability evaluation methods. Our results suggest several evaluation methods which may be useful in allowing researchers and practitioners to perform effective Web usability evaluations.

Currently, there is a lack of empirically validated usability evaluation methods that can properly be integrated during the early stages of Web development processes. This has motivated us to propose a usability inspection method called WUEP that can be integrated into different model-driven Web development processes. In previous work, we presented the operationalization and validation of WUEP in a specific process based on the Object-Oriented Hypermedia (OO-H) method. In this paper, we present further analysis of the empirical validation of the operationalization of WUEP into WebML, which is one of the most well-known industrial model-driven Web development process. The effectiveness, efficiency, perceived ease of use, and satisfaction of WUEP was evaluated in comparison to Heuristic Evaluation. The results show that WUEP is more effective and efficient than heuristic evaluation in the detection of usability problems. The inspectors were also satisfied when applying WUEP, and found it easier to use than heuristic evaluation.

The objective of this workshop is to attract professionals from academia and industry to discuss the role of requirements engineering in product line developments, including new techniques, methods and tools that will help practitioners to improve their current requirements engineering practices. The workshop will address experiences from practitioners, empirical evidence about current practices, successful deployment of novel approaches, and current obstacles and proposed solutions. Nevertheless, REPOS also aims to become a discussion forum about the state of the art and practice for practitioners and researchers on Requirements Engineering for Software Product Lines.

The challenge of developing more usable Web applications has promoted the emergence of several usability evaluation methods. However, there is a lack of empirically validated methods that can properly be integrated during the early stages of Web development processes. This has motivated us to propose a Web Usability Evaluation Method (WUEP) which can be integrated into model-driven Web development processes. This paper presents the first steps in the empirical validation of WUEP through a controlled experiment. This experiment was designed in order to evaluate the effectiveness, efficiency, perceived ease of use, and satisfaction with WUEP in comparison to a widely-used inspection method: Heuristic evaluation (HE). Results show that WUEP is more effective and efficient than HE in the detection of usability problems in artifacts obtained from a model-driven Web development process. The evaluators were also satisfied when applying WUEP, and found it easier to use than HE.

This paper describes the results of an external replication of an experiment for assessing whether the use of dynamic modeling influences the comprehension of software requirements. The results of the original experiment conducted in Italy did not confirm that there was a significant difference in the comprehension of software requirements when dynamic modeling is used. The goal of the replication was therefore to verify these findings with a group of more experienced students at the Universidad Politeacutecnica de Valencia (UPV) in Spain. The results shows that the use of dynamic modeling does significantly improve the comprehension of software requirements, thus providing evidence that dynamic modeling facilitates the interpretation and comprehension of requirements.

This paper reports on a controlled experiment that investigates the influence of stereotypes in UML sequence diagrams. The comprehension of UML sequence diagrams with and without stereotypes is analyzed from three different perspectives: semantic comprehension, retention and transfer. The experiment was carried out with 77 undergraduate students of Computer Science from the University of Bari in Italy. The results obtained show a slight tendency in favor of the use of stereotypes in facilitating the comprehension of UML sequence diagrams. Further replications are needed to obtain more conclusive results.

This paper presents a replication study that investigates the efficacy and likely adoption of a measurement procedure for sizing Web applications from conceptual models (OOmFPWeb). The goal of the replication was to provide evidence for the generalization of the results by repeating the experiment in a different environment, using different subjects. The results of the replica carried out in Austria have confirmed the results of the original experiment, which was carried out in Spain. OOmFPWeb is efficient when compared to current industry practices. It provides reproducible functional size measurements and is perceived as easy to use and useful by its users, who also expressed their intention to use OOmFPWeb in the future. The analysis further supports the validity and reliability of Moody’s Method Evaluation Model for evaluating functional size measurement methods.

In this paper, we briefly introduce a controlled experiment to investigate the selection of alternative transformation rules through which to obtain UML class diagrams from a Requirements Model. The main goal of this experiment was to determine which of the transformation rules for structural relationships between classes (association (Al), aggregation (A2) and association class (A3)) produces the UML class diagram that is easiest to understand. More details about the transformations and about the experiment are provided. We focus upon the understandability of UML class diagrams because it is well recognized that if a model is easier to understand it will be easier to maintain, reuse, etc.