Digital Health (DH) is widely considered essential for sustainable future healthcare systems. Software quality, particularly usability, is crucial for the success and adoption of most DH products. However, concerns about the effectiveness and efficiency of testing the usability of DH products have been raised. This article aims to analyse the prevalence and application contexts of usability evaluation methods in DH and to highlight potential issues related to their effectiveness and efficiency. A systematic literature review of usability evaluation studies, published by (academic) practitioners between 2016 and April 2023, was conducted. 610 primary articles were identified and analysed, utilising five major scientific databases. Our findings show a preference for inquiry (85%) and testing (63%) methods, with inspection used less frequently (17%). The published studies employed methods like questionnaires (75%); notably the SUS (49%), semi-structured interviews (25%), and heuristic evaluations (73%), with percentages based on their group. Data collection mainly encompassed the use of participant feedback (45%), audio/video recordings (44%), and system logs (20%), with both qualitative and quantitative data analyses prevalent in studies. However, several usability characteristics such as accessibility, memorability, and operability were found to be largely overlooked, and automation tools or platforms were not widely used. Among the systems evaluated were mHealth applications (70%), telehealth platforms (36%), health information technology (HIT) solutions (29%), personalized medicine (Per. Med.) (17%), wearable devices (12%), and digital therapeutics (DTx) interventions (6%), with the participation of general users, patients, healthcare providers, and informal caregivers varying based on the health condition studied. Furthermore, insights and experiences gathered from 24 articles underscored the importance of a mixed-method approach in usability evaluations, the limitations of traditional methods, the necessity for sector-specific customisation, and the potential benefits of remote usability studies. Moreover, while eye-tracking emerged as a promising evaluation technique, careful execution and interpretation are crucial to avoid data misinterpretation. The study’s findings showed that employing a combination of inquiry and testing-based methods is prevalent for evaluating DH platforms. Despite an array of DH systems, method distribution remained consistent across platforms and targeted user groups. The study also underlines the importance of involving target user groups in the process. Potentially affected cognitive abilities of participants and potential user groups of interest have to be taken into account when choosing evaluation methods, and methods might therefore need to be tailored. Complementary inspection methods might be particularly useful when recruiting representative participants is difficult. Several potential paths for future research are outlined, such as exploring novel technologies like artificial intelligence, for improved automation tool support in the usability evaluation process.

InMap is an interactive and iterative information retrieval-based automated mapping algorithm that produces code-to-architecture mapping recommendations.In its original form, InMap requires an architect to provide feedback for each code-to-architecture mapping recommendation in a given set produced (complete feedback).However, architects may delay/ defer deciding on some of the mapping recommendations provided.This leads us to ask, how would InMap perform if only a subset of the recommendations provided (partial feedback) or only a single recommendation (real-time feedback) is reviewed by the architect?Through carefully designed mapping experiments, we show that an architect giving partial or real-time feedback does not harm the recall and precision of the recommendations produced by InMap.On the contrary, we observed from the results of the systems tested a net increase of 2-5% (depending on the approach).This shows that in addition to In-Map's original complete feedback approach, the two new approaches of collecting feedback presented in this paper, i.e. partial and real-time, create flexibility in how software architecture consistency checking tool developers may choose to collect mapping feedback and how architects may opt-to provide feedback, with no harm to the recall and precision of the results.

Reflexion Modelling is a successful method used in industry for Software Architectural Consistency Checking (SACC). However, it includes a mapping step that is manual and tedious, especially for large complex systems. Various studies have shown how to successfully automate the interactive mapping of the basic units of the software’s codebase, i.e. its classes, to its architecture modules. However, their inherent drawback is that the effort required by an architect to review the mapping recommendations produced, whether during the mapping occurs or at the end of mapping, can be considerable. Subsequent studies have attempted to reduce this effort by use of a hierarchical mapping approach. These studies have demonstrated a reduction in the review effort required by a software architect; however, the gain in effort reduction occurred at the price of a lower recall and precision than similar non-hierarchical mapping approaches. In this study, we present an integrated approach of automated code-to-architecture mapping that draws from hierarchical (package mapping) and non-hierarchical (class mapping) techniques to keep effort minimal for an architect with marginal loss in recall and precision. Using the harmonic mean of f1-scores and effort reduction, our results show that with our integrated approach, we could achieve 0.90 on average, compared to 0.87 for the other two methods.

Reflexion Modelling is a popular method used in industry for Software Architectural Consistency Checking (SACC). However, it involves a mapping step that is manual and tedious. There exist techniques and tools that attempt to automate mapping, yet they are either limited in their approach or they require an initial set of manually pre-mapped entities. This study proposes a novel technique, InMap, that improves the mapping process in reflexion modelling by both providing versatility and eliminating the constraint of needing a set of manually pre-mapped entities in order to automate mapping. Using a software’s architecture descriptions, InMap applies information retrieval concepts to the software’s source code to interactively provide mapping recommendations to an architect. For the six systems InMap was evaluated on, the recommendations it provided achieved an average recall of 0.97, and an average precision of 0.82. InMap also achieved higher, f1-scores in comparison to existing techniques that require premapping. This provides a basis for improving industry tools that use reflexion modelling or similar SACC methods.

Software architecture degradation is a phenomenon that frequently occurs during software evolution. Source code anomalies are one of the several aspects that potentially contribute to software architecture degradation. Many techniques for automating the detection of such anomalies are based on source code metrics. It is, however, unclear how accurate these techniques are in identifying the architecturally relevant anomalies in a system. The objective of this paper is to shed light on the extent to which source code metrics on their own can be used to characterize classes contributing to software architecture degradation. We performed a multi-case study on three open-source systems for each of which we gathered the intended architecture and data for 49 different source code metrics taken from seven different code quality tools. This data was analyzed to explore the links between architectural inconsistencies, as detected by applying reflexion modeling, and metric values indicating potential design problems at the implementation level. The results show that there does not seem to be a direct correlation between metrics and architectural inconsistencies. For many metrics, however, classes more problematic as indicated by their metric value seem significantly more likely to contribute to inconsistencies than less problematic classes. In particular, the fan-in, a classes’ public API, and method counts seem to be suitable indicators. The fan-in metric seems to be a particularly interesting indicator, as class size does not seem to have a confounding effect on this metric. This finding may be useful for focusing code restructuring efforts on architecturally relevant metrics in case the intended architecture is not explicitly specified and to further improve architecture recovery and consistency checking tool support.

Architecture Consistency (AC) aims to align implemented systems with their intended architectures. Several AC approaches and tools have been proposed and empirically evaluated, suggesting favourable results. In this paper, we empirically examine the state of practice with respect to Architecture Consistency, through interviews with nineteen experienced software engineers. Our goal is to identify 1) any practises that the companies these architects work for, currently undertake to achieve AC; 2) any barriers to undertaking explicit AC approaches in these companies; 3) software development situations where practitioners perceive AC approaches would be useful, and 4) AC tool needs, as perceived by practitioners. We also assess current commercial AC tool offerings in terms of these perceived needs. The study reveals that many practitioners apply informal AC approaches as there are barriers for adopting more formal and explicit approaches. These barriers are: 1) Difficulty in quantifying architectural inconsistency effects, and thus justifying the allocation of resources to fix them to senior management, 2) The near invisibility of architectural inconsistency to customers, 3) Practitioners’ reluctance towards fixing architectural inconsistencies, and 4) Practitioners perception that huge effort is required to map the system to the architecture when using more formal AC approaches and tools. Practitioners still believe that AC would be useful in supporting several of the software development activities such as auditing, evolution and ensuring quality attributes. After reviewing several commercial tools, we posit that AC tool vendors need to work on their ability to support analysis of systems made up of different technologies, that AC tools need to enhance their capabilities with respect to artefacts such as services and meta-data, and to focus more on non-maintainability architectural concerns.

The article Architecture consistency: State of the practice, challenges and requirements, written by Nour Ali et al. was originally published electronically on the publisher’s internet portal on May 15, 2017 without open access. The original article has been corrected.

Context: Facing the implementation of the EU General Data Protection Regulation in May 2018, many commercial software providers will soon need to adapt their products to new privacy-related constraints. Privacy patterns defined for different aspects of the software engineering process promise to be a useful concept for this task. In this situation, it seems valuable to characterize the state of the research related to privacy patterns.Objective: To identify, characterize and classify the contributions made by published research results related to patterns in the context of considering privacy concerns in engineering software. Method: A literature review in form of a mapping study of scientific articles was performed. The resulting map structures the relevant body of work into multiple dimensions, illustrating research focuses and gaps.Results: Results show that empirical evidence in this field is scarce and that holistic approaches to engineering privacy into software based on patterns are lacking. This potentially hinders industrial adoption.Conclusion: Based on these results, we recommend to empirically validate existing privacy patterns, to consolidate them in pattern catalogues and languages, and to move towards seamless approaches from engineering privacy requirements to implementation.

Expert software engineers working on large systems often need to perform feature location when moving to work in unfamiliar areas. We hypothesise that leveraging the system-specific knowledge of these may help to improve semi-automated feature location techniques. In order to assess and understand how software nomads perform manual feature location searches, two expert professional software engineers were observed in-vivo following a think-aloud protocol while performing manual feature location on a large-scale heterogeneous system. The nomads' search actions were found to be around twice as effective as those reported in previous studies. This cannot be explained by sophisticated use of tools or complex queries. We conclude that system rules and conventions are frequently used by experts when constructing feature location search terms.

Reflexion Modelling is a well-understood technique to detect architectural violations that occur during software architecture erosion. Resolving these violations can be difficult when erosion has reached a critical level and the causes of the violations are interwoven and difficult to understand. This article outlines a novel technique to automatically detect typical causes of violations in reflexion models, based on the definition and detection of typical symptoms for these causes. Preliminary results show that the proposed technique can support software architects' navigation through reflexion models of eroded systems to understand causes of violations and to systematically take actions against them.

Reference architecture can help in enterprise architecture management to develop and operate standardized and maintainable software landscapes. Similar to the software architectures of single systems, however, they are threatened by architecture erosion, i.e. the continuous divergence between intended architectures and their actual realizations. Architecture erosion has negative effects on the maintainability of software systems and on other quality attributes. In this paper, we report on the application of a rule-based architecture conformance checking approach in an industrial case study in which we investigate an industrial reference architecture for the German public administration. The reference architecture and its constraints for implementations are formalized as architecture rules enabling automatic conformance checking tool support. The results from the case study show that the approach is capable of checking reference architecture conformance in realistic settings and helps to avoid software architecture erosion.

Detecting software architecture erosion is an important task during the development and maintenance of software systems. Even in model-driven approaches in which consistency between artifacts can partially be established by construction and consistency issues have been intensively investigated, the intended architecture and its realization may diverge with negative effects on software quality. In this article, we describe an approach to flexible architecture erosion detection for model-driven development approaches. Consistency constraints expressed by architectural aspects called architectural rules are specified as formulas on a common ontology, and models are mapped to instances of that ontology. A knowledge representation and reasoning system is then utilized to check whether these architectural rules are satisfied for a given set of models. We describe three case studies in which this approach has been used to detect architecture erosion flexibly and argue that the negative effects of architecture erosion can be minimized effectively.

Modern enterprise application systems are parts of complex IT landscapes. The architecture of such a landscape may impose constraints upon the design of single applications, for example by the mandatory use of enterprise-wide reference architectures. It is of great importance for the sake of smooth operation and easy maintaining that single applications are com-pliant to the reference architectures. Checking this compliance is highly important for the architecture management to assure the quality of application systems. Unfortunately, current tool support is not flexible enough to easily check different aspects of architectural compliance.This paper proposes a rule-based approach based upon logic programming concepts towards a formalism for architectural compliance checking. In this approach, the architecture and design are represented as logical knowledge base that can be queried for architectural compliance. Furthermore, the paper presents a case study, in which the approach was prototypically implemented and applied in an industrial context.