UML is the de facto standard notation for graphically representing software. UML diagrams are used in the analysis, construction, and maintenance of software systems. Mostly, UML diagrams capture an abstract view of a (piece of a) software system. A key purpose of UML diagrams is to share knowledge about the system among developers. The quality of the layout of UML diagrams plays a crucial role in their comprehension. In this paper, we present an automated method for evaluating the layout quality of UML class diagrams. We use machine learning based on features extracted from the class diagram images using image processing. Such an automated evaluator has several uses: (1) From an industrial perspective, this tool could be used for automated quality assurance for class diagrams (e.g., as part of a quality monitor integrated into a DevOps toolchain). For example, automated feedback can be generated once a UML diagram is checked in the project repository. (2) In an educational setting, the evaluator can grade the layout aspect of student assignments in courses on software modeling, analysis, and design. (3) In the field of algorithm design for graph layouts, our evaluator can assess the layouts generated by such algorithms. In this way, this evaluator opens up the road for using machine learning to learn good layouting algorithms. We use machine learning techniques to build (linear) regression models based on features extracted from the class diagram images using image processing. As ground truth, we use a dataset of 600+ UML Class Diagrams for which experts manually label the quality of the layout. This paper makes the following contributions: (1) We show the feasibility of the automatic evaluation of the layout quality of UML class diagrams. (2) We analyze which features of UML class diagrams are most strongly related to the quality of their layout. (3) We evaluate the performance of our layout evaluator. (4) We offer a dataset of labeled UML class diagrams. In this dataset, we supply for every diagram the following information: (a) a manually established ground truth of the quality of the layout, (b) an automatically established value for the layout-quality of the diagram (produced by our classifier), and (c) the values of key features of the layout of the diagram (obtained by image processing). This dataset can be used for replication of our study and others to build on and improve on this work. Editor’s note: Open Science material was validated by the Journal of Systems and Software Open Science Board..

Today, there is a wide range of techniques to support software comprehension. However, we do not fully understand yet what techniques really help novices, to comprehend a software system. In this paper, we present a master level project course on software evolution, which has a large focus on software comprehension. We collected data about student's experience with diverse comprehension techniques during focus group discussions over the course of two years. Our results indicate that systematic code reading can be supported by additional techniques to guiding reading efforts. Most techniques are considered valuable for gaining an overview and some techniques are judged to be helpful only in later stages of software comprehension efforts.

Motivation: Software visualization can be helpful in comprehending the architecture of large software systems. Traditionally, software visualisation focuses on representing the structural perspectives of systems. In this paper we enrich this perspective by adding the notion of role-stereotype. This rolestereotype carries information about the type of functionality that a class has in the system as well as the types of collaborations with other classes that it typically has.Objective: We propose an interactive visualization called RoleViz, that visualizes system architectures in which architectural elements are annotated with their role-stereotypes.Method: We conducted a user-study in which developers use RoleViz and Softagram (a commercial tool for software architecture comprehension) to solve two separate comprehension tasks on a large open source system. We compared RoleViz against Softagram in terms of participant's: (i) perceived cognitive load, (ii) perceived usability, and (iii) understanding of the system. Result: In total, 16 developers participated in our study. Six of the participants explicitly indicated that visualizing roles helped them complete the assigned tasks. Our observations indicate significant differences in terms of participant's perceived usability and understanding scores.Conclusion: The participants achieved better scores on completing software understanding tasks with RoleViz without any cognitive-load penalty.

Role stereotypes indicate generic roles that classes play in the design of software systems (e.g. controller, information holder, or interfacer). Knowledge about the role-stereotypes can help in various tasks in software development and maintenance, such as program understanding, program summarization, and quality assurance. This paper presents an automated machine learning-based approach for classifying the role-stereotype of classes in Java. We analyse the performance of this approach against a manually labelled ground truth for a sizable open source project (of 770+ Java classes) for the Android platform. Moreover, we compare our approach to an existing rule-based classification approach. The contributions of this paper include an analysis of which machine learning algorithms and which features provide the best classification performance. This analysis shows that the Random Forest algorithm yields the best classification performance. We find however, that the performance of the ML-classifier varies a lot for classifying different role-stereotypes. In particular its performs degrades for rare role-types. Our ML-classifier improves over the existing rule-based classification method in that the ML-approach classifies all classes, while rule-based approaches leave a significant number of classes unclassified.

The benefits of modeling the design to improve the quality and maintainability of software systems have long been advocated and recognized. Yet, the empirical evidence on this remains scarce. In this paper, we fill this gap by reporting on an empirical study of the relationship between UML modeling and software defect proneness in a large sample of open-source GitHub projects. Using statistical modeling, and controlling for confounding variables, we show that projects containing traces of UML models in their repositories experience, on average, a statistically minorly different number of software defects (as mined from their issue trackers) than projects without traces of UML models.

UML Class diagrams are commonly used to describe the designs of systems. Such designs can be used to guide the construction of software. In practice, we have identified two main types of using UML: (i) FwCD refers to diagrams are hand-made as part of the forward-looking development process; (ii) RECD refers to those diagrams that are reverse engineered from the source code; Recently, empirical studies in Software Engineering have started looking at open source projects. This enables the automated extraction and analysis of large sets of project-data. For researching the effects of UML modeling in open source projects, we need a way to automatically determine the way in which UML used in such projects. For this, we propose an automated classifier for deciding whether a diagram is an FwCD or an RECD. We present the construction of such a classifier by means of (supervised) machine learning algorithms. As part of its construction, we analyse which features are useful in classifying FwCD and RECD. By comparing different machine learning algorithms, we find that the Random Forest algorithm is the most suitable algorithm for our purpose. We evaluate the performance of the classifier on a test set of 999 class diagrams obtained from open source projects.

A recurring theme in discussions about the adoption of Model-Based Engineering (MBE) is its effectiveness. This is because there is a lack of empirical assessment of the processes and (tool-)use of MBE in practice. We conducted a multiple-case study by observing 2 two-month MBE projects from which software for a Mars rover were developed. We focused on assessing the distribution of the total software development effort over different development activities. Moreover, we observed and collected challenges reported by the developers during the execution of projects. We found that the majority of the effort is spent on the collaboration and communication activities. Furthermore, our inquiry into challenges showed that tool-related challenges are the most encountered.

One argument in the discussion about the adoption of UML in industry is the supposedly large effort it takes to do modeling. Our study explores how the creation of UML models can be understood to consist of different cognitive activities: (i) designing: thinking about the design (ideation, key-design decision making), (ii) notation expression: expressing a design in a modeling notation and (iii) layouting: the spatial organization of model elements in a diagram. We explain that these different subactivities relate to different short-term and long-term benefits of modeling. In this study we present two controlled experiments with a total of 100 subjects creating models for a small system. In these experiments we focus on software models as represented through UML class diagram. Our results show that at least 56% of the effort spent on creating a class model is actually due to designing. Notation expression is around 41% of the model creation effort and layouting is in the order of 3%. This finding suggests that a significant part of creating models is devoted to design thinking about the problem.

The Unified Modeling Language (UML) is widely taught in academia and has good acceptance in industry. However, there is not an ample dataset of UML diagrams publicly available. Our aim is to offer a dataset of UML files, together with meta-data of the software projects where the UML files belong to. Therefore, we have systematically mined over 12 million GitHub projects to find UML files in them. We present a semi-automated approach to collect UML stored in images, .xmi, and .uml files. We offer a dataset with over 93,000 UML diagrams from over 24,000 projects in GitHub.

Graphical modelling of various aspects of software and systems is a common part of software development. UML is the de-facto standard for various types of software models. To be able to research UML, academia needs to have a corpus of UML models. For building such a database, an automated system that has the ability to classify UML class diagram images would be very beneficial, since a large portion of UML class diagrams (UML CDs) is available as images on the Internet. In this study, we propose 23 image-features and investigate the use of these features for the purpose of classifying UML CD images. We analyse the performance of the features and assess their contribution based on their Information Gain Attribute Evaluation scores. We study specificity and sensitivity scores of six classification algorithms on a set of 1300 images. We found that 19 out of 23 introduced features can be considered as influential predictors for classifying UML CD images. Through the six algorithms, the prediction rate achieves nearly 96% correctness for UML-CD and 91% of correctness for non-UML CD.