Background: Taxonomies are crucial for the development of a research field, as they play a major role in structuring a complex body of knowledge and help to classify processes, approaches, and solutions. While there is an increasing interest in taxonomies in the software engineering (SE) research field, we observe that SE taxonomies are rarely evaluated. Aim: To raise awareness and provide operational guidance on how to evaluate a taxonomy, this paper presents a three step evaluation method evaluating its structure, applicability, and purpose. Method: To show the feasibility and applicability of our approach, we provide a running example and additionally illustrate our approach to a practical case study in SE research. Results and Conclusion: Our method with operational guidance enables SE researchers to systematically evaluate and improve the quality of their taxonomies and support reviewers to systematically assess a taxonomy’s quality.

Science and thereby software engineering (SE) research enables progress and innovation. SE research has high economic relevance and serves as an enabler for innovations in other research fields as well. Researchers in this field perform many activities and face corresponding challenges. They have to design and conduct research, while deciding on an appropriate method design for a scientific or practical problem. They have to review research while critically assessing research outcomes in a systematic and comparable way. They have to search for research outcomes to find research gaps to build upon. Likewise, by finding related information, they are able to justify relevance of own research work. These searching processes are done via keyword-based searches while struggling with information overload due to immensely growing number of SE papers published every year. Thus, there is the need for conceptual approaches, automated mechanisms, and a rethinking about scholarly communication to support these research activities and challenges in an effective, efficient, and comprehensible way. There are a variety of available classifications for research methods in literature. Additionally, empirical standards for research methods are continuously developed and adapted from other research fields. However, there is no consistency in the covered research methods, nor in the used terminology as well as no systematic approach to structure this body of research knowledge (e.g., template research questions, research methods, validity threats, replication types). Therefore, this doctoral thesis will provide a unified classification scheme to describe the characteristics of SE research. This further enables classification of SE papers and a corresponding queryable knowledge management system and, thus, advance state of practice.

Context: Software architecture (SA) as research area experienced an increase in empirical research, as identified by Galster and Weyns in 2016 [1]. Empirical research builds a sound foundation for the validity and comparability of the research. A current overview on the evaluation and replicability of SA research objects could help to discuss our empirical standards as a community. However, no such current overview exists.Objective: We aim at assessing the current state of practice of evaluating SA research objects and replication artifact provision in full technical conference papers from 2017 to 2021.Method: We first create a categorization of papers regarding their evaluation and provision of replication artifacts. In a systematic literature review (SLR) with 153 papers we then investigate how SA research objects are evaluated and how artifacts are made available.Results: We found that technical experiments (28%) and case studies (29%) are the most frequently used evaluation methods over all research objects. Functional suitability (46% of evaluated properties) and performance (29%) are the most evaluated properties. 17 papers (11%) provide replication packages and 97 papers (63%) explicitly state threats to validity. 17% of papers reference guidelines for evaluations and 14% of papers reference guidelines for threats to validity.Conclusions: Our results indicate that the generalizability and repeatability of evaluations could be improved to enhance the maturity of the field; although, there are valid reasons for contributions to not publish their data. We derive from our findings a set of four proposals for improving the state of practice in evaluating software architecture research objects. Researchers can use our results to find recommendations on relevant properties to evaluate and evaluation methods to use and to identify reusable evaluation artifacts to compare their novel ideas with other research. Reviewers can use our results to compare the evaluation and replicability of submissions with the state of the practice.

The rapid growth of software engineering research publications forces an amount of scholarly knowledge that needs to be managed, organized and communicated in digital libraries and scientific search engines. Thus, there is a need for classified papers to accomplish these tasks, but the classification process is cumbersome. Moreover, in case of new schemas, one would need to reclassify previously published research. We propose to automate the classification and present different possible techniques for doing so: Using natural language models, a rule-based approach, or an approach based on topic-labeling. In this proposal paper, we initially implemented a prototype for text classification of software engineering research papers.

This taxonomy paper presents a novel way of knowledge engineering in the software engineering research community. Till now, research papers are organized digitally as documents, mostly in PDF files. Not much effort is spent on effective knowledge classification, retrieval, storage, and representation. In contrast to the current paper-based approach for knowledge documentation, we present a statement-based approach, where each statement is linked to arguments and data of its evidence as well as to related statements. We argue that in this way, knowledge will be easier to retrieve, compare, and evaluate in contrast to current paper-based knowledge engineering in scientific search engines and digital libraries. Therefore, we present as a first step a novel multi-dimensional classification for statements in software engineering research. Statements are classified according to their research object, their kind (e.g., relevance), and their underlying evidence. This classification is validated and extended with a first systematic literature review. Additionally, we provide an example for illustration purpose.

Business processes need to constantly adapt due to changes in their environment and requirements. Therefore, one of the main activities in business process management is the management of changes. To effectively manage changes, there is a need for categorization of change triggers in business processes. However, existing categories of change triggers are limited to information systems and neglect the change triggers of business processes. We conducted a review with a well-defined methodology to identify categories of change triggers in business processes. This paper presents a generic categorization scheme of change triggers in business processes based on the results of the review. The new categorization scheme can serve as a checklist to elicit the possible future business process changes and, thus, support the process of change and risk management.