Nowadays, software permeates almost every aspect of our lives. To produce complex and large-scale software products, a large number of software engineers are required. Accordingly, researchers and educators recognize the importance of Software Engineering Education (SEE), and many studies related to SEE have been published in recent years. To synthesize the large amount of research in SEE, some Systematic Literature Reviews (SLRs) focusing on different areas of SEE have been conducted and reported. However, due to their limited focuses, none of these SLRs is able to depict an overall state-of-the-art for SEE. To remedy this, we conducted a tertiary study on SEE, which identifies 26 relevant SLRs published between 2004 and 2019. By classifying and positioning these SLRs in two dimensions, i.e. the education methods/tools applied for SEE and the research topics related to SEE, we present a landscape of SEE, which locates the SLRs on SEE and their research dimensions. Further, we collected the issues studied in the published research and those that need to be addressed for instructors. This paper also discusses the challenges of the current SEE research landscape.

Context: Following on other scientific disciplines, such as health sciences, the use of Grey Literature (GL) has become widespread in Software Engineering (SE) research. Whilst the number of papers incorporating GL in SE is increasing, there is little empirically known about different aspects of the use of GL in SE research.

Software development is a collective undertaking, and the team’s efficiency is critical in development. In order to reduce project management overheads and improve productivity, a global information and communication technology enterprise institutionalizes an organization wide small-team practice, called fireteams, to tackle the problems arising from human and social aspects, such as amicability, talent, skill, and communications. This paper reports a mixed-method research, which combines archive analysis, interviews and survey, to empirically investigate the characteristics and impacts of fireteam in this industrial setting. We identify three categories of fireteam in terms of its demonstrated characteristics: ordinary agile team with extensions, single-function team, and entire life-cycle team; elaborate four key activities of fireteam, i.e. team formation, maintenance, communication, and meeting. Less communication and management overheads, higher agility & concurrency, and improved personal ability are the three important contributors that increase the productivity of fireteams. Whereas management & leadership effort, divergent understanding of fireteam, and self-organized team are discovered as the three major problems associated with fireteams. Although the benefits of fireteam can be observed from its adoption, this practice does not achieve the enterprise’s anticipations very well. Some considerations and recommendations are also discussed to improve this small-team practice.

Context: Data synthesis is one of the most significant tasks in Systematic Literature Review (SLR). Software Engineering (SE) researchers have adopted a variety of methods of synthesizing data that originated in other disciplines. One of the qualitative data synthesis methods is meta-ethnography, which is being used in SE SLRs. Objective: We aim at studying the adoption of meta-ethnography in SE SLRs in order to understand how this method has been used in SE. Method: We conducted a tertiary study of the use of meta-ethnography by reviewing sixteen SLRs. We carried out an empirical inquiry by integrating SLR and confirmatory email survey. Results: There is a general lack of knowledge, or even awareness, of different aspects of meta-ethnography and/or how to apply it. Conclusion: There is a need of investment in gaining in-depth knowledge and skills of correctly applying meta-ethnography in order to increase the quality and reliability of the findings generated from SE SLRs. Our study reveals that meta-ethnography is a suitable method to SE research. We discuss challenges and propose recommendations of adopting meta-ethnography in SE. Our effort also offers a preliminary checklist of the systematic considerations for doing meta-ethnography in SE and improving the quality of meta-ethnographic research in SE.

Software Engineering (SE) community has recently been investing significant amount of effort in qualitative research to study the human and social aspects of SE processes, practices, and technologies. Ethnography is one of the major qualitative research methods, which is based on constructivist paradigm that is different from the hypothetic-deductive research model usually used in SE. Hence, the adoption of ethnographic research method in SE can present significant challenges in terms of sufficient understanding of the methodological requirements and the logistics of its applications. It is important to systematically identify and understand various aspects of adopting ethnography in SE and provide effective guidance. We carried out an empirical inquiry by integrating a systematic literature review and a confirmatory survey. By reviewing the ethnographic studies reported in 111 identified papers and 26 doctoral theses and analyzing the authors' responses of 29 of those papers, we revealed several unique insights. These identified insights were then transformed into a preliminary checklist that helps improve the state-of-the-practice of using ethnography in SE. This study also identifies the areas where methodological improvements of ethnography are needed in SE.

Synthesizing data extracted from primary studies is an integral component of the methodologies in support of Evidence Based Software Engineering (EBSE) such as System Literature Review (SLR). Since a large and increasing number of studies in Software Engineering (SE) incorporate qualitative data, it is important to systematically review and understand different aspects of the Qualitative Research Synthesis (QRS) being used in SE. We have reviewed the use of QRS methods in 328 SLRs published between 2005 and 2015. We also inquired the authors of 274 SLRs to confirm whether or not any QRS methods were used in their respective reviews. 116 of them provided the responses, which were included in our analysis. We found eight QRS methods applied in SE research, two of which, narrative synthesis and thematic synthesis, have been predominantly adopted by SE researchers for synthesizing qualitative data. Our study determines that a significant amount of missing knowledge and incomplete understanding of the defined QRS methods in the community. Our effort also identifies an initial set factors that may influence the selection and use of appropriate QRS methods in SE.

Context: The assessment of Threats to Validity (TTVs) is critical to secure the quality of empirical studies in Software Engineering (SE). In the recent decade, Systematic Literature Review (SLR) was becoming an increasingly important empirical research method in SE. One of the mechanisms of insuring the level of scientific value in the findings of an SLR is to rigorously assess its validity. Hence, it is necessary to realize the status quo and issues of TTVs of SLRs in SE. Objective: This study aims to investigate the-state-of-the-practice of TTVs of the SLRs published in SE, and further support SE researchers to improve the assessment and strategies against TTVs in order to increase the quality of SLRs in SE. Method: We conducted a tertiary study by reviewing the SLRs in SE that report the assessment of TTVs. Results: We identified 316 SLRs published from 2004 to the first half of 2015, in which TTVs are discussed. The issues associated to TTVs were also summarized and categorized. Conclusion: The common TTVs related to SLR research, such as internal validity and reliability, were thoroughly discussed in most SLRs. The threats to construct validity and external validity drew less attention. Moreover, there are few strategies and tactics being reported to cope with the various TTVs.

Context: The quality of an Systematic Literature Review (SLR) is as good as the quality of the reviewed papers. Hence, it is vital to rigorously assess the papers included in an SLR. There has been no tertiary study aimed at reporting the state of the practice of quality assessment used in SLRs in Software Engineering (SE).