Paul Ralph

dblp:71/2904 · DBLP profile ↗
← Back
40ranked-venue papers
13as first author
16since 2021 · last 2025
0000-0002-7411-0857ORCID · verified

Domains — the database's venue-derived domains; a paper can count in several

Software engineering, systems software and programming languages · 39 · 13 first-author · 16 since 2021Human-computer interaction and ubiquitous computing · 2 · 1 since 2021Databases, data management, data science and information retrieval · 1 · 1 since 2021Applied, interdisciplinary, general and emerging computing · 1
YearPublicationVenuePosition
2025 User Personas Improve Social Sustainability by Encouraging Software Developers to Deprioritize Antisocial Features
abstract
Background: Sustainable software development involves creating software in a manner that meets present goals without undermining our ability to meet future goals. In a software engineering context, sustainability has at least four dimensions: ecological, economic, social, and technical. No interventions for improving social sustainability in software engineering have been tested in rigorous lab-based experiments, and little evidence-based guidance is available. Objective: The purpose of this study is to evaluate the effectiveness of two interventions—stakeholder maps and persona models—for improving social sustainability through software feature prioritization. Method: We conducted a randomized controlled factorial experiment with 79 undergraduate computer science students. Participants were randomly assigned to one of four groups and asked to prioritize a backlog of prosocial, neutral, and antisocial user stories for a shopping mall's digital screen display and facial recognition software. Participants received either persona models, a stakeholder map, both, or neither. We compared the differences in prioritization levels assigned to prosocial and antisocial user stories using Cumulative Link Mixed Model regression. Results: Participants who received persona models gave significantly lower priorities to antisocial user stories but no significant difference was evident for prosocial user stories. The effects of the stake-holder map were not significant. The interaction effects were not significant. Conclusion: Providing aspiring software professionals with well-crafted persona models causes them to de-prioritize antisocial software features. The impact of persona modelling on sustainable software development therefore warrants further study with more experience professionals. Moreover, the novel methodological strategy of assessing social sustainability behavior through backlog prioritization appears feasible in lab-based settings.
Bimpe Ayoola, Miikka Kuutila, Rina R. Wehbe, Paul Ralph
ICSE4
2025 The Impact of Generative AI on Creativity in Software Development: A Research Agenda
abstract
As GenAI becomes embedded in developer toolchains and practices, and routine code is increasingly generated, human creativity will be increasingly important for generating competitive advantage. This article uses the McLuhan tetrad alongside scenarios of how GenAI may disrupt software development more broadly, to identify potential impacts GenAI may have on creativity within software development. The impacts are discussed along with a future research agenda comprising five connected themes that consider how individual capabilities, team capabilities, the product, unintended consequences, and society can be affected.
Victoria Jackson, Bogdan Vasilescu, Daniel Russo 0002, Paul Ralph, Rafael Prikladnicki, Maliheh Izadi, Sarah D'Angelo, Sarah Inman, Anielle Lisboa, André van der Hoek
ACM Trans. Softw. Eng. Methodol.4
2024 The Promise and Challenges of Using LLMs to Accelerate the Screening Process of Systematic Reviews
abstract
Context: Systematic review (SR) is a popular research method in software engineering (SE). However, conducting an SR takes an average of 67 weeks. Thus, automating any step of the SR process could reduce the effort associated with SRs. Objective: Our objective is to investigate the extent to which Large Language Models (LLMs) can accelerate title-abstract screening by (1) simplifying abstracts for human screeners, and (2) automating title-abstract screening entirely. Method: We performed an experiment where human screeners performed title-abstract screening for 20 papers with both original and simplified abstracts from a prior SR. The experiment with human screeners was reproduced by instructing GPT-3.5 and GPT-4 LLMs to perform the same screening tasks. We also studied whether different prompting techniques (Zero-shot (ZS), One-shot (OS), Few-shot (FS), and Few-shot with Chain-of-Thought (FS-CoT) prompting) improve the screening performance of LLMs. Lastly, we studied if redesigning the prompt used in the LLM reproduction of title-abstract screening leads to improved screening performance. Results: Text simplification did not increase the screeners’ screening performance, but reduced the time used in screening. Screeners’ scientific literacy skills and researcher status predict screening performance. Some LLM and prompt combinations perform as well as human screeners in the screening tasks. Our results indicate that a more recent LLM (GPT-4) is better than its predecessor LLM (GPT-3.5). Additionally, Few-shot and One-shot prompting outperforms Zero-shot prompting. Conclusion: Using LLMs for text simplification in the screening process does not significantly improve human performance. Using LLMs to automate title-abstract screening seems promising, but current LLMs are not significantly more accurate than human screeners. To recommend the use of LLMs in the screening process of SRs, more research is needed. We recommend future SR studies to publish replication packages with screening data to enable more conclusive experimenting with LLM screening.
Aleksi Huotala, Miikka Kuutila, Paul Ralph, Mika Mäntylä
EASE3
2024 Generative AI in Software Engineering Must Be Human-Centered: The Copenhagen Manifesto
Daniel Russo 0002, Sebastian Baltes, Niels van Berkel, Paris Avgeriou, Fabio Calefato, Beatriz Cabrero-Daniel, Gemma Catolino, Jürgen Cito, Neil A. Ernst, Thomas Fritz 0001, Hideaki Hata, Reid Holmes, Maliheh Izadi, Foutse Khomh, Mikkel Baun Kjærgaard, Grischa Liebel, Alberto Lluch-Lafuente, Stefano Lambiase, Walid Maalej, Gail C. Murphy, Nils Brede Moe, Gabrielle O'Brien, Elda Paja, Mauro Pezzè, John Stouby Persson, Rafael Prikladnicki, Paul Ralph, Martin P. Robillard, Thiago Rocha Silva, Klaas-Jan Stol, Margaret-Anne D. Storey, Viktoria Stray, Paolo Tell, Christoph Treude, Bogdan Vasilescu
J. Syst. Softw.27
2023 Post-pandemic Resilience of Hybrid Software Teams
abstract
Background. The COVID-19 pandemic triggered a widespread transition to hybrid work models (combinations of co-located and remote work) as software professionals’ demanded more flexibility and improved work-life balance. However, hybrid work models reduce the spontaneous, informal face-to-face interactions that promote group maturation, cohesion, and resilience. Little is known about how software companies can successfully transition to a hybrid workforce or the factors that influence the resilience of hybrid software development teams. Goal. The purpose of this study is to explore the relationship between hybrid work and team resilience in the context of software development. Method. Constructivist Grounded Theory was used, based on interviews of 26 software professionals. This sample included professionals of different genders, ethnicities, sexual orientations, and levels of experience. Interviewees came from eight different companies, 22 different projects, and four different countries. Consistent with grounded theory methodology, data collection, and analysis were conducted iteratively, in waves, using theoretical sampling, constant comparison, and initial, focused, and theoretical coding. Results. Software Team Resilience is the ability of a group of software professionals to continue working together effectively under adverse conditions. Resilience depends on the group’s maturity. The configuration of a hybrid team (who works where and when) can promote or hinder group maturity depending on the level of intra-group interaction it supports. Conclusion. This paper presents the first study on the resilience of hybrid software teams. Software teams need resilience to maintain their performance in the face of disruptions and crises. Software professionals strongly value hybrid work; therefore, team resilience is a key factor to be considered in the software industry.
Ronnie E. S. Santos, Gianisa Adisaputri, Paul Ralph
CHASE3
2023 Sustainability is Stratified: Toward a Better Theory of Sustainable Software Engineering
abstract
Background: Sustainable software engineering (SSE) means creating software in a way that meets present needs without undermining our collective capacity to meet our future needs. It is typically conceptualized as several intersecting dimensions or “pillars”-environmental, social, economic, technical and in-dividual. However; these pillars are theoretically underdeveloped and require refinement. Objectives: The objective of this paper is to generate a better theory of SSE. Method: First, a scoping review was conducted to understand the state of research on SSE and identify existing models thereof. Next, a meta-synthesis of qualitative research on SSE was conducted to critique and improve the existing models identified. Results: 961 potentially relevant articles were extracted from five article databases. These articles were de-duplicated and then screened independently by two screeners, leaving 243 articles to examine. Of these, 109 were non-empirical, the most common empirical method was systematic review, and no randomized controlled experiments were found. Most papers focus on ecological sustainability (158) and the sustainability of software products (148) rather than processes. A meta-synthesis of 36 qualitative studies produced several key propositions, most notably, that sustainability is stratified (has different meanings at different levels of abstraction) and multisystemic (emerges from interactions among multiple social, technical, and sociotechnical systems). Conclusion: The academic literature on SSE is surprisingly non-empirical. More empirical evaluations of specific sustainability interventions are needed. The sustainability of software development products and processes should be conceptualized as multisystemic and stratified, and assessed accordingly.
Sean McGuire, Erin Schultz, Bimpe Ayoola, Paul Ralph
ICSE4
2022 Personality Traits in Game Development
abstract
Existing work on personality traits in software development excludes game developers as a discrete group. Whilst games are software, game development has unique considerations, so game developers may exhibit different personality traits from other software professionals. We assessed responses from 123 game developers on an International Personality Item Pool Five Factor Model scale and demographic questionnaire using factor analysis. Programmers reported lower Extraversion than designers, artists and production team members; lower Openness than designers and production, and reported higher Neuroticism than production—potentially linked to burnout and crunch time. Compared to published norms of software developers, game developers reported lower Openness, Conscientiousness, Extraversion and Agreeableness, but higher Neuroticism. These personality differences have many practical implications: differences in Extraversion among roles may precipitate communication breakdowns; differences in Openness may induce conflict between programmers and designers. Understanding the relationship between personality traits and roles can help recruiters steer new employees into appropriate roles, and help managers apply appropriate stress management techniques. To realise these benefits, individuals must be distinguished from roles: just because an individual occupies a role does not mean they possess personality traits associated with that role.
Miriam Sturdee, Matthew Ivory, David A. Ellis, Patrick Stacey, Paul Ralph
EASE5
2022 What Makes Effective Leadership in Agile Software Development Teams?
abstract
Effective leadership is one of the key drivers of business and project success, and one of the most active areas of management research. But how does leadership work in agile software development, which emphasizes self-management and self-organization and marginalizes traditional leadership roles? To find out, this study examines agile leadership from the perspective of thirteen professionals who identify as agile leaders, in different roles, at ten different software development companies of varying sizes. Data from semi-structured interviews reveals that leadership: (1) is dynamically shared among team members; (2) engenders a sense of belonging to the team; and (3) involves balancing competing organizational cultures (e.g. balancing the new agile culture with the old milestone-driven culture). In other words, agile leadership is a property of a team, not a role, and effectiveness depends on agile team members' identifying with the team, accepting responsibility, and being sensitive to cultural conflict.
Lucas Gren, Paul Ralph
ICSE2
2022 Social Science Theories in Software Engineering Research
abstract
As software engineering research becomes more concerned with the psychological, sociological and managerial aspects of software development, relevant theories from reference disciplines are increasingly important for understanding the field's core phenomena of interest. However, the degree to which software engineering research draws on relevant social sciences remains unclear. This study therefore investigates the use of social science theories in five influential software engineering journals over 13 years. It analyzes not only the extent of theory use but also what, how and where these theories are used. While 87 different theories are used, less than two percent of papers use a social science theory, most theories are used in only one paper, most social sciences are ignored, and the theories are rarely tested for applicability to software engineering contexts. Ignoring relevant social science theories may (1) undermine the community's ability to generate, elaborate and maintain a cumulative body of knowledge; and (2) lead to oversimplified models of software engineering phenomena. More attention to theory is needed for software engineering to mature as a scientific discipline.
Tobias Lorey, Paul Ralph, Michael Felderer
ICSE2
2022 A Grounded Theory of Coordination in Remote-First and Hybrid Software Teams
abstract
While the long-term effects of the COVID-19 pandemic on software professionals and organizations are difficult to predict, it seems likely that working from home, remote-first teams, distributed teams, and hybrid (part-remote/part-office) teams will be more common. It is therefore important to investigate the challenges that software teams and organizations face with new remote and hybrid work. Consequently, this paper reports a year-long, participant-observation, constructivist grounded theory study investigating the impact of working from home on software development. This study resulted in a theory of software team coordination. Briefly, shifting from in-office to at-home work fundamentally altered coordination within software teams. While group cohesion and more effective communication appear protective, coordination is undermined by distrust, parenting and communication bricolage. Poor coordination leads to numerous problems including misunderstandings, help requests, lower job satisfaction among team members, and more ill-defined tasks. These problems, in turn, reduce overall project success and prompt professionals to alter their software development processes (in this case, from Scrum to Kanban). Our findings suggest that software organizations with many remote employees can improve performance by encouraging greater engagement within teams and supporting employees with family and childcare responsibilities.
Ronnie E. S. Santos, Paul Ralph
ICSE2
2022 Empirical Standards for Repository Mining
abstract
The purpose of scholarly peer review is to evaluate the quality of scientific manuscripts. However, study after study demonstrates that peer review neither effectively nor reliably assesses research quality. Empirical standards attempt to address this problem by modelling a scientific community's expectations for each kind of empirical study conducted in that community. This should enhance not only the quality of research but also the reliability and predictability of peer review, as scientists adopt the standards in both their researcher and reviewer roles. However, these improvements depend on the quality and adoption of the standards. This tutorial will therefore present the empirical standard for mining software repositories, both to communicate its contents and to get feedback from the attendees. The tutorial will be organized into three parts: (1) brief overview of the empirical standards project; (2) detailed presentation of the repository mining standard; (3) discussion and suggestions for improvement.
Preetha Chatterjee, Tushar Sharma 0001, Paul Ralph
MSR3
2022 Paving the way for mature secondary research: the seven types of literature review
abstract
Confusion over different kinds of secondary research, and their divergent purposes, is undermining the effectiveness and usefulness of secondary studies in software engineering. This short paper therefore explains the differences between ad hoc review, case survey, critical review, meta-analysis (aka systematic literature review), meta-synthesis (aka thematic analysis), rapid review and scoping review (aka systematic mapping study). These definitions and associated guidelines help researchers better select and describe their literature reviews, while helping reviewers select more appropriate evaluation criteria.
Paul Ralph, Sebastian Baltes
ESEC/SIGSOFT FSE1
2022 Sampling in software engineering research: a critical review and guidelines
Sebastian Baltes, Paul Ralph
Empir. Softw. Eng.2
2022 How Templated Requirements Specifications Inhibit Creativity in Software Engineering
abstract
Desiderata is a general term for stakeholder needs, desires or preferences. Recent experiments demonstrate that presenting desiderata as templated requirements specifications leads to less creative solutions. However, these experiments do not establish how the presentation of desiderata affects design creativity. This study, therefore, aims to explore the cognitive mechanisms by which presenting desiderata as templated requirements specifications reduces creativity during software design. Forty-two software designers, organized into 21 pairs, participated in a dialog-based protocol study. Their interactions were transcribed and the transcripts were analyzed in two ways: (1) using inductive process coding and (2) using an a-priori coding scheme focusing on fixation and critical thinking. Process coding shows that participants exhibited seven categories of behavior: making design moves, uncritically accepting, rejecting, grouping, questioning, assuming and considering quality criteria. Closed coding shows that participants tend to accept given requirements and priority levels while rejecting newer, more innovative design ideas. Overall, the results suggest that designers fixate on desiderata presented as templated requirements specifications, hindering critical thinking. More precisely, requirements fixation mediates the negative relationship between specification formality and creativity.
Rahul Mohanani, Paul Ralph, Burak Turhan, Vladimir Mandic
IEEE Trans. Software Eng.2
2021 Towards a More Structured Peer Review Process with Empirical Standards
abstract
Context. Empirical research consistently demonstrates that that scholarly peer review is ineffective, unreliable, and prejudiced. In principle, the solution is to move from contemporary, unstructured, essay-like reviewing to more structured, checklist-like reviewing. The Task Force created models—called “empirical standards”—of the software engineering community’s expectations for different popular methodologies. Objective. This paper presents a tool for facilitating more structured reviewing by generating review checklists from the empirical standards. Design. A tool that generates pre-submission and review forms using the empirical standards for software engineering research was designed and implemented. The pre-submission and review forms can be used by authors and reviewers, respectively, to determine whether a manuscript meets the software engineering community’s expectations for the particular kind of research conducted. Evaluation. The proposed tool can be empirically evaluated using lab or field randomized experiments as well as qualitative research. Huge, impractical studies involving splitting a conference program committee are not necessary to establish the effectiveness of the standards, checklists and structured review. Conclusions. The checklist generator enables more structured peer reviews, which in turn should improve review quality, reliability, thoroughness, and readability. Empirical research is needed to assess the effectiveness of the tool and the standards.
Arham Arshad, Taher Ahmed Ghaleb, Paul Ralph
EASE3
2021 Requirements Framing Affects Design Creativity
abstract
Design creativity, the originality and practicality of a solution concept, is critical for the success of many software projects. However, little research has investigated the relationship between the way desiderata are presented and design creativity. This study therefore investigates the impact of presenting desiderata as ideas, requirements or prioritized requirements on design creativity. Two between-subjects randomized controlled experiments were conducted with 42 and 34 participants. Participants were asked to create design concepts from a list of desiderata. Participants who received desiderata framed as requirements or prioritized requirements created designs that are, on average, less original but more practical than the designs created by participants who received desiderata framed as ideas. This suggests that more formal, structured presentations of desiderata are less appropriate where more innovative solutions are desired. The results also show that design performance is highly susceptible to minor changes in the vernacular used to communicate desiderata.
Rahul Mohanani, Burak Turhan, Paul Ralph
IEEE Trans. Software Eng.3
2020 Pandemic programming
abstract
Abstract Context As a novel coronavirus swept the world in early 2020, thousands of software developers began working from home. Many did so on short notice, under difficult and stressful conditions. Objective This study investigates the effects of the pandemic on developers’ wellbeing and productivity. Method A questionnaire survey was created mainly from existing, validated scales and translated into 12 languages. The data was analyzed using non-parametric inferential statistics and structural equation modeling. Results The questionnaire received 2225 usable responses from 53 countries. Factor analysis supported the validity of the scales and the structural model achieved a good fit (CFI = 0.961, RMSEA = 0.051, SRMR = 0.067). Confirmatory results include: (1) the pandemic has had a negative effect on developers’ wellbeing and productivity; (2) productivity and wellbeing are closely related; (3) disaster preparedness, fear related to the pandemic and home office ergonomics all affect wellbeing or productivity. Exploratory analysis suggests that: (1) women, parents and people with disabilities may be disproportionately affected; (2) different people need different kinds of support. Conclusions To improve employee productivity, software companies should focus on maximizing employee wellbeing and improving the ergonomics of employees’ home offices. Women, parents and disabled persons may require extra support.
Paul Ralph, Sebastian Baltes, Gianisa Adisaputri, Richard Torkar, Vladimir Kovalenko, Marcos Kalinowski, Nicole Novielli, Shin Yoo, Xavier Devroey, Xin Tan 0003, Minghui Zhou 0001, Burak Turhan, Rashina Hoda, Hideaki Hata, Gregorio Robles, Amin Milani Fard, Rana Alkadhi
Empir. Softw. Eng.1
2020 Cognitive Biases in Software Engineering: A Systematic Mapping Study
abstract
One source of software project challenges and failures is the systematic errors introduced by human cognitive biases. Although extensively explored in cognitive psychology, investigations concerning cognitive biases have only recently gained popularity in software engineering research. This paper therefore systematically maps, aggregates and synthesizes the literature on cognitive biases in software engineering to generate a comprehensive body of knowledge, understand state-of-the-art research and provide guidelines for future research and practise. Focusing on bias antecedents, effects and mitigation techniques, we identified 65 articles (published between 1990 and 2016), which investigate 37 cognitive biases. Despite strong and increasing interest, the results reveal a scarcity of research on mitigation techniques and poor theoretical foundations in understanding and interpreting cognitive biases. Although bias-related research has generated many new insights in the software engineering community, specific bias mitigation techniques are still needed for software professionals to overcome the deleterious effects of cognitive biases on their work.
Rahul Mohanani, Iflaah Salman, Burak Turhan, Pilar Rodríguez 0002, Paul Ralph
IEEE Trans. Software Eng.5
2019 The product backlog
abstract
Context: One of the most common artifacts in contemporary software projects is a product backlog comprising user stories, bugs, chores or other work items. However, little research has investigated how the backlog is generated or the precise role it plays in a project. Objective: The purpose of this paper is to determine what is a product backlog, what is its role, and how does it emerge? Method: Following Constructivist Grounded Theory, we conducted a two-year, five-month participant-observation study of eight software development projects at Pivotal, a large, international software company. We interviewed 56 software engineers, product designers, and product managers.We conducted a survey of 27 product designers. We alternated between analysis and theoretical sampling until achieving theoretical saturation. Results: We observed 13 practices and 6 obstacles related to product backlog generation. Limitations: Grounded Theory does not support statistical generalization. While the proposed theory of product backlogs appears widely applicable, organizations with different software development cultures may use different practices. Conclusion: The product backlog is simultaneously a model of work to be done and a boundary object that helps bridge the gap between the processes of generating user stories and realizing them in working code. It emerges from sensemaking (the team making sense of the project context) and coevolution (a cognitive process where the team simultaneously refines its understanding of the problematic context and fledgling solution concepts).
Todd Sedano, Paul Ralph, Cécile Péraire
ICSE2
2019 Toward Methodological Guidelines for Process Theories and Taxonomies in Software Engineering
abstract
Software engineering is increasingly concerned with theory because the foundational knowledge comprising theories provides a crucial counterpoint to the practical knowledge expressed through methods and techniques. Fortunately, much guidance is available for generating and evaluating theories for explaining why things happen (variance theories). Unfortunately, little guidance is available concerning theories for explaining how things happen (process theories), or theories for analyzing and understanding situations (taxonomies). This paper therefore attempts to clarify the nature and functions of process theories and taxonomies in software engineering research, and to synthesize methodological guidelines for their generation and evaluation. It further advances the key insight that most process theories are taxonomies with additional propositions, which helps inform their evaluation. The proposed methodological guidance has many benefits: it provides a concise summary of existing guidance from reference disciplines, it adapts techniques from reference disciplines to the software engineering context, and it promotes approaches that better facilitate scientific consensus.
Paul Ralph
IEEE Trans. Software Eng.1
2018 Construct Validity in Software Engineering Research and Software Metrics
abstract
Construct validity is essentially the degree to which our scales, metrics and instruments actually measure the properties they are supposed to measure. Although construct validity is widely considered an important quality criterion for most empirical research, many software engineering studies simply assume that proposed measures are valid and make no attempt to assess construct validity. Researchers may ignore construct validity because evaluating it is intrinsically difficult, or due to lack of specific guidance for addressing it. In any case, some research inevitably produces erroneous conclusions, because due to invalid measures. This article therefore attempts to address these problems by explaining the theoretical basis of construct validity, presenting a framework for understanding it, and developing specific guidelines for assessing it. The paper draws on a detailed example involving 15 software metrics, which ostensibly measure the size, coupling and cohesion of Java classes.
Paul Ralph, Ewan D. Tempero
EASE1
2018 Objects Count so Count Objects!
abstract
One means to determine whether a student understands the fundamentals of good object-oriented design is to assess designs the student has created. However, providing reliable assessment of designs efficiently is difficult due to the many viable designs that are possible and the high level of expertise required. Consequently, design assessment tends to be limited to identifying the most basic of design problems. We propose a technique---"object counts''---that involves counting the objects created at runtime. This is more efficient than manual grading because the data is gathered automatically and more reliable than using rubrics because it is based on objective data. The data is relevant because it captures the fundamental property of an object-oriented program---the creation of objects---and so provides good insight into the student's design decisions. This provides support for both summative and formative feedback. We demonstrate the technique on two corpora containing submissions for a typical first assignment of an introductory course on object-oriented design.
Ewan D. Tempero, Paul Denny 0001, Andrew Luxton-Reilly, Paul Ralph
ICER4
2018 The two paradigms of software development research
Paul Ralph
Sci. Comput. Program.1
2018 A framework for defining coupling metrics
Ewan D. Tempero, Paul Ralph
Sci. Comput. Program.2
2017 Perceptions of Creativity in Software Engineering Research and Practice
abstract
Software engineering, especially design and requirements engineering, is intensely creative. However, practitioners and researchers appear to perceive creativity differently, hindering knowledge transfer. To explore and understand these perceptual differences, this paper combines a systematic mapping study of SE research literature with an interview study of practitioners. The subsequent analysis of 84 primary studies and 17 semi-structured interviews reveal some agreement (e.g. creativity is a process that produces novel and useful ideas). However, it also reveals important differences in the way creativity is conceptualized, measured and improved. These differences undermine evidence-based techniques to enhance and measure creativity in SE research and practice.
Rahul Mohanani, Prabhat Ram, Ahmed Lasisi, Paul Ralph, Burak Turhan
SEAA4
2017 Software development waste
abstract
Context: Since software development is a complex socio-technical activity that involves coordinating different disciplines and skill sets, it provides ample opportunities for waste to emerge. Waste is any activity that produces no value for the customer or user. Objective: The purpose of this paper is to identify and describe different types of waste in software development. Method: Following Constructivist Grounded Theory, we conducted a two-year five-month participant-observation study of eight software development projects at Pivotal, a software development consultancy. We also interviewed 33 software engineers, interaction designers, and product managers, and analyzed one year of retrospection topics. We iterated between analysis and theoretical sampling until achieving theoretical saturation. Results: This paper introduces the first empirical waste taxonomy. It identifies nine wastes and explores their causes, underlying tensions, and overall relationship to the waste taxonomy found in Lean Software Development. Limitations: Grounded Theory does not support statistical generalization. While the proposed taxonomy appears widely applicable, organizations with different software development cultures may experience different waste types. Conclusion: Software development projects manifest nine types of waste: building the wrong feature or product, mismanaging the backlog, rework, unnecessarily complex solutions, extraneous cognitive load, psychological distress, waiting/multitasking, knowledge loss, and ineffective communication.
Todd Sedano, Paul Ralph, Cécile Péraire
ICSE2
2017 Guest editorial for special section on success and failure in software engineering
Mika Mäntylä, Magne Jørgensen, Paul Ralph, Hakan Erdogmus
Empir. Softw. Eng.3
2016 A Model for Defining Coupling Metrics
abstract
Many metrics have been proposed to measure coupling-the degree of association between modules in a system. However, most metrics are under-defined, meaning that different tool developers can reasonably implement the same metric in many ways. This gives rise to families of metrics, which are superficially similar but potentially produce different results. To understand how different these metrics are, we propose a single model of coupling based on the concept of dependencies. This model is useful for defining existing coupling metrics, analysing their differences and clarifying divergent implementations. We demonstrate its efficacy by using it to describe existing coupling metrics and inform tool development. We have applied the tool to the 112 systems in the Qualitas Corpus, generating 21 million measurements from 88 coupling metrics. The simplicity of the tool implementation and the number of metrics it supports demonstrates the usefulness of our model.
Ewan D. Tempero, Paul Ralph
APSEC2
2016 Social theory for software engineering research
abstract
Many challenges in software projects are sociological, psychological or managerial in nature. Without knowledge of social science, developers, managers and researchers may misunderstand the social aspects of their projects, leading to ineffective decisions and actions. Yet, social science theories are rarely applied to Software Engineering (SE). Furthermore, understanding a single software project frequently necessitates combining multiple theories---often from several disciplines. This paper therefore aims to illustrate how certain social theories work together in a complementary manner to understand various dynamics of a software development project. To illustrate this, seven theories to understand key dynamics --- Actor Network Theory, Theory of Boundary Objects, Complexity Theory, Theory of Cognitive Biases, Effectuation Theory, Sensemaking-Coevolution-Implementation Theory and Transactive Memory Theory --- are used to explain a longitudinal study of a software development project. This study illustrates the need for integrating more social science into SE research and curriculum.
Paul Ralph, Mike W. Chiasson, Helen Kelley
EASE1
2016 Characteristics of decision-making during coding
abstract
Code appears replete with decisions including logic, organization, presentation and library use. But are developers consciously making these decisions or does it only look that way post hoc? Analysis of 104 developer interviews indicates that developers make many decisions concerning design, architecture, scope and technology use. However, they struggle to reconstruct codefocused choices between two or more alternatives. This suggests that decision-making is not the dominant cognitive process underlying coding behavior, which raises questions about how we teach coding and how we evaluate coding performance.
Paul Ralph, Ewan D. Tempero
EASE1
2016 Practice and perception of team code ownership
abstract
Context: Team code ownership is a software development practice where any team member can modify any part of the team's code. However, many factors beyond official policy affect a developer's sense of ownership.
Todd Sedano, Paul Ralph, Cécile Péraire
EASE2
2016 Sustainable Software Development through Overlapping Pair Rotation
abstract
Context: Conventional wisdom says that team disruptions (like team churn) should be avoided. However, we have observed software development projects that succeed despite high disruption.
Todd Sedano, Paul Ralph, Cécile Péraire
ESEM2
2016 Grounded theory in software engineering research: a critical review and guidelines
abstract
Grounded Theory (GT) has proved an extremely useful research approach in several fields including medical sociology, nursing, education and management theory. However, GT is a complex method based on an inductive paradigm that is fundamentally different from the traditional hypothetico-deductive research model. As there are at least three variants of GT, some ostensibly GT research suffers from method slurring, where researchers adopt an arbitrary subset of GT practices that are not recognizable as GT. In this paper, we describe the variants of GT and identify the core set of GT practices. We then analyze the use of grounded theory in software engineering. We carefully and systematically selected 98 articles that mention GT, of which 52 explicitly claim to use GT, with the other 46 using GT techniques only. Only 16 articles provide detailed accounts of their research procedures. We offer guidelines to improve the quality of both conducting and reporting GT studies. The latter is an important extension since current GT guidelines in software engineering do not cover the reporting process, despite good reporting being necessary for evaluating a study and informing subsequent research.
Klaas-Jan Stol, Paul Ralph, Brian Fitzgerald 0001
ICSE2
2016 Software engineering process theory: A multi-method comparison of Sensemaking-Coevolution-Implementation Theory and Function-Behavior-Structure Theory
Paul Ralph
Inf. Softw. Technol.1
2015 Developing and Evaluating Software Engineering Process Theories
abstract
A process theory is an explanation of how an entity changes and develops. While software engineering is fundamentally concerned with how software artifacts change and develop, little research explicitly builds and empirically evaluates software engineering process theories. This lack of theory obstructs scientific consensus by focusing the academic community on methods. Methods inevitably oversimplify and over-rationalize reality, obfuscating crucial phenomena including uncertainty, problem framing and illusory requirements. Better process theories are therefore needed to ground software engineering in empirical reality. However, poor understanding of process theory issues impedes research and publication. This paper therefore attempts to clarify the nature and types of process theories, explore their development and provide specific guidance for their empirically evaluation.
Paul Ralph
ICSE (1)1
2015 4th SEMAT Workshop on General Theory of Software Engineering (GTSE 2015)
abstract
General theories explain the fundamental phenomena that constitute a research domain. They apply across a domain and often integrate many theories and concepts into a single cohesive view. While general theories are extremely important for education and research coordination, and common in many disciplines (e.g. sociology, criminology, electrical engineering, biology, physics), software engineering lacks a well-accepted general theory. The General Theory of Software Engineering workshop seeks to rectify this situation by promoting theory development in software engineering. The fourth workshop in this series, held in conjunction with the International Conference on Software Engineering, displayed a promising trend toward more theory development papers.
Paul Ralph, Gregor Engels, Ivar Jacobson, Michael Goedicke
ICSE (2)1
2015 The Sensemaking-Coevolution-Implementation Theory of software design
Paul Ralph
Sci. Comput. Program.1
2014 Requirements fixation
abstract
There is a broad consensus that understanding system desiderata (requirements) and design creativity are both important for software engineering success. However, little research has addressed the relationship between design creativity and the way requirements are framed or presented. This paper therefore aims to investigate the possibility that the way desiderata are framed or presented can affect design creativity. Forty two participants took part in a randomized control trial where one group received desiderata framed as “requirements” while the other received desiderata framed as “ideas”. Participants produced design concepts which were judged for originality. Participants who received requirements framing produced significantly less original designs than participants who received ideas framing (Mann-Whitney U=116.5, p=0.004). We conclude that framing desiderata as “requirements” may cause requirements fixation where designers’ preoccupation with satisfying explicit requirements inhibits their creativity.
Rahul Mohanani, Paul Ralph, Ben Shreeve
ICSE2
2014 The dimensions of software engineering success
abstract
Software engineering research and practice are hampered by the lack of a well-understood, top-level dependent variable. Recent initiatives on General Theory of Software Engineering suggest a multifaceted variable – Software Engineering Success. However, its exact dimensions are unknown. This paper investigates the dimensions (not causes) of software engineering success. An interdisciplinary sample of 191 design professionals (68 in the software industry) were interviewed concerning their perceptions of success. Non-software designers (e.g. architects) were included to increase the breadth of ideas and facilitate comparative analysis. Transcripts were subjected to supervised, semi-automated semantic content analysis, including a software developer vs. other professionals comparison. Findings suggest that participants view their work as time-constrained projects with explicit clients and other stakeholders. Success depends on stakeholder impacts – financial, social, physical and emotional – and is understood through feedback. Concern with meeting explicit requirements is peculiar to software engineering and design is not equated with aesthetics in many other fields. Software engineering success is a complex multifaceted variable, which cannot sufficiently be explained by traditional dimensions including user satisfaction, profitability or meeting requirements, budgets and schedules. A proto-theory of success is proposed, which models success as the net impact on a particular stakeholder at a particular time. Stakeholder impacts are driven by project efficiency, artifact quality and market performance. Success is not additive, e.g., ‘low’ success for clients does not average with ‘high’ success for developers to make ‘moderate’ success overall; rather, a project may be simultaneously successful and unsuccessful from different perspectives.
Paul Ralph, Paul Richard Kelly
ICSE1
2013 The illusion of requirements in software development
Paul Ralph
Requir. Eng.1