Logs are essential for understanding Continuous Integration (CI) behavior, particularly for diagnosing build failures and performance regressions. Yet their growing volume and verbosity make both manual inspection and automated analysis increasingly costly, time-consuming, and environmentally costly. While prior work has explored log compression, anomaly detection, and LLM-based log analysis, most efforts target structured system logs rather than the unstructured, noisy, and verbose logs typical of CI workflows.

AI agents are increasingly used in software development, yet their interaction with CI/CD configurations is not well studied. We analyze 8,031 agentic pull requests (PRs) from 1,605 GitHub repositories where AI agents touch YAML configurations, encompassing 711,923 file changes across five agents. CI/CD configuration files account for only 3.25% of agent changes, with significant variation across agents (Devin: 4.83%, Codex: 2.01%, p < 0.001). When agents change CI/CD, 96.77% target GitHub Actions. Agentic PRs with CI/CD changes merge slightly less often than others (67.77% vs. 71.80%), except for Copilot, whose CI/CD changes are merged 15.63 percentage points more often. Across 99,930 workflow runs, we observe comparable build success rates for CI/CD and non-CI/CD changes (75.59% vs. 74.87%), though three agents have significantly higher success when changing CI/CD. These results show that AI agents rarely change CI/CD and focus mainly on GitHub Actions, yet their configuration changes are as reliable as regular code. Copilot’s strong CI/CD performance despite lower overall acceptance suggests emerging configuration specialization, with implications for agent training and DevOps automation.

AI coding agents are reshaping software development through both autonomous and human-mediated pull requests (PRs). When developers use AI agents to generate code under their own accounts, code authorship attribution becomes critical for repository governance, research validity, and understanding modern development practices. We present the first study on fingerprinting AI coding agents, analyzing 33,580 PRs from five major agents (OpenAI Codex, GitHub Copilot, Devin, Cursor, Claude Code) to identify behavioral signatures. With 41 features spanning commit messages, PR structure, and code characteristics, we achieve 97.2% F1-score in multi-class agent identification. We uncover distinct fingerprints: Codex shows unique multiline commit patterns (67.5% feature importance), and Claude Code exhibits distinctive code structure (27.2% importance of conditional statements). These agent-specific signatures reveal that AI coding tools produce detectable behavioral patterns, suggesting potential for identifying AI contributions in software development repositories.

AI coding agents can autonomously generate pull requests (PRs), yet little is known about how their contributions compare to those of humans. We analyze 33,596 agent-generated PRs (APRs) and 6,618 human PRs (HPRs) to compare code-change characteristics and message quality. We observe that APR-introduced symbols (functions and classes) are removed much sooner than those in HPRs (median time to removal 3 vs. 34 days) and are also removed more often (symbol churn 7.33% vs. 4.10%), reflecting a focus on other tasks like documentation and test updates. Agents generate stronger commit-level messages (semantic similarity 0.72 vs. 0.68) but lag humans at PR-level summarization (PR–commit similarity 0.86 vs. 0.88). Commit message length is the best predictor of description quality, indicating reliance on individual commits over full-PR reasoning. These findings highlight a gap between agents’ micro-level precision and macro-level communication, suggesting opportunities to improve agent-driven development workflows.

Mobile apps face strong pressure for fast and reliable updates. Continuous Integration (CI) helps automate builds, tests, and releases, but its impact on mobile development remains underexplored. Despite the widespread use of CI, little is known about how it affects development activity, release speed, and user-facing outcomes in mobile projects. Existing studies mostly focus on CI adoption in general-purpose software, providing limited insight into mobile-specific dynamics, such as app store visibility and user engagement. In this paper, we analyze open-source Android apps to (1) compare CI adopters and non-adopters, (2) characterize adoption patterns using activity and bug metrics, and (3) assess pre/post adoption changes and user-facing outcomes. We observe that CI adopters are larger and more active, with faster and more regular releases. CI adoption is concentrated in integration- and reliability-intensive categories (e.g., finance and productivity) and is associated with higher Google Play Store engagement (more downloads and reviews) without lower ratings. Overall, CI adoption aligns with practices that support sustained delivery, higher project visibility, and stronger user engagement in mobile ecosystems.

Continuous Integration and Continuous Delivery (CI/CD) is a well-established practice that automatically builds, tests, packages, and deploys software systems. To adopt CI/CD, software developers need to configure their projects using dedicated YML configuration files. Mobile apps have distinct characteristics with respect to CI/CD practices, such as testing on various emulators and deploying to app stores. However, little is known about the challenges and added value of adopting CI/CD in mobile apps and how developers maintain such a practice. In this article, we conduct an empirical study on CI/CD practices in \(2{,}557\) Android apps adopting 4 popular CI/CD services, namely GitHub Actions, Travis CI, CircleCI, and GitLab CI/CD. We also compare our findings with those reported in prior research on general CI/CD practices to situate them within broader trends. We observe a lack of commonality and standardization across CI/CD services and Android apps, leading to complex YML configurations and associated maintenance efforts. We also observe that CI/CD configurations focus primarily on the build setup, with around half of the projects performing standard testing and only 9% incorporating deployment. In addition, we find that CI/CD configurations are changed bi-monthly on average, with frequent maintenance correlating with active issue tracking, project size/age, and community engagement. Our qualitative analysis of commits uncovered 11 themes in CI/CD maintenance activities, with over a third of the changes focusing on improving workflows and fixing build issues, whereas another third involves updating the build environment, tools, and dependencies. Our study emphasizes the necessity for automation and AI-powered tools to improve CI/CD processes for mobile apps and advocates creating adaptable open source tools to efficiently manage resources, especially in testing and deployment.

Continuous Integration (CI) services, such as GitHub Actions and Travis CI, are widely adopted in open-source development to automate testing and deployment. Though much existing research often examines individual CI services in isolation, it remains unclear how such multiple services are co-adopted and maintained in practice. To understand how CI adoption is evolving across services, we conducted an empirical study analyzing historical CI adoption of 1 8, 9 2 4 Java projects hosted on GitHub from January 2008 to December 2024, adopting at least one of eight CI services, namely Travis CI, AppVeyor, CircleCI, Azure Pipelines, GitHub Actions, Bitbucket, GitLab CI, and Cirrus CI. Specifically, we investigate: (1) how frequently CI services are co-adopted or replaced, and (2) how maintenance activity varies across different services. Our analysis shows that the use of multiple CI services within the same project is a recurring pattern observed in nearly one in five projects, often reflecting migration across CI services. Our study is among the first to examine multi-CI adoption and maintenance in practice, offering new insights for future research and highlighting the need for strategies and tools to support service selection, coordination, and migration in evolving CI environments.

Continuous Integration (CI) services, such as GitHub Actions, require developers to write YAML-based configurations, which can be tedious and error-prone. Despite the increasing use of Large Language Models (LLMs) to automate software engineering tasks, their ability to generate CI configurations remains underexplored. This paper presents a preliminary study evaluating six LLMs for generating GitHub Actions configurations from natural language descriptions. We assess three general-purpose foundation models (GPT-4o, Llama, and Gemma) and three code-pretrained models (GPT-4.1, Code Llama, and CodeGemma). We also introduce the first labeled dataset of its kind, constructed from GitHub Actions documentation, pairing descriptions with corresponding bestpractice YAML configurations. Zero-shot prompting achieves up to 69 % similarity with the ground truth, with only 3 % perfect matches. Code-pretrained models slightly underperform compared to general-purpose ones in YAML-based CI tasks, revealing LLM limitations for CI configuration generation. Analyzing GPT-4o outputs reveals issues like missing or renamed steps, misinterpreted descriptions, and unnecessary additions that may affect structural and contextual correctness, indicating a gap between generation quality and the precision required for executable CI configurations. Our research offers insights for improving LLM alignment with configuration languages and guiding future efforts on CI automation and tooling support.

Test suites tend to grow when software evolves, making it often infeasible to execute all test cases with the allocated testing budgets, especially for large software systems. Test suite minimization (TSM) is employed to improve the efficiency of software testing by removing redundant test cases, thus reducing testing time and resources while maintaining the fault detection capability of the test suite. Most existing TSM approaches rely on code coverage (white-box) or model-based features, which are not always available to test engineers. Recent TSM approaches that rely only on test code (black-box) have been proposed, such as ATM and FAST-R. The former yields higher fault detection rates (FDR) while the latter is faster. To address scalability while retaining a highFDR, we propose LTM (Language model-basedTest suiteMinimization), a novel, scalable, and black-box similarity-based TSM approach based on large language models (LLMs), which is the first application of LLMs in the context of TSM. To support similarity measurement using test method embeddings, we investigate five different pre-trained language models: CodeBERT, GraphCodeBERT, UniXcoder, StarEncoder, and CodeLlama, on which we compute two similarity measures: Cosine Similarity and Euclidean Distance. Our goal is to find similarity measures that are not only computationally more efficient but can also better guide a Genetic Algorithm (GA), which is used to search for optimal minimized test suites, thus reducing the overall search time. Experimental results show that the best configuration of LTM (UniXcoder/Cosine) outperforms ATM in three aspects: (a) achieving a slightly greater saving rate of testing time ($41.72\%$versus$41.02\%$, on average); (b) attaining a significantly higher fault detection rate ($0.84$versus$0.81$, on average); and, most importantly, (c) minimizing test suites nearly five times faster on average, with higher gains for larger test suites and systems, thus achieving much higher scalability.

Executing large test suites is time and resource consuming, sometimes impossible, and such test suites typically contain many redundant test cases. Hence, test case (suite) minimization is used to remove redundant test cases that are unlikely to detect new faults. However, most test case minimization techniques rely on code coverage (white-box), model-based features, or requirements specifications, which are not always (entirely) accessible by test engineers. Code coverage analysis also leads to scalability issues, especially when applied to large industrial systems. Recently, a set of novel techniques was proposed, called FAST-R, relying solely on test case code for test case minimization, which appeared to be much more efficient than white-box techniques. However, it achieved a comparable low fault detection capability for Java projects, thus making its application challenging in practice. In this paper, we propose ATM (AST-based Test case Minimizer), a similarity-based, search-based test case minimization technique, taking a specific budget as input, that also relies exclusively on the source code of test cases but attempts to achieve higher fault detection through finer-grained similarity analysis and a dedicated search algorithm. ATM transforms test case code into Abstract Syntax Trees (AST) and relies on four tree-based similarity measures to apply evolutionary search, specifically genetic algorithms, to minimize test cases. We evaluated the effectiveness and efficiency of ATM on a large dataset of 16 Java projects with 661 faulty versions using three budgets ranging from 25% to 75% of test suites. ATM achieved significantly higher fault detection rates (0.82 on average), compared to FAST-R (0.61 on average) and random minimization (0.52 on average), when running only 50% of the test cases, within practically acceptable time (1.1 - 4.3 hours, on average, per project version), given that minimization is only occasionally applied when many new test cases are created (major releases). Results achieved for other budgets were consistent.

Fast loading web apps can be a key success factor in terms of user experience. However, improving the performance of a web app is not trivial, since it requires a deep understanding of both the browser engine and the specific usage scenarios of the web app under consideration. In this paper, we present an industrial case study at 30 MHz, an agricultural technology company, in which we target a large web-based dashboard, where its performance was improved via 13 distinct interventions over a four-month period. Moreover, we conduct a user study to analyse whether web performance metrics correlate with the user perceived page load time in optimization scenarios. First, we design a replicable performance engineering plan, where the technical realization of each intervention is reported in detail along with its development effort. Second, we develop a benchmarking tool that supports 11 widely used web performance metrics. Finally, we use the benchmarking tool to quantitatively evaluate the performance of the target web app and measure the effect of 13 interventions on both desktop and mobile devices. For the user study, we record six videos of different page loads and ask participants about their opinion about the time a web page is considered ready. We calculate the correlation of the user perceived data with each web performance metric. We observe a considerable performance improvement over the course of the 13 interventions. Among others, we achieve 98.37% and 97.56% time reductions on desktop and mobile, respectively, for the First Contentful Paint metric. In addition, we achieve 48.25% and 19.85% improvements for the Speed Index (SI) metric on desktop and mobile, respectively. Our user study shows that the Lowest Time to Widget metric, a product-specific web performance metric, is perfectly correlated with perceived performance during the optimization process. This study shows the importance of a continuous focus on performance engineering in the context of large-scale web apps to improve user browsing experience. We recommend developers to carefully plan their performance engineering activities, since different interventions might require different efforts and can have different effects on the overall performance of the web application.

Software testing assures that code changes do not adversely affect existing functionality. However, a test case can be flaky, i.e., passing and failing across executions, even for the same version of the source code. Flaky test cases introduce overhead to software development as they can lead to unnecessary attempts to debug production or testing code. Besides rerunning test cases multiple times, which is time-consuming and computationally expensive, flaky test cases can be predicted using machine learning (ML) models, thus reducing the wasted cost of re-running and debugging these test cases. However, the state-of-the-art ML-based flaky test case predictors rely on pre-defined sets of features that are either project-specific, i.e., inapplicable to other projects, or require access to production code, which is not always available to software test engineers. Moreover, given the non-deterministic behavior of flaky test cases, it can be challenging to determine a complete set of features that could potentially be associated with test flakiness. Therefore, in this article, we propose Flakify, a black-box, language model-based predictor for flaky test cases. Flakify relies exclusively on the source code of test cases, thus not requiring to (a) access to production code (black-box), (b) rerun test cases, (c) pre-define features. To this end, we employed CodeBERT, a pre-trained language model, and fine-tuned it to predict flaky test cases using the source code of test cases. We evaluated Flakify on two publicly available datasets (FlakeFlagger and IDoFT) for flaky test cases and compared our technique with the FlakeFlagger approach, the best state-of-the-art ML-based, white-box predictor for flaky test cases, using two different evaluation procedures: (1) cross-validation and (2) per-project validation, i.e., prediction on new projects. Flakify achieved F1-scores of 79% and 73% on the FlakeFlagger dataset using cross-validation and per-project validation, respectively. Similarly, Flakify achieved F1-scores of 98% and 89% on the IDoFT dataset using the two validation procedures, respectively. Further, Flakify surpassed FlakeFlagger by 10 and 18 percentage points (pp) in terms of precision and recall, respectively, when evaluated on the FlakeFlagger dataset, thus reducing the cost bound to be wasted on unnecessarily debugging test cases and production code by the same percentages (corresponding to reduction rates of 25% and 64%). Flakify also achieved significantly higher prediction results when used to predict test cases on new projects, suggesting better generalizability over FlakeFlagger. Our results further show that a black-box version of FlakeFlagger is not a viable option for predicting flaky test cases.

The Continuous Integration (CI) practice allows developers to build software projects automatically and more frequently. However, CI builds may undergo long build durations or frequent build breakages, which we refer to asbuild performance. Both long durations and frequent breakages of CI builds can impede developers from engaging in other development activities. Prior research has conducted independent studies on build durations or build breakages. However, there is little attention to the possible interplay between reducing build durations and build breakages. In particular, it is unclear from prior studies (i) whether and how build performance is influenced by the context of projects; (ii) whether the actions to reduce build durations would reduce or increase build breakages; and (iii) whether fixing build breakages would lead to longer or faster builds. It is important for developers to understand the practices that make both timely and passing CI builds. In this paper, we conduct experimental and survey studies on the practices that can have dual or inverse associations with twobuild performancemeasures:build durationsandbuild breakages. To this end, we extend an existing dataset calledTravisTorrentto exclude inactive projects and collect recent builds of active projects. As a result, we study 924,616 CI builds from 588GitHubprojects that are linked withTravis CI. In addition, we survey developers who contributed to the projects in our dataset to get their feedback on our experimental observations. First, we investigate project-level metrics and find that project characteristics have a significant association with build durations and breakages. In addition, we investigate how build-level metrics are associated with both build durations and breakages and observe an evident interplay between them. In particular, we observe that actions to fix build breakages (e.g., retrying or waiting for build commands) not only increase build durations but also do not guarantee passing builds. We also find that improving the build performance of a project is dependent on the current build durations and breakages of that project. Furthermore, we analyze how build performance changes over time and observe nearly a third of projects in which one performance measure is sacrificed in favor of the other, especially when not possible to achieve both together. The majority of our experimental observations are confirmed by survey results, which provide useful insights though some survey responses disagree with some of our experimental observations. Our work (a) provides developers with development and building practices to maintain timely and passing CI builds, and (b) encourages researchers to highlight any potential dual or inverse side effects when reporting actionable findings about CI builds.

MRI brain images are always of low contrast, which makes it difficult to identify to which area the information at the boundary of brain images belongs. This can make the extraction of features at the boundary more challenging, since those features can be misleading as they might mix properties of different brain regions. Hence, to alleviate such a problem, image boundary detection plays a vital role in medical image segmentation, and brain segmentation in particular, as unclear boundaries can worsen brain segmentation results. Yet, given the low quality of brain images, boundary detection in the context of brain image segmentation remains challenging. Despite the research invested to improve boundary detection and brain segmentation, these two problems were addressed independently, i.e., little attention was paid to applying boundary detection to brain segmentation tasks. Therefore, in this paper, we propose a boundary detection-based model for brain image segmentation. To this end, we first design a boundary segmentation network for detecting and segmenting images brain tissues. Then, we design a boundary information module (BIM) to distinguish boundaries from the three different brain tissues. After that, we add a boundary attention gate (BAG) to the encoder output layers of our transformer to capture more informative local details. We evaluate our proposed model on two datasets of brain tissue images, including infant and adult brains. The extensive evaluation experiments of our model show better performance (a Dice Coefficient (DC) accuracy of up to [Formula: see text] compared to the state-of-the-art models) in detecting and segmenting brain tissue images.

Abstract Context Tangled commits are changes to software that address multiple concerns at once. For researchers interested in bugs, tangled commits mean that they actually study not only bugs, but also other concerns irrelevant for the study of bugs. Objective We want to improve our understanding of the prevalence of tangling and the types of changes that are tangled within bug fixing commits. Methods We use a crowd sourcing approach for manual labeling to validate which changes contribute to bug fixes for each line in bug fixing commits. Each line is labeled by four participants. If at least three participants agree on the same label, we have consensus. Results We estimate that between 17% and 32% of all changes in bug fixing commits modify the source code to fix the underlying problem. However, when we only consider changes to the production code files this ratio increases to 66% to 87%. We find that about 11% of lines are hard to label leading to active disagreements between participants. Due to confirmed tangling and the uncertainty in our data, we estimate that 3% to 47% of data is noisy without manual untangling, depending on the use case. Conclusion Tangled commits have a high prevalence in bug fixes and can lead to a large amount of noise in the data. Prior research indicates that this noise may alter results. As researchers, we should be skeptics and assume that unvalidated data is likely very noisy, until proven otherwise.

Context. Empirical research consistently demonstrates that that scholarly peer review is ineffective, unreliable, and prejudiced. In principle, the solution is to move from contemporary, unstructured, essay-like reviewing to more structured, checklist-like reviewing. The Task Force created models—called “empirical standards”—of the software engineering community’s expectations for different popular methodologies. Objective. This paper presents a tool for facilitating more structured reviewing by generating review checklists from the empirical standards. Design. A tool that generates pre-submission and review forms using the empirical standards for software engineering research was designed and implemented. The pre-submission and review forms can be used by authors and reviewers, respectively, to determine whether a manuscript meets the software engineering community’s expectations for the particular kind of research conducted. Evaluation. The proposed tool can be empirically evaluated using lab or field randomized experiments as well as qualitative research. Huge, impractical studies involving splitting a conference program committee are not necessary to establish the effectiveness of the standards, checklists and structured review. Conclusions. The checklist generator enables more structured peer reviews, which in turn should improve review quality, reliability, thoroughness, and readability. Empirical research is needed to assess the effectiveness of the tool and the standards.

Design patterns are generic solutions to common programming problems. Design patterns represent a typical example of design reuse. However, implementing design patterns can lead to several problems, such as programming overhead and traceability. Existing research introduced several approaches to alleviate the implementation issues of design patterns. Nevertheless, existing approaches pose different implementation restrictions and require programmers to be aware of how design patterns should be implemented. Such approaches make the source code more prone to faults and defects. In addition, existing design pattern implementation approaches limit programmers to apply specific scenarios of design patterns (e.g. class-level), while other approaches require scattering implementation code snippets throughout the program. Such restrictions negatively impact understanding, tracing, or reusing design patterns. In this paper, we propose a novel approach to support the implementation of software design patterns as an extensible Java compiler. Our approach allows developers to use concise, easy-to-use language constructs to apply design patterns in their code. In addition, our approach allows the application of design patterns in different scenarios. We illustrate our approach using three commonly used design patterns, namely Singleton, Observer and Decorator. We show, through illustrative examples, how our design pattern constructs can significantly simplify implementing design patterns in a flexible, reusable and traceable manner. Moreover, our design pattern constructs allow class-level and instance-level implementations of design patterns.

Much research has investigated the common reasons for build breakages. However, prior research has paid little attention to builds that may break due to reasons that are unlikely to be related to development activities. For example, Continuous Integration (CI) builds may break due to timeout or connection errors while generating the build. Such kinds of build breakages potentially introduce noises to build breakage data. Not considering such noises may lead to misleading results when studying CI builds. In this paper, we propose three criteria to identify build breakages that can potentially introduce noises to build breakage data. We apply these criteria to a dataset of 350,246 builds from 153 GitHub projects that are linked with Travis CI. Our results reveal that 33 percent of the build breakages are due to environmental factors (e.g., errors in CI servers), 29 percent are due to (unfixed) errors in previous builds, and 9 percent are due to build jobs that were later deemed by developers as noisy (there is an overlap of 17 percent between these three types of breakages). We measure the impact of noises in build breakage data on modeling build breakages. We observe that models that use uncleaned build breakage data can lead to misleading associations between build breakages and development activities (e.g., the role of developer). However, such associations could not be observed after eliminating noisy build breakages. Moreover, we replicate a prior study that investigates the association between build breakages and development activities using data from 14 GitHub projects. We observe that some observations reported by the prior study (e.g., pull requests cause more breakages) do not hold after eliminating the noises from build breakage data.

Software} maintainers employ reverse-engineered sequence diagrams to visually understand software behavior, especially when software documentation is absent or outdated. Much research has studied the adoption of reverse-engineered sequence diagrams to visualize program interactions. However, due to the forward-engineering nature of sequence diagrams, visualizing more complex programming scenarios can be challenging. In particular, sequence diagrams represent method invocations as unidirectional arrows. However, in practice, source code may contain compound method invocations that share values/objects implicitly. For example, method invocations can be nested, e.g., fun (foo ()), or chained, e.g., fun (). foo (). The standard notation of sequence diagrams does not have enough expressive power to precisely represent compound scenarios of method invocations. Understanding the flow of information between method invocations simplifies debugging, inspection, and exception handling operations for software maintainers. Despite the research invested to address the limitations of UML sequence diagrams, previous approaches fail to visualize compound scenarios of method invocations. In this paper, we propose sequence diagram extensions to enhance the visualization of (i) three widely used types of compound method invocations in practice (i.e., nested, chained, and recursive) and (ii) lifelines of objects returned from method invocations. We aim through our extensions to increase the level of abstraction and expressiveness of method invocation code. We develop a tool to reverse engineer compound method invocations and generate the corresponding extended sequence diagrams. We evaluate how our proposed extensions can improve the understandability of program interactions using a controlled experiment. We find that program interactions are significantly more comprehensible when visualized using our extensions.

Abstract Reverse engineering of sequence diagrams refers to the process of extracting meaningful information about the behavior of software systems in the form of appropriately generated sequence diagrams. This process has become a practical method for retrieving the behavior of software systems, primarily those with inadequate documentation. Various approaches have been proposed in the literature to produce from a given system a series of interactions that can be used for different purposes. The reason for such diversity of approaches is the need to offer sequence diagrams that can cater for the users' specific goals and needs, which can vary widely depending on the users' perception and understandability of visual representations and the target application domains. In this paper, we systematically review existing techniques in this context while focusing on their distinct purposes and potentials of providing more understandable sequence diagrams. In addition, a qualitative evaluation of such techniques is conducted to expose their adequacy and applicability for effective program comprehension. Finally, we list a set of possible extensions to theunified modeling languagesequence diagram standard that we anticipate will enhance its versatility and understandability of program control flow, followed by a number of concluding remarks.