Workflow runs record the executions of continuous integration and delivery specifications. They serve as the primary monitoring mechanism, enabling failure detection and integration reliability assessment. However, understanding run intricacies is far from trivial. Each run involves multiple interdependent execution units (e.g., attempts, jobs, and steps) whose relationships are not immediately visible. Moreover, current tools provide limited support for detecting patterns across runs, particularly in projects with high workflow execution frequency. As a result, run monitoring remains constrained to fragmented views that are cumbersome to use.

GitHub Actions (GA) has become the de facto tool that developers use to automate software workflows, seamlessly building, testing, and deploying code. Yet when GA fails, it disrupts development, causing delays and driving up costs. Diagnosing failures becomes especially challenging because error logs are often long, complex and unstructured. Given these difficulties, this study explores the potential of large language models (LLMs) to generate correct, clear, concise, and actionable contextual descriptions (or summaries) for GA failures, focusing on developers' perceptions of their feasibility and usefulness. Our results show that over 80 % of developers rated LLM explanations positively in terms of correctness for simpler/small logs. Overall, our findings suggest that LLMs can feasibly assist developers in understanding common GA errors, thus, potentially reducing manual analysis. However, we also found that improved reasoning abilities are needed to support more complex CI/CD scenarios. For instance, less experienced developers tend to be more positive on the described context, while seasoned developers prefer concise summaries. Overall, our work offers key insights for researchers enhancing LLM reasoning, particularly in adapting explanations to user expertise.

GitHub Actions (GA) is an orchestration platform that streamlines the automatic execution of software engineering tasks such as building, testing, and deployment. Although GA workflows are the primary means for automation, according to our experience and observations, human intervention is necessary to correct defects, update dependencies, or refactor existing workflow files. In fact, previous research has shown that software artifacts similar to workflows, such as build files and bots, can introduce additional maintenance tasks in software projects. This suggests that workflow files, which are also used to automate repetitive tasks in professional software production, may generate extra workload for developers. However, the nature of such effort has not been well studied. This paper presents a large-scale empirical investigation towards characterizing the maintenance of GA workflows by studying the evolution of workflow files in almost 200 mature GitHub projects across ten programming languages. Our findings largely confirm the results of previous studies on the maintenance of similar artifacts, while also revealing GA-specific insights such as bug fixing and CI/CD improvement being among the major drivers of GA maintenance. A direct implication is that practitioners should be aware of proper resource planning and allocation for maintaining GA workflows, thus exposing the “hidden costs of automation.” Our findings also call for identifying and documenting best practices for such maintenance, and for enhanced tool features supporting dependency tracking and better error reporting of workflow specifications.

GitHub Actions (GA) enjoy increasing popularity in many software development projects as a means to automate repetitive software engineering tasks by enabling programmable event-driven workflows. Researchers typically analyze GA at the raw data level using batch tools to mine and analyze actions, jobs, and steps within GA workflows. Although this approach is widely applicable, it ignores the specific context of the GA workflow domain. Consequently, researchers do not reason directly about the domain abstractions.We present our preliminary steps in building EGAD (Explorable GitHub Action Domain Model), a moldable domain-specific tool to depict and analyze detailed GA workflow data. EGAD consists of an explorable domain model of GA workflows augmented with custom, domain-specific views, and live narratives. We illustrate EGAD in action using it to explore "sticky commits" in GitHub repositories.

GitHub Actions are an event-driven tool to automatically respond to particular GitHub events. Typical events are receiving new pull requests or publishing a software release. Despite the massive and quick adoption of GitHub Actions, little is known about the incremental construction of GitHub Actions workflow by practitioners. This paper presents the result of a manual inspection of 222 commits of GitHub Actions workflows obtained from 10 popular open-source repositories. Our hierarchical taxonomy, obtained by systematically categorizing and tagging workflow modifications, reveals 11 types of modifications and presents opportunities for improvement in the way workflows are built and edited. In particular, our results highlight the need for adequate tooling to support refactoring, debugging and code editing of GitHub Actions workflows.