GHA-SmellFix: Taking Action with Agentic LLMs for Detecting and Refactoring GitHub Actions Workflow Smells

Anderson Uchôa, Jhefferson Souza, Emanuel Ávila, Gabriel Silva, Kevin Lira, Hui Sun, Nelson Araújo, Arthur Cordeiro, Rohit Gheyi, Wesley AssunçãoarXiv 2026 · arXiv

TL;DR

GHA-SmellFix is a new agentic LLM pipeline designed to automatically detect and refactor 19 types of "configuration smells" in GitHub Actions (GHA) workflows. The research reveals that while splitting detection and refactoring into separate agents doesn't always improve patch quality, it acts as a critical "gatekeeper" that helps models abstain from making unnecessary or dangerous changes to CI/CD pipelines.

What Problem Does This Paper Address?

Modern software relies heavily on GitHub Actions for CI/CD, but these workflows are often riddled with "configuration smells"—anti-patterns like unpinned action versions, missing timeouts, or unsafe artifact uploads that compromise security and reliability.

Existing solutions are mostly rule-based, making them brittle and difficult to maintain as GHA features evolve. Furthermore, most tools only report smells; they don't fix them. While Large Language Models (LLMs) are great at code transformation, using them for CI/CD is risky. A "hallucinated" edit in a workflow file could accidentally grant excessive permissions or break a deployment. The authors investigate whether an agentic approach—decomposing the task into specialized roles—can make LLM-driven refactoring safer and more effective.

Methods

The authors developed GHA-SmellFix, a pipeline implemented using the Agno framework. They compared two primary orchestration designs:

  1. Joint Design (D1): A single LLM agent receives the workflow and smell definition, then decides if a smell exists and provides a patch in one go.
  2. Decomposed Design (D2): A "Detection Agent" first classifies the smell. Only if it confirms the smell is present is a second "Refactoring Agent" invoked to generate a patch.
Fig. 2. GHA-SmellFix orchestration designs for target-conditioned smell detection and refactoring.

The pipeline was tested across four models: GPT-5.4-nano, Gemini 3.5 Flash, Claude Haiku 4.5, and DeepSeek V4 Flash. To evaluate the results, they used a "construct-aware" protocol that doesn't just check if the code changed, but specifically whether the LLM modified the correct YAML constructs (e.g., jobs.<id>.timeout-minutes) without touching unrelated, behavior-critical parts of the workflow.

Datasets

The study introduces a substantial new benchmark for GHA maintenance:

  • Ground Truth Dataset: 1,710 manually curated (workflow, smell) pairs, covering 19 distinct smell types.
  • Smell Catalog: The authors operationalized 19 smells from existing literature, ranging from "Localized" smells (visible in a single line) to "Context-Dependent" smells (requiring reasoning across the whole file).
  • Evaluation Set: A balanced subset of 570 pairs (15 positive and 15 negative cases per smell) used for cross-model comparison.
Table 1. GitHub Actions configuration smells, refined from Khatami et al. [24], and ground-truth coverage. P/N denotes positive/negative pairs; full definitions are in the replication package.

Key Insights and Ideas

  • Detection is the Bottleneck: The study found that once an LLM correctly identifies a smell, it is remarkably good at fixing it (92.6% clean removal rate). The primary failure mode isn't bad refactoring, but rather failing to see the smell or "hallucinating" one where none exists.
  • Decomposition as a Gatekeeper: Splitting the agents (D2) didn't necessarily make the patches "smarter," but it significantly improved abstention. By forcing a dedicated detection step, the system was less likely to make "unnecessary edits" on clean files.
  • The "Clean-up" Impulse: LLMs have a tendency to over-refactor. Even when fixing a specific smell, they often deleted helpful comments or changed unrelated versions. In a CI/CD context, these "cosmetic" changes are actually high-risk.
  • Efficiency Gains: Counter-intuitively, the decomposed design (D2) often reduced latency and API costs. By using a detection "gate," the system avoids calling the more expensive refactoring logic for the many workflows that are already smell-free.

Key Results

  • Improved Accuracy: The decomposed design (D2) raised the average F1-score from 86.34% to 87.55% and recall from 79.12% to 82.37%.
  • Refactoring Success: For correctly detected smells, the models achieved a 74% clean removal rate overall, which jumped to 92.6% when conditioned on correct detection.
  • Abstention Challenges: Despite the benefits of decomposition, the pipelines still produced 93 unnecessary edits on negative cases. Over half of these (54.84%) affected behavior-relevant constructs, highlighting the need for human-in-the-loop review.
Overview of the refactoring assessment protocol.

Limitations and Future Work

The authors acknowledge that their evaluation is construct-aware but not runtime-semantic. They verified that the YAML was structurally correct and the smell was removed, but they did not actually execute the workflows to ensure no subtle runtime regressions occurred.

Future work includes integrating external validators (like linters or sandboxed CI runs) into the agentic loop and exploring "multi-smell" refactoring, where an agent must coordinate fixes for several overlapping issues in a single complex workflow. For now, they recommend that GHA-SmellFix be used as a review-support tool rather than a fully autonomous "auto-fixer."

Summary updated

Abstract

Continuous integration and continuous delivery workflows are key artifacts in software projects. In GitHub Actions, they encode build, test, release, deployment, and security automation in YAML. These workflows contain configuration smells affecting security, reliability, maintainability, or operational stability. Existing approaches are mostly rule-based, with rules hard to create and evolve, and typically report findings without fixes. Agentic LLM pipelines can combine contextual detection, transformation selection, and patch generation. Specialized agents can gate refactoring on positive detection verdicts, potentially avoiding unnecessary changes, but it is unclear whether decomposition improves effectiveness or mainly changes overhead. We present GHA-SmellFix, an agentic LLM pipeline for detecting and refactoring GitHub Actions workflow smells. We compare two designs: Joint, where one agent detects and refactors, and Decomposed, where separate agents detect and then refactor. We operationalize 19 smells from an existing catalog with explicit detection criteria, refactoring expectations, exclusion rules, and safety checks, and curate a ground truth of 1,710 (workflow, smell) pairs. We evaluate the designs with four LLMs (GPT-5.4-nano, Gemini 3.5 Flash, Claude Haiku 4.5, and DeepSeek V4 Flash) on 570 sampled pairs. Results show that decomposition does not universally improve patch quality; its main benefit is controlling when patch generation is attempted. It helps models prone to missing smells and can reduce overhead by gating patches. Once a smell is correctly detected, clean refactoring is generally effective, making detection the main end-to-end bottleneck. However, false positives can trigger unnecessary edits to execution-relevant constructs. Practical CI/CD agents thus need reliable abstention and construct-aware validation alongside patch generation.
BibTeX
@misc{uchoa2026gha,
  author = {Anderson Uchôa and Jhefferson Souza and Emanuel Ávila and Gabriel Silva and Kevin Lira and Hui Sun and Nelson Araújo and Arthur Cordeiro and Rohit Gheyi and Wesley Assunção},
  title = {{GHA-SmellFix: Taking Action with Agentic LLMs for Detecting and Refactoring GitHub Actions Workflow Smells}},
  year = {2026},
  eprint = {2611.14963},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.14963},
}
Figure