A Static Change Impact Analysis Approach based on Metrics and Visualizations to Support the Evolution of Workflow Repositories

Gustavo Ansaldi Oliva, Marco Aurélio Gerosa, Fabio Kon, Virginia Smith, Dejan S. MilojicicInt. J. Web Serv. Res 2016 · DOI

TL;DR

This paper introduces a novel approach for analyzing the impact of changes in large workflow repositories, crucial for maintaining complex IT operations. By using metrics like "change scattering" and "impact," combined with visualizations like call-graphs and treemaps, it helps developers proactively identify and manage potential ripple effects, leading to more reliable and maintainable workflows.

What Problem Does This Paper Address?

Organizations increasingly rely on complex workflow management systems (WFMSs) to automate IT operations. These systems often contain thousands of interdependent workflows, making it challenging for developers to understand the consequences of modifying a single workflow. This lack of visibility can lead to unintended side effects and ripple effects, compromising the reliability and evolvability of the entire workflow repository. Existing tools often lack comprehensive analysis capabilities or are too complex for everyday use, leaving developers "blindsided" by changes. The paper aims to address this by providing a practical, static change impact analysis approach that supports workflow designers in understanding and managing these interdependencies.

Methods

The core of the proposed approach lies in static interdependency-based change impact analysis. This means the analysis is performed on the workflow schema (structure) rather than runtime instances. The key techniques include:

  • Call-Graphs: These are used to represent calling relationships between workflows. Each node is a workflow, and a directed edge from workflow F₁ to F₂ signifies that F₁ calls F₂ (F₁ is a client of F₂). The analysis builds call-graphs for each workflow to understand its dependencies.
  • Metrics:
    • Change Scattering: Measures the number of workflows that are possibly impacted by a change in a specific workflow. This is determined by traversing the call-graph.
  • Impact: Measures the number of workflows that have a high chance of being impacted by a change in a specific workflow, based on a probability threshold p. This metric considers execution paths and the position of calls within those paths.
  • Visualizations:
    • Call-Graph Visualization: Provides an interactive view of the call-graph for a selected workflow, highlighting its direct and indirect dependencies.
  • Treemaps: Used to visualize the entire repository, coloring sections based on the "red," "yellow," and "green" status of the flows they contain. This helps in identifying sections that require more attention.
  • Color Schemes: Flows are colored red (high scattering and high impact), yellow (high scattering XOR high impact), or green (low scattering and low impact). Sections are colored based on the flows they contain (red if any red flow, yellow if any yellow flow and no red, green if only green flows).

Datasets

The paper describes an exploratory study that analyzed eight HP OO customer repositories. These repositories were provided by HP Software and represent real-world usage of the HP Operations Orchestration (HP OO) tool. The study also included the HP OO out-of-the-box (OOTB) workflow repository as a baseline for comparison. No new benchmarks were introduced; the focus was on analyzing existing customer data.

Key Insights and Ideas

  • Customer Repositories are Significantly More Complex: Customer repositories often contain a much higher number of workflows with high change impact compared to the OOTB repository. This highlights the need for specialized tools to manage this complexity.
  • "Red" and "Yellow" Flows are Concentrated: In some repositories, particularly C5, high-impact (red and yellow) workflows are not evenly distributed but are concentrated in specific sections, suggesting that these sections are critical and require focused attention.
  • Dispersion Matters: The dispersion of high-impact flows across sections can indicate how many teams might be involved in maintenance or testing, or how much effort is needed to refactor or update those sections.
  • Visualizations are Crucial for Understanding: The combination of call-graphs and treemaps provides both low-level (specific workflow dependencies) and high-level (repository-wide overview) insights, making complex relationships more understandable to developers.

Key Results

  • Customer repositories showed a significant increase in workflows with high change impact compared to the OOTB repository. For example, customer repositories had 11 (+10%) to 35 (+250%) more workflows with high change impact than the OOTB's 10.
  • Repository sizes varied considerably, from 1687 (OOTB) to 3769 (C8) workflows.
  • The analysis revealed that repositories C7 and C5 had the largest ratios of red and yellow flows, indicating a higher potential for change impact issues.
  • Repository C7 had the largest absolute number of red and yellow flows (35 red flows, 3.5x more than OOTB).
  • The study demonstrated that the approach could identify specific "core" workflows (like "[ServiceComposite]" in repository C5) that have a high chance of impacting a large number of other workflows.

Limitations and Future Work

  • Assumption Regarding Workflow Schema: The current domain model might not fully represent all workflow constructs (e.g., Inclusive Gateways, Complex Gateways, Events). However, the approach is independent of activity implementation details.
  • Analysis Scope: The approach does not account for data dependencies within or between workflows, nor does it consider resource contention.
  • Triangulation of Results: The study lacked qualitative feedback from customers regarding the results.
  • Future Work:
    • Conducting qualitative studies with customers to gather feedback.
  • Discovering "safe-to-change" zones within workflows.
  • Incorporating data dependencies and runtime execution data (e.g., from execution logs) to refine impact calculations.
  • Exploring mechanisms for transparently applying workflow schema changes during runtime.

Summary updated

BibTeX
@article{oliva2016static,
  author = {Gustavo Ansaldi Oliva and Marco Aurélio Gerosa and Fabio Kon and Virginia Smith and Dejan S. Milojicic},
  title = {{A Static Change Impact Analysis Approach based on Metrics and Visualizations to Support the Evolution of Workflow Repositories}},
  journal = {Int. J. Web Serv. Res.},
  year = {2016},
  doi = {10.4018/ijwsr.2016040105},
}

Loading related content...

Figure