Testing Non-Local Interactions of Software Changes

16 concepts6 papers1 proposal

When developers review code or run tests on a new commit, their attention naturally gravitates toward the "diff"—the specific lines of code that were added, modified, or deleted. The implicit assumption is that if the new code is locally correct and the existing tests pass, the system remains stable. However, software systems are tightly interwoven webs of data and control flow. A change in one component can ripple across the codebase, triggering unexpected behaviors in distant, unmodified modules.

This phenomenon gives rise to Change Interaction Errors (CIEs): bugs that manifest not because a single modification is inherently flawed, but because multiple changes—or a new change and an existing system property—interact in unforeseen ways.

The Anatomy of Inter-Change Failures

The insight that errors stem from interconnected modifications has deep roots. In an early study simulating cognitive processes in programming, Spohrer et al. observed that software defects are frequently the side effects of a repair designed to solve a completely different problem. A sequence of local fixes can inadvertently culminate in a global error, as a patch in one area disrupts assumptions relied upon by another.

To understand why this happens at a mechanical level, consider how data flows through a program. When a patch modifies how a variable is updated, it alters the state space of the application. In Mitigating Security Risks in Linux with KLAUS: A Method for Evaluating Patch Correctness, Wu et al. formalize this through Altered Write-Read Pairs (AWRPs). An AWRP occurs when a new or modified write operation interacts unexpectedly with an existing, unmodified read operation on the same variable. Because standard analysis techniques focus primarily on the correctness of the write itself, they miss how the altered data corrupts downstream execution flows, frequently introducing severe security vulnerabilities into production environments.

[ Traditional Testing Focus ]
New/Modified Code ---> [ New Write ] 

[ Actual Bug Location (AWRP) ]
New/Modified Code ---> [ New Write ] --(Altered Data Flow)--> [ Unmodified Read ] ---> Failure

Shifting Focus to Unmodified Code

If the root cause of a regression lies in the interaction between a new change and the existing codebase, then testing only the modified lines is fundamentally insufficient. This realization has reoriented modern mutation testing strategies.

In On the use of commit-relevant mutants, Ojdanic et al. challenge the traditional practice of seeding mutants exclusively within a commit's modified lines. The authors discover that the vast majority of mutants affected by a code change actually reside in unmodified parts of the system. By shifting the testing target to "commit-relevant mutants"—unmodified code that exhibits an observable behavioral dependence on the new changes—they demonstrate a 30% increase in effectiveness at catching commit-related bugs.

Expanding on this in Mutation Testing in Evolving Systems: Studying the Relevance of Mutants to Code Evolution, Ojdanic et al. demonstrate that change-aware testing must explicitly account for these non-local behavioral dependencies. When software evolves, the pre-existing codebase contains latent sensitivities to new modifications. Focusing mutation testing solely on the delta misses the critical ripple effects where local changes impact distant components.

Tracking and Benchmarking Change Complexity

To systematically expose these errors, testing frameworks must explicitly model how changes intersect over time. In Regression tests to expose change interaction errors (ESEC/SIGSOFT FSE 2013), Böhme et al. introduce a method to construct a "Change Sequence Graph" (CSG). By mapping out the execution paths and dependencies between multiple independent modifications, the authors use directed symbolic execution to actively probe the interfaces where changes collide. This approach uncovers deep architectural bugs that traditional regression testing—which treats changes in isolation—fails to reach.

The sheer difficulty of catching and fixing these errors stems from their inherent complexity. In CoREBench: studying complexity of regression errors, Böhme et al. quantify this using a metric called Cyclomatic Change Complexity (CyCC). Their evaluation reveals that errors requiring a specific sequence of historical changes to manifest are significantly more complex to diagnose and repair than isolated faults.

This body of research underscores why traditional, diff-centric testing strategies fall short: they treat code evolution as a series of isolated snapshots rather than a continuous, interconnected web of behavioral dependencies. By focusing on the interaction boundaries between changes and unmodified code, modern testing frameworks can better safeguard evolving software systems against non-local regressions.

Key Foundational Work

Regression tests to expose change interaction errors (2013) — contributed 6 concept elements to this direction.
Simulating Student Programmers (1989) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_change_interaction_errors_2b846553,
  title = {change interaction errors},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/change-interaction-errors/rdr_2b846553dcd94d1f9e8ac020f01877d4},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_change_interaction_errors_2b846553_v1,
  title = {change interaction errors},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/change-interaction-errors/rdr_2b846553dcd94d1f9e8ac020f01877d4/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake