Understanding the Causes of Architecture Changes Using OSS Mailing Lists

Wei Ding, Peng Liang, Antony Tang, Hans van VlietInt. J. Softw. Eng. Knowl. Eng 2015 · DOI

TL;DR

This paper investigates the reasons behind software architecture changes in open-source projects by analyzing developer discussions on mailing lists. They found that "preventative changes" are the most common cause, aiming to improve future maintainability, and that a significant portion of these changes occur before a project's first stable release.

What Problem Does This Paper Address?

Understanding why software architectures change is crucial for preventing knowledge loss and architectural decay. However, in open-source software (OSS) development, this crucial "why" is often undocumented. Developers communicate about architecture decisions and changes primarily through mailing lists. This paper aims to bridge the gap between these informal discussions and the actual architectural evolution in the codebase, specifically by identifying and categorizing the causes of these changes.

Methods

The researchers employed a multi-phase approach:

  1. Data Collection and Initial Analysis: They selected two popular OSS projects, Hibernate and ArgoUML, known for their active mailing lists. They identified "architectural threads" – discussions related to architecture – from these mailing lists. Using a top-down approach, they extracted architecture changes and classified their causes based on an existing categorization from prior research [17].
  2. Source Code Verification: To ensure the identified changes were real, they cross-referenced the discussions with the projects' source code. They used a static analysis tool called "Understand" to compare different versions of the source code, confirming whether changes discussed in mailing lists corresponded to actual code modifications.
  3. In-depth Cause Analysis (Grounded Theory): For a deeper understanding of the causes, they applied a bottom-up approach using Grounded Theory (GT). This involved open coding of the communicated causes to identify core concepts and generate classification codes, which were then categorized into functional requirements, external quality requirements, and internal quality requirements.

Datasets

The study utilized the mailing list archives of two open-source projects:

  • Hibernate: Data was collected from Jan 2002 to Aug 2014.
  • ArgoUML: Data was collected from Jan 2001 to Aug 2014.

The source code of various releases for both projects was used for verification.

Key Insights and Ideas

  • Mailing Lists as a Goldmine: The study confirms that OSS mailing lists are a valuable, albeit informal, source for understanding architectural evolution and its drivers.
  • Preventative Maintenance Reigns Supreme: The most common reason for architecture changes in both Hibernate and ArgoUML is "preventative changes." This highlights a proactive approach by developers to improve the system's long-term maintainability and evolvability, rather than just reacting to immediate problems.
  • Early Bird Changes: A significant portion (over 45%) of architecture changes occur before a project releases its first stable version. This suggests that the foundational architecture is often shaped and refined during the early, more experimental phases of development.
  • Developer-Centric Quality: Internal quality requirements (like conciseness, consistency, and modularity) are significant drivers of architecture changes, indicating that developers' concerns about code structure and understandability play a crucial role.

Key Results

  • Major Cause: Preventative changes were the dominant cause of architecture changes in both Hibernate (48.6%) and ArgoUML (45.9%).
  • Subtypes of Causes:
    • Hibernate: The most frequent subtypes were "Conciseness" (internal quality) and "Modularity" (internal quality) for preventative changes, and "New component" (functional requirement) for perfective changes.
  • ArgoUML: "Modularity" (internal quality) was the leading cause for preventative changes, and "New component" (functional requirement) for perfective changes.
  • Timing of Changes:
    • Before the first stable release (BFR), preventative and perfective changes were most common.
  • After the first stable release (AFR), Hibernate saw a mix of adaptive and preventative changes, while ArgoUML continued to be driven by perfective changes.
  • Core Developer Dominance: A substantial majority of architecture changes were driven by the top two or three core developers in each project, underscoring their pivotal role as de facto architects.

Limitations and Future Work

  • Threats to Validity: The authors acknowledge potential threats to construct validity (e.g., interpretation of criteria, difficulty in verifying system-level changes) and external validity (generalizability beyond the two studied projects).
  • Future Work:
    • Surveys: Validate findings with core developers through surveys.
  • Tool Development: Create tools to assist in managing architecture changes based on their causes.
  • Broader Data Sources: Explore other communication channels like forums, commit logs, and blogs for architecture change information.
  • Automated Analysis: Employ topic modeling to automatically analyze architectural posts and compare with manual findings.

Summary updated

BibTeX
@article{ding2015understanding,
  author = {Wei Ding and Peng Liang and Antony Tang and Hans van Vliet},
  title = {{Understanding the Causes of Architecture Changes Using OSS Mailing Lists}},
  journal = {Int. J. Softw. Eng. Knowl. Eng.},
  year = {2015},
  doi = {10.1142/s0218194015400367},
}
Figure