TL;DR
This document summarizes five high-impact studies originally published in top-tier software engineering journals and presented at the ICSA 2022 "Journal First" track. The research provides a panoramic view of modern software architecture, covering adversarial machine learning in product lines, the lifecycle of Android code smells, 30 years of reference architectures, communication patterns in open-source projects, and the practical realities of microservices.
What Problem Does This Paper Address?
The "Journal First" track at ICSA 2022 serves as a bridge between deep-dive journal research and the broader software architecture community. Collectively, these five papers address several critical gaps:
- ML Reliability in Software Product Lines (SPLs): Machine learning is used to predict valid product configurations, but prediction errors can lead to costly failures.
- Mobile Technical Debt: While "code smells" are well-understood in desktop software, their evolution and removal in the Android ecosystem remain under-researched.
- Reference Architecture Maturity: Despite 30 years of use, there is a lack of systematic understanding regarding how reference architectures are actually constructed and utilized in the industry.
- Architectural Knowledge Silos: Understanding how (and why) developers communicate architectural decisions in open-source software (OSS) is vital for project sustainability.
- Microservices Complexity: There is a lack of empirical evidence on how practitioners actually design, monitor, and test microservices in the real world.
Methods
The papers utilize a diverse range of empirical and technical methodologies:
- Adversarial Attack Algorithms: One study applies state-of-the-art adversarial ML techniques to synthesize "fake" valid configurations to stress-test SPL prediction models.
- Large-Scale Empirical Mining: Researchers analyzed 324 Android apps and 561 specific "smell-removing" commits, supplemented by interviews with 25 developers.
- Systematic Mapping Study: A comprehensive review of three decades of literature to characterize the context, goals, and maturity of software reference architectures.
- Longitudinal Case Studies: An 18-year analysis of developer mailing lists from two major OSS projects (ArgoUML and Hibernate) using a "multiple case study" approach.
- Mixed-Methods Practitioner Research: A survey of 106 practitioners combined with 6 in-depth interviews to map the current state of microservices architecture (MSA).
Datasets
The studies draw from a mix of industrial, open-source, and academic data:
- Industrial & OSS Configurators: MOTIV (an industrial video synthesizer) and JHipster (a popular web-app configurator).
- Mobile Repositories: Data extracted from 324 Android applications.
- OSS Communication Archives: 18 years of mailing list data from the ArgoUML and Hibernate projects.
- Literature Corpus: Thirty years of published research on reference architectures.
- Practitioner Survey Data: Responses from 106 microservices professionals.
Key Insights and Ideas
- Adversarial Vulnerability in SPLs: ML models used for configuration can be easily fooled. Adversarial configurations can be crafted that are logically valid but cause the model to fail, providing a new foundation for quality assurance in ML-based systems.
- The "Side Effect" of Refactoring: In the Android world, code smells are rarely removed intentionally. Instead, their removal is usually a side effect of other maintenance activities, even when developers are aware the smells exist.
- Reference Architecture Maturity Gap: While industry adoption is increasing, many reference architectures are still designed without a systematic process and lack the maturity required for real-world software projects.
- The "Core Developer" Effect: Architectural communication in OSS projects is highly centralized, revolving around a few core developers, and tends to decrease significantly after the first stable release.
- Microservices Design Trends: Domain-driven design (DDD) and business capability are the dominant strategies for decomposing applications into microservices, yet dedicated solutions for managing MSA complexity are still lacking.
Key Results
- 100% Misclassification: In the SPL study, adversarial attacks achieved up to a 100% misclassification rate without violating logical constraints.
- Smell Diffuseness: High levels of mobile-specific code smells are not necessarily caused by "releasing pressure" (tight deadlines) but are persistent throughout the app lifecycle.
- Communication Focus: In OSS, "Architecture Rationale" and "Architecture Models" are the most frequently discussed elements, primarily for the purposes of negotiation and interpretation.
- MSA Patterns: API Gateway and "Backend for Frontend" (BFF) emerged as the most used patterns in microservices, while resource usage and load balancing are the primary monitoring metrics.
Limitations and Future Work
- Broadening Evidence: Several authors noted the need to move beyond academic data libraries to gather empirical evidence from more diverse industrial sources.
- Complexity Management: A recurring theme is the need for better tools to handle the inherent complexity of microservices at the design level.
- Security in MSA: Future research is specifically called for in the realm of handling security within microservices systems.
- QA Frameworks: There is a clear path forward for developing robust quality assurance frameworks specifically for machine learning components within software product lines.
Summary updated