Stepwise Migration of a Monolith to a Microservices Architecture: Performance and Migration Effort Evaluation
TL;DR
This paper provides an empirical look at the "stepwise" migration of a real-world application from a monolith to microservices via an intermediate "modular monolith" stage. The authors find that while modularization provides a necessary foundation for agility, the final leap to microservices carries a massive performance penalty—with latency increasing by up to 70x—and requires significant refactoring effort to manage data consistency and serialization overhead.
What Problem Does This Paper Address?
The software industry is currently obsessed with microservices, but the path to get there is often a "big bang" migration fraught with risk. Many legacy systems possess "rich domain models"—complex, tightly coupled webs of entities that are easy to develop initially but hard to untangle later.
The authors address the lack of empirical data regarding the trade-offs of a stepwise migration. Specifically, they want to know:
- How much effort (refactoring) is actually required to break a monolith into modules and then into services?
- What is the real-world performance cost of moving from in-memory calls to network-based inter-service communication?
Methods
The researchers performed a three-stage migration on a production system:
- Monolith: The original, tightly coupled system.
- Modular Monolith: The system is refactored into independent modules with well-defined interfaces (using Domain-Driven Design "Bounded Contexts"), but still runs in a single process sharing one database.
- Microservices: Modules are extracted into independent services communicating via REST APIs (synchronous) and message brokers (asynchronous).
Key Refactoring Tactics: * God Class Splitting: Breaking down massive classes that centralized system logic. * Anti-Corruption Layers: Using Data Transfer Objects (DTOs) to prevent one module's internal model from leaking into another. * ID-based Linking: Replacing direct object references between modules with unique identifiers. * Performance Tuning: Implementing service-level caching and "coarse-grained" APIs to reduce the number of expensive remote calls.
Datasets
The study uses the LdoD Archive, a real-world digital archive for digital humanities.
- Scale: 71 domain entities and 81 bidirectional relations.
- Codebase: Approximately 54,000 lines of code (Java, JSP, and JavaScript).
- Complexity: It features a "rich domain," meaning entities are highly interconnected, making it a challenging and representative candidate for migration.
Key Insights and Ideas
- The Modular Monolith is a "Sweet Spot": The authors argue that the modular monolith is often an end-goal in itself. It provides the team-decoupling benefits of microservices without the crushing overhead of network latency.
- Refactoring can improve performance (initially): Surprisingly, the modular monolith was sometimes faster than the original monolith. Why? Because the refactoring process forced the developers to implement better indexing and more efficient data access patterns that weren't present in the "messy" original version.
- Serialization is the Silent Killer: In microservices, the "network" isn't the only bottleneck. The authors found that serialization/deserialization (converting objects to JSON and back) accounted for a staggering 82% of total latency in some operations.
- The "Rich Domain" Tax: If your domain is highly interconnected, the effort to define DTOs and remove circular dependencies is where the bulk of the work lies. This effort is required just to reach the modular stage, long before you even touch a network protocol.
Key Results
The quantitative results highlight the steep cost of distribution:
- Migration Effort: In the "Virtual" module, 82% of domain entities had to be modified just to reach the modular monolith stage.
- Latency Explosion: Moving from a monolith to microservices caused latency to increase between 201% and 6977%. For data-heavy tasks like "Source Listing," the system became nearly 70 times slower.
- Throughput Collapse: Throughput for the "Source Listing" functionality dropped by 96% when moving to microservices.
- Consistency Complexity: The move to microservices introduced 8 different types of events to manage eventual consistency, adding significant cognitive load to the development process.
Limitations and Future Work
The primary limitation is that this is a single case study. While the LdoD Archive is a complex, real-world system, results might vary for simpler CRUD-based applications.
The authors suggest that future work should explore:
- Saga Patterns: More complex distributed transaction management.
- Operational Costs: Evaluating the effort required for CI/CD, monitoring, and orchestration (e.g., Kubernetes), which were not the focus of this developer-centric study.
- Automated Refactoring: Tools that can help identify bounded contexts and automate the generation of DTOs to lower the migration effort.
Summary updated