An Approach to Autonomizing Legacy Systems

Gail E. Kaiser, Phil Gross, Gaurav S. Kc, Janak J. Parekh, Giuseppe Valetto2005 · DOI

TL;DR

This paper proposes an externalized, "plug-and-play" infrastructure called Kinesthetics eXtreme (KX) to add autonomic capabilities (self-healing, self-management, adaptation) to existing, often heterogeneous, distributed systems. KX allows for dynamic monitoring and reconfiguration without modifying the core application code, making it ideal for complex "systems of systems."

What Problem Does This Paper Address?

Modern high-performance computing and distributed systems are increasingly composed of diverse components, including legacy systems, commercial off-the-shelf (COTS) products, and open-source software. Managing and adapting these complex "systems of systems" is a significant challenge. Traditionally, this requires highly skilled technicians to manually monitor and reconfigure systems, which is time-consuming, error-prone, and doesn't scale. Hardwiring adaptation logic directly into new systems is also problematic, leading to brittle, non-reusable solutions. The authors aim to provide a standardized, external approach to imbue these systems with autonomic capabilities, enabling them to adapt to changing environments and self-manage without human intervention or core code modification.

Methods

The core of the proposed solution is a three-tiered, externalized infrastructure:

  • Probes: These are non-invasive sensors attached to running systems to collect raw data about their activity and state. They can be passive (triggered by system events) or active (performing computations within the system context). The paper mentions using technologies like "Probelets" (mobile agent-based probes) and a standardized "Smart Event" XML Schema for probe output.
  • Gauges: These software entities interpret the data from probes. They filter, aggregate, and analyze this information against various models of the system to produce higher-level measurements. Gauges can also emit events, feeding into the next layer. The infrastructure includes components like an "Event Packager" and an "Event Distiller" to manage and interpret these events.
  • Decision/Control Layer: This layer analyzes the interpreted data from gauges and makes decisions about how to adapt or reconfigure the system. This can involve adjusting probes/gauges themselves or triggering reconfiguration actions. The paper highlights the use of a decentralized workflow system called "Workflakes" to coordinate these reconfiguration actions via "effectors."

The key novelty lies in the externalized nature of this infrastructure. It operates orthogonally to the legacy systems' business logic and communication frameworks, promoting reusability and simplifying the integration process.

Datasets

Not discussed in the paper. The paper focuses on the infrastructure and its application to existing systems rather than training on specific datasets.

Key Insights and Ideas

  • Externalization is Key: The most significant insight is that adding autonomic capabilities doesn't require invasive modification of existing code. An external, standardized infrastructure can achieve this, making it applicable to a wide range of systems, especially legacy ones.
  • "Plug-and-Play" Autonomy: The proposed infrastructure aims for a "plug-and-play" experience, where probes and gauges can be attached and removed, and reconfiguration workflows can be defined and executed, without deep knowledge of the target system's internals.
  • Separation of Concerns: The approach clearly separates the reusable infrastructure (probes, gauges, control logic) from the system-specific policies and models, leading to greater modularity and reusability.
  • Handling "Systems of Systems": The framework is explicitly designed to address the complexity of systems composed of multiple, independently developed components, offering a potential "global supervisor" for adaptation.
  • Dynamic Architecture Modeling: The paper touches upon the idea of dynamically modeling the evolving architecture of a system, which is crucial for understanding and managing complex, changing environments.

Key Results

The paper describes the development and experimental application of the Kinesthetics eXtreme (KX) infrastructure. While specific quantitative results are not detailed, the paper highlights:

  • The successful instrumentation of a variety of existing systems.
  • The ability to convey runtime information to a monitoring layer.
  • Progress in developing the "Decision" layer for control and adaptation.
  • Experimental applications in areas like load balancing, server replacement for messaging systems, and complex contingency management in geographically-based analysis systems.
  • The infrastructure's ability to support both static and dynamic architectural scenarios, enabling adaptations ranging from simple parameter changes to complex component replacements and reconfigurations.

Limitations and Future Work

  • Manual Workflow Coding: Currently, workflow notation for reconfiguration actions is coded by hand in Java, rather than using a formal process specification language. The authors are experimenting with formalisms like Little-JIL for this.
  • Ad Hoc Feedback Loop: The current feedback loop is described as "relatively ad hoc," relying on manually constructed gauge rules. Future work will focus on basing reconfiguration decisions on more sophisticated architectural models.
  • Probe/Effector Specialization: While the infrastructure is generic, specific probes and effectors still need to be specialized to the implementation technology of the target system.
  • Future Work: The authors plan to focus on improving the decision-making logic in the feedback and reconfiguration layer, leveraging more comprehensive architectural models. They also aim to build gauges that can recognize structural changes based on these models.

Summary updated

Abstract

Adding adaptation capabilities to existing distributed systems is a major concern. The question addressed here is how to retrofit existing systems with self-healing, adaptation and/or self management capabilities. The problem is obviously intensified for 'systems of systems' composed of components, whether new or legacy, that may have been developed by different vendors, mixing and matching COTS and 'open source' components. This system composition model is expected to be increasingly common in high performance computing. The usual approach is to train technicians to understand the complexities of these components and their connections, including performance tuning parameters, so that they can then manually monitor and reconfigure the system as needed. We envision instead attaching a 'standard' feedback loop infrastructure to existing distributed systems for the purposes of continual monitoring and dynamically adapting their activities and performance. (This approach can also be applied to 'new' systems, as an alternative to 'building in' adaptation facilities, but we do not address that here.) Our proposed infrastructure consists of multiple layers with the objectives of probing, measuring and reporting of activity and state within the execution of the legacy system among its components and connectors; gauging, analysis and interpretation of the reported events; and possible feedback to focus the probes and gauges to drill deeper, or when necessary - direct but automatic reconfiguration of the running system.
BibTeX
@misc{kaiser2005approach,
  author = {Gail E. Kaiser and Phil Gross and Gaurav S. Kc and Janak J. Parekh and Giuseppe Valetto},
  title = {{An Approach to Autonomizing Legacy Systems}},
  year = {2005},
  doi = {10.21236/ada437473},
}
Figure