To Adapt or Not to Adapt: A Quantification Technique for Measuring an Expected Degree of Self-Adaptation

Sven Tomforde, Martin GollerComput 2020 · DOI

TL;DR

This paper introduces a novel method to quantify the degree of self-adaptation in complex systems. It addresses the challenge of determining "how much" adaptation is optimal, moving beyond simply triggering adaptations to measuring their extent and impact, which is crucial for system stability and user acceptance.

What Problem Does This Paper Address?

Modern information and communication technology systems are becoming increasingly complex. To manage this complexity, the paradigms of self-adaptation and self-organization (SASO) have emerged, where systems can modify their own behavior and structure at runtime. While adaptivity is generally seen as desirable, the authors argue that too much adaptation can negatively impact system stability and user acceptance. Therefore, there's a need to quantify the degree of self-adaptation, not just whether it occurs. This paper aims to provide a measurement framework to quantify this degree, which is essential for understanding and controlling SASO systems, especially in situations where human interaction is involved.

Methods

The core of the proposed method is to quantify self-adaptation as an "unexpected or unpredictable change of the distribution underlying the observed samples (i.e., the configurations of the subsystems)." This is achieved using generative probabilistic models and divergence measures. Specifically, they propose using the symmetric Kullback–Leibler (KL) divergence (KL2) to compare probability distributions of subsystem configurations over different time intervals.

The approach involves:

  • System Model: Defining a subsystem as having a productive part and a control mechanism, with configurations represented by vectors of control variables.
  • Measurement Approach: Collecting configuration snapshots over time and forming probability distributions. The KL2 divergence between a reference distribution and a current distribution quantifies the degree of self-adaptation. A higher KL2 value indicates a greater change and thus a higher degree of self-adaptation.
  • Sliding Windows: Utilizing sliding windows to capture temporal changes in configurations and derive these probability distributions.

Datasets

The paper evaluates its proposed method using three distinct simulation-based use cases:

  • Urban Traffic Control: A simulation of a stadium event in Hannover, Germany, using the Aimsun Professional toolkit.
  • Swidden Farming Model: An existing simulation model of farming households adapting their behavior to environmental dynamics.
  • Organic Network Control: A simulation of mobile ad-hoc networks using the NS/2 simulation toolkit and the MASON multi-agent simulation toolkit.

The paper does not introduce new datasets but rather uses existing simulation environments to test its measurement framework.

Key Insights and Ideas

  • Adaptation is Not Always Good: The paper challenges the assumption that more adaptation is always better, highlighting potential downsides like decreased stability and user acceptance. This shifts the focus from enabling adaptation to managing and quantifying it.
  • Quantifying "How Much": The core contribution is a method to measure the degree of self-adaptation, not just its presence. This moves beyond binary "adapt or not adapt" to a continuous scale.
  • Distributional Change as a Metric: Using the change in the distribution of system configurations as a proxy for self-adaptation is a novel way to capture the overall system's dynamic behavior.
  • Generative Models and Divergence: The application of KL divergence to compare probability distributions of configurations provides a mathematically grounded way to measure the "unexpectedness" of changes.
  • Application Independence: The proposed method is designed to be generic and independent of specific application domains, relying on a common system model.

Key Results

The paper demonstrates the applicability of its KL2-based measurement approach across three diverse scenarios:

  • Traffic Control: The KL2 measure successfully indicated increased adaptation effort during peak traffic times (stadium event) compared to normal traffic, showing higher values as traffic volumes increased and during departure phases. The reference solution, which used a static strategy, showed minimal KL2 values.
  • Swidden Farming: The metric showed significant differences between sampling periods and the reference distribution, reflecting the adaptation of farming agents. The symmetric KL2 variant performed consistently regardless of which distribution was used as reference.
  • Organic Network Control: The KL2 values were generally higher in this scenario due to increased heterogeneity of configurations and a larger number of subsystems. The introduction of more agents after 5000s led to dramatically increased KL values, demonstrating the method's ability to detect changes.

Qualitatively, the results suggest that the KL2 measure can indeed detect and quantify changes in self-adaptation behavior, providing an indicator of adaptation effort.

Limitations and Future Work

The authors acknowledge several limitations and outline future research directions:

  • Absolute Value Interpretation: The absolute KL2 values are difficult to interpret directly and do not allow for direct comparison of the "severity" of change between different scenarios (e.g., "X is twice as strong as Y"). Future work aims to develop normalized values or methods for qualitative statements.
  • Feature Vector Dependence: The KL2 values are highly dependent on the feature vector used (number, type, resolution of parameters) and the adaptation frequency. This makes them application-dependent. Further studies are needed for generalized applicability.
  • High-Dimensional Spaces: The behavior of the measure in high-dimensional configuration spaces is an open question. The authors speculate that self-adaptation mechanisms themselves might reach their limits in such scenarios.
  • Anomaly Detection: The current measure quantifies changes. Future work should integrate anomaly detection mechanisms to identify "abnormally high" adaptation behavior.
  • Self-Explanation: The ultimate goal is to use these measurements to enable self-explanation of SASO behavior, identifying root causes and generating comprehensible explanations for users.
  • Stability and Situation-Dependence: Further research is needed to develop techniques for analyzing stabilization processes and determining an "optimal" degree of self-adaptation that considers user acceptance and stability, not just utility.
  • Alternative Measures: Exploring other dependency measures that might offer more qualitative statements about the determined values is also suggested.

Summary updated

BibTeX
@article{tomforde2020adapt,
  author = {Sven Tomforde and Martin Goller},
  title = {{To Adapt or Not to Adapt: A Quantification Technique for Measuring an Expected Degree of Self-Adaptation}},
  journal = {comput.},
  year = {2020},
  doi = {10.3390/computers9010021},
}
Figure