Compressed Decentralized Learning of Conditional Mean Embedding Operators in Reproducing Kernel Hilbert Spaces

Boya Hou, Sina Sanjari, Nathan Dahlin, Subhonmesh BoseAAAI 2023 · DOI

Introduction

Conditional Mean Embeddings (CMEs) offer a robust framework for representing conditional probability distributions as linear operators within Reproducing Kernel Hilbert Spaces (RKHS). This approach is particularly advantageous for modeling complex, nonlinear dynamical systems by transforming intricate high-dimensional integrals into manageable linear operations. However, deploying kernel-based CMEs in real-world decentralized environments—such as robot swarms or distributed sensor networks—presents significant challenges.

Two primary hurdles impede widespread adoption: scalability and communication overhead. Kernel methods typically scale poorly as the model size grows with the number of data points. Furthermore, in decentralized settings, transmitting raw data or full kernel models can quickly saturate network bandwidth. Existing quantized consensus methods often suffer from "drift," where compression errors cause the global average of agent estimates to diverge over time. This paper introduces a decentralized algorithm designed to compress these kernel operators while preserving the global average and ensuring stable learning.

Methodology

The proposed approach utilizes a two-stage decentralized algorithm that combines coherence-based sparsification with a modified consensus protocol to maintain model stability and accuracy under communication constraints.

Figure 2: (a) Phase portrait of the target dynamical system and (b) the 10-agent communication network used for decentralized learning.
  1. Dictionary Passing: To establish a shared basis for representation, agents first exchange "compressed" versions of their local datasets. A coherence-based compression scheme is employed, retaining only data points that are sufficiently distinct based on their kernel inner product. Within a number of iterations equal to the network's diameter, all agents converge on a "stationary dictionary"—a unified set of basis functions representing the state space.
  2. Coefficient Passing: Once the dictionary is established, agents only need to communicate the scalar weights (coefficients) associated with these basis functions, drastically reducing the required bandwidth.
  3. Modified Consensus Update: To mitigate the drift common in compressed learning, the authors implement a specific update rule featuring a correction term. This ensures that the network-wide average of the covariance operators remains invariant, providing Bounded-Input Bounded-Output (BIBO) stability for the iterative dynamics.

Experimental Evaluation

The algorithm was validated on two benchmarks:

  • The Duffing Oscillator: A nonlinear system used to test the learning of Koopman operators. A dataset of 4,900 points was distributed across 10 agents.
  • Inverted Pendulum (OpenAI Gym): A reinforcement learning task where agents performed Approximate Value Iteration to learn swing-up and balance control using 6,000 distributed state-action pairs.

Key Results and Discussion

The experiments demonstrated that the compression scheme significantly enhances efficiency without sacrificing performance. In the Duffing Oscillator task, the model required nearly 45% fewer data points (reducing the count from 1,253 to 693) while accurately identifying the system's regions of attraction.

In the Inverted Pendulum task, the decentralized agents achieved performance levels comparable to a centralized reference controller. Notably, Agent 8 in the network achieved a median score of -129, slightly outperforming the centralized reference median of -134. Qualitative analysis of the value function heatmaps confirms that the consensus algorithm successfully harmonizes disparate local models into a unified, accurate global representation.

Figure 6: Value function heatmaps for Agent 2 and Agent 10, demonstrating the successful consensus and model alignment across the network.

Conclusion

This research provides a stable and efficient framework for decentralized CME learning. By utilizing stationary dictionaries and a drift-correcting consensus rule, the authors enable complex nonlinear modeling in bandwidth-constrained environments. Future work will focus on characterizing the long-term asymptotic behavior of these estimates and extending the framework to online streaming settings where agents collect and share data in real-time. Additionally, exploring the impact of heterogeneous kernels—where agents possess different sensing capabilities—remains a promising avenue for further study.

Summary updated

BibTeX
@inproceedings{hou2023compressed,
  author = {Boya Hou and Sina Sanjari and Nathan Dahlin and Subhonmesh Bose},
  title = {{Compressed Decentralized Learning of Conditional Mean Embedding Operators in Reproducing Kernel Hilbert Spaces}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.1609/aaai.v37i7.25956},
}
Figure