Bidirectional Information Flow (BIF) - A Sample Efficient Hierarchical Gaussian Process for Bayesian Optimization

Juan David Guerra, Thomas Garbay, Guillaume Lajoie, Marco BonizzatoarXiv 2025 · DOI · arXiv

TL;DR

This paper introduces Bidirectional Information Flow (BIF), a novel hierarchical Gaussian Process framework for Bayesian Optimization. BIF enables two-way communication between parent and child models, significantly improving sample efficiency and performance on complex, structured optimization tasks compared to existing one-way hierarchical methods.

What Problem Does This Paper Address?

Optimizing complex systems with structured parameter spaces, like those found in neurostimulation or aerospace engineering, is challenging due to limited data and high-stakes constraints. Bayesian Optimization (BO) offers a principled approach, often using Gaussian Processes (GPs) as surrogate models. However, when problems have an inherent hierarchy, standard GPs become inefficient. Hierarchical GPs (H-GPs) decompose tasks into child GPs and a parent GP, but traditional H-GPs suffer from a critical limitation: information only flows up the hierarchy from children to parent. This unidirectional flow prevents the parent from informing or refining the child models, leading to suboptimal sample efficiency and slower convergence. This paper addresses this limitation by introducing a bidirectional information flow mechanism.

Methods

The core innovation of BIF is its Bidirectional Information Flow mechanism. This framework maintains the modularity of H-GPs but establishes a two-way communication channel:

  • Upwards Information Flow (Prior Maps): Child GPs create acquisition maps over their respective subspaces. These maps are then aggregated into a "prior map" which serves as a prior for the parent GP's mean. This helps the parent build a rough expectation of the state space.
  • Downwards Information Flow (Response Contribution): When the parent receives a response from the environment, a novel "contribution system" is used to decouple and distribute this response back to the children. Each child receives a percentage of the contribution based on its own predictions. This allows children to learn from the parent's experience, even if they haven't directly queried the environment.
  • Adaptive Data Rescaling: To handle data from both the true environment and the inferred contributions, an adaptive rescaling method is used to normalize data to the [0, 1] range, ensuring child models can effectively learn from both sources.
  • Online Training: BIF is designed for online learning, where models are continuously updated as new data becomes available.

Datasets

The paper evaluates BIF on three types of datasets:

  • Synthetic 2D Dataset: A hierarchical dataset generated from two children and one parent function.
  • Synthetic 3D Dataset: A higher-dimensional synthetic dataset with three children and one parent, designed to test robustness.
  • Neurostimulation Dataset: A real-world dataset for optimizing neurostimulation protocols, mimicking the setup from Laferrière et al. (2020).

Key Insights and Ideas

  • The Power of Two-Way Communication: The most significant insight is that enabling parent-to-child communication in H-GPs dramatically improves sample efficiency. This is because children can learn from the parent's higher-level understanding and feedback, accelerating their own learning.
  • Bridging the Gap in Hierarchical Learning: BIF effectively bridges the gap between modular child models and a global parent model, allowing for a more holistic and efficient optimization process.
  • Simulated Environment for Children: The contribution system effectively creates a "simulated environment" for the children, allowing them to learn and improve even when direct interaction with the real environment is limited.
  • Modularity for Transfer Learning: The modular design of BIF naturally lends itself to transfer learning, where pre-trained child modules can be efficiently deployed to new, related tasks.

Key Results

BIF consistently outperforms baseline methods (Vanilla GPBO and Laferrière et al. [2020]) across various metrics:

  • Synthetic 2D: BIF achieves comparable Relative Optimum to Laferrière but significantly higher child R² (69.26 vs. 54.51 for Laferrière).
  • Synthetic 3D: BIF shows a substantial performance boost, with Global AUC more than doubling compared to baselines. Parent R² is over four times higher than Laferrière's, and child R² improves threefold.
  • Neurostimulation: BIF achieves the highest parent and child R² scores (73.82 and 58.68 respectively), outperforming Laferrière by approximately 8 points in parent R² and nearly doubling its child R².
  • Robustness: BIF demonstrates good robustness to noise, with parent R² remaining stable. It also shows reasonable robustness to nonlinearity, though exponential nonlinearities can impact performance.

Limitations and Future Work

  • Computational Intensity: BIF can be computationally intensive, especially with continuous training and many child processes, leading to O(n³) complexity per GP.
  • Combinatorial Explosion: Training separate parent processes for each task combination risks a combinatorial explosion.
  • Future Work:
    • Develop schedulers for bidirectional updates to manage computational costs.
  • Investigate pretrained, general-purpose parent models to mitigate combinatorial explosion.
  • Further explore the impact of noise and nonlinearity, and how to mitigate them.
  • Explore the transfer learning capabilities more deeply, especially in scenarios with "bad" pretraining.

Summary updated

Abstract

Hierarchical Gaussian Process (H-GP) models divide problems into different subtasks, allowing for different models to address each part, making them well-suited for problems with inherent hierarchical structure. However, typical H-GP models do not fully take advantage of this structure, only sending information up or down the hierarchy. This one-way coupling limits sample efficiency and slows convergence. We propose Bidirectional Information Flow (BIF), an efficient H-GP framework that establishes bidirectional information exchange between parent and child models in H-GPs for online training. BIF retains the modular structure of hierarchical models - the parent combines subtask knowledge from children GPs - while introducing top-down feedback to continually refine children models during online learning. This mutual exchange improves sample efficiency, enables robust training, and allows modular reuse of learned subtask models. BIF outperforms conventional H-GP Bayesian Optimization methods, achieving up to 4x and 3x higher $R^2$ scores for the parent and children respectively, on synthetic and real-world neurostimulation optimization tasks.
BibTeX
@misc{guerra2025bidirectional,
  author = {Juan David Guerra and Thomas Garbay and Guillaume Lajoie and Marco Bonizzato},
  title = {{Bidirectional Information Flow (BIF) - A Sample Efficient Hierarchical Gaussian Process for Bayesian Optimization}},
  year = {2025},
  eprint = {2505.11294},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2505.11294},
}
Figure