Introduction
In high-stakes domains such as medical diagnostics and human-machine interaction, machine learning models frequently encounter nuisance variations. These are factors that influence the observed data ($X$) but are irrelevant to the primary classification task ($Y$). For instance, in EEG-based brain-computer interfaces, the unique biological "signature" of a specific individual acts as a nuisance variable ($S$) that can obscure the underlying task-related signals.
Standard deep learning models often struggle to generalize across different subjects or environments because they fail to explicitly account for the probabilistic interactions between $X$, $Y$, and $S$. While specialized architectures like Adversarial Variational Autoencoders (A-CVAE) have been developed to address these issues, there is no universal architecture suitable for all datasets. Currently, designing these "macro-level" architectures—deciding how encoders, decoders, and adversaries should be connected—is a manual, intuition-driven process that represents a significant bottleneck in developing robust biosignal analysis pipelines.
The AutoBayes Framework
To address this challenge, we present AutoBayes, an automated framework designed to search for the optimal Bayesian graphical model for a given dataset. Unlike traditional AutoML, which focuses on "micro" hyperparameters like layer depth or filter counts, AutoBayes explores the "macro" connectivity between fundamental functional blocks.
The framework identifies the most effective way to arrange Classifiers ($C$), Encoders ($E$), Decoders ($D$), Nuisance Estimators ($N$), and Adversaries ($A$) based on the specific probabilistic structure of the data. By tailoring the architecture to the dataset, AutoBayes builds pipelines that are inherently robust to nuisance factors.
Bayesian Graph Exploration and Methodology
The core of AutoBayes is its ability to systematically explore the space of possible Bayesian networks that factorize the joint probability of data, labels, and latent variables.
Key Methodological Components:
- Graph Generation: The system generates a variety of potential Bayesian networks.
- The Bayes-Ball Algorithm: To ensure efficiency, AutoBayes employs the Bayes-Ball algorithm to identify conditional independence between variables. This allows the system to prune redundant links and create justifiable, streamlined inference graphs.
- Adversarial Disentanglement: AutoBayes automatically determines where to attach "adversary" networks. These adversaries "censor" nuisance information from the latent representation ($Z$), ensuring that the features used for classification are task-relevant and subject-invariant.
- Ensemble Stacking: Recognizing that different graphical assumptions may capture complementary information, AutoBayes combines the predictions of multiple explored graphs using a meta-learner (e.g., a simple MLP).
Experimental Evaluation
The framework was evaluated across eight diverse datasets, primarily focusing on biosignals such as EEG (RSVP, MI, ErrP), EMG (ASL), and physiological data (Stress). It was also tested on QMNIST to assess its performance on handwritten digit recognition with writer IDs as nuisances.
Key Results:
- State-of-the-Art Performance: AutoBayes outperformed existing state-of-the-art benchmarks on nearly all biosignal datasets. Notably, it achieved 99.61% accuracy on QMNIST, demonstrating its versatility beyond medical data.
- Robustness and Subject Transfer: In "subject-transfer" scenarios, where models are tested on individuals not seen during training, AutoBayes significantly improved the "performance floor." For the Stress dataset, even the most challenging subjects achieved at least 94% accuracy, whereas standard classifiers showed much wider performance gaps.
- The Power of Ensembles: The ensemble stacking method provided a substantial performance boost—up to 19% in some cases—suggesting that the automated search successfully identifies diverse and useful model structures.
Conclusion
AutoBayes demonstrates that the macro-architecture of a machine learning model—the high-level relationship between its functional components—is critical for robustness. By automating the exploration of Bayesian graphical models, the framework removes the need for manual architectural design and consistently produces models that outperform standard baselines in complex, nuisance-heavy tasks. Future work will look toward integrating this macro-level search with traditional micro-AutoML to create fully optimized, end-to-end automated pipelines.
Summary updated