Abstract
Neural codes are reflected in complex, temporally and spatially specific patterns of activation. One popular approach to decode neural codes in electroencephalography (EEG) is multivariate decoding. This approach examines the discriminability of activity patterns across experimental conditions to test if EEG contains information about those conditions. However, conventional decoding analyses ignore aspects of neural activity which are informative. Specifically, EEG data can be decomposed into a large number of mathematically distinct features (e.g., entropy, Fourier and Wavelet coefficients) which can reflect different aspects of neural activity. We previously compared 30 such features of EEG data, and found that visual category, and participant behavior, can be more accurately predicted using multiscale spatiotemporally sensitive Wavelet coefficients than mean amplitude (Karimi-Rouzbahani et al., 2021b). Here, we considered that even this larger set of features may only partially capture the underlying neural code, because the brain could use a combination of encoding protocols within a single trial which is not reflected in any one mathematical feature alone. To check, we combined those mathematical features using state-of-the-art supervised and unsupervised feature selection procedures (n = 17). Across 3 datasets, we compared decoding of visual object category between these 17 sets of combined features, and between combined and individual features. Object category could be robustly decoded using the combined features from all of the 17 algorithms. However, the combination of features, which were equalized in dimension to the individual features, were outperformed in most of the time points by the most informative individual feature (Wavelet coefficients). Moreover, the Wavelet coefficients also explained the behavioral performance more accurately than the combined features. These results suggest that a single but multiscale encoding protocol may capture the neural code better than any combination of features. Our findings put new constraints on the models of neural information encoding in EEG.
Introduction
How does the brain package information about the world into electrical signals? For decades, neuroscientists have used "mean amplitude"—the average voltage in a window of time—to decode what a person is looking at from EEG recordings. However, the brain is far more complex; information might be hidden in the "noise" or variability of the signal, such as its entropy, frequency shifts, or complexity.
While recent studies have shown that specific features like Wavelet coefficients outperform simple mean amplitude, a major question remained: Does the brain use multiple encoding protocols simultaneously? This paper tests the "combinatorial" hypothesis—that a combination of distinct mathematical features provides the most complete picture—against the "multiscale" hypothesis, which posits that a single, multiscale feature captures the underlying neural code more effectively.
Methodology
The researchers implemented a rigorous pipeline to compare individual features against optimized combinations:
- Feature Extraction: They extracted 26 mathematically distinct features from EEG data, including statistical moments (mean, variance), complexity measures (Lempel-Ziv), entropy (Sample, Approximate), frequency-based features, and Wavelet coefficients (which capture information across multiple scales of time and space).
- Feature Selection (FS): To avoid simply throwing "everything at the wall," they used 17 different state-of-the-art algorithms (8 supervised like LASSO and Relief; 9 unsupervised like Laplacian Score) to select the top 5 most informative features at every 50ms time window.
- Dimensionality Control: Crucially, they used Principal Component Analysis (PCA) to ensure that the "combined" feature sets had the same number of dimensions as the "individual" features. This ensured a fair comparison—preventing the combined sets from winning simply because they had more variables.
- Decoding and Behavioral Prediction: They used Linear Discriminant Analysis (LDA) to decode object categories and then correlated these neural results with participants' actual reaction times to see which features best predicted human behavior.
Datasets
The study validated its findings across three diverse, previously published EEG datasets to ensure generalizability, ranging from passive fixation to active detection tasks.
Key Results
The "Whole" is Less than the Sum of its Parts
In most machine learning contexts, more features usually equal better performance. Here, the opposite was true. The "best" individual feature (Wavelets) consistently outperformed all 17 feature-selection combinations across the majority of time points in all three datasets.
Multiscale Superiority and Bayesian Evidence
Wavelet coefficients are unique because they are inherently "multiscale"—they look at the signal at different frequencies and time resolutions simultaneously. Using Bayes Factor analysis, the authors found "strong evidence" (BF > 10) that the individual Wavelet feature provided better decoding than the combined sets during peak processing times (approx. 80ms to 320ms post-stimulus).
Behavioral Relevance
The study moves beyond simple decoding accuracy. In the active task (Dataset 2), the correlation between neural decoding and reaction time was significantly stronger for Wavelets than for the combined features. Even the best-performing selection algorithm (Laplacian) failed to reach the peak predictive power of the Wavelet transform, suggesting that multiscale information is what the brain actually uses to drive behavior.
Discussion and Future Work
These results suggest that instead of searching for the perfect "cocktail" of features, researchers should focus on refined multiscale models that better mirror the brain's integrated encoding strategy. The brain’s "language" appears naturally multiscale, and attempting to reconstruct it by stitching together other features (like entropy and frequency) may introduce more noise than signal.
While this study focused on visual object categorization, it remains to be seen if other cognitive processes—like working memory or decision-making—might rely on a more combinatorial encoding strategy. Future research should investigate if these multiscale principles hold across broader cognitive domains.
Summary updated