Imagine training a machine learning model or analyzing a neuroimaging scan to detect when a subject perceives an abstract concept, such as "animacy" or "food." Your model achieves 95% accuracy. It is tempting to conclude that the system has successfully isolated the high-level semantic concept.
But what if every animal image in your dataset happens to feature high spatial frequencies (like fur or feathers), specific aspect ratios, or distinct color profiles compared to the inanimate controls? Is your model truly processing meaning, or is it simply acting as a glorified texture detector?
This fundamental ambiguity is known as low-level feature confounding. In both biological visual systems and artificial neural networks, high-level semantic categories are deeply intertwined with low-level sensory properties. Distinguishing whether a system responds to the abstract meaning of a stimulus or merely its physical structure is one of the most persistent challenges in cognitive neuroscience and computer vision.
The Multicollinearity Trap in Natural Stimuli
In the real world, visual and auditory features do not exist in isolation. As Shinji Nishimoto argues in a review of visual encoding models, natural scenes contain highly correlated features—such as the statistical coupling between specific colors and spatial positions—which severely complicates the isolation of true neural drivers (Quantitative Modeling of the Visual System (The Brain & Neural Networks 2012)).
This statistical dependency, known as multicollinearity, makes it incredibly difficult to attribute causality. When multiple descriptive features move together, standard regression and encoding models cannot definitively determine which specific attribute triggers a response.
This entanglement spans across modalities and domains:
- Visual Scenes: Monica Betta et al. note that natural visual stimuli exhibit high degrees of correlation between low-level features, geometric shapes, and semantic categories (Common spatiotemporal processing of visual features shapes object representation (Scientific 2019)).
- Image Properties: In single-trial EEG decoding, Sebastian Sieghartsleitner et al. point out that basic properties like luminance, contrast, and saturation are inherently linked, preventing researchers from attributing brain activity to a single physical feature (Decoding of image properties from single-trial visual evoked potentials recorded by ultra-high-density EEG (Scientific 2025)).
- Natural Language: The problem is not unique to vision. Leila Wehbe et al. demonstrate that in naturalistic stories, different linguistic and narrative features are highly collinear (e.g., a specific character always performing a certain type of action), which prevents models from isolating the exact drivers of neural activity (Simultaneously Uncovering the Patterns of Brain Regions Involved in Different Story Reading Subprocesses (PLoS 2014)).
Where Confounding Blurs the Lines
When low-level features and high-level semantics are correlated, models can easily exploit low-level "shortcuts." This creates significant hurdles when trying to map specialized brain regions or understand deep neural network representations.
Scene Perception vs. Spatial Geometry
What exactly are scene-selective brain areas processing? Mark D. Lescroart et al. demonstrated that three major competing theories—low-level 2D Fourier power, mid-level 3D spatial layout, and high-level object semantics—all explain brain activity remarkably well (Fourier power, subjective distance, and object categories all provide plausible models of BOLD responses in scene-selective visual areas (Frontiers Comput. Neurosci. 2015)). Because these features share massive amounts of information in natural environments, standard models cannot determine whether a brain area cares about the meaning of a scene or just its basic visual structure. Similarly, George L. Malcolm et al. emphasize that low-level features are intrinsically linked to high-level semantic categories in real-world scene understanding (Making Sense of Real-World Scenes (Trends 2016)).
Abstract Dimensions and Scales
When evaluating how the brain represents the transition from a close-up object to a wide-scale scene, Jeongho Park et al. note that changes in spatial scale naturally co-vary with texture density and high-frequency visual details (Ramp-shaped neural tuning supports graded population-level representation of the object-to-scene continuum (Scientific 2022)). Without separating these, it is unclear if the observed neural responses are driven by the concept of scale itself or by these correlated image statistics.
Categorization and Animacy
When studying how biological classes are organized in the brain, Andrew C. Connolly et al. point out that conceptual similarity frequently correlates with physical similarity (e.g., small size or articulated parts) (The Representation of Biological Classes in the Human Brain (Journal of Neuroscience 2012)). This makes it difficult to prove that the brain is performing high-level semantic grouping rather than simple pattern recognition of visual contours—a challenge that also applies to body part discrimination (Ryohei Fukuma), object identity (Rory Sayres et al.), face familiarity (M. Ida Gobbini), and food selectivity (Nidhi Jain et al.).
Mathematical and Statistical Disentanglement
To isolate true functional drivers from correlated nuisance variables, researchers deploy advanced statistical frameworks designed to handle multicollinearity.
One approach is to use regularized regression models that penalize redundant features. For instance, Ying Wu et al. utilized elastic net logistic regression to handle the high dimensionality and significant correlation between adjacent brain regions in neuroimaging data (Detection of functional and structural brain alterations in female schizophrenia using elastic net logistic regression (Brain 2021)).
Alternatively, generalized additive models can map complex, overlapping variables during unconstrained behavior. Edoardo Balzani et al. introduced a Poisson Generalized Additive Model (P-GAM) framework that automatically learns the smoothness of neural tuning, allowing it to distinguish between true behavioral drivers and correlated environmental noise (Efficient estimation of neural tuning during naturalistic behavior (neurips 2020)).
Accounting for the geometric interactions between existing variables can also expose statistical artifacts. Kathryn McClain et al. developed a Position-Theta-Phase model to demonstrate that much of what was previously interpreted as "speed modulation" in hippocampal place cells was actually a statistical artifact of how position and timing variables interact (Position-theta-phase model of hippocampal place cell activity applied to quantification of running speed modulation of firing rate (2019)).
Engineering Decoupled Stimuli
While statistical adjustments help post-hoc, the most robust solution is to break the feature correlations at the source by engineering stimulus sets where low-level properties and high-level semantics are completely orthogonal.
Natural Stimuli (Confounded)
[High-Level Category: Animal] <-- Highly Correlated --> [Low-Level Feature: High Spatial Frequency]
Engineered Stimuli (Decoupled)
[High-Level Category: Animal] <-- Independent --> [Low-Level Feature: Low Spatial Frequency]
[High-Level Category: Object] <-- Independent --> [Low-Level Feature: High Spatial Frequency]
Algorithmic Optimization
Researchers can use mathematical optimization to select natural stimuli that are statistically independent. In auditory neuroscience, Maria Araújo Barbosa et al. developed a framework to select natural sounds by minimizing the distance covariance between deep neural network representations (acoustic texture) and word embeddings (semantic meaning) (Optimal stimulus selection for dissociating acoustic and semantic processing of natural sounds (SPIRE 2024)). This optimization allows scientists to finally distinguish whether an auditory response is driven by how a sound feels or what it means. Similarly, to study animacy, the authors of Disentangling five dimensions of animacy in human brain and behaviour used a genetic algorithm to curate a stimulus set that completely uncoupled highly correlated traits like biological appearance, self-initiated movement, and unpredictability.
Frequency Tagging
Another method physically manipulates the presentation of the stimulus to isolate semantic processing. Roger Koenig‐Robert developed Semantic Wavelet-Induced Frequency-Tagging (SWIFT), a technique that periodically modulates the semantic meaning of an image at a specific frequency while keeping its low-level physical properties constant (Semantic Wavelet-Induced Frequency-Tagging (SWIFT) Periodically Activates Category Selective Areas While Steadily Activating Early Visual Areas (PLoS 2015)). This allows researchers to track the precise transition from sensory processing to semantic interpretation in the visual cortex.
Synthetic and 3D Rendering Pipelines
Standard 2D pixel-based stimuli inherently entangle geometry, illumination, and orientation. To bypass this limitation, Pavithra Elumalai et al. developed a differentiable 3D rendering pipeline that allows researchers to independently manipulate an object's underlying 3D shape, lighting, and pose (Beyond Pixels: A Differentiable Pipeline for Probing Neuronal Selectivity in 3D (arxiv 2025)). By moving from 2D pixels to 3D meshes, models can determine whether a neuron is responding to a true geometric structure or merely the specific lighting conditions of a flat image.
By combining these advanced statistical models with rigorously engineered stimulus sets, researchers can move past superficial pattern recognition and uncover the true causal mechanisms driving both biological and artificial intelligence.