Abstract
We revisit the classic signal-to-symbol barrier in light of the remarkable ability of deep neural networks to generate realistic synthetic data. DeepFakes and spoofing highlight the feebleness of the link between physical reality and its abstract representation, whether learned by a digital computer or a biological agent. Starting from a widely applicable definition of abstract concept, we show that standard feed-forward architectures cannot capture but trivial concepts, regardless of the number of weights and the amount of training data, despite being extremely effective classifiers. On the other hand, architectures that incorporate recursion can represent a significantly larger class of concepts, but may still be unable to learn them from a finite dataset. We qualitatively describe the class of concepts that can be "understood" by modern architectures trained with variants of stochastic gradient descent, using a (free energy) Lagrangian to measure information complexity. Even if a concept has been understood, however, a network has no means of communicating its understanding to an external agent, except through continuous interaction and validation. We then characterize physical objects as abstract concepts and use the previous analysis to show that physical objects can be encoded by finite architectures. However, to understand physical concepts, sensors must provide persistently exciting observations, for which the ability to control the data acquisition process is essential (active perception). The importance of control depends on the modality, benefiting visual more than acoustic or chemical perception. Finally, we conclude that binding physical entities to digital identities is possible in finite time with finite resources, solving in principle the signal-to-symbol barrier problem, but we highlight the need for continuous validation.
Introduction: The Signal-to-Symbol Barrier
In the current era of generative AI, the distinction between physical reality and synthetic representation has become increasingly blurred. DeepFakes and sophisticated spoofing techniques have exposed a fundamental vulnerability in how we link physical entities (signals) to digital identities (symbols). This research addresses the "signal-to-symbol barrier"—the theoretical challenge of ensuring that a discrete digital representation truly corresponds to a unique, persistent physical object.
While modern neural networks are exceptional pattern matchers, this paper investigates whether they truly "understand" the underlying physical mechanisms of the world or if they are merely interpolating within a bounded dataset. By applying tools from Model Theory, Turing completeness, and Information Theory, the authors analyze the representability and learnability of physical concepts within artificial architectures.
Defining Concepts as "Efflux"
To analyze how a network understands a physical object, the authors define a concept not as a static dataset, but as an infinite efflux. In this framework, a concept is the set of all possible observations (images, sounds, tactile data) that a physical object could ever produce under all possible conditions.
Understanding a concept, therefore, is not about memorizing a finite set of images, but about capturing the underlying generative law that governs this efflux. This distinction is critical: a network that has memorized a million photos of a person has not necessarily captured the "concept" of that person if it cannot predict how that person would look under a novel lighting condition or from an unseen angle.
Architectural Constraints: Feed-Forward vs. Recursive
A central finding of the research is the inherent limitation of standard feed-forward architectures, such as Convolutional Neural Networks (CNNs) and Variational Autoencoders (VAEs). The authors prove that these networks are "topologically trivial."
Specifically, Proposition 2.9 demonstrates that no feed-forward architecture can encode even basic concepts, such as the set of all even numbers, across an infinite domain. They are limited to recognizing patterns within a bounded range of the data space. Consequently, while these models are effective classifiers, they cannot truly represent the recursive or infinite nature of physical reality.
In contrast, architectures that incorporate recursion—such as Recurrent Neural Networks (RNNs) and Transformers—are Turing-complete. Theoretically, these architectures can represent any computable concept. However, the authors note a significant gap between representability and learnability: even if a Transformer can hold a concept, standard training methods like Stochastic Gradient Descent (SGD) often fail to discover the underlying recursive logic from a finite dataset.
The Activation Barrier and Information Complexity
Learning a concept is rarely a linear path of improvement. The authors use a (free energy) Lagrangian to model the "activation barrier" inherent in conceptual understanding. Often, a network must first increase its internal complexity by memorizing data points before it can discover the underlying pattern that allows for simplification and true understanding.
If the "temperature" (noise) of the training process is too low, the network remains trapped in a state of memorization (overfitting). Reaching a state of understanding requires overcoming this informational barrier, a process that is currently difficult to guarantee with standard optimization techniques.
Active Perception: The Defense Against Spoofing
The paper highlights a critical difference between passive observation and active perception. A passive observer—one that merely processes an incoming stream of data—is always vulnerable to a sufficiently sophisticated spoofer. In the "cat-and-mouse game" of DeepFakes, a static image or a pre-recorded video can eventually be faked to bypass a passive classifier.
However, an active agent—one that can interact with the environment by changing viewpoints, altering illumination, or requesting specific movements—can bind a physical object to a digital identity in finite time. By "poking" the environment and observing if the efflux responds according to the laws of physics (e.g., how shadows move or how occlusions change), the agent can verify the physical reality of the subject.
This benefit is modality-specific. The authors show that visual perception benefits significantly more from active control than acoustic perception. This is because sound waves combine linearly (superposition), making them easier to spoof even with active intervention, whereas visual objects occlude each other in a non-linear, geometry-dependent fashion that is much harder to simulate perfectly in real-time.
Conclusion: The Need for Continuous Validation
The research concludes that while binding physical entities to digital identities is theoretically possible with finite resources, it is not a "one-and-done" process. Because an external agent can never be 100% certain that a network’s internal understanding is perfect based on a finite interaction, security requires continuous validation.
Systems must move beyond simple authentication and toward a model of persistent challenge-response. To truly bridge the signal-to-symbol barrier, AI must transition from being a passive recipient of data to an active participant in the physical world, constantly verifying the reality of the symbols it processes.
Summary updated