In many real-world sensing tasks, we only see a fraction of the truth. A LiDAR scanner captures only the side of a car facing the sensor; a CT scan takes limited projections to reduce radiation; a camera view of a building is blocked by a tree. Traditionally, filling these gaps required "ground truth"—massive datasets of complete objects to teach a model what the missing parts should look like.
However, a growing body of research suggests we can reconstruct the whole from the part without ever seeing a "clean" example. The key lies in symmetry and equivariance: the mathematical observation that most signals are redundant. If we know a chair is bilateral or a texture is periodic, the visible data contains the blueprint for the hidden data.
The Nullspace Problem
To understand why this works, we must frame reconstruction as an inverse problem. A sensor acts as an operator that maps a high-dimensional signal (a full 3D object) to a lower-dimensional measurement (a 2D image). This process inevitably loses information, creating a nullspace—a set of signal components that the sensor simply cannot see.
Julián Tachella, Dongdong Chen, and Mike Davies formalized a solution to this in Equivariant Imaging: Learning Beyond the Range Space. They realized that if a signal is invariant to certain transformations (like rotation or translation), we can use those transformations to "move" information out of the nullspace and into the sensor's observable range.
Imagine a sensor that only sees vertical lines. If you have a signal with both vertical and horizontal lines, the horizontal ones are in the nullspace. But if you know the signal is rotationally symmetric, you can virtually rotate the measurement. The horizontal lines become vertical, the sensor "sees" them, and the model learns the full structure. Victor Sechaud and colleagues extended this in Equivariant Splitting, proving that enforcing this consistency allows networks to converge to optimal estimators using only noisy, incomplete measurements.
Geometric Symmetry as a Structural Prior
While equivariance handles abstract mathematical transformations, geometric symmetry provides a concrete anchor for 3D reconstruction. Most man-made objects and biological entities possess bilateral or axial symmetry.
In 3D SLAM and object detection, sensors often capture only a "single-sided" view. Ziwei Liao and Xinge Zhu demonstrate that by identifying a symmetry plane, a system can mirror visible point cloud data to reconstruct occluded surfaces. This isn't just a heuristic; it's a powerful constraint. Konstantinos Rematas showed in ShaRF that incorporating a symmetry loss function allows a Neural Radiance Field (NeRF) to accurately hallucinate the "far side" of an object from a single image.
This principle is particularly effective for deformable objects. Shangzhe Wu and the DOVE team learned to reconstruct 3D animals from monocular videos by enforcing bilateral symmetry on a canonical "rest pose." Because the animal's left and right sides are roughly identical, the model uses the visible side in one frame to learn the geometry of the hidden side in another.
Beyond Mirroring: Self-Similarity and Flow
Symmetry isn't always a perfect reflection. In many cases, it manifests as self-similarity—the repetition of patterns across a signal.
- Internal Redundancy: Thommen Korah showed that building facades are "near-regular textures." If a tree blocks three windows, the model can look at the ten visible windows to reconstruct the missing ones.
- Patch Retrieval: Bo Sun introduced PatchRD, which fills 3D gaps by retrieving and deforming high-resolution patches from the same object. This ensures that the reconstructed part matches the unique style of the original, rather than a generic average.
- Appearance Flow: Instead of generating new pixels, Tinghui Zhou proposed learning an "appearance flow." The model learns to "borrow" pixels from visible regions to fill occluded ones, preserving sharpness and texture that generative models often blur.
Symmetry as a Guide, Not a Rule
A major challenge in this field is that real-world symmetry is rarely perfect. Lighting changes, perspective distortion, and natural variations (like a person's gaze) break strict mathematical symmetry.
Jiankang Deng addressed this in UV-GAN by using symmetry as a secondary input channel rather than a direct copy-paste. The model sees the mirrored version of a face as a hint, but a Generative Adversarial Network (GAN) decides how to synthesize the final texture, allowing it to account for lighting and subtle asymmetries. Similarly, Tal Hassner found that while mirroring helps reconstruct facial shape, it can create artifacts in the eyes, necessitating "symmetry-aware" masks that apply the prior only where it makes sense.
By treating symmetry and equivariance as inductive biases rather than rigid rules, researchers like Christopher Wewer (with SimNP) are moving toward models that can autonomously discover and exploit the inherent structure of the world to see what is otherwise invisible.