In the standard deep learning paradigm, convolutional neural networks (CNNs) learn thousands of filters to extract features from data. While effective, these "black box" models require massive amounts of data to learn basic geometric properties like translation invariance and stability to small deformations.
The Wavelet Scattering Transform (WST), pioneered by Stéphane Mallat and Joan Bruna in Classification with Scattering Operators, offers a "white box" alternative. It replaces learned convolutional layers with a cascade of fixed, mathematically designed wavelet transforms. By doing so, it provides a representation that is inherently invariant to translations and robust to the "stretching" or "warping" of signals, all without requiring a single training parameter in its feature extraction layers.
The Problem: Why Averaging Isn't Enough
To make a representation invariant to small translations (like shifting an image by a few pixels), the simplest mathematical operation is a local average (low-pass filtering). However, averaging is a destructive process. If you average a high-frequency signal, you lose all the fine-grained structural information—the very details needed for classification.
The Scattering Transform solves this by using a three-step recursive mechanism:
- Wavelet Transform: Instead of a simple average, the signal is convolved with a bank of wavelets at different scales and orientations. This captures the high-frequency details that an average would kill.
- Modulus Nonlinearity: The complex modulus is applied to the wavelet coefficients. This acts like a "rectifier" that removes the phase (which changes rapidly with translation) but keeps the amplitude (the energy of the signal at that scale).
- Low-pass Filtering: The resulting modulus is then averaged to create an invariant coefficient.
Recovering Lost Information through Cascades
The genius of the scattering transform lies in its deep architecture. When we apply the low-pass filter in step 3, we still lose the high-frequency information of the modulus itself. To recover this, the WST "scatters" the signal into a second layer.
As Michael Eickenberg and colleagues demonstrated in Second Order Scattering Descriptors Predict fMRI Activity Due to Visual Textures, these second-order coefficients capture complex patterns like textures and "vibrato" that first-order transforms miss. By cascading these operations, the network builds a hierarchical representation that preserves signal energy while becoming increasingly invariant to transformations.
Stability to Deformations
Beyond simple shifts, real-world data is often "deformed." A handwritten '7' might be slightly slanted, or an ECG signal might be stretched due to a changing heart rate. Standard Fourier transforms are notoriously unstable to these small "diffeomorphisms"—a tiny stretch in time can lead to a massive change in high-frequency Fourier coefficients.
Stéphane Mallat showed in Deep Learning by Scattering that scattering transforms are Lipschitz continuous to deformations. This means that if the input changes slightly, the representation changes only slightly. This stability is why Ruizhe Zheng and others used the transform in ScatterFormer to detect epileptic discharges across different patients; the transform ignores the "noise" of individual physiological differences while focusing on the underlying signal structure.
Hybrid Networks: The Best of Both Worlds
A recurring theme in recent research is that we don't need to choose between wavelets and deep learning. Instead, we can use the Scattering Transform as a fixed "front-end" for a neural network.
- Data Efficiency: Edouard Oyallon showed in Scaling the Scattering Transform: Deep Hybrid Networks that by replacing the first few layers of a CNN with a scattering transform, models can match ImageNet performance while being significantly more robust in data-scarce environments.
- Speed and Memory: In Compressing the Input for CNNs with the First-Order Scattering Transform, Eugene Belilovsky and colleagues demonstrated that using scattering as a preprocessing step allows for significant reductions in spatial resolution and memory usage without sacrificing accuracy.
- Interpretability: Because the filters are fixed wavelets, the features have physical meaning. Joakim Andén and Vincent Lostanlen leveraged this in Classification with Joint Time-Frequency Scattering to capture spectrotemporal patterns like chirps in audio that are difficult for standard mel-spectrograms to resolve.
Beyond Images: Molecules and Manifolds
The principles of scattering are now being applied to non-Euclidean data. Nicolas Poilvert and Matthew J. Hirn applied these transforms to quantum chemistry in Quantum Energy Regression using Scattering Transforms, using the transform's inherent rotation and translation invariance to predict molecular energy levels. Similarly, Joyce A. Chew extended the idea to high-dimensional point clouds in The Manifold Scattering Transform, proving that the intuition of "wavelet + modulus + average" is a universal tool for stable feature extraction across any domain.