Implicit Neural Representations (INRs) have fundamentally changed how we store and process continuous signals like images, 3D shapes, and CT scans. Instead of storing data in a discrete grid of pixels or voxels, we train a neural network to map a coordinate (like $x, y, z$) to a signal value (like color or density).
However, a persistent challenge haunts this field: spectral bias. Standard neural networks are naturally biased toward learning low-frequency, smooth functions. While this makes them robust to noise, it also makes them struggle to capture the sharp edges and fine textures that define high-quality data.
The Frequency Scale Dilemma
To overcome spectral bias, researchers often use "positional encodings" or periodic activation functions (like sine waves) to inject high-frequency information into the network. But this introduces a fragile trade-off. As Amer Essakine and colleagues point out in their comprehensive survey, these models are extremely sensitive to the "frequency scale" hyperparameter.
If the scale is too low, the reconstruction is blurry. If it is too high, the network becomes unstable, fitting to high-frequency noise and creating "salt-and-pepper" artifacts. Guangming Zang and the team behind IntraTomo observed this directly in medical imaging: without precise tuning, coordinate-based networks either smooth out critical anatomical details or introduce artificial patterns that could lead to misdiagnosis.
Learning in Layers: The Progressive Approach
One way to solve this is to stop treating all frequencies as equal. Zoe Landgraf and her team proposed Progressive Implicit Networks (PINs) to handle this multi-scale nature of signals.
Instead of a single MLP, PINs use a "ladder" architecture. The model starts by learning the coarsest version of a signal using low-frequency features. As training progresses, it feeds higher-frequency Fourier features into subsequent stages of the ladder. This allows the network to build a foundation of global structure before attempting to resolve fine-grained details, effectively bypassing the noise-fitting issues of high-frequency initializations.
Variable Periodicity and Bias Initialization
Rather than changing the architecture, Hao Zhu and the creators of FINER++ looked at the activation functions themselves. They realized that standard periodic activations have a fixed "rhythm."
By introducing "variable-periodic" functions and carefully initializing the network's bias, they allow different neurons to specialize in different frequency bands. In this framework, the bias isn't just an offset; it acts as a selector that shifts the input into a region of the activation function with a specific periodicity. This allows the network to adaptively "choose" the right frequency for a specific part of the signal, capturing complex textures without requiring the user to manually tune the global frequency scale.
The Sampling Theory Perspective
Why do these frequency-based methods work so well? Hemanth Saratchandran and his colleagues provide a theoretical bridge to classical signal processing. They argue that an INR is essentially a continuous interpolator, and according to the Nyquist-Shannon sampling theorem, the ideal tool for reconstruction is the sinc function.
By using sinc activations with a tunable bandwidth parameter, the network acts as a controllable low-pass filter. This provides a principled way to handle noise: if you know your signal shouldn't have information above a certain frequency, you can constrain the bandwidth of the activation functions. This mathematical grounding suggests that the future of INRs lies in treating neural weights not just as abstract parameters, but as coefficients in a sophisticated, adaptive filter bank.