Periodic Activations for Implicit Neural Representations

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 2256 papers321 conceptsfrozen 2026-10-04

Traditional neural networks are remarkably good at finding patterns in discrete data, but they often struggle when asked to represent the world as a continuous, high-fidelity signal. Whether it is the fine texture of a 3D-scanned face, the complex oscillations of a brain wave, or the sharp gradients of a fluid simulation, standard architectures tend to produce "blurry" results. This phenomenon, known as spectral bias, occurs because common activation functions like ReLU are biased toward learning low-frequency components first.

To overcome this, a growing body of research has shifted toward Implicit Neural Representations (INRs)—specifically those utilizing periodic activation functions. By replacing piecewise-linear activations with sine waves, researchers can map coordinates (like $x, y, z, t$) directly to signal values with unprecedented detail.

The Problem of Spectral Bias

When a standard Multi-Layer Perceptron (MLP) uses ReLU activations, it approximates functions as a collection of connected linear segments. While this is powerful for classification, it is mathematically "clunky" for representing signals with high-frequency detail. As noted by Sameera Ramasinghe and Simon Lucey in Beyond Periodicity: Towards a Unifying Framework for Activations in Coordinate-MLPs, these networks suffer from a lack of high-rank feature projection, effectively smoothing out the very edges and textures that define a high-resolution signal.

Why Periodic Activations Work

The breakthrough in this field, often centered around the Sinusoidal Representation Network (SIREN), involves using the sine function as an activation. This works for two intuitive reasons:

  1. Fourier Equivalence: A network with sine activations can be viewed as a learned, hierarchical Fourier decomposition. As Marc Rußwurm demonstrates in Geographic Location Encoding with Spherical Harmonics and Sinusoidal Representation Networks, these networks automatically discover optimal frequency-based embeddings, allowing them to capture fine-grained spatial patterns that flat-map projections miss.
  2. Infinite Differentiability: The derivative of a sine wave is a cosine wave (another sine wave with a phase shift). This means the network's derivatives are as well-behaved as the network itself. For physics-based tasks, where the loss function depends on gradients (like the Laplacian in fluid dynamics), this property is transformative.

Capturing High-Frequency Physics

The ability to represent complex derivatives makes periodic activations the engine of choice for modern physics-informed models. In Hamiltonian Neural PDE Solvers through Functional Approximation, Anthony Zhou and Amir Barati Farimani use sinusoidal activations to compute higher-order functional derivatives, ensuring that physical simulations conserve energy and remain stable over time.

Similarly, in the study of wavefields, Tariq Alkhalifah and Xinquan Huang show that integrating wave-specific basis functions—like the product of Gaussian envelopes and sinusoidal waves—allows networks to solve the Helmholtz equation without the low-frequency bias that plagues traditional PINNs (Physics-informed neural wavefields with Gabor basis functions).

From Medical Imaging to Generative Art

The versatility of this approach is evident in its rapid adoption across disparate fields:

  • 3D Vision: Eric R. Chan and colleagues developed pi-GAN, which uses periodic activations to synthesize 3D-consistent images with fine-grained textures like hair and skin.
  • Medical Diagnostics: Researchers like Simone Saitta have applied these networks to 4D flow MRI, treating blood flow as a continuous function of space and time to reduce measurement errors by 50%.
  • Biological Signals: In Leveraging sinusoidal representation networks to predict fMRI signals from EEG, Yamin Li demonstrates that periodic activations are uniquely suited to the oscillatory nature of brain dynamics, outperforming standard units in predicting subcortical activity.

The Evolving Landscape of Activations

While the sine function is the most prominent tool in this research direction, the field is expanding toward other specialized activations. Lizhong Zhao and B. Aditya Prakash introduced PINNsFormer, which uses wavelet-based activations to create a smoother optimization landscape for solving PDEs. Others are exploring Gabor filters and learnable periodic terms to handle multiscale oscillations.

Regardless of the specific function used, the core intuition remains the same: to represent the continuous world accurately, our neural networks must speak the language of frequencies and derivatives, moving beyond the "blurry" constraints of piecewise-linear logic.

Papers in this version

Showing 60 of 256 papers.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 122 papers
  2. Version 1 · 2026-10-04 · 256 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_periodic_activation_functions_sinusoidal_2c6f12ee,
  title = {periodic activation functions (sinusoidal representation networks)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/periodic-activation-functions-sinusoidal-representation-networks/rdr_2c6f12ee390b43d5b7891463d30c4060},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_periodic_activations_for_implicit_neural_2c6f12ee_v1,
  title = {Periodic Activations for Implicit Neural Representations},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/periodic-activations-for-implicit-neural-representations/rdr_2c6f12ee390b43d5b7891463d30c4060/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}