Overview
CoordX is a novel Multi-Layer Perceptron (MLP) architecture designed to significantly accelerate Implicit Neural Representations (INRs). By processing coordinate dimensions (such as $x, y,$ and $z$) independently before fusing them, this "split" approach achieves up to a 2.9x speedup in training and inference across images, videos, and 3D shapes, all while maintaining high reconstruction accuracy.
The Computational Tax of Implicit Neural Representations
Implicit Neural Representations have revolutionized how we store and render digital media. Instead of saving an image as a discrete grid of pixels, an MLP is trained to act as a continuous function: input a spatial coordinate $(x, y)$, and it outputs a color. This paradigm allows for infinite resolution and high memory efficiency.
However, this comes with a massive "computation tax." To render a single $1024 \times 1024$ image, the MLP must be queried over a million times. For 3D scenes (like NeRFs), this cost explodes, often requiring hours of training and seconds per frame for inference. While previous solutions have utilized complex data structures like octrees to skip "empty space," they often add memory overhead or are highly task-specific. CoordX tackles this bottleneck directly at the architectural level, aiming for a general-purpose speedup.
The CoordX Architecture: Splitting the Bill
The core innovation of CoordX is its Split MLP Architecture.
In a standard baseline model (coordMLP), every single coordinate point is treated as an independent input vector. CoordX observes that points along the same axis inherently share structural information.
The architecture optimizes processing through the following steps:
- Input Decomposition: Instead of feeding $(x, y)$ together, the model splits the input. One branch processes the $x$-coordinates, and another processes the $y$-coordinates.
- Split & Shared Layers: The initial layers are split into parallel branches. To keep the parameter count low, many of these layers are shared across branches, allowing the model to learn correlations between dimensions without doubling the weights.
- Feature Fusion (The Outer Product): To reconstruct the full signal, CoordX uses an outer product to fuse the features from the separate branches. This mathematically mirrors how a matrix can be decomposed into the product of two vectors (similar to Singular Value Decomposition), allowing the network to learn a compressed representation of the signal.
- Final Refinement: A few standard fully connected layers process the fused features to produce the final output (e.g., RGB color or 3D occupancy).
Because the initial layers process $H + W$ inputs instead of $H \times W$ inputs, the computational load drops dramatically.
Core Insights
- Dimensional Locality: Standard MLPs ignore the fact that neighboring pixels share coordinate values. By splitting the architecture, CoordX explicitly leverages this locality, treating an image more like a decomposable matrix than a flat list of isolated points.
- The Efficiency of Sums vs. Products: For an $H \times W$ image, CoordX reduces the primary input processing from $O(H \times W)$ to $O(H + W)$. As resolution increases, the speedup becomes exponentially more pronounced.
- Orthogonality: CoordX is an architectural change, not a data-structure change. This means it can be seamlessly combined with other optimization techniques (like sparse octrees or multi-scale decompositions) to stack even more performance gains.
Performance and Results
The authors evaluated CoordX across several standard benchmarks, including 2D RGB images (DIV2K), 10-second video clips, 3D shape occupancy networks (Stanford 3D Scanning Repository), and Novel View Synthesis (Blender scenes).
- Speedups: The architecture demonstrated a speedup of up to 2.92x for image tasks and 2.81x for 3D shape rendering.
- NeRF Acceleration: In a simplified NeRF model, CoordX achieved a 1.81x training speedup and a staggering 4.41x rendering speedup with less than a 0.5 dB drop in quality.
- Accuracy Retention: While there is a slight "accuracy tax" (roughly a 3dB PSNR drop in some high-frequency image tasks), this can be mitigated by increasing the "reduction dimension" ($R$), allowing users to trade a fraction of speed for higher fidelity.
- Memory: Unlike many acceleration methods, CoordX introduces no additional memory overhead.
Limitations and Future Directions
While highly effective for multi-dimensional data, CoordX provides no benefit for unidimensional signals (like audio) because there are no dimensions to split. Additionally, the decomposition approach can sometimes struggle to capture extremely fine, high-frequency details in images compared to a "brute-force" MLP.
Furthermore, standard NeRF uses irregular ray sampling. To make CoordX work in these environments, the authors utilized a grid-interpolation workaround. Future work could involve making the split architecture more natively flexible for non-grid-aligned data and exploring its application in even higher-dimensional data modeling beyond 3D and video.
Summary updated