Nonlinear Prediction of Multidimensional Signals via Deep Regression with Applications to Image Coding

Xi Zhang, Xiaolin Wu

Introduction

Sequential prediction is a cornerstone of image processing, particularly in lossless image coding. The core task involves estimating the value of a current pixel based on its "causal context"—the set of previously processed neighboring pixels. Traditionally, this task has been dominated by linear prediction models. While computationally efficient, linear models rely on assumptions of stationarity and Gaussianity that natural images, characterized by complex textures and sharp edges, frequently violate.

Recent advancements like the MDL-PAR model attempted to address these limitations using adaptive piecewise autoregression. However, such methods are computationally prohibitive, often requiring hours to process a single standard-definition image. This paper introduces PredNet, a deep convolutional neural network (DCNN) framework that leverages nonlinear regression to achieve superior prediction accuracy with significantly reduced computational overhead.

The PredNet Architecture

The authors propose a two-stage framework designed to capture the intricate, nonlinear dependencies within image data. The primary component is the PredNet, a DCNN architecture optimized for pixel-level regression.

Stage 1: Multi-Norm PredNets

The first stage consists of three separate PredNet models. Each model shares the same structural backbone but is trained using a different error norm to capture diverse statistical properties of the image signal:

  • $\ell_1$-norm: Optimized to minimize prediction entropy, the primary metric for compression efficiency.
  • $\ell_2$-norm: The standard least-squares approach for general-purpose prediction.
  • $\ell_\infty$-norm: Focused on minimizing the maximum possible error, ensuring robustness against outliers.
The PredNet architecture consists of a convolutional module with 16 residual units followed by a regression module for pixel value prediction.

As shown in the architecture diagram, the network utilizes a series of 16 residual units within a convolutional module to extract high-level features from the causal context. This is followed by a regression module that flattens the feature maps to produce a final predicted pixel value. A key technical innovation is the use of sparse regularization ($\ell_1$-norm on weights), which enables the network to automatically identify the most relevant pixels within the causal neighborhood, effectively solving the context selection problem.

Stage 2: Refinement Regression (PredNet-R)

The second stage introduces a refinement network, PredNet-R. This model takes the outputs from the three Stage 1 predictors as input and "fuses" them. By learning how to weigh these different perspectives based on local image content, PredNet-R produces a final prediction that is more precise than any single-norm model.

Experimental Results

The models were trained on millions of patches from high-resolution datasets (DIV2K, CLIC, and Flickr2K) and evaluated on the Kodak lossless image dataset.

Quantitative Performance

PredNet-R demonstrates a significant leap in compression performance. It achieved a self-entropy of 4.25 bits/pixel, surpassing the previous state-of-the-art MDL-PAR (4.40 bits/pixel) and standard linear predictors like GAP (4.69 bits/pixel).

Crucially, this performance gain is accompanied by a massive reduction in inference time. While MDL-PAR requires approximately 8 hours to process a 512×512 image, PredNet-R completes the same task in just 30 seconds. This 1,000x speedup shifts the computational burden from the end-user to the offline training phase, making high-end nonlinear prediction practical for the first time.

Qualitative Analysis

The effectiveness of the nonlinear approach is most visible in the "residual images"—the difference between the original image and the predicted version.

Comparison of residual images for 'motocross bikes' and 'window' scenes. PredNet-R (d) shows significantly less structural information in the residuals compared to GAP (b) and MDL-PAR (c).

As illustrated in the comparison above, the residuals produced by PredNet-R are remarkably "clean," containing almost no discernible structures or edges. This indicates that the model has successfully captured nearly all the meaningful information from the causal context, leaving behind only unpredictable noise. In contrast, the residuals from linear predictors like GAP retain significant structural artifacts, highlighting their inability to model complex image features.

Conclusion and Future Work

PredNet represents a paradigm shift in sequential signal prediction by replacing traditional linear optimization with deep nonlinear regression. By leveraging diverse loss functions and sparse regularization, the framework achieves state-of-the-art lossless compression results with practical execution times.

While 30 seconds per image is a vast improvement, further optimization is required for real-time applications. Future research directions include extending this framework to 1D audio and 3D video signals, as well as adapting the high-precision regression techniques for lossy compression scenarios.

Summary updated

BibTeX
@inproceedings{zhang2019nonlinear,
  author = {Xi Zhang and Xiaolin Wu},
  title = {{Nonlinear Prediction of Multidimensional Signals via Deep Regression with Applications to Image Coding}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2019},
  url = {https://openreview.net/forum?id=g3h6AUgzpe},
}
Figure