<title>Two-dimensional sequence compression using MPEG</title>

Ming-Hoe Kiu, Xiaosong Du, Robert Moorhead, David C. Banks, Raghu MachirajuProceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE 1998 · DOI

TL;DR

This paper proposes using the MPEG-2 video compression standard to compress 2D arrays of images for image-based rendering (IBR). By adapting MPEG-2's spatial and temporal compression techniques to a spatial grid of images, they significantly reduce storage requirements for IBR applications, making them more accessible on standard hardware.

What Problem Does This Paper Address?

Interactive image-based rendering (IBR) techniques allow users to navigate virtual environments by interpolating between pre-acquired images. While IBR offers computational advantages over traditional 3D model-based rendering, it suffers from a major drawback: the immense storage needed for the large number of pre-acquired images. Previous methods like vector quantization (VQ) offered some compression but were limited by I/O bottlenecks on standard workstations. This paper tackles the challenge of efficiently compressing these image datasets to make IBR practical for everyday users.

Methods

The core of this paper is the application of the MPEG-2 compression standard to a 2D grid of images, treating spatial relationships as analogous to temporal ones in video. Instead of encoding images sequentially in time, they are organized into "2D Groups of Pictures" (GOPs) that span across spatial dimensions.

Key techniques include:

  • MPEG-2 Compression: Leveraging MPEG-2's established methods for intra-frame (I-picture) and inter-frame (P- and B-picture) compression.
  • Spatial "Temporal" Redundancy: Exploiting similarities between adjacent images in the 2D grid, similar to how MPEG-2 exploits temporal redundancy in video frames.
  • 2D GOP Structures: Designing novel arrangements of I-, P-, and B-pictures within a 2D spatial context to optimize compression and reduce latency during interpolation. They explore different 2D GOP patterns (Figure 8) and analyze their performance.
  • Motion Compensation (Adapted): While MPEG-2 uses motion compensation for temporal changes, here it's adapted to predict differences between spatially adjacent images.
  • Discrete Cosine Transform (DCT): Used for spatial redundancy reduction within individual images.
  • Quantization and Entropy Coding: Standard MPEG-2 steps to further reduce data size.

Datasets

The paper evaluates their approach using two test images: "lion" and "buddha." These appear to be standard datasets for image compression research, as they are compared against results from a previous study [1] which also used these images. No new benchmarks are introduced.

Key Insights and Ideas

  • MPEG-2 is surprisingly effective for spatial compression: The paper demonstrates that MPEG-2's inherent ability to exploit redundancy can be effectively repurposed for spatial image arrays, achieving higher compression ratios than prior methods like VQ.
  • 2D GOP design is critical for IBR performance: The arrangement of I-, P-, and B-pictures in a 2D grid significantly impacts latency and the number of frames needed for interpolation. The authors identify specific 2D GOP patterns (like Figure 8a) that offer better performance.
  • Trade-off between compression and latency: While higher compression is desirable, the paper highlights that complex GOP structures or larger GOPs can increase latency when interpolating for new viewpoints, especially at boundaries.
  • Self-contained GOPs are preferred: GOPs where all necessary reference frames are within the same GOP (like Figure 8a-c) are shown to be more efficient than those requiring external references (like Figure 11).

Key Results

  • High Compression Ratios: The proposed MPEG-2 based scheme achieved compression ratios significantly higher than vector quantization. For the "lion" image, ratios of 164:1 to 165:1 were reported, compared to 118:1 for Levoy and Hanrahan [1]. For "buddha," ratios reached 56:1, compared to 45:1.
  • Comparable Image Quality: The Peak Signal-to-Noise Ratio (PSNR) of the reconstructed images was comparable to or slightly better than existing methods, indicating minimal image degradation. For "lion," PSNRs were around 29-31, and for "buddha," around 36-38.
  • Latency Analysis: The paper provides detailed tables (Tables 2, 3, and 4) analyzing the number of frames that need to be reconstructed for interpolation in various "jump-in" and progressive movement scenarios. This analysis helps in selecting optimal 2D GOP patterns for different navigation behaviors.

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that while MPEG-2 offers high compression, the computational cost of decoding (specifically the Inverse Discrete Cosine Transform - IDCT) can be higher than VQ. However, they argue this is mitigated by the availability of dedicated hardware.
  • The analysis of latency is extensive but focuses on specific scenarios. Real-world user navigation might present more complex movement patterns.
  • The paper primarily addresses "object-image sequences" and does not cover "panoramic-image sequences."
  • Future Work:
    • The authors suggest extending the work to handle panoramic image sequences.
  • Testing the approach on a wider variety of image sequences is recommended.
  • Further exploration of different 2D GOP structures and their impact on latency and compression is implied.

Summary updated

Abstract

This paper presents a technique to use the MPEG-2 compression scheme to compress a 2D array of images. The technique operates over two spatial dimensions, instead of the normal temporal dimension. This compression is crucial to interactive image-based rendering techniques, which allow virtual walk-throughs of texture-rich scenes. Since the MPEG schemes can achieve high compression with minimal image degradation, the large storage problems are reduced. This increase the applicability of image-based rendering on standard workstations or personal computers. Results are compared against previously presented compression schemes for image-based rendering.
BibTeX
@inproceedings{kiu1998lt,
  author = {Ming-Hoe Kiu and Xiaosong Du and Robert Moorhead and David C. Banks and Raghu Machiraju},
  title = {{\<title\>Two-dimensional sequence compression using MPEG\</title\>}},
  booktitle = {Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE},
  year = {1998},
  doi = {10.1117/12.298404},
}
Figure