A compact random-access representation for urban modeling and rendering

Zhengzheng Kuang, Bin Chan, Yizhou Yu, Wenping WangACM Trans. Graphics 2013 · DOI

TL;DR

This paper introduces Non-Uniform Textures (NUTs), a novel, highly memory-efficient representation for urban buildings that drastically reduces data size compared to traditional mesh models. NUTs enable real-time rendering of massive urban scenes on GPUs, paving the way for more detailed and interactive urban visualizations.

What Problem Does This Paper Address?

Visualizing large-scale urban environments presents a significant challenge due to the sheer volume of data required for detailed 3D models and high-resolution textures. Traditional mesh-based representations, while versatile, consume enormous amounts of memory and bandwidth, making it difficult to transmit and render complex cities in real-time, especially for applications like interactive fly-throughs or street-level exploration. Existing methods often compromise on detail for performance or struggle with the diversity of individual building designs, assuming repetitive elements which is not always realistic. The authors aim to bridge this gap by proposing a representation that is both compact and capable of capturing individual building details efficiently.

Methods

The core innovation is the Non-Uniform Texture (NUT) representation. Unlike conventional textures that use uniform pixel grids, NUTs employ an axis-aligned grid where pixel sizes can vary adaptively. This allows NUTs to align pixel boundaries with architectural features like corners and edges, significantly reducing the number of pixels needed to represent a building.

Key aspects of their method include:

  • Hierarchical Grid-Based Modeling: Buildings are represented as a hierarchical structure of 3D grids. This allows for efficient representation of repetitive elements at different levels, similar to how textures wrap.
  • Non-Uniform Texture (NUT): This is the fundamental data structure. It's essentially a 2D image encoding of 3D internal solid information, not just surface details. It stores coordinates, dimensions, and material properties in a specialized layout.
  • Random Access and Metadata Array: To enable efficient GPU lookup, NUTs use a metadata array. This array guides the shader to the correct pixel without requiring multiple texture lookups, a common bottleneck in traditional texture mapping. A heuristic approach is employed to balance accuracy and performance.
  • NUT Header and Data Component: The NUT structure is organized into a header (containing metadata like size, dimensions, and redirection information) and a data component (describing cell geometry, material, and texture mapping).
  • Texture Mapping: NUTs support limited texture mapping, primarily for decals or material textures. Textures are mapped to NUT cells with fixed physical dimensions, and a global texture atlas is used for efficient GPU access.
  • HNI Compilation: Multiple NUTs can be compiled into a single Hierarchical NUT Image (HNI) for efficient batch processing on the GPU. This involves a bin-packing algorithm to optimize the layout.
  • Semi-Automatic Mesh-to-NUT Converter: A tool is provided to convert existing mesh models into NUTs, involving solid conversion, block extraction, and simplification steps. This process aims to leverage the hierarchical nature of NUTs for compression.
  • GPU Ray Casting: The rendering is performed using a custom GPU ray casting shader that traverses the NUT hierarchy. This approach avoids mesh reconstruction and allows for efficient rendering directly on the GPU.

Datasets

The paper uses two primary urban scene datasets for evaluation:

  • A city with 5,061 high-rise buildings.
  • A European-style city with 40,400 distinct buildings.

These datasets are used to demonstrate the efficiency and scalability of the NUT representation compared to traditional methods. The authors do not introduce new benchmark datasets but rather use existing urban scene structures to showcase their method's performance.

Key Insights and Ideas

  • Breaking the Mesh Paradigm for Urban Scenes: The paper convincingly argues that for large-scale urban environments, the traditional mesh representation is fundamentally inefficient. NUTs offer a compelling alternative by leveraging the inherent geometric properties of buildings (orthogonality, repetition) in a fundamentally different way.
  • Image-Based Representation for 3D Geometry: The core idea of encoding 3D geometric and material information within 2D images (NUTs) is powerful. This allows for direct GPU processing without complex intermediate data structures, aligning well with modern graphics hardware.
  • Adaptive Pixel Grids are Key: The "non-uniform" aspect of NUTs is crucial. By allowing pixel sizes to adapt to feature boundaries, they achieve significant data compression compared to uniform grids, which is a major advantage for detailed urban models.
  • Hierarchical Structure for Scalability: The hierarchical nature of NUTs is essential for managing complexity. It allows for efficient Level-of-Detail (LOD) management and enables the representation of complex buildings from both close-up and distant views.
  • Real-time Rendering of Massive Cities: The demonstration of rendering tens of thousands of buildings at interactive frame rates is a significant achievement, highlighting the practical viability of NUTs for large-scale visualization.

Key Results

  • Massive Data Reduction: NUTs achieve data sizes that are "more than two orders of magnitude less" than conventional mesh representations. For example, a city model that would be 37GB as a mesh is compressed to 10.6MB PNG for NUTs.
  • High Rendering Performance: The proposed method achieves high frame rates, with examples showing 49 fps for a modern city and 30 fps for a larger European city at 1280x720 resolution on a GTX580 GPU.
  • Efficiency Gains: Comparisons show NUTs are significantly more space-efficient than instancing and mesh-based models. Rendering speed is comparable or better than other methods, especially for large scenes where mesh performance degrades.
  • Mesh-to-NUT Conversion: The semi-automatic converter can reduce data size by up to 98.2% for certain models.

Limitations and Future Work

  • Axis-Aligned and Regular Nature: The authors acknowledge that NUTs are not well-suited for highly irregular architectural structures (e.g., Sydney Opera House, Gherkin) or complex organic shapes like trusses, sky domes, or anticlastic tensile structures. Meshes remain a better choice for these.
  • Ray Casting Performance: While efficient, the ray casting approach is computationally intensive. Future work includes developing a geometric shader for on-the-fly NUT-to-mesh conversion and exploring adaptive ray casting for better anti-aliasing.
  • Mesh-to-NUT Conversion Limitations: The current converter primarily works with mesh models and can be sensitive to model cleanliness. Future research aims to develop methods for direct NUT construction from photos or laser scans.
  • Anti-aliasing: The current rendering relies on post-processing anti-aliasing. Developing an adaptive ray caster that directly supports anti-aliasing for NUTs is a desired future direction.

Summary updated

BibTeX
@article{kuang2013compact,
  author = {Zhengzheng Kuang and Bin Chan and Yizhou Yu and Wenping Wang},
  title = {{A compact random-access representation for urban modeling and rendering}},
  journal = {ACM Trans. Graphics},
  year = {2013},
  doi = {10.1145/2508363.2508424},
}
Figure