Geometry Aware Evaluation for Omnidirectional Media

This direction is merged into another direction: equirectangular projection distortion (spherical projection distortion). Its last version stays readable and citable here.

Version 1 of 120 papers30 conceptsfrozen 2026-10-04

When we map the surface of the Earth onto a flat paper map, the regions near the poles appear massive while the equator looks compressed. This is the classic cartographic distortion of the equirectangular projection. In the world of 360-degree (omnidirectional) video and virtual reality, this distortion creates a fundamental problem: traditional computer vision metrics, designed for flat 2D images, become mathematically dishonest when applied to spherical data.

Researchers are moving away from standard pixel-wise comparisons toward geometry-aware evaluation, a framework that accounts for the non-uniform information density of spherical projections to better reflect how humans actually perceive immersive content.

The Failure of Flat Metrics

Standard metrics like Peak Signal-to-Noise Ratio (PSNR) or Structural Similarity (SSIM) treat every pixel in an image as equally important. However, in an equirectangular projection—the standard "unrolled" format for 360-degree content—pixels near the top and bottom edges represent much smaller physical areas on the sphere than pixels in the center.

As Roberto Gerson de Albuquerque Azevedo and colleagues explain in Visual Distortions in 360-degree Videos, this leads to a "pole bias." If a model produces a blur at the north pole of a panorama, a standard metric will penalize it heavily because that blur covers many pixels. In reality, when viewed through a VR headset, those pixels are pinched together into a tiny point, making the error almost invisible. Conversely, critical artifacts like stitching seams or viewport-specific rendering errors may be under-penalized if they occur in the less-sampled equatorial regions.

Restoring the Spherical Balance

To fix this, researchers have introduced spherical weighting. The intuition is simple: if a pixel represents a smaller area on the unit sphere, its contribution to the total error score should be scaled down proportionally.

In Study of 3D Virtual Reality Picture Quality, Meixu Chen and the LIVE lab team demonstrate that incorporating spherical geometry—often through weighted-to-sphere (WS) calculations—significantly improves the correlation between automated metrics and subjective human quality ratings. By applying a weight (typically based on the cosine of the latitude), these metrics ensure that the "stretched" poles do not disproportionately influence the final score.

This principle extends beyond quality assessment to model training. For instance, Hongying Liu and collaborators utilized a latitude-aware loss function in A Single Frame and Multi-Frame Joint Network for 360-degree Panorama Video Super-Resolution, forcing the neural network to prioritize visual fidelity in the regions that matter most to the viewer.

Evaluation via Reprojection

While weighting pixels helps, it doesn't solve the problem of "seams" or the fact that standard feature extractors (like those used in FID) are trained on rectilinear photographs, not warped panoramas. A second major strategy involves reprojecting the spherical data into undistorted views before evaluation.

  1. Cubemaps and Tangent Planes: Instead of looking at the warped equirectangular map, researchers project the sphere onto the faces of a cube or a series of flat tangent planes. Anders Christensen and Nooshin Mojab introduced OmniFID in Geometry Fidelity for Spherical Images, which uses cubemap projections to detect geometric distortions and field-of-view reductions that standard FID misses.
  2. Perceptual Viewports: Since humans only see a small portion of the 360-degree field at any given time, Mai Xu and others argue in their State-of-the-art survey that metrics should be "viewport-aware." This involves sampling the views a user would actually see while wearing a Head-Mounted Display (HMD), ensuring the evaluation aligns with the immersive experience.

Defining Metrics on the Manifold

The most robust approach is to move the math entirely off the 2D grid and onto the spherical manifold itself. This is particularly vital for tasks like object detection and tracking, where "bounding boxes" become curved shapes.

In Spherical Criteria for Fast and Accurate 360° Object Detection, Pengyu Zhao and colleagues define evaluation metrics directly on the spherical surface. By calculating the Intersection over Union (IoU) based on true surface area rather than pixel counts, they eliminate the geometric bias that makes standard detectors appear less accurate than they truly are. Similarly, Yi Zhang 0076 introduced a geometry-adapted structural similarity metric ($S_\alpha^{360}$) in SHD360 to accurately measure the consistency of human detection in panoramic videos.

By treating the sphere as a unique geometric entity rather than a distorted image, these researchers are building a more rigorous foundation for the next generation of immersive media. Readers can now trust that a "high quality" score for a VR scene actually corresponds to what they will see when they put on the headset.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_geometry_aware_evaluation_for_omnidirect_25131941,
  title = {Geometry Aware Evaluation for Omnidirectional Media},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/geometry-aware-evaluation-for-omnidirectional-media/rdr_251319410e874454b120e6f2dff4139c},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_geometry_aware_evaluation_for_omnidirect_25131941_v1,
  title = {Geometry Aware Evaluation for Omnidirectional Media},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/geometry-aware-evaluation-for-omnidirectional-media/rdr_251319410e874454b120e6f2dff4139c/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}