CLaRe: Compact near-lossless Latent Representations of High-Dimensional Object Data

Emma Zohner, Edward Gunning, Giles Hooker, Jeffrey S. MorrisarXiv 2025 · DOI · arXiv

Abstract

Latent feature representation methods play an important role in the dimension reduction and statistical modeling of high-dimensional complex data objects. However, existing approaches to assess the quality of these methods often rely on aggregated statistics that reflect the central tendency of the distribution of information losses, such as average or total loss, which can mask variation across individual observations. We argue that controlling average performance is insufficient to guarantee that statistical analysis in the latent space reflects the data-generating process and instead advocate for controlling the worst-case generalization error, or a tail quantile of the generalization error distribution. Our framework, CLaRe (Compact near-lossless Latent Representations), introduces a systematic way to balance compactness of the representation with preservation of information when assessing and selecting among latent feature representation methods. To facilitate the application of the CLaRe framework, we have developed GLaRe (Graphical Analysis of Latent Representations), an open-source R package that implements the framework and provides graphical summaries of the full generalization error distribution. We demonstrate the utility of CLaRe through three case studies on high-dimensional datasets from diverse fields of application.

The Problem with Average Performance

When compressing high-dimensional data—such as medical images or genomic sequences—into a lower-dimensional latent space using techniques like Principal Component Analysis (PCA) or Autoencoders, success is traditionally measured by average reconstruction error.

However, relying solely on central tendency is a dangerous oversight. An excellent average error can mask a subset of data that is reconstructed poorly. If these latent features are subsequently used for downstream tasks, such as disease prediction or object classification, the resulting models may fail significantly for those poorly represented individuals. To ensure a latent representation is truly "near-lossless," researchers must examine the entire distribution of errors and guarantee that a high percentage of observations (the "attainment rate") fall below a specific tolerance level ($\epsilon$).

Comparison of average information loss versus the distribution of individual losses, highlighting how average metrics can mask poor worst-case performance.

The CLaRe Framework

The authors introduce CLaRe (Compact near-lossless Latent Representations) and an accompanying R package, GLaRe (Graphical Analysis of Latent Representations), to address these limitations. The framework shifts the focus from aggregate scores to individual-level reliability through three core components:

  1. Cross-Validation for Individuals: Instead of a single aggregate score, the framework uses cross-validation to estimate the reconstruction error for every individual observation in the dataset.
  2. The Qualifying Dimension ($qd$): This metric is defined as the smallest number of latent features required to ensure that a user-defined proportion of the data (e.g., 95%) meets a specific error tolerance ($\epsilon$).
  3. Visualization Suite: The GLaRe package provides "Summary Plots" (tracking min, max, and quantiles of error), dotplots of the full distribution, and heatmaps to visualize the error landscape across different dimensions.
The GLaRe package provides a suite of visualizations, including summary statistics, dotplots of individual losses, and heatmaps of the error distribution.

Case Studies and Comparative Analysis

The paper evaluates the framework across four diverse high-dimensional datasets: Glaucoma images (mechanical strain), Proteomic Gels (electrophoresis images), MNIST handwritten digits, and Phoneme audio signals.

Key Insights

  • Methodological Variability: The optimal representation method depends heavily on the dataset. PCA excelled on Glaucoma data, while Autoencoders were superior for MNIST.
  • Flexibility vs. Sample Size: Flexible methods like Autoencoders and PCA learn structure from data. While powerful with large sample sizes ($N$), they struggle or overfit when $N$ is small relative to the number of features ($T$).
  • The Utility of Fixed Bases: For the Proteomic Gels dataset (high $T$, low $N$), the Discrete Wavelet Transform (DWT) outperformed learned methods. Because DWT uses a fixed mathematical basis, it remains effective even with limited samples.
  • Visualizing the Tail: At low dimensions, "average" error may appear acceptable while "worst-case" error remains near 1.0, indicating zero information retention for specific samples.

Results Summary

  • Glaucoma: PCA achieved the qualifying criterion with 41 features (351:1 compression). DWT and Autoencoders failed to meet the criteria efficiently.
  • Proteomic Gels: Only DWT met the near-lossless criteria, requiring 7,801 features (71:1 compression). PCA was limited by the small sample size ($N=53$).
  • MNIST: Autoencoders were most compact ($qd=101$), outperforming PCA ($qd=201$) and DWT ($qd=321$).

Discussion and Limitations

The current CLaRe framework assumes independent observations. Future work is needed to extend the framework to dependent data, such as longitudinal or spatial observations. Additionally, while CLaRe prioritizes compactness and losslessness, it does not explicitly account for feature interpretability or the computational costs associated with training complex models like Autoencoders. Finally, in datasets with high white noise, achieving "near-lossless" status may be impossible; in such cases, CLaRe serves as a tool for quantifying the exact extent of information loss.

Software Availability: The GLaRe R package is available on GitHub at github.com/edwardgunning/GLaRe.

Summary updated

Abstract

Latent feature representation methods play an important role in the dimension reduction and statistical modeling of high-dimensional complex data objects. However, existing approaches to assess the quality of these methods often rely on aggregated statistics that reflect the central tendency of the distribution of information losses, such as average or total loss, which can mask variation across individual observations. We argue that controlling average performance is insufficient to guarantee that statistical analysis in the latent space reflects the data-generating process and instead advocate for controlling the worst-case generalization error, or a tail quantile of the generalization error distribution. Our framework, CLaRe (Compact near-lossless Latent Representations), introduces a systematic way to balance compactness of the representation with preservation of information when assessing and selecting among latent feature representation methods. To facilitate the application of the CLaRe framework, we have developed GLaRe (Graphical Analysis of Latent Representations), an open-source R package that implements the framework and provides graphical summaries of the full generalization error distribution. We demonstrate the utility of CLaRe through three case studies on high-dimensional datasets from diverse fields of application. We apply the CLaRe framework to select among principal components, wavelets and autoencoder representations for each dataset. The case studies reveal that the optimal latent feature representation varies depending on dataset characteristics, emphasizing the importance of a flexible evaluation framework.
BibTeX
@misc{zohner2025clare,
  author = {Emma Zohner and Edward Gunning and Giles Hooker and Jeffrey S. Morris},
  title = {{CLaRe: Compact near-lossless Latent Representations of High-Dimensional Object Data}},
  year = {2025},
  eprint = {2502.07084},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2502.07084},
  howpublished = {arXiv},
}
Figure