Optimizing the Information Retrieval Trade-off in Data Visualization Using $α$-Divergence

Ehsan Amid, Onur Dikmen, Erkki Oja

Introduction

Dimensionality reduction is a fundamental task in data science, yet it inherently involves a trade-off between two competing Information Retrieval (IR) goals: recall (preserving high-dimensional neighbors in the low-dimensional map) and precision (ensuring low-dimensional proximity reflects true high-dimensional similarity). While existing methods like Stochastic Neighbor Embedding (SNE) prioritize recall, they often suffer from "crowding." Conversely, methods that allow tuning this trade-off, such as NeRV, require manual parameter selection. This paper introduces $\alpha$-SNE, a framework that automates this trade-off using $\alpha$-divergence and a statistical framework called Exponential Divergence with Augmentation (EDA).

The $\alpha$-SNE Objective

The core innovation lies in replacing the standard Kullback-Leibler (KL) divergence with the more general $\alpha$-divergence. The authors demonstrate that minimizing $\alpha$-divergence is mathematically equivalent to maximizing the geometric mean of precision and recall. This provides a more robust and principled objective than the arithmetic weighted sums used in previous literature, as the geometric mean offers a tighter bound on overall visualization quality.

Gradient Analysis and Force Balancing

A critical aspect of visualization performance is the "gradient" or the forces that move points during optimization. Standard SNE is characterized by strong attractive forces that can lead to cluster collapse. In contrast, $\alpha$-SNE provides a more balanced interplay between attraction and repulsion.

Comparison of gradients between standard SNE (left) and $\alpha$-SNE (right).

As shown in the gradient plots, $\alpha$-SNE (with $\alpha=0.8$) exhibits a more nuanced force distribution. This prevents the "crowding" effect common in SNE without resulting in the "over-separation" sometimes observed in t-SNE, leading to cleaner and more interpretable data representations.

Automated Parameter Tuning via EDA

To eliminate the "guess-and-check" nature of parameter tuning, the authors introduce Exponential Divergence with Augmentation (EDA). By treating the visualization as a statistical estimation problem, the EDA framework—which approximates the Tweedie distribution—can automatically estimate the optimal $\alpha$ value for a specific dataset.

The EDA framework identifies a clear optimal $\alpha$ for the Texture dataset.

The framework analyzes the distribution of the input data to find the $\alpha$ that yields the most faithful representation. Empirical results show that the $\alpha$ values selected by EDA are remarkably close to the "best" values found through exhaustive manual searching.

Results and Performance

The researchers evaluated $\alpha$-SNE against 13 diverse datasets, including MNIST, Olivetti Faces, and various specialized image datasets. Using the Area Under the ROC Curve (AUC) as a holistic measure of precision and recall, $\alpha$-SNE consistently outperformed established methods like PCA, Isomap, t-SNE, and NeRV across all benchmarks. On the "Texture" dataset specifically, $\alpha$-SNE produced well-separated clusters that still reflected their true proximity, avoiding the isolated "islands" often created by t-SNE.

Conclusion

$\alpha$-SNE represents a significant step forward in data-driven visualization. By leveraging $\alpha$-divergence and the EDA framework, it provides a mathematically grounded way to balance precision and recall automatically. Future work will explore incorporating heavy-tailed distributions to further enhance cluster separation and optimizing computational efficiency for larger datasets.

Summary updated

Abstract

Data visualization is one of the major applications of nonlinear dimensionality reduction. From the information retrieval perspective, the quality of a visualization can be evaluated by considering the extent that the neighborhood relation of each data point is maintained while the number of unrelated points that are retrieved is minimized. This property can be quantified as a trade-off between the mean precision and mean recall of the visualization. While there have been some approaches to formulate the visualization objective directly as a weighted sum of the precision and recall, there is no systematic way to determine the optimal trade-off between these two nor a clear interpretation of the optimal value. In this paper, we investigate the properties of $α$-divergence for information visualization, focusing our attention on a particular range of $α$ values. We show that the minimization of the new cost function corresponds to maximizing a geometric mean between precision and recall, parameterized by $α$. Contrary to some earlier methods, no hand-tuning is needed, but we can rigorously estimate the optimal value of $α$ for a given input data. For this, we provide a statistical framework using a novel distribution called Exponential Divergence with Augmentation (EDA). By the extensive set of experiments, we show that the optimal value of $α$, obtained by EDA corresponds to the optimal trade-off between the precision and recall for a given data distribution.
BibTeX
@misc{amid2015optimizing,
  author = {Ehsan Amid and Onur Dikmen and Erkki Oja},
  title = {{Optimizing the Information Retrieval Trade-off in Data Visualization Using $α$-Divergence}},
  year = {2015},
  eprint = {1505.05821},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1505.05821},
}

Loading related content...

Figure