Introduction
In modern data science, uncertainty is frequently relegated to a final step—often represented simply as error bars on a completed analysis. However, many real-world datasets are inherently uncertain and are better modeled as probability distributions rather than static points. While existing visualization frameworks like D3 or Matplotlib excel at rendering final outputs, they lack integrated support for the intermediate stages of the data pipeline, such as dimensionality reduction or complex transformations. When standard techniques like Principal Component Analysis (PCA) are applied to uncertain data, the underlying distribution often "collapses," losing vital context.
UADAPy addresses this gap by providing a comprehensive Python-based toolbox designed for the entire uncertainty-aware visualization pipeline. It enables researchers to model, transform, and render data while preserving the critical nuances of its underlying probability distributions.
The UADAPy Architecture
The cornerstone of UADAPy is its Distribution class. This class provides a unified interface for various types of uncertain data, whether represented through Gaussian models or Kernel Density Estimation (KDE). By centering the library around a "Distribution" object, the authors ensure that any operation—be it scaling, rotation, or projection—is applied directly to the probability model rather than just a point estimate.
The toolbox follows a structured workflow:
- Create: Users can generate distributions from raw samples or define them using mathematical models.
- Transform: High-dimensional distributions are processed using specialized algorithms.
- Visualize: The resulting data is mapped to visual representations based on its dimensionality.
Uncertainty-Aware Transformations and Visual Mapping
A key technical contribution of UADAPy is the integration of Uncertainty-Aware Principal Component Analysis (UAPCA) and Uncertainty-Aware Multidimensional Scaling (UAMDS). Unlike traditional dimensionality reduction, these methods project high-dimensional probability distributions into 2D space while maintaining the "shape" and spread of the uncertainty.
The toolbox offers a hierarchy of visual mappings tailored to the data's dimensionality:
- 1D Visualizations: Includes standard but uncertainty-focused plots like box plots, violin plots, and swarm plots.
- 2D Visualizations: Utilizes scatter plots for samples and isolines or isobands to represent distribution quantiles.
- nD Visualizations: Employs plot matrices (small multiples) to display pairs of dimensions or univariate distributions.
The API is designed for accessibility, allowing complex uncertainty-aware visualizations to be generated with minimal code, thus lowering the barrier for researchers who are not visualization experts.
Key Insights and Results
UADAPy successfully bridges the gap between scientific computing (SciPy) and rendering (Matplotlib). By providing a consistent API for disparate methods like UAPCA and UAMDS, it eliminates the need for researchers to implement these complex algorithms from scratch.
The toolbox includes a curated collection of example datasets from landmark uncertainty visualization publications, providing a "ready-to-use" environment for testing new features. The primary achievement is the release of a functional, integrated software package available via GitHub and PyPI, which demonstrates that n-dimensional uncertain samples can be modeled, reduced in dimensionality, and visualized as contours within a single Python environment.
Limitations and Future Directions
While UADAPy provides a robust foundation, several areas for expansion remain:
- Interactivity: The current reliance on Matplotlib favors static, publication-quality figures. Future versions aim to support interactive frameworks like Bokeh or Plotly.
- Temporal Data: Support for uncertain time series is under development, specifically through Uncertainty-Aware Seasonal-Trend Decomposition (UASTL).
- Ecosystem Integration: The authors plan to provide R bindings to better serve the statistics community and are encouraging the integration of more community-developed uncertainty-aware algorithms.
Project Link: https://github.com/UniStuttgart-VISUS/uadapy
Summary updated