Abstract
Contains procedures for depth-based supervised learning, which are entirely non-parametric, in particular the DDalpha-procedure (Lange, Mosler and Mozharovskyi, 2014). The training data sample is transformed by a statistical depth function to a compact low-dimensional space, where the final classification is done. It also offers an extension to functional data and routines for calculating certain notions of statistical depth functions. 50 multivariate and 5 functional classification problems are included. (Pokotylo, Mozharovskyi and Dyckerhoff, 2019).
Introduction
The ddalpha package provides a comprehensive R-based toolkit for non-parametric classification using the concept of data depth. By transforming complex, high-dimensional data into a low-dimensional "depth space," it enables robust supervised learning for both multivariate and functional data (curves) without requiring strict parametric assumptions.
The Challenge of High-Dimensional Classification
In traditional supervised learning, many classifiers assume that data follows a specific distribution, such as a Gaussian distribution. However, real-world data is often asymmetric, heavy-tailed, or contains outliers, causing these parametric models to fail. While non-parametric methods offer a solution by making fewer assumptions, they often suffer from the "curse of dimensionality" or high computational costs.
The ddalpha package addresses these challenges by leveraging statistical data depth—a measure of how "central" a point is relative to a data cloud. The core innovation lies in efficiently calculating these depths and using them to build fast, robust classifiers for diverse data types, including functional data where observations are continuous curves.
Methodology: The DD$\alpha$-Procedure
The package centers on the DD$\alpha$-procedure, which follows a two-step workflow:
- Depth Transformation: Every data point in a $d$-dimensional space is transformed into a $q$-dimensional "depth vector," where $q$ is the number of classes. Each component represents the point's depth relative to one of the training classes. This maps the data into a compact unit cube $[0, 1]^q$, regardless of the original dimensionality.
- The $\alpha$-Procedure: In this simplified depth space, the algorithm constructs a linear separation rule, potentially using polynomial extensions for complex boundaries.
Functional Data Extension
For functional data, the package implements the DD$\alpha$f-classifier. This approach discretizes curves and applies functional depth notions (e.g., Band Depth or h-mode depth) before proceeding with the classification pipeline.
The training process for functional data involves specific parameters to handle the list-based structure of functional observations and options for data adjustment, such as PCA-based subspace projection.
Benchmarking and Datasets
A significant strength of ddalpha is its extensive library of built-in benchmarks:
- 50 Multivariate Datasets: Includes classic problems like
iris,diabetes, and banknote authentication. - 5 Functional Datasets: Real-world curve data such as the Berkeley growth study (
growth) and Drosophila gene expression (geneexp). - Simulated Data Generators: Tools for creating synthetic functional data to test model performance under controlled conditions.
Key Insights
- Dimension Reduction via Centrality: Classification is performed in a reduced "depth space" (often 2D or 3D), simplifying the problem significantly.
- Inherent Robustness: Geometric depth measures are naturally resistant to outliers.
- Outsider Handling: Points outside the training data's convex hull (zero depth) are managed via fallback methods like k-Nearest Neighbors (kNN) or LDA.
Results and Performance
Research by Pokotylo et al. (2019) demonstrates that the DD$\alpha$-classifier is highly competitive, often matching or exceeding the accuracy of state-of-the-art non-parametric classifiers across multivariate benchmarks. Furthermore, the package provides "DD-plots" for visual interpretation of class separation in depth space.
Limitations
While powerful, calculating exact depths (like Tukey depth) can be computationally intensive in very high dimensions. Additionally, the classifier's performance on "outsiders" depends heavily on the chosen fallback treatment. Future work involves integrating more modern depth notions and optimizing multi-class procedures.
Summary updated