Introduction
In high-dimensional data analysis, identifying the most relevant features—Dimension Selection (DS)—is critical for interpretability, cost-effective experimental design, and model robustness. Unlike dimensionality reduction techniques like PCA, which transform data into abstract latent spaces, DS preserves the original features. This paper introduces a novel unsupervised DS framework that leverages Graph Signal Processing (GSP) and a synthetic "blue noise" probe to rank feature importance without requiring ground-truth labels.
Methodology
The proposed method treats a dataset as a $k$-nearest neighbor graph, where nodes represent individual samples. The core innovation is the use of a Blue Noise Probe, a synthetic signal defined in the graph's spectral domain.
The Blue Noise Probe
As shown in the spectral domain definition, the probe signal is designed to have zero energy in low frequencies (the "zero region") and constant energy in high frequencies. This makes the signal inherently "rough" or unpredictable relative to the graph's topology.
The Smoothness Hypothesis
The authors hypothesize that an important feature is one that significantly contributes to the underlying graph structure. When such a feature is masked (set to a constant), the graph's topology changes. If the feature was critical, this perturbation causes the previously "rough" blue noise signal to appear "smoother" relative to the new graph. Spectrally, this manifests as energy "leaking" into the previously empty low-frequency zero region.
Importance Scoring
The importance of a feature is quantified by measuring this spectral leakage. A high-reliability feature (one that is important) will show significant energy in the zero region after masking, whereas a low-reliability feature will show minimal change. Because each feature is evaluated independently against the same probe, the algorithm is highly parallelizable, offering significant speed advantages over iterative selection methods.
Experimental Evaluation
The method was benchmarked against state-of-the-art techniques using:
- PMLB (Penn Machine Learning Benchmark): Diverse datasets (e.g., Breast Cancer, Musk Clean1, Mfeat) with varying sample sizes and dimensions.
- MNIST: A subset of 1,000 images used to evaluate image topology recovery through pixel masking.
Key Results
The Blue Noise Spectrum approach demonstrated superior efficiency and accuracy across multiple tasks:
- Feature Efficiency: On the
Mfeat-Fourierdataset, the method achieved an F1-score of 0.8 using only 10 features, compared to 40 features required by Principal Coordinate Analysis. - Noise Filtering: In the
Musk Clean1dataset, it achieved a near-perfect F1-score (0.99) using just 3 features. Notably, performance was higher with these 3 selected features than with the full set of 168, indicating successful removal of "distractor" dimensions. - Topology Preservation: In image masking experiments on MNIST, the method was approximately 20% more accurate than existing dimension masking techniques at recovering the original neighborhood structure.
Conclusion and Future Work
By reframing dimension selection as a spectral perturbation problem, this work provides a robust, scalable, and unsupervised solution for high-dimensional data analysis. Future research will focus on the method's sensitivity to initial graph construction, automated tuning of the spectral "zero region" size, and applications in uncertainty quantification and label propagation.
Summary updated