Introduction
In fields ranging from bioinformatics to mobile health, researchers frequently encounter the "high $p$, low $n$" problem: datasets where the number of features ($p$) vastly exceeds the number of samples ($n$). This "curse of dimensionality" often leads to unstable feature selection, where standard algorithms like Chi-square or Mutual Information (MI) pick up noise rather than signal, resulting in poor model generalization. This paper introduces FS-FPG, a meta-feature selection approach that leverages the Frequent Pattern Growth (FP-Growth) algorithm to identify stable, high-performing feature sets.
The FS-FPG Methodology
The proposed FS-FPG algorithm acts as a consensus layer over traditional feature selection methods. The process involves five key stages:
- Multi-Method Selection: Five standard methods (Chi-square, MI, F-test, Random Forest importance, and Randomized Logistic Regression) are applied via 10-fold cross-validation.
- Normalization: Feature scores are scaled to a 0–1 range, and a threshold (0.5) is applied to retain top features.
- Transaction Mapping: Each set of top features from a specific fold and method is treated as a "transaction."
- FP-Growth Mining: The algorithm identifies "frequent itemsets"—groups of features that consistently appear together across different folds and methods.
- Filtering: Feature sets are refined based on support and confidence thresholds to ensure stability.
Experimental Evaluation
The study compared FS-FPG against standard baselines on two distinct datasets:
- D1 (High $p$, Low $n$): 7,731 features from 140 students (smartphone/Fitbit data) to predict loneliness scores.
- D2 (Low $p$, High $n$): 225 features from 3,880 instances (ExtraSensory Dataset) to predict physical activity.
Results and Discussion
The effectiveness of FS-FPG is highly dependent on the dataset's dimensionality ratio. In the sparse D1 dataset, standard methods performed poorly. For instance, as shown in the baseline results, using Mutual Information with a Support Vector Machine (SVM) yielded an accuracy of only 0.31.
By contrast, FS-FPG significantly improved performance on D1. The frequent feature set [f4, f1] boosted SVM accuracy to 0.73, while the set [f1, f2, f3] enabled XGBoost to reach 0.71. These results demonstrate that mining for feature synergy and stability can overcome the limitations of univariate selection in sparse data environments.
However, for the more balanced D2 dataset, FS-FPG showed diminishing returns. While standard methods achieved accuracies as high as 0.97 (e.g., XGBoost with MI), the FS-FPG sets maxed out at 0.70. This suggests that when sample sizes are sufficient, the meta-selection process may discard useful information that standard methods already successfully capture.
Conclusion
FS-FPG offers a robust solution for identifying a "democratic consensus" among features in extreme data-sparse environments. By mining frequent sets rather than individual features, it captures synergistic relationships that improve predictive power. Future work will focus on optimizing support and confidence thresholds to further refine model robustness across diverse high-dimensional domains.
Summary updated