Abstract
The mlr3filters package extends the mlr3 machine learning ecosystem by providing a comprehensive suite of filter methods for feature selection. Beyond standalone filter algorithms, it supports built-in importance methods from any machine-learning learner and enables partial scoring for computationally intensive multivariate filters. This unified framework facilitates the seamless integration and tuning of feature selection within automated ML pipelines.
Introduction
In high-dimensional machine learning tasks, feature selection is a critical prerequisite for building robust, interpretable models. By identifying the most relevant features, practitioners can reduce computational overhead, mitigate the "curse of dimensionality," and improve generalization performance. Filter methods are particularly advantageous in this context; they are computationally efficient and model-agnostic, scoring features based on intrinsic data properties rather than the performance of a specific downstream classifier.
However, the implementation of these methods in the R programming language has historically been fragmented across numerous packages with inconsistent APIs. mlr3filters addresses this challenge by providing a standardized, extensible framework within the mlr3 ecosystem. It allows for the easy comparison, benchmarking, and automated tuning of feature selection strategies.
Methodology
mlr3filters is built upon the R6 object-oriented programming paradigm, ensuring a consistent and modular interface. The package centralizes all available algorithms in the mlr_filters dictionary, allowing users to retrieve and initialize filters using simple helper functions like flt().
Algorithm Support and Extensibility
The package supports a diverse range of filtering techniques:
- Univariate Filters: Includes standard statistical metrics such as ANOVA F-tests, Pearson/Spearman correlation, and variance-based methods.
- Multivariate Filters: Integrates complex mutual information algorithms (e.g., MRMR, JMI, CMIM) via the
praznikpackage. These methods often employ greedy forward selection to rank features by their marginal contribution. - Embedded Importance: Any
mlr3learner with a native variable importance calculation (e.g., Random Forests or Gradient Boosting) can be wrapped and used as a filter.
Advanced Features
- Numerical Stability: For p-value based filters, the package automatically applies a $-log_{10}(p)$ transformation. This ensures that scores are numerically stable and that higher scores consistently represent better features.
- Partial Scoring: To address the computational cost of multivariate filters on high-dimensional data,
mlr3filterssupports anfeatargument. This ensures that only the top $N$ features are fully scored, optimizing performance. - Data Integrity: The package explicitly tracks the "missings" property of each filter, ensuring that users are aware of which algorithms can handle missing values without prior imputation.
Pipeline Integration
A core strength of mlr3filters is its integration with mlr3pipelines. By using the PipeOpFilter node, feature selection becomes a component of a larger computational graph. This allows the selection method and the number of features to be treated as hyperparameters that can be optimized via cross-validation or other tuning strategies.
Results and Applications
The package provides robust tools for various machine learning tasks:
- Classification and Regression: Supports standard metrics like Information Gain and AUC.
- Survival Analysis: Includes specialized filters such as the Correlation-Adjusted Survival (CARS) score and Univariate Cox models.
- Heuristic Methods: Integrates the Boruta algorithm, which uses shadow features to distinguish between "confirmed" and "tentative" feature importance.
- Predictive Performance Filter: Allows features to be scored based on the cross-validated performance of a simple proxy learner (e.g., a decision tree).
Conclusion and Future Work
mlr3filters provides a unified and scalable solution for feature selection in R. While multivariate methods can be computationally intensive and univariate methods may ignore complex feature interactions, the package's modular design and support for parallel processing provide a flexible foundation for modern ML workflows. Future developments will focus on expanding the mlr_filters dictionary to include emerging selection techniques and further enhancing integration with third-party libraries.
This paper summarizes the technical capabilities of the mlr3filters R package.
Summary updated