TL;DR
This paper introduces FIRM, a novel "universal" and "objective" feature importance measure that can be applied to any machine learning model, unlike previous methods restricted to specific model types. FIRM goes beyond simple feature weighting by considering feature correlations, allowing it to identify truly important features even when they are not explicitly represented in the model or are obscured by noise.
What Problem Does This Paper Address?
A core goal in machine learning, beyond just accurate predictions, is understanding why a model makes certain predictions. This is particularly crucial for researchers in application domains who need to interpret model behavior and gain insights into their data. While many methods exist to identify "important" features, they often come with significant limitations:
- Linear Models Only: Many methods are restricted to linear models or kernel expansions, failing to provide insights for more complex, non-linear models.
- Feature Selection Instability: Methods that enforce sparsity (e.g., Lasso) can be unstable, meaning a slightly different subset of features might yield the same accuracy, making it hard to pinpoint true importance.
- Rescaling Sensitivity: Simple feature weighting can be easily manipulated by simply rescaling input features, leading to arbitrary changes in perceived importance without affecting the model's predictions.
- Ignoring Correlations: Most methods treat features independently, failing to account for how correlated features might influence each other and the overall prediction.
This paper aims to bridge this gap by proposing a feature importance measure that is universal (works with any model), objective (invariant to irrelevant transformations), and intelligent (accounts for feature correlations).
Methods
The authors introduce the Feature Importance Ranking Measure (FIRM), which operates in two steps:
- Conditional Expected Score ($q_f(t)$): For a given feature $f$ and its value $t$, FIRM first calculates the expected score of the learning machine's output $s(X)$ given that the feature $f$ takes on the value $t$. This is denoted as $q_f(t) = E[s(X) | f(X) = t]$. A feature is considered important if this conditional expectation changes significantly as $t$ varies.
- Feature Importance ($Q_f$): FIRM then quantifies the importance of feature $f$ by measuring the variability of its conditional expected score function $q_f(t)$. Specifically, it uses the standard deviation of $q_f(t)$ across the distribution of $f(X)$: $Q_f = \sqrt{Var[q_f(f(X))]}$.
Key aspects of FIRM:
- Universality: It can be applied to any learning machine that provides a real-valued output function, regardless of whether it's linear, kernel-based, or something else.
- Objectivity: It is designed to be invariant to irrelevant transformations like feature translation and, in many cases, feature rescaling.
- Intelligence: FIRM explicitly accounts for the correlation structure between features. This allows it to identify features that might be implicitly represented or that are important due to their relationship with other features, even if they aren't directly used by the learning machine.
- Approximations: For practical computation, especially with normally distributed features, FIRM can be approximated using first-order Taylor expansions, requiring only the covariance structure of the inputs.
Datasets
The paper uses synthetic datasets for illustration and evaluation:
- Boolean Formula Data: A toy example involving learning a Boolean formula ($x_1 \lor (\neg x_1 \land \neg x_2)$) using an SVM with a polynomial kernel.
- Gaussian Data: A binary classification task with three real-valued dimensions, where the first two dimensions are discriminative and the third is random noise. A linear SVM classifier is trained.
- Sequence Data: A toy dataset of random DNA sequences with a planted motif (GATTACA) that is subject to mutations. An SVM with a WDS kernel is used.
The paper does not introduce new benchmark datasets.
Key Insights and Ideas
- Beyond Simple Weights: FIRM's core innovation is moving beyond simple feature weights. By considering how a feature changes the expected output, it captures a more nuanced understanding of importance.
- Correlation is Key: The ability of FIRM to incorporate feature correlations is a major advantage. This allows it to disentangle the effects of related features and avoid the instability often seen in methods that treat features independently.
- "Intelligent" Feature Discovery: FIRM can identify the importance of features that are not explicitly part of the model's internal representation (e.g., higher-order polynomial features for a linear model, or motifs longer than those explicitly searched for in sequence data).
- Universality as a Game-Changer: The ability to apply a single, consistent importance measure across vastly different machine learning models (linear, kernel, etc.) is a significant step towards unified model interpretation.
- Robustness to Noise and Mutation: FIRM demonstrates a remarkable ability to identify true underlying patterns even when data is noisy (Gaussian example) or when the "true" signal is mutated (sequence example), outperforming traditional feature weighting.
Key Results
- Boolean Data: FIRM correctly identifies the most important features for the Boolean formula, aligning better with the underlying logic than the raw SVM weights, which were biased by the kernel's structure.
- Gaussian Data: FIRM successfully identifies the two discriminative dimensions as important and the third (noise) dimension as uninformative, mirroring the true data structure.
- Sequence Data: FIRM clearly identifies the GATTACA motif and its slightly mutated variants as important, while raw SVM weights fail to distinguish the true motif from random sequences. This highlights FIRM's ability to handle noisy or imperfect signals.
- Theoretical Properties: FIRM is shown to be invariant to feature translation and reasonably invariant to feature rescaling, making it more objective than many existing methods.
Limitations and Future Work
- Distribution Dependency: FIRM's definition relies on the distribution of input features, which is generally unknown. The paper proposes approximations that can be computed from data, but their accuracy depends on how well the underlying assumptions (e.g., normality) are met.
- Computational Cost: While approximations exist, computing FIRM can still be computationally intensive, especially for complex models or high-dimensional data.
- Future Work: The authors suggest further research into exploiting FIRM's ability to smooth away label errors, extend the set of labeled data to the entire space, and explicitly utilize distributional information. They also propose exploring how FIRM can be used without a prior learning step, directly on raw training data.
Summary updated