DiFA: Differentiable Feature Acquisition

Aritra Ghosh, Andrew S. LanAAAI 2023 · OpenReview

Abstract

Feature acquisition in predictive modeling is an important task in many practical applications. For example, in patient health prediction, we do not fully observe their personal features and need to dynamically select features to acquire. Our goal is to acquire a small subset of features that maximize prediction performance. Recently, some works reformulated feature acquisition as a Markov decision process and applied reinforcement learning (RL) algorithms, where the reward reflects both prediction performance and feature acquisition cost. However, RL algorithms only use zeroth-order information on the reward, which leads to slow empirical convergence, especially when there are many actions (number of features) to consider. For predictive modeling, it is possible to use first-order information on the reward, i.e., gradients, since we are often given an already collected dataset. Therefore, we propose differentiable feature acquisition (DiFA), which uses a differentiable representation of the feature selection policy to enable gradients to flow from the prediction loss to the policy parameters. We conduct extensive experiments on various real-world datasets and show that DiFA significantly outperforms existing feature acquisition methods when the number of features is large.

Introduction

In many real-world supervised learning scenarios, such as medical diagnostics or personalized education, acquiring all possible features for every data point is prohibitively expensive or impractical. Instead, a model must sequentially select the most informative features based on the information already gathered. This is known as the Dynamic Feature Acquisition task.

Previous approaches have largely relied on Reinforcement Learning (RL) to learn acquisition policies. However, RL methods are limited by their reliance on "zeroth-order" information—they observe a scalar reward (like final accuracy) and must infer how to improve through trial and error. This leads to slow convergence, particularly in high-dimensional spaces where the number of possible actions (features to acquire) is large. DiFA addresses this by making the acquisition process differentiable, allowing the use of "first-order" gradient information to optimize the policy directly.

Methodology

DiFA transforms the discrete feature selection problem into a differentiable framework, enabling end-to-end training via backpropagation. The framework consists of three primary components:

  1. Differentiable Selection: To bridge the gap between discrete feature acquisition and continuous optimization, DiFA utilizes the Gumbel-softmax operator in conjunction with a straight-through estimator. This allows gradients from the prediction loss to flow back through the selection masks to the policy parameters $\phi$.
  2. Imputation Model: Because the model operates on partial feature sets, it employs a Variational Autoencoder with Arbitrary Conditioning (VAEAC). This model "fills in" unobserved features, providing the policy with probabilistic side information (such as expected utility and variance) to inform the next acquisition step.
  3. End-to-End Training: The acquisition policy and the predictor are trained simultaneously. The policy learns to select features that specifically minimize the predictor's loss function.
Differentiable Feature Acquisition mechanism showing the flow of gradients from the prediction loss back to the policy parameters.

Experiments and Results

The authors evaluated DiFA against state-of-the-art baselines, including JAFA, GSMRL, and EDDI, across a variety of datasets:

  • Image Classification: MNIST, FashionMNIST, SVHN, and CIFAR10 (3072 features).
  • Tabular/Static Data: Grid (binary classification), Parkinson (regression), and PhysioNet Mortality (imbalanced classification).

Key Findings

  • Superior Efficiency: DiFA demonstrates significant sample efficiency. On FashionMNIST, DiFA achieved the same predictive performance using only 32 features that RL-based methods required 64 features to reach—a 50% reduction in acquisition cost.
  • Scalability to High Dimensions: While RL methods often struggle to converge as dimensionality increases, DiFA scales effectively. On the SVHN dataset with 128 features, DiFA reached ~94.5% accuracy, outperforming the best RL baseline by nearly 3%.
  • Consistent Performance: Across almost all benchmarks, DiFA (represented by the red line in performance plots) consistently maintains a performance lead or parity with existing methods, particularly as the number of acquired features $K$ increases.
Performance comparison of DiFA against baselines across multiple datasets, demonstrating superior accuracy and efficiency.

Conclusion

By leveraging first-order gradient information, DiFA overcomes the convergence bottlenecks inherent in RL-based feature acquisition. The use of a differentiable selection mechanism and a robust imputation model allows DiFA to achieve higher accuracy with fewer features, especially in high-dimensional tasks like image classification. Future work may involve jointly training the imputation model with the acquisition policy and extending the framework to handle temporal dynamics in time-series data.

Summary updated

Abstract

Feature acquisition in predictive modeling is an important task in many practical applications. For example, in patient health prediction, we do not fully observe their personal features and need to dynamically select features to acquire. Our goal is to acquire a small subset of features that maximize prediction performance. Recently, some works reformulated feature acquisition as a Markov decision process and applied reinforcement learning (RL) algorithms, where the reward reflects both prediction performance and feature acquisition cost. However, RL algorithms only use zeroth-order information on the reward, which leads to slow empirical convergence, especially when there are many actions (number of features) to consider. For predictive modeling, it is possible to use first-order information on the reward, i.e., gradients, since we are often given an already collected dataset. Therefore, we propose differentiable feature acquisition (DiFA), which uses a differentiable representation of the feature selection policy to enable gradients to flow from the prediction loss to the policy parameters. We conduct extensive experiments on various real-world datasets and show that DiFA significantly outperforms existing feature acquisition methods when the number of features is large.
BibTeX
@inproceedings{ghosh2023difa,
  author = {Aritra Ghosh and Andrew S. Lan},
  title = {{DiFA: Differentiable Feature Acquisition}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2023},
  url = {https://openreview.net/forum?id=bQyHNoXjEW},
}
Figure