Beyond Deep Learning: An Evolutionary Feature Engineering Approach to Tabular Data Classification

Hengzhe Zhang, Qi Chen, Aimin Zhou, bing xue, Yan Wang, mengjie zhangICLR 2023 · OpenReview

Abstract

In recent years, deep learning has achieved impressive performance in the computer vision and natural language processing domains. In the tabular data classification scenario, with the emergence of the transformer architecture, a number of algorithms have been reported to yield better results than conventional tree-based models. Most of these methods attribute the success of deep learning methods to the expressive feature construction capability of neural networks. Nonetheless, in real practice, manually designed high-order features with traditional machine learning methods are still widely used because neural-network-based features can be easy to over-fitting. In this paper, we propose an evolution-based feature engineering algorithm to imitate the manual feature construction process through trial and improvement. Importantly, the evolutionary method provides an opportunity to optimize cross-validation loss, where gradient methods fail to do so. On a large-scale classification benchmark of 119 datasets, the experimental results demonstrate that the proposed method outperforms existing fine-tuned state-of-the-art tree-based and deep-learning-based classification algorithms.
BibTeX
@inproceedings{zhang2023beyond,
  author = {Hengzhe Zhang and Qi Chen and Aimin Zhou and bing xue and Yan Wang and mengjie zhang},
  title = {{Beyond Deep Learning: An Evolutionary Feature Engineering Approach to Tabular Data Classification}},
  booktitle = {International Conference on Learning Representations},
  year = {2023},
  url = {https://openreview.net/forum?id=3C9Eqd0hCrr},
}

Loading related content...

Figure