Reinforcement Learning for Automated Feature Transformation

This direction is merged into another direction: combinatorial feature selection complexity. Its last version stays readable and citable here.

Version 1 of 115 papers15 conceptsfrozen 2026-10-04

In machine learning, the raw data we collect is rarely in the form most conducive to prediction. A model might struggle to learn from "Length" and "Width" individually, but easily grasp the concept of "Area" if we provide the product of the two. This process—feature transformation—is the art of constructing high-level features from base attributes using mathematical operations.

Historically, this was a manual "grind." Data scientists would use domain expertise to guess which combinations might work. However, as datasets grow in dimensionality, the number of possible transformations explodes. If you have $n$ features and a handful of mathematical operators, the search space for combinations grows exponentially, making exhaustive search computationally impossible.

To solve this, researchers have moved toward treating feature engineering not as a manual task, but as a search problem navigated by intelligent agents.

The Combinatorial Explosion

The fundamental challenge identified as early as 1989 by Christopher J. Matheus and Larry A. Rendell in Constructive Induction On Decision Trees is the "combinatorial explosion." When we allow features to be added, multiplied, or transformed by functions like $log$ or $sin$, the number of potential "candidate features" quickly exceeds the number of atoms in the universe.

Early systems like CITRE attempted to manage this by using domain knowledge to prune the search space, filtering out irrelevant combinations before they were even tested. But as data became more complex and domain knowledge less certain, the field needed a way to "learn" how to search.

Feature Generation as a Sequential Decision

Modern approaches frame feature construction as a Reinforcement Learning (RL) problem. Instead of trying every combination, an RL agent "walks" through the space of transformations, receiving a reward based on how much the new feature improves a downstream model's accuracy.

In Group-wise Reinforcement Feature Generation, Dongjie Wang and Yanjie Fu introduced the GRFG framework. Rather than treating every feature as an independent candidate—which is still too slow—they clustered features into groups. Cascading RL agents then decide which groups to "cross" and which operations to apply. This hierarchical approach allows the system to discover complex relationships (like interaction terms in a regression) without getting bogged down in the high-dimensional noise of individual variables.

While early RL methods focused on discrete choices (e.g., "Pick Feature A, then apply Plus, then Pick Feature B"), recent work has sought to make this search even more fluid.

  1. Continuous Optimization: In Reinforcement-Enhanced Autoregressive Feature Transformation, Dongjie Wang and Meng Xiao proposed MOAT. This system represents mathematical transformations as "postfix expressions" (like $x_1 x_2 +$) and maps them into a continuous embedding space. This allows the agent to use gradient-based search to find the "perfect" transformation, effectively smoothing out a previously jagged, discrete search landscape.
  2. Evolutionary LLMs: Nanxu Gong and Chandan K. Reddy have pushed the boundary further by incorporating Large Language Models (LLMs). In Evolutionary Large Language Model for Automated Feature Transformation, they use an LLM to "reason" about which features might be related, using those insights to seed an evolutionary algorithm. This combines the "intuition" of a language model with the iterative rigor of evolution.

Why It Works: Efficiency and Traceability

The goal of these systems isn't just to find any feature that works, but to do so efficiently and transparently.

Efficiency is gained through techniques like sample compression. In Toward Efficient Automated Feature Engineering, Kafeng Wang demonstrated that using MinHash to compress data samples allows agents to evaluate the "quality" of a new feature much faster, leading to a 2x speedup in discovery.

Traceability is equally vital, especially in scientific domains. Unlike "black-box" deep learning where features are hidden in neural layers, the features generated by frameworks like MAFG (Reinforcement Learning-based Feature Generation Algorithm for Scientific Data) are explicit mathematical formulas. As Meng Xiao notes, this allows researchers to not only improve their models but also understand the underlying scientific relationships the AI has uncovered.

By automating the navigation of the combinatorial explosion, these frameworks transform feature engineering from a manual bottleneck into a self-optimizing engine for discovery.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_reinforcement_learning_for_automated_fea_a3402bcd,
  title = {Reinforcement Learning for Automated Feature Transformation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/reinforcement-learning-for-automated-feature-transformation/rdr_a3402bcd6a7248ea8fc748f4be983ffa},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_reinforcement_learning_for_automated_fea_a3402bcd_v1,
  title = {Reinforcement Learning for Automated Feature Transformation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/reinforcement-learning-for-automated-feature-transformation/rdr_a3402bcd6a7248ea8fc748f4be983ffa/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}