SELA: Tree-Search Enhanced LLM Agents for Automated Machine Learning

Yizhou Chi, Yizhang Lin, Duyi Pan, Bangbang Liu, Guanghao Mei, Yaying Fei, Tianqi Pang, Sirui Hong, Jacky Kwok, Ceyao Zhang, Chenglin WuSubmission to ICLR 2025 · Rejected

TL;DR

SELA (Tree-Search Enhanced LLM Agents) combines large language model (LLM) agents with Monte Carlo Tree Search (MCTS) to systematically explore and refine machine learning pipelines. By structuring pipeline generation into a stage-wise decision tree and guiding search via experimental feedback, SELA outperforms both traditional AutoML systems and existing LLM agents across 20 diverse tabular datasets.

What Problem Does This Paper Address?

Automated Machine Learning (AutoML) traditionally relies on static search spaces—primarily optimizing hyperparameters and model ensembling (e.g., Auto-Sklearn, AutoGluon) while leaving data preprocessing and feature engineering largely underexplored.

Recent LLM-based agents attempt to automate end-to-end data science by generating code from natural language prompts. However, existing agents suffer from one of two extremes:

  1. Single-pass, stage-wise planning (e.g., Data Interpreter), which fails to iteratively refine solutions once executed.
  2. Monolithic iterative refinement (e.g., AIDE), which modifies an entire script greedily in one step without stage-specific structure.

Human experts do neither; they iterate through discrete pipeline stages—exploring hypotheses, running experiments, and backtracking based on intermediate performance. SELA bridges this gap by marrying stage-wise planning with guided tree search.

Methods

SELA decomposes the AutoML workflow into three interacting components:

[ Problem & Data Info ] ──> [ Insight Proposer (LLM) ] ──> Search Space (Tree)
                                                                 │
                                                                 ▼
[ Solution Output ] <── [ Backpropagation ] <── [ Executor Agent ] <── [ MCTS (UCT-DP) ]
  1. LLM Insight Proposer: Given dataset metadata and sample records, the LLM proposes candidate strategies across five ML stages: Exploratory Data Analysis, Data Preprocessing, Feature Engineering, Model Training, and Evaluation. These form a discrete search space pool ($\Lambda$).
  2. MCTS with Depth-Preferred UCT (UCT-DP): The search space is modeled as a tree where each depth level corresponds to an ML stage, and each branch represents an insight. Because training ML models is computationally expensive, SELA modifies the standard Upper Confidence Bound for Trees (UCT) into UCT-DP, using a hyperparameter ($\alpha_{\text{unvisited}}$) to prioritize reaching deeper nodes (evaluations) early rather than exhaustively expanding shallow nodes.
  3. Execution & Code Caching: An executor agent (based on Data Interpreter) translates selected paths into code and executes them sequentially. To prevent redundant computation and LLM variance, SELA caches code at the stage level. Subsequent branches reuse upstream stage code, making rollouts faster and cheaper.

Datasets

The authors evaluated SELA across 20 tabular datasets spanning classification (13) and regression (7), split 6:2:2 into train, validation, and test sets:

  • AutoML Benchmark (AMLB) / OpenML: Includes benchmarks like credit-g, Click_prediction_small, jasmine, mfeat-factors, boston, colleges, and diamonds.
  • Kaggle Competitions: Datasets including house-prices, titanic, smoker-status, and software-defects.

Evaluation used metric-appropriate targets: F1 / weighted F1 for classification and RMSE for regression (mapped into a 0–1 Normalized Score). No new datasets were introduced.

Key Insights and Ideas

  • Tree search over natural language insights, not raw code: Instead of searching over low-level code edits, the MCTS operates over high-level ML strategies proposed by an LLM, delegating tactical coding to an executor agent.
  • Stage caching slashes agent execution cost: Reusing code from parent nodes reduces token consumption and stabilizes execution. Counterintuitively, SELA ($0.05 per task over 10 rollouts) is *cheaper* than vanilla Data Interpreter ($0.07 per task) despite executing 10 search rollouts.
  • Search space structure matters as much as the search algorithm: In ablation experiments, even a random walk over the LLM-generated insight tree beat Data Interpreter (58.6% vs. 56.4% Normalized Score), while MCTS guided by validation feedback achieved the highest performance (60.9%).

Key Results

SELA was compared against traditional systems (AutoGluon, AutoSklearn) and agent frameworks (Data Interpreter, AIDE) using DeepSeek-V2.5 as the default base model:

  • Win Rates: SELA achieved a 65% to 80% win rate against each individual baseline across all 20 datasets.
  • Score and Ranking: SELA attained the highest Average Best Normalized Score (54.7%) and the best Average Best Rank (2.7), outperforming AutoGluon (4.4), AIDE (5.3), AutoSklearn (6.1), and Data Interpreter (6.4).
  • Top-1 Finishes: SELA produced the highest test score on 7 out of 20 datasets, compared to 4 for AutoGluon, 5 for AutoSklearn, and 2 each for AIDE and Data Interpreter.
  • LLM Agnostic: SELA generalized across LLM backbones, achieving strong Normalized Scores with Claude-3.5-Sonnet (57.9%) and GPT-4o (62.3%).

Limitations and Future Work

  • Tabular Scope: Evaluation was restricted strictly to tabular classification and regression. The authors did not assess multimodal, NLP, or computer vision tasks.
  • Search Horizon: Experiments were constrained to 10 rollouts per dataset due to compute limits. The scalability of UCT-DP over larger rollout budgets and wider insight spaces remains unverified.
  • Interpretability: The framework lacks automated mechanisms to explain why specific paths succeeded or failed beyond scalar validation scores. Future work aims to incorporate explicit natural-language post-hoc explanations and extend the tree-search formulation to broader domains like scientific discovery and robotics.

Summary updated

Abstract

Automated Machine Learning (AutoML) approaches encompass traditional methods that optimize fixed pipelines for model selection and ensembling, as well as newer LLM-based frameworks that autonomously build pipelines. While LLM-based agents have shown promise in automating machine learning tasks, they often generate low-diversity and suboptimal code, even after multiple iterations. To overcome these limitations, we introduce Tree-$\textbf{S}$earch $\textbf{E}$nhanced $\textbf{L}$LM $\textbf{A}$gents ($\textbf{SELA}$), an innovative agent-based system that leverages Monte Carlo Tree Search (MCTS) to optimize the AutoML process. By representing pipeline configurations as trees, our framework enables agents to conduct experiments intelligently and iteratively refine their strategies, facilitating a more effective exploration of the machine learning solution space. This novel approach allows SELA to discover optimal pathways based on experimental feedback, improving the overall quality of the solutions. In an extensive evaluation across 20 machine learning datasets, we compare the performance of traditional and agent-based AutoML methods, demonstrating that SELA achieves a win rate of 65\% to 80\% against each baseline across all datasets. These results underscore the significant potential of agent-based strategies in AutoML, offering a fresh perspective on tackling complex machine learning challenges. The code will be open-sourced upon publication.
BibTeX
@misc{chi2024sela,
  author = {Yizhou Chi and Yizhang Lin and Duyi Pan and Bangbang Liu and Guanghao Mei and Yaying Fei and Tianqi Pang and Sirui Hong and Jacky Kwok and Bang Liu and Ceyao Zhang and Chenglin Wu},
  title = {{SELA: Tree-Search Enhanced LLM Agents for Automated Machine Learning}},
  year = {2024},
  eprint = {2410.17238},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2410.17238},
}
Figure