Tabular Foundation Models in Financial Predictions

Ed deHaan, Allen H. HuangarXiv 2026 · arXiv

TL;DR

This paper introduces Tabular Foundation Models (TFMs)—specifically Google’s TabFM—to the world of financial forecasting. By leveraging "in-context learning" and pretraining on purely synthetic data, TabFM outperforms both fine-tuned Large Language Models (LLMs) and traditional tree-based machine learning models in predicting corporate earnings changes, all while being significantly more computationally efficient than LLMs.

What Problem Does This Paper Address?

Empirical accounting and finance research rely heavily on structured tabular data (rows of firm-years, columns of financial ratios). Traditionally, researchers use tree-based models like Random Forests or XGBoost, which require extensive hyperparameter tuning and retraining for every new task.

Recently, Large Language Models (LLMs) have been fine-tuned for these tasks, but they come with two major headaches:

  1. Computational Cost: Fine-tuning billions of parameters is expensive and slow.
  2. Look-ahead Bias: Because LLMs are trained on vast swaths of the internet, they may have already "seen" the future outcomes of the firms they are trying to predict, leading to artificially inflated performance.

The authors investigate whether TFMs can provide a "best of both worlds" solution: the reasoning capabilities of a foundation model with the efficiency and bias-resistance needed for financial data.

Methods

The study centers on Tabular Foundation Models (TFMs), with a primary focus on Google Research’s TabFM.

Unlike traditional models, TFMs use In-Context Learning (ICL). Instead of updating the model's weights (training), you simply provide the model with a "context" consisting of historical examples (e.g., past financial statements and their resulting earnings) followed by the new case you want to predict.

Key Technical Highlights: * Synthetic Pretraining: TabFM is pretrained on hundreds of millions of synthetic datasets generated by causal models. Because it never saw real company data during pretraining, it cannot "memorize" real-world outcomes, effectively eliminating look-ahead bias. * Attention Mechanism: The architecture applies attention alternately across rows and columns, allowing the model to understand how a specific account (like Net Income) relates to other accounts within the same firm and how it compares to other firms in the sample. * Zero-Shot Utility: It works "out of the box" without the need for the user to perform a configuration or hyperparameter search.

Datasets

The authors evaluate the models using a dataset provided by prior research (Dou & Fu 2026):

  • Scope: 62,796 firm-year observations from 2005 to 2024.
  • Predictors: 130 variables, including income statement and balance sheet data from the current and prior two years.
  • Task: Predicting whether a firm's net income will increase in the following year.
  • Evaluation Window: Specifically focused on 2023 and 2024 to ensure a fair comparison against LLMs released in late 2024.

Key Insights and Ideas

  • The "Learning to Learn" Advantage: TFMs use meta-learning to encode a general procedure for extracting predictive relationships. This allows them to generalize to financial data even though they were trained on purely mathematical synthetic tables.
  • Superiority in Data Scarcity: One of the most surprising findings was TabFM's performance on small samples. Even when given only 5% of the training data (~2,800 observations), TabFM’s accuracy was higher than tree-based models trained on the entire dataset.
  • Resilience to Feature Loss: TabFM proved remarkably robust. Dropping the 20 most "important" predictors only resulted in a minor 1.3 percentage point drop in accuracy, suggesting the model is excellent at recovering information from correlated secondary variables.
  • The Speed-Accuracy Trade-off: While TabFM is much faster than fine-tuning an LLM, tree-based models are still faster at making "incremental" predictions once they are already trained.

Key Results

  • Predictive Accuracy: TabFM achieved a pooled AUC of 69.53%, significantly higher than the fine-tuned Llama model (67.88%) and the suite of tree-based models (which ranged from 62.75% to 63.78%).
  • Computational Efficiency: Producing forecasts with TabFM took about 12 minutes on a single GPU. In contrast, fine-tuning the Llama model took approximately 108 hours on a high-end NVIDIA H200.
  • FLOPs Comparison: In terms of raw floating-point operations, fine-tuning Llama required ~6,700 times more computation than running TabFM's forecasts.

Limitations and Future Work

The authors are careful to note that this is a "technical note" based on a single benchmark task.

  • Generalization: It remains to be seen if these results hold for other financial tasks like fraud detection or credit risk assessment.
  • Feature Scaling: The study used a maximum of 130 predictors; how TFMs scale to datasets with thousands of features is currently unknown.
  • Interpretability: Like many deep learning models, TabFM acts as a "black box." Future work is needed to "peek under the hood" and understand which accounting relationships the model is actually prioritizing.
  • Technological Churn: The field is moving so fast that newer open-weight LLMs or next-generation TFMs (like TabFM 1.1.0 or TabPFN) might shift the rankings quickly.

Summary updated

Abstract

This technical note introduces tabular foundation models (TFMs) to accounting research and evaluates their performance in one common benchmark task: predicting firms' next-year earnings changes. TFMs produce numeric predictions and learn in context, without post-release training or tuning, making them natural and easy to use for forecasting. The TFMs we study are pretrained on synthetic tabular data, so they avoid look-ahead bias from memorizing firms' outcomes. We compare Google Research's TabFM with a fine-tuned open-weight large language model developed in recent accounting research and with several tree-based models. TabFM is more accurate than all comparison models. It is also faster and less computationally intensive than the large language model. Several tree-based specifications produce predictions more quickly than TabFM, so choosing between them involves a trade-off between accuracy and speed. We conclude that TFMs have promise as a new tool for forecasting in accounting and finance. However, our evaluation is based on a single benchmark task, and no class of models is likely to dominate in all settings.
BibTeX
@misc{dehaan2026tabular,
  author = {Ed deHaan and Allen H. Huang},
  title = {{Tabular Foundation Models in Financial Predictions}},
  year = {2026},
  eprint = {2611.13550},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.13550},
}
Figure