While deep learning has mastered the generation of images and text, the "bread and butter" of industry—tabular data—remains a stubborn outlier. Tabular datasets are often a messy mix of continuous numbers, categorical labels, and missing values. Standard deep generative models like GANs or VAEs struggle here, requiring extensive preprocessing and delicate hyperparameter tuning to handle these heterogeneous structures.
A growing body of research suggests a more natural fit: using tree ensembles as the engine for generative modeling. By adapting the recursive partitioning logic of decision trees to density estimation and synthesis, researchers are building models that are faster, more robust, and often more accurate than their neural counterparts.
The Intuition of Local Independence
The core challenge of generative modeling is learning the joint probability distribution of a dataset. In Smooth densities and generative modeling with unsupervised random forests, David Watson, Marvin N. Wright, and colleagues describe a mechanism called Adversarial Random Forests (ARF).
The intuition is simple: a decision tree's job is to partition data into boxes (leaves). If you partition the data enough, the variables within each leaf eventually become independent of one another. This state of "local independence" is the holy grail for synthesis. Once the complex global dependencies are broken down into simple local ones, generating new data is as easy as sampling from each variable independently within a leaf and then aggregating the results.
To find these regions, ARFs use an adversarial setup. A forest is trained to distinguish real data from synthetic "noise" data. As the trees learn to split the data to better identify the real samples, they naturally find the regions where the features are most strongly correlated, eventually isolating them into locally independent clusters.
Replacing Neurons with Leaves in Diffusion
Diffusion models have become the gold standard for generation by learning to reverse a noise process. Traditionally, a neural network is used to estimate the "score function"—the gradient of the log-density that points toward the data. However, neural networks are notoriously poor at handling the discrete jumps and missing values common in tabular data.
Treeffuser, developed by Alessandro Antonio Grande, David Blei, and others, replaces the neural network with Gradient-Boosted Trees (GBTs). Because GBTs are non-parametric and naturally handle categorical variables, they can estimate score functions for complex, multimodal, and skewed distributions without the need for specialized hardware like GPUs. This allows for high-fidelity probabilistic predictions on a standard laptop CPU, making generative AI accessible for large-scale tabular datasets in resource-constrained environments.
Efficiency and Fidelity in Practice
The shift toward tree-based generative models is driven by three practical advantages:
- Structural Fit: As noted in Generative Forests by Richard Nock and Mathieu Guillame-Bert, trees inherently handle mixed data types and non-linear dependencies without the heavy preprocessing (like one-hot encoding or scaling) that neural networks demand.
- Computational Speed: Models like arfpy demonstrate that tree-based synthesis can be two orders of magnitude faster than deep learning alternatives, scaling linearly with sample size.
- Energy-Based Optimization: Methods like NRGBoost by João Bravo bring the power of energy-based models to GBTs, optimizing log-likelihood approximations to achieve discriminative performance that rivals XGBoost while remaining generative.
Beyond Synthesis: Explanations and Privacy
The utility of these models extends beyond just creating "fake" data. In CountARFactuals, Susanne Dandl and Kristin Blesch show how ARFs can generate realistic "what-if" scenarios. By sampling from the learned distribution, they can produce counterfactual explanations that are not just mathematically valid, but actually plausible within the context of the real-world data distribution.
Furthermore, these models are proving vital for privacy-sensitive fields. Jan Kapar and colleagues demonstrated in a 2025 replication study that ARF-generated synthetic data can successfully replicate real-world epidemiological findings. This suggests that tree-based generative AI can serve as a high-fidelity, privacy-preserving proxy for sensitive medical records, enabling data sharing that was previously impossible due to regulation.