Subsurface schematisation from sparse CPT data using the tabular foundation model TabPFN

Taiga Saito, Fabian Campos Montero, Bruno Zuada Coelho, Ana Martins Teixeira, Stephen Wu, Timo Schweckendiek, Yu OtakearXiv 2026 · arXiv

This paper explores a fascinating shift in how we might model the ground beneath our feet. Traditionally, geotechnical engineers have relied on geostatistical methods like Kriging or, more recently, specialized deep-learning models to "fill in the blanks" between sparse soil measurements. This study asks a provocative question: Can a general-purpose AI, trained on abstract tabular data, outperform specialized geological models at reconstructing subsurface sections?

TL;DR

Researchers successfully used TabPFN, a foundation model for tabular data, to reconstruct 2D soil profiles from sparse Cone Penetration Test (CPT) data without any initial geotechnical training. Surprisingly, this general-purpose model outperformed a specialized "geological" AI (SchemaGAN) on real-world field data, suggesting that foundation models offer a flexible, "zero-shot" path for subsurface characterization.

What Problem Does This Paper Address?

Geotechnical engineering is a game of "connect the dots" played with very few dots. Engineers use CPTs—vertical probes that measure soil properties—to understand the subsurface. However, CPTs are expensive and sparse, leaving massive gaps in our knowledge of layer boundaries and soil variability.

The challenge, known as subsurface schematisation, is to predict soil properties (like the Soil Behaviour Type index, $I_c$) in the spaces between these probes. Traditional interpolation (Kriging) often struggles with complex, non-stationary geological features. Recent deep-learning approaches, like SchemaGAN, show promise but are "brittle"—they require massive amounts of representative synthetic training data and can struggle when real-world conditions don't match their training set.

Methods

The authors compare four methods for 2D $I_c$ reconstruction:

  1. TabPFN (The Newcomer): A Transformer-based foundation model. Unlike typical models, it uses in-context learning. You provide it with a "context" (the $(x, z)$ coordinates and $I_c$ values of observed points) and "queries" (the coordinates where you want a prediction). It makes predictions based on patterns learned during its massive pretraining on synthetic tabular datasets, without updating its weights for a specific site.
  2. SchemaGAN (The Specialist): A Generative Adversarial Network specifically trained on 16,000 synthetic geotechnical sections to "image-complete" sparse data.
  3. Kriging: A standard geostatistical interpolation method.
  4. Nearest Neighbour: A simple baseline.
Figure 1: Input representations of SchemaGAN and TabPFN for subsurface schematisation from six input columns. (a) Synthetic test section and sampled observations; cyan bars mark the observed depth ranges. (b) Image input and reconstruction with SchemaGAN; unobserved pixels are filled with zero to mark missing input, and the input image is then normalised as described in Section 2.4. (c) Tabular input and reconstruction with TabPFN; the context set C contains the observed (x, z, Ic) rows, and the query set Q contains only the (x, z) coordinates of all grid pixels, including input locations. Both reconstructed sections show point predictions ˆyj at the grid locations. The vertical scale and input-column widths are exaggerated for clarity.

As shown in Figure 1, SchemaGAN treats the section as an image with missing pixels, while TabPFN treats it as a table of coordinates and values.

Datasets

  • Synthetic Data: 24,000 2D sections generated using GeoSyn, which simulates realistic geological layering and random field variability.
  • Field Data: Real-world CPT data from two sites in the Netherlands: Delft–Schiedam and Groningen–Delfzijl. These represent challenging, real-world stratigraphy.
Figure 2: Locations of the field CPTs and their assignment to the two sections at each site. Panel (a) shows the locations of the two sites in the Netherlands, and panels (b) and (c) show CPT locations overlaid on aerial photographs. In panels (b) and (c), white and orange triangles mark the CPTs in sections 1 and 2, respectively. Axes show RD New (EPSG:28992) coordinates in kilometres. Aerial photographs (2025): Beeldmateriaal Nederland; province boundaries: Kadaster; both obtained via PDOK under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Land and lakes: Natural Earth (public domain).

Key Insights and Ideas

  • The Power of Generalization: TabPFN, which has never "seen" a rock or a layer of clay during its primary training, was able to recover the main layer geometry of geological sections simply by recognizing abstract spatial patterns in the data.
  • The "Specialization Trap": The authors tried fine-tuning TabPFN on the same synthetic geotechnical data used to train SchemaGAN. While this made the model better at predicting synthetic tests, it actually made it worse on real field data. This suggests that the synthetic training data was too "clean" or lacked the specific types of variability found in the real world, causing the model to overfit to unrealistic geological "rules."
  • In-Context Flexibility: Because TabPFN is a tabular model, it naturally handles any number of CPTs. SchemaGAN, by contrast, showed increased error when given more than the six CPTs it was originally designed to handle.

Key Results

  • Accuracy: On synthetic data, TabPFN and SchemaGAN performed similarly (Mean Absolute Error of ~0.12). However, on real field sites, TabPFN was the clear winner, achieving lower MAE and RMSE than SchemaGAN, Kriging, and Nearest Neighbour.
  • Uncertainty Quantification: TabPFN provides a predictive distribution, not just a single number. Its 90% prediction intervals were remarkably well-calibrated, covering the true value roughly 89.2% of the time. SchemaGAN’s intervals (generated via Monte Carlo dropout) were far too narrow, providing poor coverage.
  • Data Scaling: TabPFN’s performance improved consistently as more CPTs were added to the context, demonstrating its utility for sequential site investigations where data arrives in stages.

Limitations and Future Work

  • Spatial Coherence: While TabPFN is accurate point-by-point, it doesn't have an explicit "geological" prior. It might occasionally predict disconnected patches where a human engineer would see a continuous layer.
  • Feature Engineering: This study only used $(x, z)$ coordinates. Future work could incorporate richer data like topography, borehole logs, or geophysical measurements as additional tabular features.
  • Resolution: The study used a $32 \times 512$ grid. Real-world applications may require much higher vertical resolution to capture thin, critical soil layers.
  • Geological Context: The model currently lacks "domain knowledge" (e.g., depositional history). Finding ways to provide this context without the pitfalls of over-specialization remains an open challenge.

Summary updated

Abstract

Subsurface schematisation supports geotechnical decisions but remains challenging because sparse site-investigation data leave substantial gaps in information about ground conditions. This study explores TabPFN, a foundation model pretrained on general synthetic tabular data, for this task. Using only grid coordinates and soil behaviour type index observations from sparse cone penetration tests (CPTs), we first reconstruct two-dimensional sections without geotechnical fine-tuning or predefined layer boundaries. We evaluate the approach on synthetic sections and at two field sites in the Netherlands, comparing it with SchemaGAN, a conditional generative adversarial network trained on synthetic geotechnical sections, and with reference interpolation methods. In the baseline comparisons, TabPFN achieved mean errors close to SchemaGAN's on synthetic sections and lower mean errors at both field sites. On synthetic sections, TabPFN's mean absolute error decreased at each tested increase in CPT count, and, under the baseline input setting, its pointwise prediction intervals had mean coverage close to nominal levels. Subsequent fine-tuning on the synthetic sections used to train SchemaGAN improved synthetic test accuracy but increased mean errors at both field sites. These findings indicate that general-purpose tabular foundation models offer a promising route for updating subsurface predictions from sparse and evolving site-investigation data without retraining.
BibTeX
@misc{saito2026subsurface,
  author = {Taiga Saito and Fabian Campos Montero and Bruno Zuada Coelho and Ana Martins Teixeira and Stephen Wu and Timo Schweckendiek and Yu Otake},
  title = {{Subsurface schematisation from sparse CPT data using the tabular foundation model TabPFN}},
  year = {2026},
  eprint = {2611.16088},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.16088},
}
Figure