CCD: Context-Calibrated Open-Entity Out-of-Distribution Detection in In-Context Tabular Foundation Models

Zhangkai Wu, Xu Zhang, Zhihao Hao, Zhihong Cui, Yongmin Yoo, Xianxun Zhu, Kaize ShiarXiv 2026 · arXiv

In the rapidly evolving world of Tabular Foundation Models (TFMs) like TabPFN, "in-context learning" is the new gold standard. These models can predict outcomes for new data points simply by looking at a provided "context" of labeled examples, without any parameter updates. However, a new paper reveals a critical blind spot: when these models are deployed in the wild—like detecting fraud for a new merchant or recommending products to a new user—they often mistake a "new entity" for an "unknown category."

TL;DR

Existing out-of-distribution (OOD) detectors for tabular models fail in "open-entity" scenarios because they can't distinguish between a new user (a harmless covariate shift) and a truly unknown class (a dangerous semantic shift). This paper introduces Context-Calibrated Detection (CCD), which uses internal "group-out" swaps to calibrate model scores, outperforming 16 existing detectors on a new, more realistic benchmark called NEO-Tab.

What Problem Does This Paper Address?

When we evaluate OOD detectors, we usually use "random splits," where examples from the same entity (e.g., the same user or seller) appear in both the training (context) and test sets. In real-world Web applications, this is rarely the case. We often need to make predictions for entirely new entities.

The authors identify a phenomenon called entity-over-class bias. Because TFMs generate scores based on how "unfamiliar" a row is relative to the context, a known-class row from a new entity often looks more "out-of-distribution" than an unknown-class row from a familiar entity.

Figure 1: Open-entity OOD detection in a marketplace, with sellers as entities and product categories as classes. A high score of a TFM-based detector indicates OOD. In classic OOD detection (faded row), it can only signal an unknown class because the entity is in the context. In open-entity OOD detection (highlighted row), known-class (ID) and unknown-class (OOD) listings both receive a high score because test rows come from new entities. The detector must separate them, yet existing detectors raise a false alarm on the ID listing.

This leads to an entity-class trade-off:

  1. Confidence-based scores (like MSP) tend to miss unknown classes because the model's label-oriented output might not capture the features that make the class "new."
  2. Distance-based scores (like k-nearest neighbors) are great at catching unknown classes but are hyper-sensitive to new entities, flagging them as outliers simply for being "far" from the context.

Methods

The core innovation of Context-Calibrated Detection (CCD) is that it uses the provided context itself to learn what a "new entity" looks like.

  1. Group-out Swaps: The algorithm partitions the context by entity (or time block). It systematically holds out one group and scores it against the remaining context. This simulates the "new entity" experience using only in-distribution data.
  2. Multi-Branch Calibration: CCD calculates four different scores: three types of distances (feature-space, label-subspace, and TFM embedding) and the model's predictive confidence. It then calibrates these scores against the reference distributions generated by the group-out swaps.
  3. Smart Fusion: Not all scores are equally useful. CCD weights the distances by their sensitivity (how much they react to new entities) and the confidence by its sharpness (how certain the model remains when an entity is removed).

Crucially, CCD keeps the underlying TFM frozen and requires no external OOD data for training.

Figure 3: The CCD framework: (1) group-out swaps give a new-entity reference, (2) distances and confidence are calibrated against it and weighted, and (3) the confidence is added with sharpness a.

Datasets

The authors argue that standard benchmarks hide detector failures due to the "entity-split gap." To fix this, they introduced NEO-Tab (New-Entity OOD detection on Tables).

  • Composition: 14 grouped and temporal tables derived from the BeyondArena benchmark and other public sources.
  • Key Difference: Unlike standard benchmarks, NEO-Tab ensures that all test rows come from entities or time periods that were never seen in the context. This forces the detector to handle the covariate shift of new entities while still identifying the semantic shift of unknown classes.

Key Insights and Ideas

  • The Entity-Split Gap is Real: Switching from random splits to entity-based splits causes standard detectors to lose between 5 and 16 AUROC points. Random splits make OOD detection look much easier than it actually is in production.
  • In-Context Models are Naturally Biased: The paper proves that for kernel-weighted in-context predictors, confidence is essentially a measure of familiarity. This explains why new entities—by definition unfamiliar—suffer from lower confidence scores regardless of their class.
  • Calibration is the Cure: By using "group-out" swaps, we can estimate the "null distribution" of a new entity. This allows us to "center" our expectations: if we know a new entity should look a bit distant, we don't overreact when it does.

Key Results

CCD demonstrated dominant performance across the board:

  • Superior Accuracy: On the NEO-Tab benchmark, CCD achieved a mean AUROC of 81.8, a significant jump over the best-tuned kNN baseline (74.4).
  • Robustness: It outperformed 16 other detectors, including specialized TFM-output scores and pre-trained outlier detectors.
  • Backbone Agnostic: The lead held firm across three different TFM backbones: TabPFN v2, TabICLv2, and TabDPT.
Table 1: Detection on the 14 benchmark tables (TabPFN v2, held-out trials 35–39). Rank: average rank among 17 detectors; Worst: lowest per-table AUROC; Random: random-split protocol; 40–44: resampling check; Best: tables won; last two columns: CCD minus the detector in AUROC with tables won, and Holm-corrected Wilcoxon p. The top three values per column are shaded.

Limitations and Future Work

  • Group Labels Required: CCD relies on having a "group" or "entity" identifier (like a User ID or a timestamp) in the context data to perform its swaps. While most web data has this, it might not be available in all tabular datasets.
  • Computational Overhead: Performing group-out swaps requires multiple forward passes through the TFM. While this is done only once per context, it could be a bottleneck for extremely latency-sensitive applications.
  • Theory vs. Practice: While the authors provide theoretical guarantees for their fusion method, these rely on assumptions like joint normality of scores, which may not perfectly hold for all real-world data distributions.

For researchers building the next generation of tabular models, this paper serves as a vital reminder: a model that is "confident" isn't always right, and a data point that is "far away" isn't always wrong. Context is everything.

Summary updated

Abstract

In-context tabular foundation models (TFMs), such as TabPFN, predict from a labelled context without updating their parameters and are increasingly used for out-of-distribution (OOD) detection in Web applications, whose test rows come from entities absent from that context. Such applications need open-entity OOD detection: a new entity must not be mistaken for an unknown class, and an unknown class must still be caught. Existing detectors fail in this setting, and an entity-split gap hides the failure: random splits also place the test entities in the context. The failure arises because every TFM score reflects how unfamiliar a row is to the context. This produces an entity-over-class bias: known-class rows of new entities look more OOD than familiar ones, sometimes more than unknown classes. Specifically, the two OOD scores most widely used with in-context TFMs face an entity-class trade-off: the TFM's predictive confidence misses unknown classes, whereas the distance of a test row to the context flags new entities. To address this problem, we hold a group of entities out of the context and score them with the remaining context, which turns them into new entities of known classes and tells us how such rows should score. To measure detection on new entities, we build NEO-Tab (New-Entity OOD detection on Tables), 14 grouped and temporal tables on which all test rows come from new entities. We then propose Context-Calibrated Detection (CCD), which uses these group-out swaps on in-distribution data to calibrate three distances and the confidence against new-entity references. CCD weights them by their sensitivity and sharpness, keeps the model frozen and uses no OOD data. On NEO-Tab, CCD outperforms 16 existing detectors on every metric we report, and its lead holds on three TFM backbones.
BibTeX
@misc{wu2026ccd,
  author = {Zhangkai Wu and Xu Zhang and Zhihao Hao and Zhihong Cui and Yongmin Yoo and Xianxun Zhu and Kaize Shi},
  title = {{CCD: Context-Calibrated Open-Entity Out-of-Distribution Detection in In-Context Tabular Foundation Models}},
  year = {2026},
  eprint = {2611.01489},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.01489},
}
Figure