Cluster Size Matters: A Comparative Study of Notip and pARI for Post Hoc Inference in fMRI

Nils Peyrouset, Pierre Neuvial, Bertrand ThirionarXiv 2025 · DOI · arXiv

Abstract

All Resolutions Inference (ARI) is a post hoc inference method for functional Magnetic Resonance Imaging (fMRI) data analysis that provides valid lower bounds on the proportion of truly active voxels within any, possibly data-driven, cluster. As such, it addresses the paradox of spatial specificity encountered with more classical cluster-extent thresholding methods. It allows the cluster-forming threshold to be increased in order to locate the signal with greater spatial precision without overfitting, also known as the drill-down approach. Notip and pARI are two recent permutation-based extensions of ARI designed to increase statistical power by accounting for the strong dependence structure typical of fMRI data. A recent comparison between these papers based on large voxel clusters concluded that pARI outperforms Notip. We revisit this conclusion by conducting a systematic comparison of the two. Our reanalysis of the same fMRI data sets from the Neurovault database demonstrates the existence of complementary performance regimes: while pARI indeed achieves higher sensitivity for large clusters, Notip provides more informative and robust results for smaller clusters. In particular, while Notip supports informative "drill-down" exploration into subregions of activation, pARI often yields non-informative bounds in such cases, and can even underperform the baseline ARI method.

Introduction

In functional Magnetic Resonance Imaging (fMRI) analysis, researchers typically group active voxels into clusters to identify brain regions responding to stimuli. However, this approach faces a "spatial specificity paradox": while a massive cluster indicates general activity, it lacks precise localization. Attempting to "drill down" by raising statistical thresholds to isolate highly active sub-regions risks "double-dipping" and increased false positives.

Post hoc inference methods, such as All Resolutions Inference (ARI), address this by providing mathematically valid lower bounds on the True Discovery Proportion (TDP)—the percentage of truly active voxels—for any cluster, regardless of how it was selected. While the baseline ARI is conservative due to its assumptions about voxel independence, newer methods like Notip and pARI use data randomization (permutations) to account for spatial dependencies and boost statistical power.

Methods: Notip vs. pARI

Both Notip and pARI utilize Joint Error Rate (JER) calibration to learn spatial dependencies from the data. The fundamental difference lies in their "templates," which dictate how statistical power is distributed across cluster sizes:

  • pARI: Employs a parametric template with a hyperparameter (typically $\delta=27$), which inherently prioritizes larger clusters at the expense of smaller ones.
  • Notip: Uses a data-driven template based on empirical quantiles of null statistics, allowing it to adapt to the data without a rigid size cutoff.

Results and Discussion

The comparative analysis reveals a "no free lunch" scenario. While pARI shows higher sensitivity for very large clusters at low thresholds, its performance degrades significantly as researchers attempt to localize signals in smaller clusters.

TDP lower bounds as a function of cluster size k for ARI, Notip, and pARI.

As shown in the figure above, pARI's TDP lower bound peaks for larger clusters but drops sharply for smaller ones ($k < 10^3$). In contrast, Notip maintains high TDP bounds across a wider range of cluster sizes, particularly outperforming pARI as the cluster size decreases. At high thresholds ($z=5$), where signals are strongest and clusters are smallest, Notip consistently identifies high TDP (often 100%), whereas pARI frequently yields uninformative bounds (TDP = 0).

Comparison of brain activation clusters and their corresponding TDP lower bounds for ARI, Notip, and pARI.

The table in the figure above illustrates this for specific clusters. Across 37 fMRI contrasts, Notip correctly identified pure signal (TDP=1) in 481 high-threshold clusters, while pARI only did so for a single cluster. This demonstrates that pARI's design actively penalizes the "drill-down" workflow essential for precise spatial localization.

Conclusion

While pARI is effective for broad, large-scale activation patterns, Notip is the more robust and practical tool for researchers who need to explore fMRI data at multiple resolutions. Notip's data-driven approach avoids the arbitrary size penalties inherent in pARI, making it a more versatile choice for precise brain mapping.

Summary updated

Abstract

All Resolutions Inference (ARI) is a post hoc inference method for functional Magnetic Resonance Imaging (fMRI) data analysis that provides valid lower bounds on the proportion of truly active voxels within any, possibly data-driven, cluster. As such, it addresses the paradox of spatial specificity encountered with more classical cluster-extent thresholding methods. It allows the cluster-forming threshold to be increased in order to locate the signal with greater spatial precision without overfitting, also known as the drill-down approach. Notip and pARI are two recent permutation-based extensions of ARI designed to increase statistical power by accounting for the strong dependence structure typical of fMRI data. A recent comparison between these papers based on large voxel clusters concluded that pARI outperforms Notip. We revisit this conclusion by conducting a systematic comparison of the two. Our reanalysis of the same fMRI data sets from the Neurovault database demonstrates the existence of complementary performance regimes: while pARI indeed achieves higher sensitivity for large clusters, Notip provides more informative and robust results for smaller clusters. In particular, while Notip supports informative ``drill-down'' exploration into subregions of activation, pARI often yields non-informative bounds in such cases, and can even underperform the baseline ARI method.
BibTeX
@misc{peyrouset2025cluster,
  author = {Nils Peyrouset and Pierre Neuvial and Bertrand Thirion},
  title = {{Cluster Size Matters: A Comparative Study of Notip and pARI for Post Hoc Inference in fMRI}},
  year = {2025},
  eprint = {2511.02422},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2511.02422},
  howpublished = {arXiv},
}
Figure