Exploratory data analysis for extreme values using non-parametric kernel methods

Boris Béranger, Tarn Duong, Sarah Perkins‐Kirkpatrick, Scott A. SissonarXiv 2016 · arXiv

Abstract

Accurately modeling the extreme tail behavior of random processes is critical across various disciplines, including finance, insurance, and climate science. While non-parametric density estimation is a preferred tool for exploratory data analysis due to its visualization properties and lack of model specification bias, standard kernel density estimators (KDEs) often falter at the tails. They frequently exhibit boundary bias and produce spurious "bumps" in sparse, heavy-tailed data. This paper develops a transformation kernel density estimator that maps truncated tail data onto the entire real line before smoothing. We demonstrate that this estimator's Mean Integrated Squared Error (MISE) efficiency is equivalent to standard KDEs while providing superior visualization and accuracy for extreme values. The method's utility is illustrated through numerical studies and an analysis of global climate models' ability to reproduce observed temperature extremes in Sydney, Australia.

Introduction

In many risk-sensitive fields, the primary interest lies not in the mean behavior of a process, but in its extremes—the values exceeding a high threshold $u$. Traditionally, researchers rely on parametric models like the Generalized Extreme Value (GEV) distribution or the Generalized Pareto Distribution (GPD). However, these models carry a risk of misspecification bias if the underlying data does not strictly follow the assumed mathematical form.

Non-parametric methods, such as histograms and standard KDEs, offer a more flexible alternative. Yet, they are not without flaws. Histograms are discrete and highly sensitive to bin width and anchor points. Standard KDEs, meanwhile, perform poorly at boundaries (where data is truncated at $u$) and often create artificial modes in heavy-tailed distributions where observations are sparse. This paper introduces a transformation-based KDE specifically designed to mitigate these issues, providing a robust tool for exploratory tail analysis.

Methodology: The Transformation Tail Density Estimator

The core innovation of this research is a three-step process designed to "stretch" the tail and eliminate boundary effects:

  1. Transformation: Instead of estimating the density directly on the tail data $X$ (where $X > u$), a monotonic transformation is applied. Typically, the authors use $t(x) = \log(x - u)$, which maps the semi-infinite interval $(u, \infty)$ to the entire real line $(-\infty, \infty)$.
  2. Smoothing: Standard kernel density estimation is performed in this transformed space. Because the data is no longer truncated at a lower boundary, the traditional boundary bias is eliminated.
  3. Back-transformation: The resulting estimate is mapped back to the original space to provide a smooth, continuous density function of the extremes.

Theoretical Efficiency

A critical contribution of this work is the proof of the estimator's efficiency. The authors demonstrate that this approach achieves a Mean Integrated Squared Error (MISE) convergence rate of $O(n^{-4/(d+4)})$. This rate is asymptotically as efficient as standard KDE applied to unbounded data and significantly faster than standard KDE applied directly to truncated data. Furthermore, the authors adapted bandwidth selection methods, such as Plug-in and Cross-validation, to function within this transformation framework.

Experimental Validation

The estimator was tested against both synthetic and real-world datasets to evaluate its performance in univariate and multivariate settings.

Simulation Studies

The authors used univariate samples from Fréchet, Gumbel, and GPD distributions, as well as multivariate samples from Asymmetric Negative Logistic and Hüsler-Reiss distributions. In these blind tests, the transformation KDE consistently achieved lower $L_2$ and $\chi^2$ errors than both histograms and standard KDEs. Notably, the transformation KDE correctly identified the underlying model in over 90% of cases, whereas histograms frequently failed, particularly with heavy-tailed Fréchet data.

Climate Model Evaluation

The method was applied to daily maximum and minimum temperature records for Sydney, Australia (1911–2005). The goal was to assess how well 22 Global Climate Models (GCMs) from the CMIP5 ensemble could reproduce observed extremes. By using the transformation KDE as a "surrogate" for the true unknown density, the authors could rank the GCMs based on their discrepancy from the non-parametric estimate.

Key Insights and Results

  • Eliminating Spurious Bumps: By stretching the sparse data in the far tail, the log-transformation allows the kernel to smooth the data more appropriately. This prevents the "spurious bumps" that occur when a standard kernel is applied to isolated extreme points.
  • Model Ranking: The analysis identified the MPI-ESM-LR and MPI-ESM-MR models as the top performers for reproducing univariate temperature extremes in Sydney.
  • Joint Extremes: The study revealed that models which perform well on univariate data (e.g., NorESM1-M) often struggle to predict the joint behavior of maximum and minimum temperatures. The transformation KDE provided much clearer visualizations of these joint extremes compared to the noisy contours of 2D histograms.

Limitations and Future Work

While the transformation KDE reduces bias, the extreme far tail remains somewhat sensitive to the choice of the base kernel; for instance, a Gaussian kernel may eventually impose its own tail behavior on the estimate. Additionally, while the log-transformation is effective for many distributions, other transformations like the shifted power family might be more appropriate for specific data types. Future research may explore local polynomial adjustments to further refine behavior at the lower boundary of the tail and address the "curse of dimensionality" as the number of variables increases.

Conclusion

The transformation kernel density estimator represents a significant advancement for exploratory data analysis of extremes. By combining the visual smoothness of KDE with a robust mathematical framework that handles truncation and sparsity, it provides researchers with a more accurate tool for validating parametric models and assessing rare, high-impact events.

Summary updated

Abstract

In many settings it is critical to accurately model the extreme tail behaviour of a random process. Non-parametric density estimation methods are commonly implemented as exploratory data analysis techniques for this purpose as they possess excellent visualisation properties, and can naturally avoid the model specification biases implied by using parametric estimators. In particular, kernel-based estimators place minimal assumptions on the data, and provide improved visualisation over scatterplots and histograms. However kernel density estimators are known to perform poorly when estimating extreme tail behaviour, which is important when interest is in process behaviour above some large threshold, and they can over-emphasise bumps in the density for heavy tailed data. In this article we develop a transformation kernel density estimator, and demonstrate that its mean integrated squared error (MISE) efficiency is equivalent to that of standard, non-tail focused kernel density estimators. Estimator performance is illustrated in numerical studies, and in an expanded analysis of the ability of well known global climate models to reproduce observed temperature extremes in Sydney, Australia.
BibTeX
@misc{beranger2016exploratory,
  author = {Boris Béranger and Tarn Duong and Sarah Perkins‐Kirkpatrick and Scott A. Sisson},
  title = {{Exploratory data analysis for extreme values using non-parametric kernel methods}},
  year = {2016},
  eprint = {1602.08807},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1602.08807},
}
Figure