Regular variation in Hilbert spaces and principal component analysis for functional extremes

Stéphan Clémençon, Nathan Huet, Anne Sabourin

TL;DR

This paper introduces a new statistical framework for performing Principal Component Analysis (PCA) on "extreme" functional data (like energy spikes or air quality crises) within Hilbert spaces ($L^2$). By focusing on the "angular" component of data that exceeds a high threshold, the authors provide a mathematically rigorous way to reduce the dimensionality of complex, heavy-tailed functional observations while maintaining strong statistical guarantees.

What Problem Does This Paper Address?

In fields like energy management or environmental monitoring, we often deal with functional data—observations that are essentially continuous curves over time. A major challenge arises when we want to study "extremes" (e.g., a massive spike in electricity demand).

Standard Extreme Value Theory (EVT) usually works with finite-dimensional vectors or continuous functions using a "maximum value" (supremum) norm. However, many real-world applications care more about the "total energy" of a spike, which is better captured by the $L^2$ norm (the square root of the integral of the squared function).

Furthermore, standard dimension reduction tools like PCA rely on the existence of second moments (variance). Extreme data is often "heavy-tailed," meaning these moments might not even exist, causing standard PCA to fail or produce misleading results. This paper fills the gap by providing a PCA framework specifically for $L^2$ functional extremes.

Methods

The authors propose a two-step theoretical and practical approach:

  1. Regular Variation in Hilbert Spaces: They extend the concept of "Regular Variation"—the mathematical language used to describe heavy-tailed distributions—to separable Hilbert spaces. Crucially, they prove that you can characterize this complex infinite-dimensional property simply by looking at the convergence of one-dimensional projections (real-valued variables).
  2. Angular PCA for Extremes: In EVT, an observation $X$ can be decomposed into its magnitude (norm $\|X\|$) and its "direction" or "shape" (the pseudo-angle $\Theta = X/\|X\|$).
  3. The authors focus on the Peaks-Over-Threshold (POT) framework: they only look at observations where the norm exceeds a very high threshold.
  4. They perform PCA on the angular components of these extremes. Since the angles are always on a unit sphere, they have finite moments by definition, bypassing the heavy-tail problem of the raw data.
  5. Statistical Guarantees: The paper provides non-asymptotic bounds on the estimation error of the covariance operator. They show that the empirical "extreme" covariance operator converges to the true one at a rate of $1/\sqrt{k}$, where $k$ is the number of extreme observations.

Datasets

The authors evaluate their method using two primary sources:

  • Simulated Toy Dataset: A multiplicative model designed to have a specific "extreme" structure (a 2D subspace) hidden within a more complex 4D dataset.
  • Electricity Demand Dataset: Real-world data (thursdaydemand from the R package fds) containing half-hourly electricity demand records from Adelaide, Australia, spanning nearly a decade (508 observations, each a vector of size 48).

Key Insights and Ideas

  • Shape vs. Magnitude: For extremes, the magnitude often follows a predictable Pareto distribution. The real "information" about the nature of the extreme event is hidden in its shape (the angle).
  • L2 is More Flexible than C[0,1]: Most previous work used the space of continuous functions ($C[0,1]$). The authors show that $L^2$ is a broader framework that can handle "rougher" data paths, which is common in high-frequency sensor data.
  • Uncovering Hidden Sparsity: Standard PCA is often "distracted" by the bulk of the data (the average days). The proposed "Extreme PCA" can uncover patterns that only appear during rare, high-magnitude events. For example, in their simulations, standard PCA saw a 4D structure, while Extreme PCA correctly identified that the extremes only lived in a 2D subspace.

Key Results

  • Consistency: The authors proved that as the sample size $n$ and the number of extremes $k$ grow, the empirical extreme PCA eigenspaces converge to the true theoretical eigenspaces.
  • Reconstruction Accuracy: On the Adelaide electricity dataset, the authors demonstrated that performing PCA specifically on the "tail" (the top 100 most extreme days) significantly reduced the reconstruction error for new extreme events compared to standard PCA performed on the full dataset.
  • Visual Diagnostics: The paper provides practical "Hill plots" and projection stability plots that allow practitioners to verify if their functional data actually follows the required "Regular Variation" assumptions before applying the model.

Limitations and Future Work

  • Choosing 'k': Like many methods in Extreme Value Theory, the results are sensitive to the choice of $k$ (the number of observations considered "extreme"). While the authors provide visual diagnostics, choosing $k$ remains a non-trivial task for the user.
  • Temporal Independence: The current framework assumes that functional observations (e.g., demand on one Thursday vs. the next) are independent. Future work could extend this to handle functional time series where extremes might be clustered in time.
  • Asymptotic Distributions: While they provided concentration inequalities (upper bounds), the authors suggest that future research could derive the full asymptotic distribution of the covariance operator to allow for formal hypothesis testing.

Summary updated

Abstract

Motivated by the increasing availability of data of functional nature, we develop a general probabilistic and statistical framework for extremes of regularly varying random elements X in L2[0,1]. We place ourselves in a Peaks-Over-Threshold framework where a functional extreme is defined as an observation X whose L2-norm ‖X‖ is comparatively large. Our goal is to propose a dimension reduction framework resulting into finite dimensional projections for such extreme observations. Our contribution is double. First, we investigate the notion of Regular Variation for random quantities valued in a general separable Hilbert space, for which we propose a novel concrete characterization involving solely stochastic convergence of real-valued random variables. Second, we propose a notion of functional Principal Component Analysis (PCA) accounting for the principal ‘directions’ of functional extremes. We investigate the statistical properties of the empirical covariance operator of the angular component of extreme functions, by upper-bounding the Hilbert–Schmidt norm of the estimation error for finite sample sizes. Numerical experiments with simulated and real data illustrate this work.
BibTeX
@inproceedings{clemencon2024regular,
  author = {Stéphan Clémençon and Nathan Huet and Anne Sabourin},
  title = {{Regular variation in Hilbert spaces and principal component analysis for functional extremes}},
  booktitle = {Stochastic},
  year = {2024},
  doi = {10.1016/j.spa.2024.104375},
}

Loading related content...

Figure