TL;DR
This paper introduces a novel statistical model for understanding extreme events in complex systems, like river networks, where some data points might be missing (latent variables). They propose a method to reliably estimate the dependence structure even with these missing values, provided that the unobserved locations have sufficient connections.
What Problem Does This Paper Address?
Modeling extreme events (like floods or financial crashes) across multiple related variables is crucial in many fields. Traditional methods often assume all variables are observed. However, in real-world scenarios, such as monitoring river levels, some measurement stations might be offline or non-existent, leading to "latent" or unobserved variables. This paper tackles the challenge of inferring the dependence structure of extreme events when some variables are latent. Specifically, they focus on a structured Hüsler-Reiss distribution, which is a powerful tool for modeling multivariate extremes, and investigate how to estimate its parameters when some of the underlying variables are unobservable. The core problem is to determine if and how we can still reliably understand the extreme dependence patterns in the system despite these data gaps.
Methods
The authors build upon the concept of Hüsler-Reiss distributions and introduce a "structured" version where the dependence parameters are linked to the structure of an undirected tree. This tree represents the relationships between variables, inspired by graphical models. A key innovation is the development of an "identifiability criterion" for the model parameters. This criterion states that if a node corresponding to a latent variable has a degree of at least three (meaning it's connected to at least three other nodes), then the parameters can still be uniquely estimated from the observed data.
They propose three estimation procedures:
- Method of Moments (MME): An adaptation of existing techniques that uses sample counterparts of statistical quantities.
- Composite Likelihood Estimator (CLE): This method involves maximizing a composite likelihood function, which is a product of likelihoods defined on subtrees.
- Pairwise Extremal Coefficients Estimator (ECE): This approach is based on minimizing the difference between non-parametric estimates and fitted parametric stable tail dependence functions.
All these methods are designed to work with observed data only, while accounting for the presence of latent variables.
Datasets
The primary dataset used for illustration and validation is water level data from the Seine river network in France. This dataset includes measurements from five locations (Paris, Meaux, Melun, Nemours, and Sens) spanning from January 1987 to April 2019. The authors specifically model this network as a tree with seven nodes, where two nodes (associated with Meaux and Nemours) are treated as latent variables.
Key Insights and Ideas
- Latent variables don't always break inference: The most significant insight is that the presence of latent variables doesn't necessarily prevent us from understanding the extremal dependence structure.
- Degree matters: The "degree" of a latent variable's node in the network structure is crucial. If a latent variable node has a degree of at least three, its parameters are identifiable. This provides a clear, actionable criterion for model applicability.
- Structured Hüsler-Reiss is key: The paper demonstrates how a Hüsler-Reiss distribution with a structure induced by a tree can effectively model extremal dependence in networked systems.
- Multiple estimators offer robustness: The development and comparison of three distinct estimation methods (MME, CLE, ECE) provide a robust framework for practical application and allow for cross-validation of results.
- Flow-connectedness influences dependence: The application to the Seine river data highlights that flow connectivity between locations can be a stronger driver of extremal dependence than simple geographical distance.
Key Results
The paper successfully applies the proposed methodology to the Seine river data, demonstrating that the six edge parameters of the structured Hüsler-Reiss distribution are identifiable despite two latent variables. The three proposed estimators (MME, CLE, and ECE) provide estimates and confidence intervals for these parameters. The goodness-of-fit is assessed by comparing model-based extremal coefficients and Pickands dependence functions with their non-parametric counterparts, showing a good fit. The analysis also confirms that flow-connected locations exhibit stronger tail dependence.
Limitations and Future Work
- Identifiability criterion is specific: The identifiability criterion is derived for the specific structured Hüsler-Reiss distribution and tree structures. Extending this to other dependence structures or graphical models remains an open question.
- Choice of neighborhoods for estimators: For the MME and CLE, the performance can depend on the choice of "neighborhoods" (subsets of observed variables) used in the estimation. While they illustrate this with an example, a more systematic approach to selecting these neighborhoods could be beneficial.
- Extensions to other families: The authors suggest exploring extensions of their framework to other parametric families of max-stable distributions beyond Hüsler-Reiss.
- Unknown structure: Future work could investigate scenarios where not only variables but also parts of the network structure itself are unknown.
Summary updated