Abstract
Motivated by the problem of computing investment portfolio weightings, we investigate various methods of clustering as alternatives to traditional mean-variance approaches. Such methods can have significant benefits from a practical point of view since they remove the need to invert a sample covariance matrix, which can suffer from estimation error and will almost certainly be non-stationary. The general idea is to find groups of assets which share similar return characteristics over time and treat each group as a single composite asset. We then apply inverse volatility weightings to these new composite assets. In the course of our investigation, we devise a method of clustering based on triangular potentials and we present associated theoretical results as well as various examples based on synthetic data.
Introduction
In quantitative finance, the Markowitz mean-variance model remains a foundational framework for portfolio construction. However, its practical application is often hindered by the requirement to invert a sample covariance matrix. These matrices are notoriously sensitive to noise and non-stationarity; small estimation errors in asset correlations can lead to extreme and erratic fluctuations in investment weightings.
To mitigate these issues, researchers have explored simpler alternatives like $1/n$ (equal) weighting. While robust, equal weighting fails to account for high correlations between assets, potentially leading to unintended concentration in specific market factors. This paper proposes a middle ground: clustering. By grouping similar assets into "composite" assets and applying inverse volatility weighting to these groups, investors can achieve meaningful diversification without the mathematical fragility of matrix inversion. The primary challenge lies in performing this clustering dynamically in real-time as market conditions evolve.
Methodology
The authors propose a Graphical Model Approach that moves beyond standard spectral clustering, which is often unstable when applied to noisy financial time series.
The Ensemble Model and Triangular Potentials
The clustering is represented as a set of binary random variables $C_{i,j}$, where $C_{i,j} = 1$ if assets $i$ and $j$ belong to the same cluster. A critical innovation of this work is the introduction of triangular potentials. In a standard probabilistic model, one might independently estimate the relationships between pairs of assets. However, clustering requires logical transitivity: if asset $A$ is clustered with $B$, and $B$ is clustered with $C$, then $A$ must be clustered with $C$. The triangular potential function enforces this by assigning a probability of zero to any configuration that violates this transitive property, ensuring the resulting graph is a disjoint union of cliques.
Similarity Measures
The model's performance depends on the choice of similarity function between time series. The authors explore different norms to define these similarities, as shown in the table below.
Inference and Temporal Smoothing
Finding the optimal cluster assignment (the MAP assignment) is proven to be NP-hard (Theorem 1), effectively reducing to the $k$-clique problem. To overcome this, the authors employ a Coagulation-Fragmentation Markov Chain Monte Carlo (MCMC) process, which explores the space of valid clusterings by splitting or joining groups.
Furthermore, to ensure the clusters are "plausible" for investment purposes, the model is wrapped in a Hidden Markov Model (HMM). This incorporates a temporal prior that favors stability, preventing clusters from shifting erratically due to daily market noise.
Experimental Results
The proposed framework was validated using synthetic datasets consisting of 5,000 periods of returns for $n=3$ assets. The data was generated with a known underlying cluster structure that shifts over time, overlaid with independent Gaussian noise.
The results demonstrate several key advantages:
- Stability: While standard spectral methods produced highly unstable results, the HMM-based approach successfully tracked the hidden state transitions with significantly higher consistency.
- Logical Consistency: The triangular potentials effectively ruled out "implausible" clustering configurations that lacked transitivity, a common failure mode in simpler exponential models.
- Tracking Accuracy: As shown in the comparison between true and estimated states, the algorithm accurately recovers the underlying structure of the synthetic assets despite the presence of noise.
Conclusion and Future Work
This research highlights that framing clustering as a logical constraint problem via triangular potentials can yield more stable and useful groupings for financial applications. However, the NP-hard nature of the problem presents scalability challenges for large portfolios. Future work will focus on optimizing MCMC convergence for high-dimensional asset spaces and testing the model on real-world market data. Additionally, the authors suggest that incorporating non-uniform priors based on sector or industry knowledge could further guide the algorithm toward economically meaningful clusters.
Summary updated