Estimating the Fractal Dimension of the S&P 500 Index using Wavelet Analysis

Erhan Bayraktar, H. Vincent Poor, Ronnie Sircar

Abstract

S&P 500 index data sampled at one-minute intervals over the course of 11.5 years (January 1989- May 2000) is analyzed, and in particular the Hurst parameter over segments of stationarity (the time period over which the Hurst parameter is almost constant) is estimated. An asymptotically unbiased and efficient estimator using the log-scale spectrum is employed. The estimator is asymptotically Gaussian and the variance of the estimate that is obtained from a data segment of $N$ points is of order $\frac{1}{N}$. Wavelet analysis is tailor made for the high frequency data set, since it has low computational complexity due to the pyramidal algorithm for computing the detail coefficients. This estimator is robust to additive non-stationarities, and here it is shown to exhibit some degree of robustness to multiplicative non-stationarities, such as seasonalities and volatility persistence, as well. This analysis shows that the market became more efficient in the period 1997-2000.

Introduction

Standard financial models, such as the Black-Scholes framework, typically assume that stock prices follow a random walk (Brownian motion). In such an ideal efficient market, price changes are independent, and the Hurst parameter ($H$)—a measure of long-range dependence (LRD)—is exactly 0.5. However, empirical studies often reveal "fractal memory" or LRD, where $H > 0.5$, suggesting that past trends influence future price movements.

Accurately measuring $H$ in real-world financial time series is challenging due to various non-stationarities. These include additive trends (shifting price levels), seasonalities (daily cycles), and multiplicative noise like volatility clustering. Traditional methods like Rescaled Range (R/S) analysis often fail to distinguish between true LRD and these non-stationary artifacts. This paper proposes a robust wavelet-based approach to estimate the fractal dimension of high-frequency market data.

Methodology and Data

The study utilizes 11.5 years of S&P 500 index data at one-minute intervals (January 1989 to May 2000), comprising over 1.1 million observations. To estimate the Hurst parameter, the authors employ a log-scale spectrum methodology based on the Discrete Wavelet Transform (DWT).

Wavelets are particularly suited for this task because their "vanishing moments" property makes them naturally insensitive to polynomial trends (additive non-stationarities). The signal is decomposed into different scales using a computationally efficient pyramidal algorithm. The variance of the wavelet coefficients at each scale $j$ is then analyzed to determine the scaling behavior.

As shown in the log-scale spectrum, the relationship between the log-variance of the coefficients and the scale is linear. The slope of this line provides an asymptotically unbiased and efficient estimate of $H$. To handle temporal changes in $H$, the data is partitioned into "segments of stationarity"—windows of approximately $2^{14}$ minutes (about two months) where $H$ remains relatively constant.

Key Results

The analysis reveals that for most of the 1990s, the S&P 500 exhibited significant long-range dependence, with the Hurst parameter averaging around 0.6. This indicates a persistent, trend-following behavior that contradicts the pure random walk hypothesis.

The study also demonstrates the robustness of the wavelet estimator to multiplicative non-stationarities, such as the volatility clusters typical of financial markets. By applying a minimum variance unbiased linear filter to the segment-based estimates, the authors tracked the evolution of market efficiency over time.

A critical finding is the shift observed starting around 1997. During the period from 1997 to 2000, the estimated $H$ began to decline, approaching the 0.5 mark. This suggests that the market became increasingly efficient toward the end of the millennium. The authors hypothesize that this transition was driven by the "Internet Revolution," which democratized market access, increased trading frequency, and accelerated the dissemination of information.

Conclusion

Wavelet-based log-scale spectrum analysis provides a powerful tool for estimating the fractal dimension of high-frequency financial data. By effectively filtering out non-stationary "noise," this method reveals that the S&P 500's underlying dynamics are not static. The observed trend toward $H=0.5$ in the late 1990s highlights how technological shifts can fundamentally alter market behavior, moving it closer to the theoretical ideal of an efficient market. Future research could extend this analysis to the high-frequency trading era to see if these efficiency gains persisted or if new forms of fractal complexity emerged.

Summary updated

Abstract

S&P 500 index data sampled at one-minute intervals over the course of 11.5 years (January 1989- May 2000) is analyzed, and in particular the Hurst parameter over segments of stationarity (the time period over which the Hurst parameter is almost constant) is estimated. An asymptotically unbiased and efficient estimator using the log-scale spectrum is employed. The estimator is asymptotically Gaussian and the variance of the estimate that is obtained from a data segment of $N$ points is of order $\frac{1}{N}$. Wavelet analysis is tailor made for the high frequency data set, since it has low computational complexity due to the pyramidal algorithm for computing the detail coefficients. This estimator is robust to additive non-stationarities, and here it is shown to exhibit some degree of robustness to multiplicative non-stationarities, such as seasonalities and volatility persistence, as well. This analysis shows that the market became more efficient in the period 1997-2000.
BibTeX
@article{bayraktar2004estimating,
  author = {Erhan Bayraktar and H. Vincent Poor and Ronnie Sircar},
  title = {{Estimating the Fractal Dimension of the S\&P 500 Index using Wavelet Analysis}},
  journal = {International Journal of Theoretical and Applied Finance},
  year = {2004},
  doi = {10.1142/s021902490400258x},
}

Loading related content...

Figure