Incorporating prior financial domain knowledge into neural networks for implied volatility surface prediction

Yu Zheng, Yongxin Yang, Bowei Chen

TL;DR

This paper introduces a novel neural network architecture for predicting implied volatility surfaces, a crucial financial metric. By embedding prior financial domain knowledge directly into the network's design and training process, the model achieves superior prediction accuracy and aligns with established financial theories, addressing the "black box" problem of traditional machine learning in finance.

What Problem Does This Paper Address?

Machine learning models, particularly deep neural networks, have shown immense promise in financial prediction tasks. However, their application in finance is often hampered by their "black box" nature, making them less interpretable and often misaligned with well-established financial theories and empirical observations. This lack of alignment can be a significant barrier for financial institutions hesitant to adopt these technologies for critical business operations. Specifically, predicting the implied volatility surface—a complex, multi-dimensional representation of market expectations about future asset price volatility—is a challenging task. Traditional models struggle with capturing the nuanced dynamics and empirical regularities of this surface, such as the "volatility smile." This paper aims to bridge this gap by developing a neural network that explicitly incorporates financial domain knowledge, making it both more accurate and more interpretable within the financial context.

Methods

The authors propose a novel neural network architecture that integrates prior financial domain knowledge in two key ways:

  • Custom Activation Function: A new activation function, termed the "smile function," is designed to directly incorporate the empirical observation of the "volatility smile" into the network. This function is applied to nodes processing the underlying asset price, helping the network learn this characteristic shape of implied volatility.
  • Constraint Embedding in Loss Function: Crucial financial conditions, such as the absence of arbitrage, boundary conditions, and asymptotic slope behaviors of the implied volatility surface, are embedded directly into the loss function used for training the neural network. This ensures that the model's predictions adhere to these fundamental financial principles, even when dealing with unseen data.

The overall architecture is a multi-model approach, combining several "single networks" that predict implied volatility independently. A "weighting network" then learns to combine the outputs of these single networks, effectively creating an ensemble that leverages diverse learned patterns.

Datasets

The study utilizes option data for the S&P 500 index, sourced from OptionMetrics, covering a period of 20 years (04/01/1996 to 29/04/2016). Zero-coupon yield curve data from OptionMetrics and Overnight Index Swap (OIS) data from Bloomberg were also used to construct the forward price. The data underwent significant cleaning, excluding options with very small bid-ask spreads and short maturities (less than 2 days). The ground truth implied volatility values were computed by inverting the Black-Scholes option pricing formula.

Key Insights and Ideas

  • Domain Knowledge as a Design Principle: The paper demonstrates that explicitly incorporating financial domain knowledge into neural network architecture and training can lead to significant improvements in both predictive performance and theoretical consistency. This moves beyond simply using ML as a black box.
  • The "Volatility Smile" is Learnable: The introduction of a custom activation function specifically designed to mimic the volatility smile shows that this empirical phenomenon can be effectively learned by neural networks when guided by appropriate design choices.
  • Constraints Drive Robustness: Embedding financial constraints (like no-arbitrage) into the loss function not only ensures theoretical consistency but also appears to improve the model's robustness and stability, especially when dealing with challenging datasets or out-of-sample predictions.
  • Ensemble of Specialized Networks: The multi-model architecture, where a weighting network combines outputs from specialized single networks, proves effective, suggesting that breaking down a complex problem into smaller, domain-aware components can be beneficial.

Key Results

The proposed "Multi" model significantly outperformed benchmark models, including the widely used Surface SVI (SSVI) model from mathematical finance and other neural network architectures.

  • Mean Average Percentage Error (MAPE): The Multi model achieved the lowest MAPEs for predicted implied volatilities in both training (1.74%) and test (3.34%) sets, outperforming SSVI (2.59% training, 3.73% test) and other neural network variants.
  • Option Price Prediction: When using the predicted implied volatilities to price options, the Multi model also showed superior performance, with MAPEs of 5.97% (training) and 10.64% (test), again outperforming SSVI and other models.
  • Empirical Satisfaction of Conditions: The paper provides evidence that the financial conditions embedded in the loss function are empirically satisfied by the trained model, with very low percentages of test samples violating these conditions.

Limitations and Future Work

  • Computational Cost: While the model achieves superior performance, the ensemble nature and the embedding of multiple constraints likely increase computational complexity during training.
  • Generalizability to Other Markets: The study focuses on the S&P 500 index. Further research could explore the model's performance on other asset classes or markets.
  • Interpretability of Weights: While the model is more aligned with financial theory, the specific learned weights within the ensemble and weighting network could be further analyzed for deeper insights.

The authors suggest that the developed framework for incorporating prior financial domain knowledge can be extended to other computational problems in finance and business analytics, such as inventory pricing and revenue management.

Summary updated

Abstract

In this paper we develop a novel neural network model for predicting implied volatility surface. Prior financial domain knowledge is taken into account. A new activation function that incorporates volatility smile is proposed, which is used for the hidden nodes that process the underlying asset price. In addition, financial conditions, such as the absence of arbitrage, the boundaries and the asymptotic slope, are embedded into the loss function. This is one of the very first studies which discuss a methodological framework that incorporates prior financial domain knowledge into neural network architecture design and model training. The proposed model outperforms the benchmarked models with the option data on the S&P 500 index over 20 years. More importantly, the domain knowledge is satisfied empirically, showing the model is consistent with the existing financial theories and conditions related to implied volatility surface.
BibTeX
@inproceedings{zheng2021incorporating,
  author = {Yu Zheng and Yongxin Yang and Bowei Chen},
  title = {{Incorporating prior financial domain knowledge into neural networks for implied volatility surface prediction}},
  booktitle = {ACM SIGKDD Conference on Knowledge Discovery and Data Mining},
  year = {2021},
  doi = {10.1145/3447548.3467115},
}
Figure