Abstract
This paper presents a novel approach to optimizing profit margins in non-life insurance markets through a gradient descent-based method, targeting three key objectives: 1) maximizing profit margins, 2) ensuring conversion rates, and 3) enforcing fairness criteria such as demographic parity (DP). Traditional pricing optimization, which heavily lean on linear and semi definite programming, encounter challenges in balancing profitability and fairness. These challenges become especially pronounced in situations that necessitate continuous rate adjustments and the incorporation of fairness criteria. Specifically, indirect Ratebook optimization, a widely-used method for new business price setting, relies on predictor models such as XGBoost or GLMs/GAMs to estimate on downstream individually optimized prices. However, this strategy is prone to sequential errors and struggles to effectively manage optimizations for continuous rate scenarios. In practice, to save time actuaries frequently opt for optimization within discrete intervals (e.g., range of [-20%, +20%] with fix increments) leading to approximate estimations. Moreover, to circumvent infeasible solutions they often use relaxed constraints leading to suboptimal pricing strategies. The reverse-engineered nature of traditional models complicates the enforcement of fairness and can lead to biased outcomes. Our method addresses these challenges by employing a direct optimization strategy in the continuous space of rates and by embedding fairness through an adversarial predictor model. This innovation not only reduces sequential errors and simplifies the complexities found in traditional models but also directly integrates fairness measures into the commercial premium calculation. We demonstrate improved margin performance and stronger enforcement of fairness highlighting the critical need to evolve existing pricing strategies.
Introduction
In the non-life insurance industry, such as automobile or home insurance, determining the optimal commercial premium is a complex balancing act. Actuaries must maximize profit margins while maintaining competitive conversion rates and ensuring ethical fairness. Traditional "Ratebook optimization" methods often fall short because they rely on discrete price adjustments and indirect modeling, which can introduce sequential errors and fail to address biases that emerge at the commercial pricing layer.
The Limitations of Traditional Pricing
Conventional methods typically involve searching through a grid of fixed price points (e.g., testing price changes in 1% increments). This discrete approach is computationally inefficient and often results in suboptimal, approximate solutions. Furthermore, even when the underlying risk models are fair, the optimization process for profit and conversion can reintroduce bias. For instance, segments of the population that are less price-elastic—often more vulnerable groups—may be inadvertently targeted with higher margins, a phenomenon known as the "commercial bias trap."
The OptiGrad Framework
To overcome these hurdles, we propose OptiGrad, a framework that shifts insurance pricing into a fully differentiable, continuous optimization space.
1. Continuous Optimization via Gradient Descent
Instead of a grid search, OptiGrad utilizes gradient descent to find the optimal commercial coefficients. By ensuring that the risk (pure premium) and conversion models are differentiable—using architectures like Generalized Linear Models (GLMs) or neural networks—we can apply the chain rule to understand exactly how marginal price changes impact the total portfolio.
2. Bounded Constraints
To maintain realistic price fluctuations (e.g., within a [-20%, +20%] range), OptiGrad employs a transformed sigmoid function. This ensures the output remains within safe operational bounds while preserving the mathematical smoothness required for gradient-based optimization.
3. Adversarial Fairness (Fair-OptiGrad)
The core innovation of our approach is the integration of an adversarial neural network. This adversary attempts to predict sensitive attributes (such as age or employment status) from the final proposed price. The pricing model is trained to maximize profit while simultaneously minimizing the adversary's success. We utilize the Hirschfeld-Gebelein-Rényi (HGR) coefficient to detect and mitigate both linear and complex non-linear dependencies, ensuring a higher standard of fairness than traditional correlation metrics.
Experimental Results
We evaluated OptiGrad using the Atoti Dataset, a collection of 46,129 automobile insurance quotes. Our results demonstrate that OptiGrad consistently outperforms traditional indirect optimization methods.
As shown in the efficiency frontier, OptiGrad achieves a superior Global Written Margin (GWM) for given levels of customer conversion compared to baseline models. Furthermore, the model successfully navigates the trade-off between fairness and performance.
By adjusting the fairness penalty ($\lambda$), we reduced the HGR unfairness score significantly—from 0.15 to 0.07 for age—with only a minimal impact on the efficiency frontier. This indicates that insurers can achieve ethical pricing without sacrificing substantial profitability.
Conclusion
OptiGrad bridges the gap between traditional actuarial science and modern machine learning. By framing pricing as a differentiable optimization problem and embedding adversarial fairness directly into the commercial layer, we provide a more efficient and ethical path forward for the insurance industry. Future work will explore additional fairness criteria, such as Equalized Odds, and the behavior of these models in highly competitive, multi-actor markets.
Summary updated