Abstract
Deep Learning is a consolidated, state-of-the-art Machine Learning tool to fit a function when provided with large data sets of examples. However, in regression tasks, the straightforward application of Deep Learning models provides a point estimate of the target. In addition, the model does not take into account the uncertainty of a prediction. This represents a great limitation for tasks where communicating an erroneous prediction carries a risk. In this paper we tackle a real-world problem of forecasting impending financial expenses and incomings of customers, while displaying predictable monetary amounts on a mobile app. In this context, we investigate if we would obtain an advantage by applying Deep Learning models with a Heteroscedastic model of the variance of a network's output. Experimentally, we achieve a higher accuracy than non-trivial baselines. More importantly, we introduce a mechanism to discard low-confidence predictions, which means that they will not be visible to users. This should help enhance the user experience of our product.
Introduction
Forecasting personal financial expenses is notoriously difficult. Unlike physical systems, human spending habits produce "short and noisy" series—often just a few dozen data points prone to erratic spikes or sudden changes. Standard Deep Learning (DL) models for regression typically provide a point estimate: a single number representing the forecast. However, these models lack a mechanism to communicate their confidence level.
In a production environment like a mobile banking application, showing a wildly incorrect prediction is worse than showing nothing at all because it erodes user trust. The challenge lies in creating a model that can "know when it doesn't know," allowing the system to filter out unreliable forecasts before they reach the consumer.
Methodology: Modeling Heteroscedastic Uncertainty
The core innovation of this research is the move beyond standard Mean Squared Error (MSE) loss toward a probabilistic approach. Instead of predicting a single value $y$, the network is designed to predict the parameters of a probability distribution.
Heteroscedastic Aleatoric Uncertainty
The authors focus on "aleatoric" uncertainty—noise inherent in the data itself. Specifically, they model heteroscedastic uncertainty, where the noise level varies depending on the input ($x$). As shown in the architectural comparison below, the network utilizes two "heads": one to predict the expected value (the forecast) and another to predict the uncertainty or scale ($b_{ale}$).
While many probabilistic models assume a Normal (Gaussian) distribution, this study opts for a Laplace distribution. The Laplace distribution was found to be more stable during training for noisy financial data, as it is less sensitive to extreme outliers than the squared-error nature of Gaussian-based loss functions.
Data and Experimental Setup
The study utilizes a massive, real-world industrial dataset from BBVA, comprising 2 million anonymized monthly time series for training and 1 million for testing. Each series consists of 24 months of aggregated financial data per category (e.g., groceries, rent, salary). Data was normalized using a scale-invariant transformation to ensure the model learned generalizable patterns rather than absolute currency values, which vary significantly across the user base.
The researchers tested both standard Dense Networks and Long Short-Term Memory (LSTM) networks, the latter being specifically suited for sequential time-series data.
Key Results and the Error-Reject Trade-off
The proposed LSTMHet (LSTM with Heteroscedastic uncertainty) outperformed all baselines, including the Random Forest model previously used in production.
- Raw Performance: At a 100% "keep" rate (predicting for all users), the LSTMHet achieved a Mean Absolute Error (MAE) of 218.1, compared to 232.1 for the Random Forest baseline.
- The Power of Rejection: The most significant finding is the "Error-Keep" curve. By ranking predictions by their predicted uncertainty and setting a threshold to "reject" (hide) the most uncertain forecasts, accuracy improves dramatically. When the model was allowed to select only the most confident 25% of cases, the MAE plummeted to 5.04.
This confirms that while some financial behaviors are essentially random, a significant subset is highly predictable if the model can successfully identify them.
Conclusion
By accounting for aleatoric noise, the model becomes more robust and provides a reliable metric for filtering forecasts. This "rejection mechanism" ensures that only high-confidence predictions are displayed in the mobile app, directly enhancing the user experience by prioritizing reliability over quantity. Future work will explore combining aleatoric uncertainty with epistemic uncertainty (model-weight uncertainty) to further refine the system's ability to handle "unknown unknowns."
Summary updated