Data-driven Neural Architecture Learning For Financial Time-series Forecasting

Dat Thanh Tran, Juho Kanniainen, Moncef Gabbouj, Alexandros IosifidisarXiv 2019 · arXiv

TL;DR

Financial time-series forecasting is notoriously difficult due to nonstationarity, complex nonlinearities, and severe class imbalances. This paper adapts a progressive neural architecture learning algorithm called HeMLGOP to automatically learn tailored, heterogeneous network structures for mid-price movement prediction. By replacing standard neurons with Generalized Operational Perceptrons (GOPs) and introducing a class-balanced objective function, the model outperforms traditional vector-based and complex tensor-based methods on a massive Limit Order Book dataset.

What Problem Does This Paper Address?

Predicting financial market movements—such as mid-price shifts in High-Frequency Trading (HFT)—is highly challenging. Traditional statistical models like ARIMA rely on unrealistic assumptions about data stationarity. While deep learning models can capture complex nonlinear relationships, standard architectures are typically hand-crafted using heuristics, forcing a rigid, uniform structure that may not fit the data.

Furthermore, financial datasets are heavily imbalanced; stock prices remain stationary far more often than they jump up or down. Standard neural network training optimization tends to bias models toward this majority class, leading to high accuracy but terrible predictive power for actual price movements. The authors address the need for an automated, data-driven way to learn optimal neural network topologies that can handle both complex nonlinearities and severe class imbalances without the massive inference costs associated with ensemble methods.

Methods

The authors adapt the Heterogeneous Multilayer Generalized Operational Perceptron (HeMLGOP) framework to financial time-series forecasting.

  • Generalized Operational Perceptrons (GOPs): Instead of standard McCulloch-Pitts neurons (which simply perform a weighted sum followed by an activation function), GOPs break down a neuron's transformation into three distinct operations: nodal (modifying incoming signals with weights), pooling (summarizing the modified signals), and activation (thresholding). Each GOP block selects its operators from a predefined set of diverse mathematical functions, enabling a much richer space of nonlinear transformations.
  • Progressive Architecture Learning: The network topology is grown step-by-step. The algorithm sequentially adds blocks of GOPs to a layer, searching for the best operator set using a randomized approach. Growth within a layer, or the addition of new layers, automatically terminates when performance improvements saturate below a defined threshold ($\epsilon$).
  • Class-Balanced Objective Function: To tackle class imbalance during the architecture search, the authors modify the framework's internal optimization step. When evaluating potential operator sets, the output layer weights are calculated using a re-weighted least-squares problem. The mean squared error contribution of each training sample is scaled inversely proportional to the popularity of its target class, preventing the architecture search from favoring majority-class structures.

Datasets

  • FI-2010 Dataset: A large-scale, public Limit Order Book (LOB) benchmark containing over 4 million limit orders from 5 Finnish stocks traded on the Nasdaq Nordic exchange over 10 working days. The dataset provides 144-dimensional feature vectors extracted from order events. The task evaluated is predicting the mid-price movement (up, down, or stationary) over a short-term horizon of the next $H=10$ events. The authors utilize the standard 9-fold anchored forward cross-validation split setup on a day-by-day basis.

Key Insights and Ideas

  • Structural Heterogeneity Wins: Allowing different hidden layers and blocks to utilize distinct mathematical operations allows the network to adapt to the specific, localized nonlinearities of financial data far better than uniform, standard networks.
  • Architecture Search Requires Class Awareness: If a progressive network searches for its own topology using standard Mean Squared Error on an imbalanced dataset, it will naturally select structures that excel only at predicting the majority class. Embedding class weights directly into the randomized search phase ensures the algorithm selects structurally robust models for minority classes.
  • Data Efficiency vs. Structural Flexibility: Interestingly, the vector-based HeMLGOP model (which only looks at the 10 most recent events) outperformed tensor-based methods that utilize a much longer history (100+ past events). This suggests that structural flexibility in the neuron model can be more powerful than simply feeding a rigid model longer historical sequences.

Key Results

Performance was evaluated primarily using the F1 score to properly account for the class imbalance.

  • HeMLGOP achieved the highest overall F1 score of 49.43%.
  • It outperformed the second-best vector-based method (Neural Bag-of-Features, or N-BoF) by nearly 8% in average F1 score.
  • Crucially, it beat advanced tensor-based methods like Weighted Multilinear Tensor Regression (WMTR, 47.87%) and Multilinear Class-specific Discriminant Analysis (MCSDA, 46.72%), despite those methods having access to a significantly larger window of historical data.
  • While some standard progressive methods (like BLS and S-ELM) showed higher raw accuracy (~89%), their recall was extremely low (~33-34%), proving they simply defaulted to predicting the majority class. HeMLGOP maintained a healthy balance between precision (48.57%) and recall (50.67%).

The complete performance breakdown can be seen in Table 1:

Table 1. Prediction performance (%) on FI-2010. Asterisk (*)

Limitations and Future Work

  • Predefined Operator Pools: While the network automatically selects the best combination of operators, the pool of available mathematical functions must still be predefined by humans; the model cannot invent entirely new mathematical operations from scratch.
  • Computational Complexity of Search: Evaluating all combinations of nodal, pooling, and activation operators via the randomized approach can be computationally demanding on massive datasets, though mitigated by block-sharing constraints.
  • Future Directions: The authors note that extending this automated architecture learning framework to natively handle tensor inputs—combining structural learning with long-sequence temporal modeling—could yield even higher forecasting performance.

Summary updated

Abstract

Forecasting based on financial time-series is a challenging task since most real-world data exhibits nonstationary property and nonlinear dependencies. In addition, different data modalities often embed different nonlinear relationships which are difficult to capture by human-designed models. To tackle the supervised learning task in financial time-series prediction, we propose the application of a recently formulated algorithm that adaptively learns a mapping function, realized by a heterogeneous neural architecture composing of Generalized Operational Perceptron, given a set of labeled data. With a modified objective function, the proposed algorithm can accommodate the frequently observed imbalanced data distribution problem. Experiments on a large-scale Limit Order Book dataset demonstrate that the proposed algorithm outperforms related algorithms, including tensor-based methods which have access to a broader set of input information.
BibTeX
@misc{tran2019data,
  author = {Dat Thanh Tran and Juho Kanniainen and Moncef Gabbouj and Alexandros Iosifidis},
  title = {{Data-driven Neural Architecture Learning For Financial Time-series Forecasting}},
  year = {2019},
  eprint = {1903.06751},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1903.06751},
}
Figure