Learning Random Forest from Histogram Data Using Split Specific Axis Rotation

Ram B. Gurung, Tony Lindgren, Henrik BoströmInternational Journal of Machine Learning and Computing 2018 · DOI

Abstract

Machine learning algorithms for data containing histogram variables have not been explored to any major extent. In this paper, an adapted version of the random forest algorithm is proposed to handle histogram data by treating histograms as cohesive units rather than independent bins. By applying local Principal Component Analysis (PCA) to find optimal splitting hyperplanes at each node, the proposed method significantly improves predictive performance on complex industrial datasets, such as truck sensor failures.

Introduction

In modern industrial applications, data is frequently aggregated into histograms to summarize high-frequency sensor readings or time-series behavior efficiently. Standard machine learning approaches, including traditional Random Forests, typically treat each histogram bin as an independent numerical feature. This approach is fundamentally flawed as it ignores the inherent spatial or temporal dependencies between adjacent bins. A shift in a value from one bin to the next is perceived by standard models as two unrelated changes, failing to capture the underlying physical phenomenon. To address this, we propose a model that treats the entire histogram as a single, multi-dimensional entity.

Methodology: Split Specific Axis Rotation

The core innovation of this research is the Split Specific Axis Rotation technique, which allows the Random Forest to find more informative decision boundaries within the multi-dimensional space of a histogram.

1. Sliding Window Grouping

To maintain local dependencies, the algorithm utilizes a sliding window to group subsets of related bins. For example, a window size of 3 might group bins 1-3, 2-4, and so on. This ensures that the model evaluates the relationship between neighboring data points rather than isolated bins.

Sliding window approach for histogram and matrix variables.

2. Local PCA and Axis Rotation

When evaluating a split at a specific node, the algorithm randomly selects a subset of these bin groups and applies Principal Component Analysis (PCA) locally to the data present at that node. PCA identifies the principal components—the directions of maximum variance. By rotating the feature space based on these components, the algorithm can implement "diagonal" splits. Unlike standard trees that are restricted to axis-parallel splits (splitting on one bin at a time), this method considers multiple bins simultaneously to find the most discriminative hyperplane.

The process of identifying optimal splitting hyperplanes using local data distribution.

3. Shadow Variables for Temporal Dynamics

For time-series data, the state of a histogram at a single point in time is often less predictive than how it changes. We introduced "shadow variables" representing the Simple Average Rate (SAR) and Weighted Average Rate (WAR) of change for each bin over time. These variables capture the evolution of the histogram, providing critical context for predictive maintenance tasks.

Experimental Results

The proposed algorithm was evaluated against standard Random Forests on both synthetic and real-world datasets:

  • Synthetic Data: On linear synthetic datasets, the Histogram RF (PCA) achieved an AUC of 0.9839, significantly outperforming the standard Random Forest's 0.9633. Notably, the PCA-based trees were nearly 50% smaller, suggesting that the rotated splits are far more informative and efficient at partitioning the data.
  • Real-World Application (Scania NOx Sensors): Using a dataset of ~12,500 heavy-duty trucks, the goal was to predict NOx sensor failures. The Histogram RF with PCA and Weighted Average Rates achieved the highest performance with an AUC of 0.8809, compared to 0.8519 for the standard model. Variable importance analysis highlighted that the rate of change in engine load histograms was the most significant predictor of failure.

Conclusion

By treating histograms as integrated units and utilizing local PCA for axis rotation, the proposed Random Forest adaptation offers a more robust framework for analyzing aggregated sensor data. The method not only improves predictive accuracy but also results in more interpretable, compact tree structures. Future work will explore extending this logic to Random Survival Forests to predict specific time-to-failure metrics for industrial assets.

Summary updated

Abstract

Machine learning algorithms for data containing histogram variables have not been explored to any major extent. In this paper, an adapted version of the random forest algorithm is proposed to handl ...
BibTeX
@article{gurung2018learning,
  author = {Ram B. Gurung and Tony Lindgren and Henrik Boström},
  title = {{Learning Random Forest from Histogram Data Using Split Specific Axis Rotation}},
  journal = {International Journal of Machine Learning and Computing},
  year = {2018},
  doi = {10.18178/ijmlc.2018.8.1.666},
}
Figure