Leveraging Machine Learning Techniques and Engineering of Multi-Nature Features for National Daily Regional Ambulance Demand Prediction

Adrian Lin, Andrew Fu Wah Ho, Kang Hao Cheong, Zengxiang Li, Wentong Cai, Marcel Lucas Chee, Yih Yng Ng, Xiaokui Xiao, Marcus Eng Hock OngInternational Journal of Environmental Research and Public Health 2020 · DOI

TL;DR

This paper tackles the challenge of predicting daily ambulance demand at a regional level by developing a novel feature engineering approach that captures complex, multi-nature dependencies. They found that the LightGBM model, leveraging features like historical demand and region ID, offers the best prediction accuracy, highlighting the importance of both temporal and spatial factors in ambulance demand.

What Problem Does This Paper Address?

Accurately predicting ambulance demand is crucial for efficient emergency service operations, ensuring timely patient access to care and optimizing resource allocation. However, this prediction is complex due to several factors:

  • Spatial Dependencies: Demand varies significantly by region, influenced by demographics like the elderly population or the specific function of an area (e.g., financial district vs. residential).
  • Temporal Dependencies: Demand exhibits periodic patterns based on the day of the week, month, and even short-term historical trends (e.g., following an outbreak or event).
  • Non-linear Dynamics: The interplay of these factors creates a complex, non-linear system that is difficult to model with traditional methods.

Existing research often focuses on city-wide predictions or uses simplified spatial grids, failing to capture the nuanced regional variations and multi-nature dependencies that are critical for effective planning. This paper aims to bridge this gap by developing a comprehensive approach to model these dependencies for regional-level prediction.

Methods

The core of this research lies in a two-stage approach: Feature Engineering and Machine Learning Prediction.

  1. Feature Engineering: This is where the novelty lies. The authors go beyond simple historical data to engineer features that capture various aspects of ambulance demand:
  2. Attributes: These include:
  3. Spatial: Region ID (Development Guide Plan region).
  4. Temporal: Day of week, day of month, month of year.
  5. Demographic: Number of people aged 50 and above in a region.
  6. Short-term Historical Demands: Demand from the previous 7 days.
  7. Long-term Historical Aggregated Demands: Total demand over the past 7, 30 days, and the current week/month.
  8. Socioeconomic Features (explored but not ultimately beneficial): Data on transportation habits, employment status, and housing tenure.
  9. Spatiotemporal Composite Features (explored but not ultimately beneficial): Combinations of region ID with temporal features.

    They utilize a sliding-window approach to extract these historical demand features, ensuring that the model learns from relevant past data.

  10. Machine Learning Prediction: Several state-of-the-art machine learning models were evaluated for their predictive performance:

  11. Regional Moving Average (baseline)
    • Linear Regression
    • Support Vector Regression (SVR)
    • Multi-layer Perceptron (MLP)
    • Radial Basis Function Network (RBFN)
    • Light Gradient Boosting Machine (LightGBM): This gradient-boosting decision tree algorithm was found to be the most effective.

The models were trained on data from 2006-2015 and validated on 2016 data.

Datasets

The primary dataset used is a comprehensive 10-year (2006-2016) ambulance call dataset from the Singapore Civil Defence Force (SCDF). This dataset includes detailed information about each incident, such as time, location, classification, and patient demographics.

To enrich the dataset, they also incorporated:

  • 2010 Planning Area Census: Used to map incidents to specific Development Guide Plan (DGP) regions.
  • Polyclinic data: Provided demographic information like population counts by age.
  • 2010 Census of Population: Offered socioeconomic data.

The paper introduces the SCDF-Engineered dataset, which is the processed version of the SCDF data with the engineered features. They also created SCDF-Engineered-Socio (including socioeconomic features) and SCDF-Engineered-ST (including spatiotemporal composite features) for comparative analysis.

Key Insights and Ideas

  • Multi-nature dependencies are key: The success of the model hinges on capturing the interplay of spatial, temporal, and demographic factors, not just historical demand.
  • Region ID is a powerful predictor: Differentiating demand based on the specific region is crucial, as different areas have vastly different demand characteristics. This suggests that localized planning is essential.
  • Long-term historical demand matters: Aggregated demand over longer periods (30 days) is more informative than just recent spikes, helping to smooth out temporary anomalies and reveal underlying trends.
  • Demographics provide valuable context: The number of older individuals (aged 50+) in a region is a significant predictor, aligning with the intuition that age is a factor in emergency service needs.
  • Socioeconomic data might be redundant: Including socioeconomic features did not improve prediction accuracy, possibly because their influence is already captured by the region ID and demographic features, or because they are static over time.
  • Spatiotemporal features can be noisy: While intuitively appealing, engineered spatiotemporal features did not improve performance and may have introduced noise for the LightGBM model.

Key Results

  • LightGBM outperforms other models: LightGBM achieved the best accuracy across multiple metrics, including Weighted Absolute Percentage Error (WAPE), Mean Absolute Error (MAE), and Mean Squared Error (MSE).
  • Top performing model (LightGBM) WAPE: 24.5% (comparable to Linear Regression at 24.5% but with better stability).
  • Most important features:
    1. Total demand over the past 30 days.
  • Total demand over the past 7 days.
  • Region ID.
  • Total number of people aged 50 and above.
  • Day of month, day of week, and month of year.
  • Socioeconomic features did not improve accuracy: WAPE remained at 22.0% for both the dataset with and without socioeconomic features.
  • Spatiotemporal composite features worsened accuracy: WAPE increased to 24.7% when these features were added.

Limitations and Future Work

  • Prediction Accuracy: While the achieved WAPE of ~24.5% is considered satisfactory, it's noted that this is higher than predictions for other services like taxis. This is attributed to the lower average daily demand for ambulances and less pronounced periodicity.
  • External Factors: The model does not currently incorporate external factors like weather conditions, which could influence ambulance demand.
  • Extreme Events: The current model is designed for typical conditions and may not accurately predict demand during large-scale incidents like epidemics or haze events.
  • Granularity: The study predicts daily demand. Future work could explore hourly predictions, but this may lead to data sparsity issues.
  • Data Sparsity: Predicting at finer granularities (e.g., hourly) could exacerbate data sparsity, making accurate training challenging.
  • Operational Constraints: The practical implementation of optimized ambulance placement or staffing might be limited by rigid shift patterns in EMS systems.

Future work suggested includes:

  • Incorporating external data sources (weather, etc.).
  • Developing models for extreme event prediction.
  • Investigating finer-grained demand prediction (e.g., hourly).
  • Exploring insights from other vehicle datasets (taxis, public transport) to understand broader medical demand patterns and accessibility.

Summary updated

Abstract

The accurate prediction of ambulance demand provides great value to emergency service providers and people living within a city. It supports the rational and dynamic allocation of ambulances and hospital staffing, and ensures patients have timely access to such resources. However, this task has been challenging due to complex multi-nature dependencies and nonlinear dynamics within ambulance demand, such as spatial characteristics involving the region of the city at which the demand is estimated, short and long-term historical demands, as well as the demographics of a region. Machine learning techniques are thus useful to quantify these characteristics of ambulance demand. However, there is generally a lack of studies that use machine learning tools for a comprehensive modeling of the important demand dependencies to predict ambulance demands. In this paper, an original and novel approach that leverages machine learning tools and extraction of features based on the multi-nature insights of ambulance demands is proposed. We experimentally evaluate the performance of next-day demand prediction across several state-of-the-art machine learning techniques and ambulance demand prediction methods, using real-world ambulatory and demographical datasets obtained from Singapore. We also provide an analysis of this ambulatory dataset and demonstrate the accuracy in modeling dependencies of different natures using various machine learning techniques.
BibTeX
@article{lin2020leveraging,
  author = {Adrian Lin and Andrew Fu Wah Ho and Kang Hao Cheong and Zengxiang Li and Wentong Cai and Marcel Lucas Chee and Yih Yng Ng and Xiaokui Xiao and Marcus Eng Hock Ong},
  title = {{Leveraging Machine Learning Techniques and Engineering of Multi-Nature Features for National Daily Regional Ambulance Demand Prediction}},
  journal = {International Journal of Environmental Research and Public Health},
  year = {2020},
  doi = {10.3390/ijerph17114179},
}
Figure