Data-Driven Structural Fire Risk Prediction for City Properties

Rupasree Dey, Alan FernAAAI 2024 · DOI

TL;DR

This paper introduces a data-driven approach to predict structural fire risk for city properties, enabling fire departments to optimize inspection resources. By leveraging historical fire incident data and property characteristics with XGBoost, the model significantly outperforms random rankings, offering a more principled way to identify high-risk areas.

What Problem Does This Paper Address?

Fire departments play a crucial role in public safety by conducting inspections to prevent fires and ensure compliance with safety regulations. However, with limited resources and a vast number of properties to inspect, prioritizing these inspections effectively is a significant challenge. Current methods often rely on intuition, experience, or simplistic rule-based heuristics, which may not fully capture the complex factors influencing fire risk. This leads to a pressing need for a more robust, data-driven approach that can objectively identify properties with a higher likelihood of experiencing a fire incident, thereby optimizing the allocation of valuable inspection resources.

Methods

The core of this research lies in applying machine learning, specifically XGBoost, to predict a fire risk score for individual properties. The problem is framed as a ranking problem, where the goal is to estimate the probability of a property experiencing a fire event.

Key methodological contributions include:

  • Feature Engineering: A comprehensive set of 24 features (14 numeric, 10 categorical) were engineered, including aggregated time-variant features (e.g., number of inspections, code violations) and a geohash feature for location encoding. Categorical features were one-hot encoded, resulting in a total of 698 features.
  • Handling Class Imbalance: Fires are rare events, leading to a highly skewed class distribution (only 1.3% of properties had fire incidents in the dataset). The authors addressed this by adjusting class weights in the loss function and using resampling-based approaches, rather than synthetic oversampling methods like SMOTE, which they argue could introduce bias.
  • Handling Missing Data: Instead of imputing missing values, which can introduce assumptions and bias, the authors utilized XGBoost's native "Sparsity Aware Split-finding Mechanism" to handle missing features directly.
  • Model Choice: XGBoost was chosen for its scalability, stability, and fast runtimes, and its native support for missing features and imbalanced class distributions. The model's performance was evaluated using the Area Under the Curve (AUC) metric, which is well-suited for ranking problems and less affected by class imbalance.

Datasets

The study utilized a dataset from a South Dakota fire department, comprising over 72,000 properties and historical fire incident data. This dataset included information from multiple sources, such as:

  • Property characteristics (e.g., Age, FCODE, Construction Type, Occupancy, SQFT, Land Value)
  • Code Case Violations
  • Crime Data
  • Utility Disconnections
  • Rental Registrations
  • Fire Incident Data
  • Fire Inspections
  • Parcel Data

The paper notes that while other similar systems exist, their datasets are often not publicly available, making direct comparison challenging.

Key Insights and Ideas

  • XGBoost's Strength with Missing Data: The research highlights that XGBoost's ability to handle missing values directly is a significant advantage, leading to better performance compared to models that require imputation or data removal. Removing properties with missing values substantially degraded performance.
  • Feature Importance Reveals Intuitive Risk Factors: The analysis of feature importance revealed that factors like dwelling type ("Multi-Family Dwelling"), specific geohash regions (indicating high-density areas), ZIP codes, crime volume, prior inspection counts, utility disconnections, construction types (especially wood-framed), and fire code violations are strong predictors of fire risk. This aligns well with intuition, building trust in the model.
  • Class Imbalance Handling: The authors found that adjusting class weights in XGBoost (using a "positive scaling weight") was less impactful than expected when dealing with missing data, suggesting that XGBoost's inherent mechanisms for handling both missing values and imbalance are quite effective.
  • Ranking is Key: The primary goal is not just predicting a binary fire/no-fire outcome, but accurately ranking properties by their risk. The cumulative fire curves clearly demonstrate that the XGBoost model, even with a weight of 1, significantly outperforms random rankings, indicating a more efficient allocation of inspection resources.

Key Results

  • Superior Ranking Performance: The data-driven approach using XGBoost significantly outperformed random rankings. For example, among the top 100 ranked properties, the model identified 16 properties with fire incidents, compared to only 1 for a random ranking.
  • AUC Scores:
    • On the full dataset (with missing values), XGBoost achieved a test AUC of 0.757.
  • When properties with missing values were removed, XGBoost's AUC dropped to 0.598 (without scaling) and 0.603 (with scaling), while ADABoost and Random Forest performed better (around 0.69). This underscores the value of XGBoost's missing data handling.
  • Feature Importance: Features related to property characteristics (dwelling type, location via geohash/ZIP code), historical inspection data, and violation records were identified as most important for predicting fire risk.

Limitations and Future Work

  • Limited Feature Subset Exploration: The authors acknowledge that they did not perform an extensive exploration of feature subsets, which could be a potential avenue for future work to further optimize the feature set.
  • Interplay of Missing Data and Scaling: The observed negative interaction between the positive scaling parameter and missing data handling in XGBoost warrants further investigation.
  • Alternative Models and Imputation: While XGBoost performed well on the full dataset, ADABoost and Random Forest showed better performance on the reduced dataset. Future work could explore combining these models with advanced imputation techniques for the full dataset.
  • Broader Prediction Scope: Extending the analysis to predict other fire categories, such as severity or cause, could provide more nuanced insights and aid in developing targeted prevention strategies.
  • Evaluation Metrics: Exploring alternative evaluation metrics that better align with the specific needs and priorities of fire departments is suggested.
  • Deployment Challenges: Maintaining the models with updated data and retraining them will require ongoing investigation from a cost-benefit perspective.

Summary updated

BibTeX
@inproceedings{dey2024data,
  author = {Rupasree Dey and Alan Fern},
  title = {{Data-Driven Structural Fire Risk Prediction for City Properties}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i21.30325},
}
Figure