Housing Prices Prediction with a Deep Learning and Random Forest Ensemble

Bruno Klaus de Aquino Afonso, Luckeciano C. Melo, Willian Dihanster Gomes de Oliveira, Samuel Sousa, Lilian Berton2019 · DOI

Abstract

The development of a housing prices prediction model can assist a house seller or a real estate agent to make better-informed decisions based on house price valuation. Only a few works report the use of machine learning (ML) algorithms to predict the values of properties in Brazil. This study analyzes a dataset composed of 12,223,582 housing advertisements, collected from Brazilian websites from 2015 to 2018. Each instance comprises twenty-four features of five different data types: integer, date, string, float, and image. To predict the property prices, we ensemble two different ML architectures, based on Random Forest (RF) and Recurrent Neural Networks (RNN). This study demonstrates that enriching the dataset and combining different ML approaches can be a better alternative for prediction of housing prices in Brazil.

Introduction

Predicting real estate prices is a complex task because property value is driven by a combination of structured data (e.g., number of rooms, square footage) and unstructured information found in listing descriptions and photos. In the Brazilian market, this challenge is exacerbated by a lack of published research and datasets plagued by significant missing values. For instance, attributes like the number of floors were missing in over 90% of the records in the dataset used for this study.

Missing rates for various features in the dataset.

The researchers sought to build a robust system capable of handling "messy" real-world data at scale, specifically targeting the Brazilian market from 2015 to 2018.

Methods: An Ensemble Approach

The study utilized an ensemble of two distinct architectures to leverage the strengths of different machine learning paradigms:

1. Enriched Random Forest (RF)

This model focused on tabular and numeric data. To address the high rate of missing values, the authors used the Google Maps API to "geocode" addresses, filling in missing coordinates. They also enriched the dataset with Brazilian Institute of Geography and Statistics (IBGE) metadata, adding city-level economic indicators such as average monthly salary and GDP per capita.

2. The KISS Model ("Keep It Simple, Stupid")

The KISS model is a multi-modal deep learning architecture designed to process various data types simultaneously:

  • Textual Features: Bidirectional Long Short-Term Memory (BiLSTM) networks were used to process listing titles and descriptions, capturing linguistic context.
  • Images: Features were extracted from property photos using NASNetMobile (a pre-trained convolutional neural network) combined with a Multilayer Perceptron via transfer learning.
  • Integration: Tabular, text, and image features were fed into a shared embedding layer and a dense layer for final regression.
The multi-modal KISS model architecture.

The final prediction was calculated as a weighted average of the Enriched RF and KISS models.

Dataset and Enrichment

The study utilized a massive dataset of 12,223,582 housing advertisements provided by Properati BR for the 2019 EEF Data Science Challenge.

  • Data Composition: 8.5 million training samples and 3.6 million testing samples.
  • Feature Engineering: Beyond geocoding and IBGE enrichment, the authors used regular expressions to extract price values mentioned directly within listing titles or descriptions, effectively capturing "leaked" data to boost accuracy.
  • Preprocessing: A logarithmic transformation was applied to the target price to handle the long-tail distribution of housing values, facilitating model convergence.

Results

The ensemble approach secured first place in the EEF 2019 competition, significantly outperforming individual models and other competitors. The ensemble's Root Mean Squared Log Error (RMSLE) was 0.23847, which was 8.75% lower than the KISS model alone and far superior to the competition's top 10 average of 0.45141.

Data enrichment played a crucial role; adding external economic data and fixing coordinates led to a 34.8% decrease in error for the Random Forest component.

Conclusion

This research highlights that while Random Forests excel at deriving rules from numeric features, they struggle with unstructured data like text and images. Conversely, deep learning models like KISS are adept at representing unstructured data but may be less precise with pure numerics. Combining these approaches allows the strengths of one to compensate for the weaknesses of the other. Future work will focus on feature selection to reduce computational intensity and exploring weakly supervised learning to further scale the training process.

Summary updated

Abstract

The development of a housing prices prediction model can assist a house seller or a real estate agent to make better-informed decisions based on house price valuation. Only a few works report the use of machine learning (ML) algorithms to predict the values of properties in Brazil. This study analyzes a dataset composed of 12,223,582 housing advertisements, collected from Brazilian websites from 2015 to 2018. Each instance comprises twenty-four features of five different data types: integer, date, string, float, and image. To predict the property prices, we ensemble two different ML architectures, based on Random Forest (RF) and Recurrent Neural Networks (RNN). This study demonstrates that enriching the dataset and combining different ML approaches can be a better alternative for prediction of housing prices in Brazil.
BibTeX
@misc{afonso2019housing,
  author = {Bruno Klaus de Aquino Afonso and Luckeciano C. Melo and Willian Dihanster Gomes de Oliveira and Samuel Sousa and Lilian Berton},
  title = {{Housing Prices Prediction with a Deep Learning and Random Forest Ensemble}},
  year = {2019},
  doi = {10.5753/eniac.2019.9300},
}
Figure