Predicting the Past: Estimating Historical Appraisals with OCR and Machine Learning

Mihir Bhaskar, Jun Tao Luo, Zihan Geng, Asmita Hajra, Junia Howell, Matthew R. GormleyarXiv 2025 · DOI · arXiv

Introduction

In the 1930s, United States housing policies institutionalized "redlining," a discriminatory practice that embedded racial bias into property appraisals and mortgage lending. While the long-term social and demographic consequences of these policies are well-documented, the precise financial impact—specifically how these appraisals affected individual property values over decades—remains a "missing variable" in economic history. The necessary data exists, but it is largely inaccessible, trapped on hundreds of thousands of physical index cards stored in county auditor basements.

An example of a historical property appraisal card from Hamilton County, Ohio, showing the complex handwritten tabular data.

Manually transcribing these records is a prohibitively expensive and slow process for researchers. Furthermore, many jurisdictions have not yet scanned their physical archives, creating a significant data gap. This research addresses these hurdles by developing a cost-effective pipeline to digitize 1930s property appraisal records using a combination of computer vision, deep learning, and predictive modeling.

Methodology: A Two-Pronged Approach

The researchers developed a hybrid system designed to handle both existing scanned records and properties where physical records are missing or unscanned.

The proposed pipeline for digitizing historical values using both OCR for scanned records and machine learning for missing data.

1. The Two-Stage OCR Pipeline

For counties with scanned images, the team implemented a specialized Optical Character Recognition (OCR) workflow. Standard OCR tools like Tesseract often struggle with the noisy, handwritten digits found on aging property cards.

  • Alignment and Segmentation: Historical cards are frequently scanned at irregular angles or with varying margins. To solve this, the team used ORB (Oriented FAST and Rotated BRIEF) feature matching. By comparing messy scans to a blank template, they calculated homography matrices to "warp" the scans into a standardized grid. This precision allows for automated "cell-level" cropping, ensuring the recognition model knows exactly where to look.
  • Handwriting Recognition: The team fine-tuned TrOCR, a Transformer-based encoder-decoder model, specifically for this task. They trained it on a custom dataset of over 12,000 manually labeled cells from Hamilton County, supplemented by broader handwritten digit datasets like CAR-B and DIDA.

2. Predictive Regression Modeling

To address the "missing scan" problem, the authors built a Random Forest Regressor. This model estimates historical 1933 appraised values using contemporary property data as a proxy. By utilizing modern features such as square footage, room counts, and building materials, the model can infer historical values for properties where physical records are lost or unavailable.

Data and Insights

The primary testing ground for this system was Hamilton County (Cincinnati), Ohio, involving approximately 56,000 scanned property cards. The researchers also tested generalization in Franklin County (Columbus), Ohio.

A key insight from the study is the effectiveness of "Contemporary Proxies." Because the core structural characteristics of pre-1930 homes (such as foundation type or primary building material) rarely change significantly over time, modern records serve as surprisingly reliable inputs for historical estimation.

Furthermore, the team leveraged TrOCR’s internal confidence scores as a quality filter. By retaining only the top 90% most confident predictions, they achieved a near-human accuracy rate of 99.4% for specific data extraction tasks. This allows researchers to prioritize high-quality data while identifying records that might require manual verification.

Results and Economic Impact

The pipeline demonstrated strong performance across both OCR and predictive tasks:

  • OCR Performance: For comprehensive data extraction, the model achieved an $R^2$ of 0.76, with 96.5% of predictions falling within 5% of the ground truth.
  • Regression Accuracy: The Random Forest model predicted 1933 values with a Mean Absolute Percentage Error (MAPE) of 17.48%. While error rates increased slightly when generalizing from Cincinnati to Columbus (22.72% MAPE), the model remains a robust baseline.
  • Bias Mitigation: Crucially, the researchers verified that the model's error rates did not correlate with a neighborhood's racial or socioeconomic composition, ensuring the tool's fairness for use in policy analysis.

The most significant result, however, is the economic feasibility. The automated pipeline reduced data extraction costs by 78% to 89% compared to traditional manual entry. This shift makes large-scale social science research into historical housing inequality possible where it was previously financially out of reach.

Conclusion and Future Directions

While the current system is highly effective, its primary limitation is regional generalizability. Appraisal styles and construction costs varied across the U.S. in the 1930s, meaning a model trained in the Midwest might require additional fine-tuning for other regions.

Future work will focus on expanding the training datasets to include more diverse geographic areas and incorporating richer historical features into the regression models. By automating the "template" creation process for different county formats, the researchers hope to create a universal tool for unlocking the financial history hidden in local archives.

Summary updated

BibTeX
@misc{bhaskar2025predicting,
  author = {Mihir Bhaskar and Jun Tao Luo and Zihan Geng and Asmita Hajra and Junia Howell and Matthew R. Gormley},
  title = {{Predicting the Past: Estimating Historical Appraisals with OCR and Machine Learning}},
  year = {2025},
  eprint = {2505.24676},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2505.24676},
}
Figure