Abstract
Bayesian Improved Surname Geocoding (BISG) is a ubiquitous tool for predicting race and ethnicity using an individual's geolocation and surname. Here we demonstrate that statistical dependence of surname and geolocation within racial/ethnic categories in the United States results in biases for minority subpopulations, and we introduce a raking-based improvement. Our method augments the data used by BISG—distributions of race by geolocation and race by surname—with the distribution of surname by geolocation obtained from state voter files. We validate our algorithm on state voter registration lists that contain self-identified race/ethnicity.
The Problem: The Independence Assumption
In critical fields like lending, healthcare, and election law, measuring racial disparities is essential but often hindered by a lack of direct data. Researchers frequently rely on Bayesian Improved Surname Geocoding (BISG) to estimate race probabilities. However, standard BISG relies on a "conditional independence" assumption: it assumes that given a person's race, their surname and geolocation are independent factors.
In reality, this assumption is false. Ethnic surnames often cluster in specific neighborhoods. By ignoring these local clusters, standard BISG systematically undercounts minority groups in many areas, leading to biased results in the very fairness audits it is meant to support. For example, in Florida, standard BISG was found to underestimate Hispanic populations in 88% of counties.
Methodology: The Raking Innovation
To address these systematic biases, we propose a calibrated version of BISG using a statistical technique called raking (or iterative proportional fitting). While standard BISG uses two "margins" from the Census (race by surname and race by geolocation), our innovation augments this with two additional data sources:
- Joint Surname-Geolocation Distribution: We use state voter registration files to determine the exact number of people with a specific surname living in a specific county.
- Statewide Race Totals: We use the Current Population Survey (CPS) to establish an accurate "target" for the total racial breakdown of a state.
The algorithm starts with standard BISG predictions and mathematically adjusts (rakes) them until the results match these known margins. This ensures that final predictions are consistent with both local neighborhood data and overall state demographics.
Datasets and Validation
The study validates this method using 2020 voter registration lists from Florida and North Carolina. These datasets are unique because they include self-identified race/ethnicity, providing a "ground truth" for evaluation. These were combined with U.S. Census data and the CPS Voter Supplement to provide the necessary calibration margins.
Key Results
The improvement in accuracy is substantial across all tested states. The raking method is "unbiased by design," meaning the mean error across an entire state is zero.
- Florida (2020): The mean absolute error for the raking method was only 4%, compared to 10% for standard BISG.
- North Carolina (2020): Raking achieved a mean absolute error of 3%, while standard BISG lagged at 7%.
As shown in the figures, standard BISG consistently struggles with "relative accuracy" for smaller subpopulations, such as Asian and Pacific Islanders (API), often underestimating them by 25% or more. The raking method acts as a "safety rail," preventing the model from drifting into biased territory.
Limitations and Future Work
While highly effective, the method relies on the availability of joint surname-geolocation data. While accessible via voter files in many states, this data may be harder to obtain in other administrative domains. Future work could improve accuracy further by using finer-grained geographic data (such as census tracts) and incorporating additional features like first names or sex into the predictive model.
Summary updated