Bias Reduction via End-to-End Shift Learning: Application to Citizen Science

Di Chen, Carla P. Gomes

Abstract

Citizen science platforms like eBird generate vast amounts of biodiversity data, but these datasets are often plagued by spatial sampling bias due to human behavior. This paper introduces the Shift Compensation Network (SCN), an end-to-end deep learning framework designed to correct for such covariate shifts. By simultaneously learning the distribution shift between biased training samples and unbiased target distributions, SCN improves species distribution modeling accuracy. Experimental results on high-dimensional satellite imagery and landscape features demonstrate that SCN outperforms existing state-of-the-art methods in both predictive performance and stability.

Introduction: The Challenge of Biased Citizen Science Data

Citizen science has revolutionized biodiversity monitoring, with platforms like eBird collecting millions of observations annually. However, this data is rarely a representative sample of the natural world. Instead, it reflects human convenience: volunteers tend to record observations near urban centers, major roads, or easily accessible parks.

Spatial bias in citizen science data across the United States, showing heavy clustering in populated areas.

In machine learning, this phenomenon is known as covariate shift. While the relationship between environmental features ($x$) and species presence ($y$)—defined by $p(y|x)$—is assumed to be consistent, the distribution of sampled environments in the training data ($p(x)$) differs significantly from the actual landscape distribution ($q(x)$). Models trained on such biased data often fail to generalize to unbiased, scientifically rigorous test sets. The objective is to estimate a "shift factor" $w(x) = q(x)/p(x)$ to re-weight training samples, effectively "re-balancing" the dataset to match the target distribution.

The Shift Compensation Network (SCN)

To address this, the authors propose the Shift Compensation Network (SCN). Unlike traditional two-stage methods that estimate weights and then train a classifier, SCN integrates these tasks into a single, end-to-end neural architecture.

The SCN architecture, featuring a shared Feature Extractor (G), a Discriminator (D) for shift estimation, and a Classifier (C).

The SCN architecture comprises three primary modules:

  1. Feature Extractor ($G$): Maps raw inputs, such as high-dimensional satellite imagery or landscape vectors, into a latent feature space.
  2. Discriminator ($D$): Trained to distinguish between samples from the biased training set and the unbiased (but unlabeled) test set.
  3. Classifier ($C$): Predicts species presence labels.

Two-Step Learning Scheme

The SCN employs a dual-objective learning process:

  • Shift Estimation: The network utilizes a discriminative loss to estimate the shift factor $w(x)$. To regularize this process and prevent the discriminator from overfitting, the authors introduce Feature Space Mean Matching (FSMM). FSMM ensures that the weighted mean of training features aligns with the mean of test features in the latent space.
  • Weighted Classification: The classifier is trained using the labels $Y_P$, with each training sample's contribution weighted by the shift factor learned in the first step.

To maintain stability during mini-batch training on large-scale datasets, the authors implement moving averages for feature mean estimation, reducing the noise inherent in stochastic gradients.

Experimental Setup and Datasets

The framework was evaluated using data from eBird, focusing on the 50 most frequently observed bird species. Two types of input features were tested:

  • NLCD Data: 16-dimensional vectors representing landscape composition (e.g., forest, urban, water).
  • Satellite Imagery: $256 \times 256$ pixel Google Earth images of observation sites.

The training set (~79k samples) contained natural human-induced bias. In contrast, the test and validation sets (~11k samples each) were constructed to be unbiased by sampling observations from a uniform spatial grid.

Key Insights and Results

The SCN was benchmarked against several baselines, including "vanilla" models (no correction) and established shift-correction techniques like KLIEP, KDE, and DFW.

  • Performance Gains: SCN consistently outperformed all competitors across multiple metrics, including AUC, Average Precision, and F1 score.
  • Handling High-Dimensionality: On the complex satellite image dataset, SCN achieved an AUC of 83.80%, a significant improvement over the vanilla model (80.93%) and the best-performing baseline, DFW (81.99%).
  • Synergy and Stability: The combination of discriminative loss and FSMM proved crucial. While the discriminator constrains the weight solution space, FSMM acts as a powerful regularizer. Furthermore, the end-to-end nature of SCN allows the feature extractor to adapt to both tasks simultaneously, resulting in a more robust compensation mechanism compared to disjoint two-stage approaches.

Conclusion

The Shift Compensation Network provides a scalable and effective solution for correcting covariate shift in large-scale citizen science datasets. By leveraging deep learning for end-to-end weight estimation and classification, SCN successfully bridges the gap between biased human-collected data and unbiased scientific requirements. Future work may explore more complex weight application strategies within deep architectures and further investigate the convergence properties of shift-weighted stochastic gradient descent.

Summary updated

Abstract

Citizen science projects are successful at gathering rich datasets for various applications. However, the data collected by citizen scientists are often biased — in particular, aligned more with the citizens’ preferences than with scientific objectives. We propose the Shift Compensation Network (SCN), an end-to-end learning scheme which learns the shift from the scientific objectives to the biased data while compensating for the shift by re-weighting the training data. Applied to bird observational data from the citizen science project eBird, we demonstrate how SCN quantifies the data distribution shift and outperforms supervised learning models that do not address the data bias. Compared with competing models in the context of covariate shift, we further demonstrate the advantage of SCN in both its effectiveness and its capability of handling massive high-dimensional data.
BibTeX
@inproceedings{chen2019bias,
  author = {Di Chen and Carla P. Gomes},
  title = {{Bias Reduction via End-to-End Shift Learning: Application to Citizen Science}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2019},
  doi = {10.1609/aaai.v33i01.3301493},
}
Figure