Human-Algorithm Interaction Biases in the Big Data Cycle: A Markov Chain Iterated Learning Framework

Olfa Nasraoui, Patrick ShaftoarXiv 2016 · arXiv

Abstract

Modern machine learning applications operate within a "Big Data Cycle," where algorithms and human users continuously influence one another. This paper introduces a theoretical framework based on Markov Chain Iterated Learning to model this feedback loop. By formalizing the dependencies between algorithmic filtering and human response, the authors demonstrate how these systems can converge on "blind spots"—regions of the data space that are systematically ignored. The framework provides mathematical definitions for these biases and explores potential "antidotes" and reactive learning strategies to restore informational diversity.

Introduction

The foundational assumption of traditional machine learning is that training data is sampled independently and identically distributed (i.i.d.) from the real world. However, in the context of modern recommendation engines and information filters, this assumption is increasingly invalid. Data is generated through a cyclical interaction: an algorithm selects a subset of information for a user, and the user’s reaction to that specific subset becomes the training data for the next iteration of the model.

This cycle introduces several critical biases. Selection bias occurs when algorithms only present users with content they are predicted to like. Human action bias arises because users do not provide feedback on all items; their decision to ignore content is often a non-random signal driven by their own preferences or the effort required to interact. Together, these factors create "filter bubbles" or "blind spots," where relevant information is permanently obscured from both the user and the learning system.

Theoretical Framework: The Markov Chain Iterated Learning Model

To analyze these dynamics, the authors adapt the concept of Iterated Learning from the study of language evolution. In this framework, the "hypothesis" (the machine learning model) evolves over successive generations of interaction.

Comparison of (a) traditional sequential learning and (b) the iterated learning framework where the model's hypothesis $h_{t-1}$ actively selects the next data point $x_t$.

As shown in Figure 1, the key distinction in the proposed model is the feedback loop from the previous hypothesis ($h_{t-1}$) to the next data point ($x_t$). In a standard model (a), data arrives independently. In the iterated learning cycle (b), the model's current state determines what data is "seen" ($p_{seen}$), effectively gating the learning process. If the model deems an item irrelevant, the probability of a human interacting with it drops to near zero, preventing the model from ever correcting its potential error regarding that item.

Modeling Human Action and Blind Spots

The framework incorporates Luce Choice (a softmax-based utility model) to represent human feedback. This acknowledges that "missing data" in collaborative filtering is rarely random; it is a deliberate choice by the user influenced by the algorithm's presentation and the perceived utility of the item.

The authors formally define:

  • Human Blind Spots ($\delta^H_b$): Information the user cannot access due to algorithmic filtering.
  • Algorithm Blind Spots ($\delta^A_b$): Regions of the feature space where the model lacks sufficient labels to make accurate predictions.
  • Blind Spot Prevalence ($\rho$): A metric to quantify the extent of the data universe being ignored by the system.

Key Insights: Filtering vs. Active Learning

A primary insight of this work is the fundamental tension between filtering and active learning. Recommender systems typically prioritize filtering—showing users what the model is most certain they will enjoy. Conversely, active learning requires showing users items the model is uncertain about to improve its internal representation. Current commercial systems favor the former, leading to a "confirmation bias" that reinforces existing model hypotheses at the expense of discovery.

Results and Mitigation Strategies

The paper mathematically proves that iterated learning under filter bias converges to a mixture of the initial prior belief and the bias induced by the filtering mechanism. To combat this convergence toward blind spots, the authors propose two categories of interventions:

  1. Antidotes: Post-learning adjustments, such as "reverse-Rocchio" algorithms that broaden relevance boundaries or ensemble methods that maintain a diversity of model perspectives.
  2. Reactive Learning: In-loop interventions, including Inverse Sampling (weighting rare labels more heavily) and Uncertainty Sampling with Diversity Maximization, which force the algorithm to explore its own blind spots during the data collection phase.

Conclusion and Future Directions

This theoretical framework highlights the structural risks inherent in the Big Data Cycle. While the paper provides a rigorous mathematical foundation, the authors note the lack of empirical datasets that record the full history of recommendations, non-actions, and presentation order. Future research must focus on developing these "interaction-aware" datasets and conducting behavioral experiments to validate how human-algorithm feedback loops manifest in real-world environments.

Summary updated

Abstract

Early supervised machine learning algorithms have relied on reliable expert labels to build predictive models. However, the gates of data generation have recently been opened to a wider base of users who started participating increasingly with casual labeling, rating, annotating, etc. The increased online presence and participation of humans has led not only to a democratization of unchecked inputs to algorithms, but also to a wide democratization of the "consumption" of machine learning algorithms' outputs by general users. Hence, these algorithms, many of which are becoming essential building blocks of recommender systems and other information filters, started interacting with users at unprecedented rates. The result is machine learning algorithms that consume more and more data that is unchecked, or at the very least, not fitting conventional assumptions made by various machine learning algorithms. These include biased samples, biased labels, diverging training and testing sets, and cyclical interaction between algorithms, humans, information consumed by humans, and data consumed by algorithms. Yet, the continuous interaction between humans and algorithms is rarely taken into account in machine learning algorithm design and analysis. In this paper, we present a preliminary theoretical model and analysis of the mutual interaction between humans and algorithms, based on an iterated learning framework that is inspired from the study of human language evolution. We also define the concepts of human and algorithm blind spots and outline machine learning approaches to mend iterated bias through two novel notions: antidotes and reactive learning.
BibTeX
@misc{nasraoui2016human,
  author = {Olfa Nasraoui and Patrick Shafto},
  title = {{Human-Algorithm Interaction Biases in the Big Data Cycle: A Markov Chain Iterated Learning Framework}},
  year = {2016},
  eprint = {1608.07895},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1608.07895},
}
Figure