In traditional human-led systems, decision-making is naturally decentralized. If you apply for a loan at five different banks, you are evaluated by five different loan officers, each with their own idiosyncratic perspectives, biases, and interpretations of your data. While this variability can feel inconsistent, it provides a crucial social safety net: a "no" from one institution does not guarantee a "no" from all others.
As machine learning systems become ubiquitous, this diversity is collapsing. We are entering an era of algorithmic monoculture, where many independent decision-makers rely on the same underlying models and datasets. This convergence creates a systemic risk known as outcome homogenization: the danger that the same individuals will be systematically rejected by every system they encounter, effectively locking them out of the market.
The Loss of Decision Process Diversity
The shift from human to algorithmic judgment represents a fundamental change in "decision process diversity." As Muhammad Bilal Zafar, Krishna P. Gummadi, and colleagues argue in On Fairness, Diversity and Randomness in Algorithmic Decision Making, replacing a diverse group of humans with a single, monolithic algorithm eliminates the variety inherent in human judgment.
In a human system, errors are often uncorrelated. In an algorithmic system, errors become structural. If a model has a specific blind spot or a rigid boundary, that boundary is applied uniformly to everyone. When multiple organizations adopt that same model, they create a "centralized point of failure" for society.
The Foundation Model Bottleneck
This monoculture is driven by the structure of the AI supply chain. Rather than building models from scratch, most developers now rely on a small number of "foundation models" (like GPT-4 or Llama) as a base. Rishi Bommasani, Percy Liang, and others at Stanford have highlighted this in On the Opportunities and Risks of Foundation Models, noting that any flaw or bias in the base model is inherited by every downstream application.
The scale of this concentration is immense. Yujie Ma and colleagues found in Understanding the Supply Chain and Risks of Large Language Model Applications that nearly half of all LLM applications rely on a tiny cluster of root models. This creates a massive "blast radius": a single vulnerability or bias in a root model can propagate through thousands of derivative systems across entirely different sectors, from hiring to healthcare.
Picking on the Same Person
The most acute harm of monoculture is systemic exclusion. In Picking on the Same Person: Does Algorithmic Monoculture lead to Outcome Homogenization?, Rishi Bommasani and Kathleen Creel formalize the risk that specific individuals are "failed" by every system they encounter.
In a "polyculture" of diverse evaluations, a qualified candidate who is a "near-miss" for one firm might be a "top-hit" for another. However, as Kenny Peng and Nikhil Garg demonstrate in Monoculture in Matching Markets, when firms use the same algorithm, they all use the same "yardstick." A single erroneous assessment—perhaps due to a noisy data point or a model's inductive bias—becomes a permanent barrier to entry across the entire industry. There is no "second chance" because there is no second opinion.
Correlated Errors and Systemic Fragility
This is not just a problem of fairness; it is a problem of stability. We often assume that using many different models provides a safety net, but Elliot Myunghoon Kim reveals in Correlated Errors in Large Language Models that model errors are highly correlated. As models get larger and more accurate, they actually tend to fail on the same difficult questions.
This correlation leads to "herd effects" in critical sectors:
- Finance: Yuxin Zhang warns in An Exploration of the Dual Impacts of Artificial Intelligence Credit Scoring on Banking System Risk that if every bank uses the same AI for credit scoring, a single market signal could trigger a synchronized credit contraction, potentially causing a liquidity crisis.
- Science: Marcel Binz and Carl T. Bergstrom argue in How should the advent of large language models affect the practice of science? that relying on a few proprietary models for research creates "epistemic monocultures," where scientific inquiry is restricted by the inherent biases of a few commercial algorithms.
Toward Algorithmic Pluralism
To combat these risks, researchers are proposing a shift toward Algorithmic Pluralism. Shomik Jain, Ashia Wilson, and Vinith Suriyakumar argue in Algorithmic Pluralism: A Structural Approach Towards Equal Opportunity that we must evaluate the ecosystem of models, not just individual ones. They suggest that a diversity of decision-making systems is necessary to ensure that no single model becomes a "bottleneck" for opportunity.
One provocative solution is the introduction of intentional randomness. In Position: Scarce Resource Allocations That Rely On Machine Learning Should Be Randomized, Shomik Jain and Kathleen Creel argue that for high-stakes decisions like hiring, we should use weighted lotteries. By breaking the deterministic "top-k" selection process, we can restore the "second chances" that were lost when we traded human idiosyncrasy for algorithmic monoculture.