Societal and technological progress as sewing an ever-growing, ever-changing, patchy, and polychrome quilt

Joel Z. Leibo, Alexander Sasha Vezhnevets, William A. Cunningham, Sébastien Krier, Manfred Diaz, Simon Osindero

Abstract

Traditional AI alignment research often operates under the "Axiom of Rational Convergence," the belief that increased intelligence and information will eventually lead agents to a singular, universal set of ethical truths. This paper challenges that paradigm, arguing that human values are fundamentally diverse, culturally "thick," and resistant to universalization. We propose the Appropriateness Framework, a decentralized approach to AI development that prioritizes context-aware behavior over global alignment. By shifting the metaphor of progress from that of an astronomer mapping the heavens to a tailor sewing a "patchwork quilt," we advocate for polycentric governance and community-driven customization as the primary safeguards for future AI systems.

Introduction: The Convergence Myth

In the field of AI safety, a dominant assumption persists: that moral disagreement is a "Mistake Theory" problem. This view suggests that if agents possessed perfect information and infinite reasoning capacity, they would eventually converge on a single, correct ethical framework. Consequently, much of the alignment literature focuses on finding a "universal" value set to bake into a monolithic superintelligence.

However, this approach fails to account for the reality of human social progress. Drawing on Conflict Theory and Pragmatism, we argue that many disagreements are not errors in reasoning but reflections of fundamentally different, enduring values. Forcing AI to align with a single global standard risks creating systems that are coercive, culturally insensitive, and ultimately fragile. Instead of seeking a singular truth, we should view societal progress as the ongoing effort to "sew" together diverse groups into a functional, albeit patchy, quilt of coexistence.

From Astronomers to Tailors

The current state of AI alignment can be compared to the work of Astronomers. Researchers attempt to observe and map a pre-existing, objective "map of human values" as if it were a celestial body. This paper argues for a shift toward the role of the Tailor. A tailor does not discover a map; they skillfully handle messy, diverse materials—the "thick" moralities of specific cultures, roles, and communities—to stitch together a functional garment.

In this metaphor, the "quilt" represents human society. Each patch has its own texture, color, and pattern. An AI that is "aligned" in one patch (e.g., a corporate legal assistant) would be entirely "inappropriate" in another (e.g., a creative writing partner or a local community mediator). Success should not be measured by how well an AI adheres to a thin set of universal principles, but by its appropriateness within its specific social context.

The Appropriateness Framework

To move beyond the limitations of universal alignment, we propose a four-pillar framework for AI development:

  1. Contextual Grounding: AI systems must be equipped with rich data regarding their specific social environment, including local norms, roles, and cultural practices.
  2. Community Customization: Rather than a top-down imposition of values, different groups and communities should have the agency to shape the norms and behaviors of the AI systems they utilize.
  3. Continual Adaptation: Human culture is "causally opaque"; we often follow norms because they facilitate cooperation, not because they are derived from first principles. AI must similarly learn through ongoing social feedback and sanctions, adapting as human norms evolve.
  4. Polycentric Governance: Drawing on the work of Elinor Ostrom, we suggest that AI oversight should be distributed across multiple, overlapping centers of authority—users, developers, and local regulators—rather than a single global entity.

Rethinking Power and Safety

This decentralized approach offers a new perspective on the risks of power-seeking AI. Rather than attempting the likely impossible task of "deleting" power-seeking drives within an individual model, safety is achieved through polycentric institutions. By ensuring that no single AI or human-AI coalition can concentrate overwhelming power, we create a system that is robust to individual failures.

Furthermore, this framework addresses the "fragility of long chains of reasoning." Many existential risk arguments rely on abstract, logical deductions that are highly sensitive to initial premises. By grounding AI safety in empirical, context-specific feedback loops, we create a more resilient and less solipsistic path toward beneficial AI.

Challenges and Future Directions

Transitioning to an Appropriateness Framework introduces new technical and ethical challenges. Contextual grounding requires access to sensitive data, necessitating a "contextual integrity" approach to privacy that utilizes secure enclaves and privacy-preserving machine learning. Additionally, there is the risk that powerful models might attempt to manipulate the feedback loops intended to train them. The solution lies in building decentralized, robust feedback mechanisms that are designed to be adversarial-resistant.

Ultimately, the goal is to move away from a bland, "corporate-speak" AI that attempts to avoid offense by saying nothing of substance. By embracing pluralism and the "patchwork quilt" of human experience, we can develop AI systems that are not just aligned, but truly appropriate for the diverse world they inhabit.

Summary updated

Abstract

Artificial Intelligence (AI) systems are increasingly placed in positions where their decisions have real consequences, e.g., moderating online spaces, conducting research, and advising on policy. Ensuring they operate in a safe and ethically acceptable fashion is thus critical. However, most solutions have been a form of one-size-fits-all "alignment". We are worried that such systems, which overlook enduring moral diversity, will spark resistance, erode trust, and destabilize our institutions. This paper traces the underlying problem to an often-unstated Axiom of Rational Convergence: the idea that under ideal conditions, rational agents will converge in the limit of conversation on a single ethics. Treating that premise as both optional and doubtful, we propose what we call the appropriateness framework: an alternative approach grounded in conflict theory, cultural evolution, multi-agent systems, and institutional economics. The appropriateness framework treats persistent disagreement as the normal case and designs for it by applying four principles: (1) contextual grounding, (2) community customization, (3) continual adaptation, and (4) polycentric governance. We argue here that adopting these design principles is a good way to shift the main alignment metaphor from moral unification to a more productive metaphor of conflict management, and that taking this step is both desirable and urgent.
BibTeX
@misc{leibo2025societal,
  author = {Joel Z. Leibo and Alexander Sasha Vezhnevets and William A. Cunningham and Sébastien Krier and Manfred Diaz and Simon Osindero},
  title = {{Societal and technological progress as sewing an ever-growing, ever-changing, patchy, and polychrome quilt}},
  year = {2025},
  eprint = {2505.05197},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2505.05197},
}
Figure