LLM Alignment should go beyond Harmlessness–Helpfulness and incorporate Human Agency

Usman Naseem, Tanmoy Chakraborty, Kai-Wei Chang, Mark Dras, Preslav Nakov, Nanyun Peng, Soujanya Poria

Abstract

Large Language Models (LLMs) are increasingly integrated into critical decision-making and communication workflows. However, the current alignment paradigm—primarily focused on the static "Harmless, Helpful, Honest" (HHH) triad—often results in models that are overcautious, culturally biased, and lacking in nuance. In this position paper, we argue that alignment failures are frequently the result of operational choices in training that prioritize post-training constraints over user agency. We propose a shift toward a dynamic, participatory approach: the Flourishing–Justice–Autonomy (FJA) framework. By moving alignment from static filters to inference-time negotiation, we can better safeguard pluralism and human flourishing.

The Limitations of Static Alignment

The AI community has reached a critical juncture with the standard HHH alignment paradigm. While these goals are well-intentioned, their implementation via Reinforcement Learning from Human Feedback (RLHF) often creates "monocultural" models. Because alignment is typically "baked in" during post-training, models tend to flatten diverse human values into a single, majority-oriented norm—often reflecting a narrow Western, liberal perspective.

This approach leads to several significant failure modes:

  • Over-alignment: Models frequently refuse benign queries (e.g., providing a recipe involving alcohol or basic medical information) due to an abundance of caution, rendering the tool less useful.
  • The "Alignment Tax": There is a measurable decline in creativity, reasoning depth, and stylistic flair when models are forced to adhere to rigid safety constraints.
  • Cultural Erasure: Models may ignore or mischaracterize non-Western traditions because their underlying "constitutions" were curated by a non-representative pool of annotators.

The FJA Framework: A New Paradigm

To address these challenges, we propose the FJA framework, which shifts the focus from rigid post-training filters to dynamic, inference-time alignment that prioritizes human agency.

The Flourishing–Justice–Autonomy (FJA) framework for LLM alignment.

The Three Pillars of FJA:

  1. Flourishing: Alignment should move beyond immediate "helpfulness" to support long-term human well-being and critical thinking. For example, instead of simply providing an answer to a student, a model should guide them through the reasoning process.
  2. Justice: Models must actively amplify marginalized voices and ensure equitable outcomes across different languages and cultures, rather than defaulting to a universalized Western standard.
  3. Autonomy: Users should have the power to see and adjust the model’s underlying "rules" for a specific interaction, treating the user as an active participant rather than a passive recipient of safe text.

Operationalizing Agency at Inference-Time

The FJA framework envisions a system that identifies user objectives at the moment of prompting. This involves a hierarchical approach to rules:

  • Global Constitutions: Immutable rules regarding core safety (e.g., preventing violence or illegal acts).
  • Local Constitutions: Adjustable rules regarding cultural etiquette, stylistic preferences, or specific domain requirements (e.g., medical or legal contexts).

Under this model, users can engage in Participatory Editing, where they can challenge a model’s refusal or suggest amendments to the local constitution. To prevent adversarial abuse, such as "jailbreaking," the authors suggest implementing computational or rate-limiting costs for these adjustments.

Interaction Traces and Results

While conceptual, the paper provides "Interaction Traces" to demonstrate how FJA differs from HHH in practice:

  • Cultural Specificity: In providing guidance on Ramadan, an HHH model might offer a generic health warning. An FJA model provides culturally specific advice while explicitly deferring medical diagnosis to professionals.
  • Indigenous Knowledge: FJA models are designed to acknowledge specific traditions, such as Māori birthing practices, without defaulting to a Western "universal" medical standard.
  • Creative Integrity: FJA allows for the exploration of complex themes like grief or conflict in fiction—themes that rigid HHH filters often flag as "harmful"—thereby preserving the model's utility for art and literature.

Challenges and Future Directions

Transitioning to an FJA-based system involves significant hurdles. Running a "mini-alignment" process for every prompt increases computational costs. Furthermore, the governance of the "Global" constitution remains a complex multi-stakeholder problem. Future research must focus on developing "reasoning-aware" reward models that are robust enough to handle user agency without compromising core safety. By treating alignment as an ongoing, participatory process, we can move toward AI systems that truly reflect the diversity and depth of human values.

Summary updated

Abstract

Large Language Models are transforming communication, research, and decision-making, but misalignment – when models diverge from human values, safety requirements, or user intent – poses serious risks. In this position paper, we argue that many alignment failures stem from operational choices in training and deployment. We posit that alignment should shift from static, post-training constraints toward dynamic, participatory approaches that safeguard pluralism, autonomy, and human flourishing. We outline forward-looking directions, including pluralistic evaluation, transparency, and the Flourishing–Justice–Autonomy (FJA) framework, and present a roadmap for advancing alignment research and practice.
BibTeX
@inproceedings{naseem2026llm,
  author = {Usman Naseem and Tanmoy Chakraborty and Kai-Wei Chang and Mark Dras and Preslav Nakov and Nanyun Peng and Soujanya Poria},
  title = {{LLM Alignment should go beyond Harmlessness–Helpfulness and incorporate Human Agency}},
  booktitle = {Cognitive},
  year = {2026},
  doi = {10.1007/s12559-026-10568-9},
}
Figure