Abstract
Large Language Models (LLMs) are increasingly integrated into critical decision-making and communication workflows. However, the current alignment paradigm—primarily focused on the static "Harmless, Helpful, Honest" (HHH) triad—often results in models that are overcautious, culturally biased, and lacking in nuance. In this position paper, we argue that alignment failures are frequently the result of operational choices in training that prioritize post-training constraints over user agency. We propose a shift toward a dynamic, participatory approach: the Flourishing–Justice–Autonomy (FJA) framework. By moving alignment from static filters to inference-time negotiation, we can better safeguard pluralism and human flourishing.
The Limitations of Static Alignment
The AI community has reached a critical juncture with the standard HHH alignment paradigm. While these goals are well-intentioned, their implementation via Reinforcement Learning from Human Feedback (RLHF) often creates "monocultural" models. Because alignment is typically "baked in" during post-training, models tend to flatten diverse human values into a single, majority-oriented norm—often reflecting a narrow Western, liberal perspective.
This approach leads to several significant failure modes:
- Over-alignment: Models frequently refuse benign queries (e.g., providing a recipe involving alcohol or basic medical information) due to an abundance of caution, rendering the tool less useful.
- The "Alignment Tax": There is a measurable decline in creativity, reasoning depth, and stylistic flair when models are forced to adhere to rigid safety constraints.
- Cultural Erasure: Models may ignore or mischaracterize non-Western traditions because their underlying "constitutions" were curated by a non-representative pool of annotators.
The FJA Framework: A New Paradigm
To address these challenges, we propose the FJA framework, which shifts the focus from rigid post-training filters to dynamic, inference-time alignment that prioritizes human agency.
The Three Pillars of FJA:
- Flourishing: Alignment should move beyond immediate "helpfulness" to support long-term human well-being and critical thinking. For example, instead of simply providing an answer to a student, a model should guide them through the reasoning process.
- Justice: Models must actively amplify marginalized voices and ensure equitable outcomes across different languages and cultures, rather than defaulting to a universalized Western standard.
- Autonomy: Users should have the power to see and adjust the model’s underlying "rules" for a specific interaction, treating the user as an active participant rather than a passive recipient of safe text.
Operationalizing Agency at Inference-Time
The FJA framework envisions a system that identifies user objectives at the moment of prompting. This involves a hierarchical approach to rules:
- Global Constitutions: Immutable rules regarding core safety (e.g., preventing violence or illegal acts).
- Local Constitutions: Adjustable rules regarding cultural etiquette, stylistic preferences, or specific domain requirements (e.g., medical or legal contexts).
Under this model, users can engage in Participatory Editing, where they can challenge a model’s refusal or suggest amendments to the local constitution. To prevent adversarial abuse, such as "jailbreaking," the authors suggest implementing computational or rate-limiting costs for these adjustments.
Interaction Traces and Results
While conceptual, the paper provides "Interaction Traces" to demonstrate how FJA differs from HHH in practice:
- Cultural Specificity: In providing guidance on Ramadan, an HHH model might offer a generic health warning. An FJA model provides culturally specific advice while explicitly deferring medical diagnosis to professionals.
- Indigenous Knowledge: FJA models are designed to acknowledge specific traditions, such as Māori birthing practices, without defaulting to a Western "universal" medical standard.
- Creative Integrity: FJA allows for the exploration of complex themes like grief or conflict in fiction—themes that rigid HHH filters often flag as "harmful"—thereby preserving the model's utility for art and literature.
Challenges and Future Directions
Transitioning to an FJA-based system involves significant hurdles. Running a "mini-alignment" process for every prompt increases computational costs. Furthermore, the governance of the "Global" constitution remains a complex multi-stakeholder problem. Future research must focus on developing "reasoning-aware" reward models that are robust enough to handle user agency without compromising core safety. By treating alignment as an ongoing, participatory process, we can move toward AI systems that truly reflect the diversity and depth of human values.
Summary updated