TL;DR
This paper introduces SOLAR, a novel framework that uses Large Language Models (LLMs) to characterize individual subjectivity by analyzing value conflicts and trade-offs in user-generated text. SOLAR demonstrates that LLMs can effectively infer personal moral judgments, especially in low-data scenarios and controversial situations, by leveraging past comments and abstract value representations.
What Problem Does This Paper Address?
While LLMs have shown impressive capabilities in objective reasoning and even mimicking general personas, understanding and modeling individual-level subjectivity remains a significant challenge. Existing approaches often generalize across demographic groups or roles, failing to capture the nuanced, personal moral compass that guides an individual's judgments. This gap is becoming increasingly important as personalized AI becomes more prevalent. The authors aim to bridge this by developing a method to infer an individual's subjective ground—their underlying principles and values—from their online interactions.
Methods
The core of SOLAR is a Retrieval-Augmented Generation (RAG) framework. When presented with a new situation, SOLAR first retrieves relevant past comments from a specific user. This retrieval process is enhanced by two key strategies:
- Situation-based Retrieval: Finds past comments on situations similar to the current one.
- Value-based Retrieval: Identifies past comments that reflect similar abstract values or value trade-offs, even if the situations themselves are topically different. This leverages Schwartz's theory of basic human values and a novel "value abstraction" process where LLMs identify and cluster value conflicts from text.
The retrieved information (past comments, abstract values, and value trade-offs) is then used as few-shot examples to prompt an off-the-shelf LLM (GPT-4.1) to predict the user's judgment on the new situation. This approach avoids the need for fine-tuning LLMs for each individual, making it more scalable.
Datasets
The study utilizes a dataset collected from the Reddit community r/AmITheAsshole (r/AITA). This community is characterized by users posting situations where they've had conflicts and asking for judgments on their behavior, with other users providing comments and verdicts. The authors created a truncated dataset of approximately 1.7K unique situations and 100 active redditors for training and inference, ensuring it reflects the broader population distribution.
Key Insights and Ideas
- Individual Subjectivity is Distinct: The research validates the hypothesis that individuals exhibit unique subjectivity patterns, making it a distinct challenge from general commonsense reasoning.
- Value Trade-offs as a Proxy for Subjectivity: Modeling how individuals navigate conflicts between different values (e.g., "Personal Autonomy" vs. "Family Roles") is an effective way to capture their moral judgments.
- Abstract Values Enhance Retrieval: Using abstract value representations for retrieval is more effective than relying solely on topical similarity, especially for users with limited data or in complex scenarios.
- RAG Outperforms Fine-tuning for Individual Subjectivity: The RAG approach, which leverages past examples without explicit per-user fine-tuning, is more robust to data scarcity and individual judgment skewness compared to traditional fine-tuning methods.
- LLMs Can Explain Judgments: The framework not only predicts judgments but also provides insights into the underlying value preferences that drive them.
Key Results
- SOLAR achieves strong performance in predicting individual moral judgments, particularly for users with limited past comments ("bottom 50% redditors") and in controversial situations.
- Compared to baseline fine-tuned models, SOLAR's RAG-based approach shows significantly better performance, especially when data is scarce. For instance, using comment and value trade-offs as input improved F1 scores by 2.69% for the bottom 50% redditors.
- The performance of fine-tuned models degrades with fewer instances and more skewed judgment patterns, whereas RAG-based models remain relatively stable.
- Qualitative analysis (heatmap visualization) of value trade-offs among active redditors reveals diverse and distinct preference patterns, supporting the core hypothesis of individual subjectivity.
Limitations and Future Work
The authors acknowledge several limitations:
- Oversimplified Subjectivity Model: Relying solely on past comments from a single community might not capture the full spectrum of an individual's subjectivity. Future work could incorporate information from other communities or broader online activities.
- Assumption of Consistent Judgments: The study assumes individuals judge situations consistently over time. Analyzing potential value shifts would be an interesting direction.
- Subreddit Specificity: The framework was tested on r/AmITheAsshole. Validating its effectiveness across other communities requiring subjective judgment is crucial.
- Suboptimal Performance: While SOLAR shows improvements, its overall performance in predicting judgments is still suboptimal, especially for non-controversial situations.
- Value Generation Validation: The LLM-generated abstract values need further validation for their soundness and quality, potentially through human evaluations.
Future work includes expanding the scope of data sources, exploring temporal value shifts, and conducting direct human evaluations of the generated value representations.
Summary updated