TL;DR
While self-reflection prompts are widely used to make Large Language Models (LLMs) safer and less biased, we have rarely been able to see how this mechanism alters their internal representations. This paper introduces ReBias-Lens, an interpretability framework revealing that while self-reflection successfully smooths out biases on a macro level across deeper network layers, it can stubbornly lock in and even amplify specific, localized social biases.
What Problem Does This Paper Address?
When prompted to "think twice" or "ensure your answer is unbiased," modern LLMs show an impressive ability to correct their own stereotypical outputs. However, evaluating this purely through behavioral outputs (the final text generated) leaves the underlying process as a black box.
This behavioral view can mask critical vulnerabilities, such as "exaggerated safety" (where a model over-refuses benign queries out of hyper-caution) or hidden bias entrenchment. Furthermore, existing internal probing methods typically rely on explicit toxicity metrics, which fail to capture the subtle, implicit value inclinations that underpin social biases. The authors address this gap by looking directly at how self-reflection reconfigures internal embedding spaces across different layers.
Methods
The authors propose ReBias-Lens, a framework that operationalizes social bias as measurable shifts along an internal valence spectrum (the semantic association of a word with pleasantness or unpleasantness).
- Intersectional Context Prompting: The framework sets up controlled prompt templates that substitute opposing demographic attributes (e.g., "white" vs. "black") while keeping the rest of the context identical. It then measures how the embedding of a trailing neutral attribute (like "person") shifts.
- Valence Projection: To avoid geometric distortions common in raw embedding spaces, they train a linear Support Vector Classifier (SVC) on pleasant and unpleasant seed words to define a robust valence vector. They introduce Valence Fluctuation (VF) to measure bias: Global-VF tracks macroscopic trends across all contexts, while Local-VF zooms into category-specific variations.
- Layer-wise Probing & Weight Ablation: They track Global-VF layer-by-layer to pinpoint the exact "transition layer" ($l^*$) where the model begins to diverge from its baseline stereotypical encoding. To test if this layer is uniquely responsible for bias mitigation, they perform localized weight ablations (zeroing out weights) within a window around $l^*$.
Datasets
- Projector Training & Validation: The valence projector was trained using 25 pleasant and 25 unpleasant seed words from prior literature and validated against Bellezza’s valence lexicon.
- Bias Evaluation Contexts: The authors curated prompt sets across 12 Western societal bias categories: age, social class, weight, race, height, sexual orientation, intelligence, religion, education, gender, literacy, and sex.
- Models Evaluated: Llama-3-8B-Instruct, Llama-3.1-8B-Instruct, Mistral-7B-Instruct-v0.3, and Qwen2.5-7B-Instruct.
Key Insights and Ideas
- Hierarchical "Valence Smoothing": Bias mitigation is not uniformly distributed across the model. Instead, it is a distinct hierarchical process that initiates sharply at a middle-to-deep transition layer ($l^*$), leading to a neutralized valence representation in the final layers.
- Distributed Structural Robustness: Surprisingly, zeroing out the weights of the transition layers does not break the model's self-correction capabilities. The bias mitigation effect is highly robust to local network perturbations, suggesting that self-reflection relies on a distributed mechanism or parameter redundancy within the network.
- The Selective Correction Trap: On a macro scale, self-reflection looks like a massive success. However, looking closer at Local-VF reveals a stubborn, category-specific selectivity. Self-reflection easily neutralizes some biases (like height) but can perversely lock in or amplify others (like sexual orientation).
Key Results
- Macro Bias Reduction: On average across all tested models and templates, self-reflection reduced Global-VF by more than 73%. For example, Qwen-2.5 transitioned 90% of its behavioral choices from biased options to neutral ones.
- Micro Bias Amplification: Local analysis exposed alarming regressions. In Llama-3-8B, while height prompts successfully pivoted toward neutral reasoning, prompts involving homosexual identity triggered a severe bias-amplification effect. Only 48.1% of homosexual identity prompts shifted to neutral evaluations, compared to 86.5% for heterosexual prompts.
Limitations and Future Work
- Single-Category Constraints: The current framework evaluates social categories in isolation. Real-world biases are intersectional (e.g., combining race and gender), which the authors plan to investigate in future work.
- One-Dimensional Probe: ReBias-Lens relies entirely on the valence dimension. However, human social judgments involve multi-dimensional representations, such as competence and warmth, which extend beyond pleasantness alone.
- Task Scope: The study focused exclusively on discriminative tasks (e.g., multiple-choice evaluations). Whether these internal mechanics behave similarly during long-term inference or open-ended generation remains to be verified.
Summary updated