TL;DR
Instead of relying on passive external tools to monitor complex, entangled LLM representations, this paper introduces TELLME (Transparency Enhancement of LLMs without External modules). By actively reorganizing the internal representation space via contrastive learning, TELLME makes LLMs inherently easier to monitor and safer, achieving significant gains in both monitorability and self-detoxification across various model scales and modalities.
What Problem Does This Paper Address?
As Large Language Models (LLMs) grow more powerful, understanding their internal decision-making processes becomes critical for safety. Traditional safety frameworks rely on external monitors to screen inputs/outputs or use post-hoc interpretability tools like Sparse Autoencoders (SAEs) to decode internal states.
However, these approaches are inherently passive. If an LLM's internal features are deeply entangled, external tools struggle to decode them effectively, often incurring massive computational costs. Furthermore, behavioral methods like Chain-of-Thought (CoT) prompting can be unfaithful or even deceptive. This paper tackles a fundamental question: Can we actively modify the LLM itself to make its internal thinking process cleaner and easier to monitor?
Methods
The authors propose TELLME, a method that directly reshapes the LLM's latent representation space during post-training using Low-Rank Adaptation (LoRA). Instead of altering the architecture, TELLME fine-tunes a target layer (typically at 80% depth) using a joint objective function:
- Disentangle Loss ($\mathcal{L}_d$): Utilizing a classical InfoNCE contrastive loss, this component constructs positive pairs (similar behaviors, like two different examples of "bias") and negative pairs (distinct behaviors, like "violence" vs. "deception"). It maximizes representation similarity for similar concepts and minimizes it for distinct ones.
- Retain Loss ($\mathcal{L}_r$): To ensure the model doesn't lose its general capabilities or conversational stability, TELLME enforces an $\ell_2$ norm constraint on intermediate representations and a KL divergence penalty on output probabilities against the original unaltered model.
Datasets
The framework relies on two types of datasets:
- Retain Sets (General Capabilities): UltraChat for text models and RLHF-V for multimodal models.
- Disentangle & Evaluation Sets (Safety/Risks): BeaverTails (for Multi-Risk and Safe-or-Harmful tasks), SPA-VL (for multimodal safety), and a dataset from Taylor et al. (2025) for detecting reward-hacking misalignment.
- General Benchmarks: MMLU, MATH, CoLA (GLUE), and SiQA were used to verify that general capabilities remained intact.
Key Insights and Ideas
- Active vs. Passive Monitoring: Actively structuring the internal representation space is far more efficient than training massive external feature extractors on top of a messy, unaltered latent space.
- The "Hidden Bonus" of Self-Detoxification: Simply separating safe and harmful behaviors in the representation space automatically improves the model's safety generation. Crucially, this happens without explicit preference data (like DPO) or explicitly instructing the model on which behavior is preferred.
- Theoretical Alignment with Optimal Transport: The authors ground TELLME's success in optimal transport theory. They demonstrate that aggregating similar behaviors reduces the k-variance of the behavior distributions, which mathematically minimizes the generalization bound for behavior classification.
Key Results
- Enhanced Monitorability: Across Llama-3.1-8B, Qwen2.5-7B, and Mistral-7B, TELLME achieved a 10.2% average accuracy improvement on representation-based monitors (Self-Similarity and Linear Probing).
- Efficiency Over Baselines: On linear probing tasks, TELLME outperformed strong SAE and Transcoder baselines by 2.5% on average, despite utilizing 26× fewer trainable parameters (~41M LoRA parameters vs. ~1.09B for the SAE baseline).
- Active Detoxification: TELLME delivered an absolute average improvement of 8.1 points in safety rates across the BeaverTails and SaladBench benchmarks.
- Robust Scalability: The method scales seamlessly across parameter sizes (from 7B up to 72B for Qwen2.5) and architectures, showing robust performance gains even on massive Mixture-of-Experts (MoE) multimodal models like Qwen3-VL-235B.
Limitations and Future Work
- Knowledge Constraints: The effectiveness of representation disentanglement is inherently bounded by the LLM's pre-trained knowledge. Unseen or highly niche behaviors not covered during the fine-tuning process may remain entangled.
- Post-Training Restriction: Due to computational resource limits, TELLME was only evaluated as a post-training intervention. The authors note that integrating TELLME directly into the pre-training stage could unlock even greater structural transparency and stronger generalization bounds.
Summary updated