TL;DR
By framing the layer-wise dynamics of Large Language Models (LLMs) as Iterated Function Systems (IFS), this paper demonstrates that transformer layers act as contractive mappings converging to concept-specific latent "Attractors." Exploiting these geometric attractors enables lightweight, training-free interventions across diverse tasks—including concept guardrailing/unlearning, code translation, toxicity reduction, vision-language hallucination reduction, and synthetic data generation—without requiring fine-tuning or retention data.
What Problem Does This Paper Address?
Despite having vastly different surface tokens or syntax, semantically related prompts (such as queries about the Lord of the Rings universe or Python programming) tend to collapse into compact, localized regions in intermediate transformer layers. While prior work observed elements of this representation convergence, a unifying mathematical framework explaining why it occurs and how to practically exploit it across different downstream applications was lacking.
Addressing challenges like machine unlearning, style/behavior steering, multimodal hallucinations, and diverse synthetic data generation often requires heavy fine-tuning, auxiliary steering models, or multi-pass sampling. The authors ask: Can we model this internal convergence using dynamical systems theory and manipulate these intermediate concept attractors directly without retraining?
Methods
- Dynamical Systems & IFS Modeling: The authors model the forward pass through layers as an Iterated Function System (IFS). Successive layer transformations act as contractive operators governed by the Hutchinson operator and the Collage Theorem, mapping inputs into invariant compact sets called semantic attractors ($\mathcal{A}_l^C$) at concept-dependent layer depths.
- Attractor Estimation & Layer Selection: For a concept $C$, its attractor $\mathbf{a}^{(l)}$ is estimated as the mean hidden state across concept samples at layer $l$. The optimal layer $l^*$ is identified by maximizing Separation (the difference between inter-concept distance and intra-concept dispersion under cosine similarity).
- Attractor-Based Interventions:
- Concept Guardrailing (Unlearning): Compute cosine similarity between the current hidden state at layer 24 and target concept attractors (e.g., forgotten authors). If similarity exceeds threshold $\tau$, generation is blocked.
- Attractor Traversals (Steering): Add or subtract attractor vectors via forward hooks during decoding. Detoxification is achieved by subtracting the toxic attractor vector at layer 16; zero-shot code transpilation (e.g., Python $\rightarrow$ C++) is achieved by shifting representations toward the target language attractor at layer 19.
- Visual Attractor Grounding: Mitigate vision-language drift by re-adding the initial visual attractor vector to hidden states across decoding steps.
- Attractor Perturbation for Data Generation: Instead of using high temperature (which introduces noise), perturb the single sample-wise attractor at layer 16 to generate diverse, high-fidelity synthetic data.
Datasets
- TOFU Benchmark: Fictitious author profiles (
forget01,forget05,forget10) used to evaluate machine unlearning and retain/utility metrics. - ParaDetox: Over 10,000 toxic/non-toxic parallel sentence pairs used to evaluate toxicity reduction and text quality.
- LeetCode Solutions: 100 solutions across Python, Java, C++, and JavaScript used to estimate programming language attractors and test transpilation.
- MSCOCO / CHAIR & POPE: Vision-language benchmarks used to measure object hallucination rates and discriminative accuracy on multimodal models.
- BoolQ & AG News: Reading comprehension and news classification datasets used for few-sample synthetic data generation.
- Celebrity & Historical Figures Dataset: 35 personalities used to evaluate synthetic factuality.
Key Insights and Ideas
- Different Concepts Settle at Different Depths: Semantic convergence is layer-specific. For example, in Llama 3.1-8B, programming language attractors solidify around layer 19, fictional worlds and TOFU authors at layer 24, and natural languages at layer 27.
- Fractal and Sub-concept Organization: Complex concepts decompose hierarchically. Arithmetic tasks form sub-clusters first by number of digits, then by operation type ($+$ vs. $-$), reflecting self-similar fractal geometries predicted by IFS theory.
- No Retention Data Needed: Traditional steering or unlearning techniques typically require paired contrastive sets or retention corpora to avoid catastrophic forgetting. Directly operating on target concept attractors bypasses the need for retention data while maintaining general model utility.
Key Results
- Unlearning (TOFU): On the difficult
forget10split, attractor thresholding blocked $>90\%$ of forgotten author queries while retaining model utility (61.34 Utility, 19.54 Forget ROUGE), outperforming trainable baselines like NPO (49.97 Utility) and ECO (62.35 Utility). - Detoxification: Attractor subtraction dropped toxicity on ParaDetox from 84.58 down to 37.66 with a ROUGE score of 61.27, outperforming LoRA (49.94 toxicity) and matching In-Context Vectors without needing paired examples.
- Zero-Shot Transpiler: Shifting activations toward target language attractors enabled zero-shot code translation across Python, Java, C++, and JS, scoring an average of 4.56/5.00 via GPT-4o evaluation (compared to 1.01 for ICV and 4.34 for Recursive Feature Machines).
- Hallucination Mitigation: Grounding the visual attractor in Llava-1.5 and InstructBLIP significantly lowered generative caption hallucinations (CHAIR) without degrading discriminative question answering (POPE) or requiring multi-pass decoding (like VCD or AVISC).
- Synthetic Data Factuality: Perturbing attractors improved factuality of generated claims by $>20\%$ on average compared to standard temperature sampling, improving downstream accuracy when training smaller student models (Qwen2.5-0.5B, GPT-Neo-1.3B).
Limitations and Future Work
- White-Box Requirement: The method requires direct access to intermediate activations and the ability to insert forward hooks, making it inapplicable to black-box commercial APIs.
- Scale Constraints: Experiments focused predominantly on open-weight models up to 14B parameters (Llama 3.1-8B, Qwen 2.5 3B–14B). Demonstrating whether similar IFS attractor properties persist identically in 70B+ or mixture-of-experts (MoE) architectures remains future work.
- Multi-Map Approximations: While a single affine map provides a strong first-order approximation, capturing intricate, multi-modal concept geometries will require fitting richer multi-map IFS systems.
Summary updated