Abstract
As Large Language Models (LLMs) are increasingly integrated into high-stakes domains, addressing their inherent vulnerabilities—such as hallucinations, bias, and safety risks—is paramount. MASteer is an end-to-end framework designed to automate the repair of these trustworthiness issues through representation engineering. By leveraging a multi-agent system to generate high-quality contrastive samples and an adaptive selection mechanism during inference, MASteer provides a training-free solution that significantly enhances model reliability without compromising general capabilities.
Introduction
The deployment of LLMs in sectors like healthcare and finance is often hindered by their tendency to produce biased or factually incorrect outputs. Existing mitigation strategies typically fall into two categories: post-training methods (e.g., SFT, RLHF), which are computationally intensive and prone to catastrophic forgetting, and prompt engineering, which often lacks the robustness required for complex scenarios.
Representation Engineering (RepE) offers a promising alternative by steering model behavior via internal activation adjustments. However, traditional RepE relies on manually curated datasets and static steering vectors. MASteer addresses these limitations by "agentizing" the entire repair lifecycle, from sample generation to dynamic inference-time intervention.
The MASteer Framework
MASteer is composed of two primary modules: AutoTester and AutoRepairer, which together automate the discovery and correction of trustworthiness flaws.
AutoTester: Controllable Sample Generation
To effectively steer a model, high-quality contrastive samples are required. AutoTester employs a multi-agent pipeline to generate these:
- Analyst: Decomposes broad issues (e.g., "Safety") into specific, orthogonal test scopes.
- Retriever: Sources real-world data to ensure factual grounding.
- Writer: Produces AB-test style QA pairs (e.g., truthful vs. hallucinated).
- Reviewer: Filters samples based on relevance, steerability, and learnability.
AutoRepairer: Adaptive Strategy Construction
The AutoRepairer module builds the intervention mechanism:
- Scholar: Maintains a library of steering algorithms, such as PCA, Mean Difference, and Logistic Regression.
- Proposer: Identifies optimal internal layers for intervention and constructs anchor vectors—activation patterns that represent specific concepts.
Inference-Time Adaptive Intervention
During inference, MASteer does not apply a fixed steering vector. Instead, it dynamically adapts to the user's query.
When a query is processed, the system compares the model's internal activations against its library of anchor vectors. It then selects the most relevant steering strategy and applies a "repair vector" (calculated as $\alpha \cdot v$) to the activations. This "surgical" approach allows the model to self-correct its behavior in real-time based on the specific context of the input.
Key Insights and Results
Experimental evaluations on LLaMA-3.1 and Qwen-3 models across benchmarks like TruthfulQA, BBQ, and SafeEdit revealed several critical insights:
- The "Middle Layer" Sweet Spot: Interventions are most effective in middle layers (e.g., Layer 13 for LLaMA-3.1-8B). Early layers are too tied to input features, while late layers are too close to token prediction; middle layers capture abstract "concepts" ideal for steering.
- Performance Gains: MASteer improved trustworthiness metrics by 15.36% on LLaMA-3.1-8B-Chat and 4.21% on Qwen-3-8B-Chat.
- Preserving Intelligence: Unlike fine-tuning, MASteer maintained or slightly improved scores on general benchmarks like MMLU and AlpacaEval, demonstrating that specific repairs do not degrade core reasoning capabilities.
- Semantic Compensation: The framework is particularly effective for weaker models, where steering vectors help bridge internal reasoning gaps.
Conclusion
MASteer represents a significant step toward automated, robust LLM maintenance. By combining multi-agent sample generation with adaptive representation engineering, it offers a scalable and efficient path to repairing model trustworthiness without the overhead of traditional fine-tuning. Future work will focus on reducing inference latency and expanding the "Scholar" agent's ability to autonomously discover new steering algorithms.
Summary updated