Beyond Linear Activation Steering: Invertible Latent Transformations for Controlling LLM Behavior

Tuc Nguyen, Thai LearXiv 2026 · arXiv

TL;DR

INNSTEER is a novel activation steering framework that moves beyond rigid linear adjustments to control LLM behavior using nonlinear, input-dependent interventions. By leveraging lightweight Invertible Neural Networks (INNs), it maps internal activations into a latent space where behaviors are linearly separable, applies a simple translation, and maps them back exactly. This approach achieves performance close to parameter-efficient fine-tuning (PEFT) at inference time, without modifying a single model weight.

What Problem Does This Paper Address?

Activation steering is a popular, lightweight technique for controlling Large Language Model (LLM) behaviors—such as increasing truthfulness or enforcing safety constraints—by modifying internal activation vectors during inference.

However, standard steering methods (like CAA, RepE, or ITI) rely on a restrictive geometric assumption: they treat behavioral control as a global, linear, additive offset. They calculate a single "steering vector" and add it to every input. This fails when behavioral features lie on curved or complex manifolds in the activation space, where the optimal intervention depends heavily on the specific input.

Motivation for INNSTEER using synthetic concentric rings data. Linear steering fails on curved manifolds, whereas INNSTEER straightens the space out in a latent coordinate system.

Methods

To overcome the limitations of linear steering, the authors introduce INNSTEER. Instead of searching for a better global vector in the original activation space, INNSTEER learns a layer-specific, lightweight Invertible Neural Network (INN) $\phi$ based on RealNVP architecture (affine coupling layers interleaved with ActNorm).

The training process uses a geometry-aware objective function with three key components:

  1. Gaussian Latent Likelihood ($L_{NLL}$): Regularizes the transformed activations under a standard Gaussian prior to keep the latent space well-structured.
  2. Directional Separation ($L_{dir}$): Maximizes the distance between contrastive class means (e.g., "safe" vs. "unsafe") in the latent space, making them linearly separable.
  3. Log-Determinant Regularization ($L_{det}$): Penalizes extreme local volume changes, ensuring that the inverse mapping remains stable and predictable after a latent shift.

At inference time, an activation vector $h$ is intercepted, normalized, and passed through $\phi$ into the latent space. A simple linear translation ($\alpha v_\phi$) is applied, and the steered latent vector is mapped back through the exact inverse transformation $\phi^{-1}$. Because of the nonlinear nature of $\phi$, this simple latent translation pulls back to a highly flexible, input-dependent update in the original activation space.

Datasets

The authors evaluated INNSTEER across multiple tasks using several established datasets:

  • Anthropic's Persona Dataset: Used to extract contrastive activation pairs for six target behaviors: Conscientiousness, Religion Following, Self-Awareness, Self-Improvement, Alliance Building, and Impact Maximization.
  • Safety Benchmarks: Evaluated on standard Refusal (unsafe prompts) and Hallucination (prompts with false premises) benchmarks.
  • Vision-Language Benchmarks: Tested on the POPE benchmark and the Object Hallucination Dataset (OHD) using LLaVA-1.5-7B to evaluate cross-modality generalization.

Key Insights and Ideas

  • Exact Invertibility is Crucial: Unlike standard autoencoders or non-invertible projection methods, an INN guarantees a bijective, one-to-one mapping. This prevents information loss and reconstruction errors, which is vital because downstream frozen LLM layers are highly sensitive to arbitrary perturbations.
  • Input-Dependent Modulation via the Jacobian: Mathematically, the effective update in the original space is modulated by the local inverse Jacobian of the network. This means the model automatically scales and rotates the steering direction based on where the current input sits on the activation manifold.
  • Robustness to Layer Selection: Linear steering methods are notoriously finicky about where you apply them, usually showing narrow performance peaks around a few specific middle layers. INNSTEER is significantly more robust, achieving competitive steering performance across a much wider span of layers.
Average layer-wise alignment across six tasks, demonstrating INNSTEER's robustness to layer selection compared to highly sensitive linear baselines.

Key Results

INNSTEER was tested against a wide array of baselines (including CAA, RepE, ITI, and ODESteer) across LLaMA and Qwen model families ranging from 3B to 32B parameters.

  • Superior Alignment: INNSTEER consistently outperformed all steering baselines, improving average alignment probability over the strongest baseline by 15.9% on LLaMA and 13.42% on Qwen.
  • The PEFT-Level Performance Trade-off: On LLaMA-3-3B, INNSTEER reached an alignment probability of 93.41%—coming within 3.64 percentage points of LoRA fine-tuning while running 393x faster and requiring no weight updates.
  • Fluency Preservation: The method successfully steered model behaviors while maintaining generation fluency, showing only minimal changes in text perplexity compared to the base models.
Figure 3: Alignment–runtime trade-off on LLAMA-3-3B-INSTRUCT, averaged across tasks. INNSTEER is within 3.64 percentage points of PEFT while running 393ˆ faster.

Limitations and Future Work

While powerful, INNSTEER currently requires training a separate INN for each specific target behavior and intervention layer. This introduces a minor training overhead prior to deployment. Additionally, its performance remains bounded by the quality and diversity of the contrastive activation data used during training.

For future work, the authors suggest exploring multi-attribute INNs (a single network capable of handling multiple behaviors), automatic layer selection algorithms, and expanding evaluations to more complex settings like multi-turn conversations and human-preference benchmarks.

Summary updated

Abstract

Activation steering provides a lightweight inference-time mechanism for controlling large language models (LLMs) by modifying their internal activation vectors toward desired behaviors. Most existing methods compute a fixed steering direction in the original activation space, typically from pairs of contrastive examples using mean differences, linear probes, or arbitrary separability criteria. While effective to a certain extent, these methods treat behavioral control as a global, linear, additive offset: the same direction is applied across inputs, and behaviors are linearly separable. This can be restrictive when behavioral features vary nonlinearly across the activation space or lie on curved and anisotropic manifolds, where the optimal intervention may be input-dependent. To address this limitation, we propose INNSteer, a nonlinear activation steering framework based on invertible latent transformations. Rather than searching for a better steering vector in the original representation space, INNSteer learns a lightweight invertible neural network $ϕ$ that maps an LLM's activations into a latent space where behavioral classes are more amenable to linear control. At inference time, activations are mapped through $ϕ$, steered in the latent space, and mapped back through the exact inverse transformation $ϕ^{-1}$. This makes a simple latent-space translation become a nonlinear, input-dependent intervention in the original activation space. Across experiment settings on multiple LLM families, scales, behavioral traits, and safety benchmarks, INNSteer consistently improves model control over linear, transport-based, and nonlinear steering baselines while largely preserving generation fluency.
BibTeX
@misc{nguyen2026beyond,
  author = {Tuc Nguyen and Thai Le},
  title = {{Beyond Linear Activation Steering: Invertible Latent Transformations for Controlling LLM Behavior}},
  year = {2026},
  eprint = {2606.08454},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2606.08454},
}

Loading related content...

Figure