Towards the Readability of LLM-Generated Codes through Multitask Representation Engineering

Huifan Gao, Liuhua He, Yinghui Pan, Shenbao Yu, Yifeng Zeng, Shengchao Qin, Weidi SunarXiv 2026 · arXiv

TL;DR

While large language models (LLMs) excel at generating functionally correct code, tailoring code style and readability to diverse developer preferences remains a major challenge. This paper introduces Multitask Representation Engineering (MRepE), a lightweight framework that simultaneously steers multiple readability metrics (like comment density and logic complexity) inside an LLM's hidden states without retraining. Crucially, the authors provide mathematical bounds to safely manage the trade-off between stylistic adjustments and functional code correctness.

What Problem Does This Paper Address?

Code quality isn't just about passing test cases; it's about maintainability and human comprehension. However, code readability is highly subjective—a novice programmer might prefer verbose, explicit comments, whereas a seasoned engineer might prioritize concise naming and simplified control flows.

Fig. 1. For different users with varying experience levels, the format of comments, naming conventions, and cyclomatic complexity can lead to differences in code readability. While User 1 (2 or 3) deems it clear and easy to understand, User 4 insists that its readability is still insufficient, advocating for the addition of a formal docstring.

Existing methods to alter code style rely heavily on prompt engineering (which is notoriously unstable) or full model fine-tuning (which is computationally expensive and rigid). Representation Engineering (RepE) offers a cleaner alternative by injecting "steering vectors" directly into the model's hidden layers during inference.

However, prior RepE techniques only handled single tasks. Readability is inherently multi-dimensional, spanning comment density, naming conventions, and cyclomatic complexity. If you independently extract and inject multiple steering vectors at the same time, they interfere with each other in the model's activation space, leading to chaotic code structures or breaking functional correctness entirely.

Methods

The authors propose the Multitask Representation Engineering (MRepE) framework to coordinate multiple stylistic adjustments simultaneously through two core innovations:

  • MOC-JPCA (Joint Principal Component Analysis with Multidimensional Orthogonal Constraints): Instead of calculating standard PCA for each readability metric independently, this new algorithm extracts steering vectors while enforcing pairwise orthogonality on a Stiefel manifold. This explicitly suppresses cross-metric interference, ensuring that increasing comment density doesn't accidentally scramble naming conventions or overcomplicate control flows.
  • Theoretical Trade-off Bounds: The paper mathematically formalizes the relationship between readability steering and code correctness. They establish a lower bound for readability improvement (Theorem 2) and an upper bound for correctness degradation (Theorem 3), giving developers a predictable "control dial" to balance style versus functionality.

Datasets

The experiments were grounded using the MBPP (Mostly Basic Python Problems) benchmark, which contains roughly 1,000 crowd-sourced Python problems. The authors extended 300 samples from MBPP to create two new evaluation datasets:

  • MBPP-CR (Readability): A dataset containing paired contrastive examples mapping high vs. low features for Comment Density (CD), Naming Conventions (NC), and Cyclomatic Complexity (CC).
  • MBPP-CC (Correctness): A multiple-choice evaluation dataset designed to test whether the steered models can still identify functionally correct code snippets.

Key Insights and Ideas

  • Disentanglement is Essential: Simply adding multiple steering vectors together causes cross-talk. Forcing geometric orthogonality during the vector extraction phase is vital for stable multitask control.
  • Readability as a Proxy Task: The authors discovered a strong positive correlation between a model's understanding of readability (evaluated via binary judgment tasks) and its actual generation of readable code.
  • Predictable Degradation: Stylistic steering acts like a controlled injection of noise relative to the model's core capabilities. Because this noise can be statistically bounded, engineers can aggressively optimize code style while keeping functional damage under a strict, predetermined ceiling.

Key Results

The MRepE framework was validated across three major models: DeepSeek-R1-14b, Qwen2.5-Coder-14b-Instruct, and CodeLlama-13b-Instruct.

  • DeepSeek-R1-14b: Achieved significant readability improvements across all three metrics while restricting code correctness degradation to a mere 0.79%.
  • Qwen2.5-Coder-14b-Instruct: Successfully optimized readability while keeping correctness loss under 0.95%.
  • CodeLlama-13b-Instruct: Maintained an incredibly low correctness penalty of just 0.28%.

The experimental data tightly matched the theoretical surfaces predicted by the authors' equations, proving that the trade-off is highly stable and controllable.

Limitations and Future Work

  • Scope of Metrics: The study is currently restricted to three primary dimensions of readability (comments, naming, complexity). The authors plan to extend MRepE to more nuanced attributes like modularity, maintainability, and specific documentation styles (e.g., docstring formats).
  • Scale of Evaluation: The framework was tested on a benchmark of basic, isolated Python problems. Validating its efficacy in large-scale, multi-file enterprise software engineering environments remains an open challenge.

Summary updated

Abstract

Correctness and readability are key measures of code quality, respectively ensuring functional fidelity and ease of comprehension. While most existing research focuses on improving the correctness of large language models~(LLMs) generated codes, readability remains under-addressed. Enhancing readability through targeted control is challenging due to its subjective nature. In this article, we employ representation engineering~(RepE) as the targeted control method given its characteristics of low data dependency and low computational cost. Prior work on RepE has primarily focused on the targeted control for a single task, but improving the code readability requires the control across multiple tasks. Accordingly we proposes the multitask RepE framework and theoretically discuss the impact of the multitask steering method on the tradeoff between the code readability and correctness. We further provide comprehensive experiments in support. All the relevant implementations are open-source and available upon request.
BibTeX
@misc{gao2026towards,
  author = {Huifan Gao and Liuhua He and Yinghui Pan and Shenbao Yu and Yifeng Zeng and Shengchao Qin and Weidi Sun},
  title = {{Towards the Readability of LLM-Generated Codes through Multitask Representation Engineering}},
  year = {2026},
  eprint = {2606.06214},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2606.06214},
}
Figure