RSIV-Harness: A Recursive Self-Improvement Harness Framework for Open-Domain Conversational Text-to-Visualization

Jiawei Lu, Chen Zhang, Jinwei Lu, Yuanwei Song, Yuanfeng Song, Raymond Chi-Wing WongarXiv 2026 · arXiv

TL;DR

RSIV-Harness is a novel framework that enables conversational text-to-visualization agents to improve themselves by evolving their "harness"—the prompts, tools, and logic surrounding a frozen Large Language Model (LLM)—based on their own failure traces. Alongside this method, the authors introduce ODAC-Vis, a rigorous new benchmark that challenges agents to discover relevant data across hundreds of databases, resolve ambiguous user requests, and refine charts through multi-turn dialogue.

What Problem Does This Paper Address?

Creating a data visualization from a natural language request is rarely a one-shot process. In real-world enterprise settings, users often don't know which database contains their data, their requests are frequently ambiguous (e.g., "show sales trend" could mean gross or net sales), and they often want to iterate on a chart once they see it.

Current agents typically rely on a static "harness"—a fixed set of system prompts, tool interfaces, and control policies. When these agents fail, they keep failing in the same way because the harness never changes. While one could retrain the underlying LLM, this is computationally expensive and often unnecessary. The authors argue that the bottleneck isn't just the model's weights, but the rigid environment (the harness) in which the model operates.

Methods

The authors propose a bi-level recursive self-improvement (RSI) architecture:

  1. Inner Task Loop: This is the "acting" layer. It uses a ReAct (Reason + Act) pattern where the agent analyzes a request, uses tools to discover data, asks clarifying questions if needed, and eventually generates visualization code (Python/Plotly).
  2. Outer Self-Improvement Loop: This is the "learning" layer. It examines traces of the agent's failures across many tasks. It identifies recurring failure patterns and autonomously proposes revisions to the harness (e.g., updating a prompt to be more specific or adding a new "skill" to the middleware).
Architecture of RSIV-Harness showing the inner task-solving loop and the outer self-improvement loop.

To ensure this "evolution" doesn't break things that were already working, they implement a four-layer stability framework:

  • Anchor Checks: The new harness must maintain or improve performance on a fixed validation set.
  • Memory Checks: It uses a "regression memory" to ensure the agent doesn't lose the ability to solve tasks it previously mastered.
  • Refine/Restart Rules: If a proposed change fails these checks, the system reverts to the last "champion" harness.

Datasets

The paper introduces ODAC-Vis (Open-Domain, Ambiguous, and Conversational text-to-visualization), a benchmark consisting of 640 tasks across 206 SQLite databases.

Unlike previous benchmarks that give the agent the correct table upfront, ODAC-Vis requires:

  • Open-Domain Discovery: Finding the right database among hundreds.
  • Ambiguity Resolution: Handling eight types of ambiguity (e.g., column selection, aggregation granularity, chart type).
  • Conversational Refinement: Supporting "drilldown" (deepening analysis) or "iteration" (changing the chart) scenarios.
Figure 1: Three challenges in real-world text-to-visualization. The assistant agent needs to discover relevant databases, resolve ambiguous requests before charting, and refine the chart across dialogue turns.

Key Insights and Ideas

  • Harness Evolution vs. Weight Tuning: You can achieve massive performance gains by optimizing the instructions and tools given to an LLM rather than the LLM itself. This keeps the core model "frozen" and efficient.
  • Stability is Non-Negotiable: Open-ended self-improvement is risky. The authors' formal stability guarantees (like anchor monotonicity) are crucial for preventing the agent from "drifting" into worse behaviors.
  • Cross-Backbone Transfer: Interestingly, a harness evolved using one model (e.g., DeepSeek) can be successfully transferred to a different model (e.g., GPT-6 Luna), suggesting that the learned "skills" and "prompts" capture general best practices for the task.

Key Results

The researchers found that just a single round of recursive self-improvement led to dramatic gains:

  • Executability: Jumped by +28.02 points compared to a static seed harness.
  • Chart Structure: Improved by +34.01 points, meaning the final visualizations were much closer to the user's actual intent.
  • Stance Accuracy: The agent became significantly better at knowing when to ask a clarifying question versus when to just write the code.
Figure 3: An illustrative task of ODAC-Vis. The first request is unambiguous, so the model retrieves the relevant data and writes a bar chart. A later iteration leaves the measure unspecified; the model asks a clarification question and emits the line chart after the user selects weight.

In case studies, RSIV-Harness was the only method capable of correctly joining multiple databases while respecting specific user constraints like excluding certain data points and encoding variables with specific colors.

Limitations and Future Work

  • Single Round Evolution: The experiments primarily focused on one round of improvement. The long-term dynamics of many successive rounds (potential for "overfitting" the harness to the training panel) remain to be explored.
  • Computational Overhead: While cheaper than retraining an LLM, the outer loop requires running many tasks to generate failure traces, which can be time-consuming.
  • Dependency on Proposal Model: The quality of the harness revision depends on the LLM used in the outer loop to analyze failures. If the "analyst" model is weak, the proposed improvements may be suboptimal.

Future work could involve extending this to even more complex data types (like NoSQL or unstructured data lakes) and exploring multi-agent harness evolution where different agents specialize in different parts of the visualization pipeline.

Summary updated

Abstract

Users often request charts without knowing which database or table holds the relevant data. Such requests may be ambiguous, and users may ask follow-up questions to deepen their analysis or iteratively refine existing charts. A useful assistant must therefore discover relevant data sources among heterogeneous databases, ask clarifying questions when needed, and track evolving user intent while preserving constraints established in earlier turns. Yet recent text-to-visualization agents typically rely on static agent harnesses that are predefined and remain unchanged despite recurring failures across tasks. To address this limitation, we introduce RSIV-Harness, which keeps the language model weights fixed and applies Recursive Self-Improvement to revise the surrounding harness from the agent's own failure traces. To balance flexible self-improvement with stability, we introduce a four-layer framework that evaluates proposed harness revisions through anchor and memory checks, refining those that pass and restoring the best validated harness otherwise. We further introduce ODAC-Vis, a benchmark with 640 tasks for Open-Domain, Ambiguous, and Conversational text-to-visualization that jointly evaluates open-domain data discovery, ambiguity resolution, multi-turn visualization refinement, and executable chart generation. Experiments show that a single round of RSIV-Harness improves all six evaluation metrics relative to the seed harness, with particularly large gains in executability (+28.02 points) and chart structure (+34.01 points), demonstrating the ability to learn from task-solving experience and turn recurring failures across tasks into substantial performance improvements.
BibTeX
@misc{lu2026rsiv,
  author = {Jiawei Lu and Chen Zhang and Jinwei Lu and Yuanwei Song and Yuanfeng Song and Raymond Chi-Wing Wong},
  title = {{RSIV-Harness: A Recursive Self-Improvement Harness Framework for Open-Domain Conversational Text-to-Visualization}},
  year = {2026},
  eprint = {2610.35316},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.35316},
}
Figure