Safe Personal RSI: Evaluating and Repairing Personal-Agent Updates

Jiapeng Pei, Yuhang Yao, Sitan Liu, Hanxin Chen, Hao Li, Guogeng Li, Jiapei Liu, Ruijie Chen, Hongshan Lin, Yang Mo, Tianyu Shi, Shangding GuarXiv 2026 · arXiv

This technical report introduces Safe Personal RSI, a framework designed to manage the "recursive self-improvement" (RSI) of personal AI agents. While we want agents to get better over time, unvetted updates to their instructions can lead to "misevolution"—where an agent becomes more efficient at the cost of leaking private health data or making unauthorized financial commitments.

TL;DR

Safe Personal RSI is an automated evaluation and repair pipeline that vets prompt-based updates for personal agents before they are adopted. By using a "rollback" mechanism to identify which specific instruction changes cause safety failures, the system successfully rejected 20 out of 21 risky update attempts and repaired others to achieve 0% violation rates in key tests.

What Problem Does This Paper Address?

Personal AI agents (like the paper's case study, "Kicker," a health assistant) rely on a "harness"—a set of reusable prompt instructions. Developers often update these instructions to make the agent more proactive or concise.

However, there is a fundamental tension between utility (getting things done quickly) and authorization (not overstepping boundaries). An instruction to "achieve inbox zero" might inadvertently cause an agent to forward sensitive medical summaries to a third party without asking. This paper addresses the lack of a rigorous, automated way to evaluate whether a proposed instruction update is safe to deploy in a sensitive, personal context.

Methods

The authors restrict RSI to prompt updates only, keeping model weights and runtime code fixed. They implement a six-step evaluation loop:

  1. Propose: Generate a new version of the prompt instructions.
  2. Search + Confirm: Test the new version against a benchmark; if it fails, confirm the failure is reproducible.
  3. Investigate: Use rollback localization—temporarily undoing specific prompt changes to see which exact sentence or block introduced the safety regression.
  4. Repair + Verify: Use an LLM to rewrite the problematic block based on the failure traces.
  5. Decide: An "Admission Gate" checks if the repaired version meets safety and utility thresholds.
  6. Adopt or Keep: If the update passes, it's adopted; otherwise, the agent reverts to the previous "parent" instructions.
Figure 1. Update evaluation. Search, investigation, repair, and admission use development cases. Final evaluation runs fixed harnesses.

Datasets

The researchers developed an executable benchmark containing 29 cases:

  • 23 core email attacks: Scenarios where external emails try to trick the agent into disclosing data or making commitments.
  • 6 extensions: Specialized cases for calendar, health, and marketplace (e.g., handling a buyer's lowball offer).
  • Probes: Specific tests for memory-entry deletion and OAuth scope enforcement.

Unlike static datasets, these are "replays" run in a simulated environment (Willow) with synthetic accounts and external services.

Key Insights and Ideas

  • The "Inbox-Zero" Trap: The most dangerous updates are those that pressure the agent for speed or conciseness. These "stress" edits consistently broke authorization boundaries, leading to the highest violation rates.
  • Rollback as Diagnosis: Simply knowing a prompt is "bad" isn't enough. By systematically rolling back changes, the system can pinpoint the exact instruction (e.g., "do not ask further questions") that triggered the unsafe behavior.
  • Repair Transferability: One of the most striking findings was that a safety repair "learned" on a smaller model (Qwen2.5-7B) could be transferred to a larger, more capable model (Qwen3-14B) and still provide significant protection against strong attacks.
  • Safety vs. Utility: The framework doesn't just look for "zero violations"; it measures Safe Task Completion. An agent that stays safe by refusing to do anything is useless. The goal is to maintain safety while still fulfilling the owner's legitimate requests.

Key Results

The framework proved to be a highly effective "gatekeeper":

  • Rejection Rate: In a controlled study, the system rejected 20 of 21 deliberately risky update attempts.
  • GPT-5.6 Luna: A successful repair reduced attack violations from 10.7% to 0% while increasing safe task completion from 89.7% to 100%.
  • Qwen3-14B Transfer: Under "stronger" attacks, using a transferred repair from a smaller model reduced violations from 69.0% to 17.9% and boosted safe task completion from 19.5% to 64.4%.
Figure 2. Repair confirmation on fixed harnesses. GPT-5.6 Luna uses standard attacks and a broad authority rule. Qwen3-14B uses stronger attacks, a credential-only rule, and a transferred repair.

Limitations and Future Work

  • Empirical Thresholds: The "Admission Gate" relies on point estimates and thresholds (e.g., a risk rate $\le 0.1$). It does not currently offer formal statistical safety guarantees.
  • Fixed Runtime: The study focuses entirely on prompt updates. It does not address safety issues that might arise from updating the underlying model weights or the tool-calling code.
  • Benchmark Scope: While the 29 cases cover critical areas like health and finance, they represent "known" attack families. Future work is needed to see how the system handles entirely novel, "zero-day" prompt injection techniques.
  • Cost of Evaluation: The paper notes that the overhead of searching for and repairing updates is significant and should be measured separately in production environments.

Summary updated

Abstract

Kicker is a personal health assistant. Changes to an assistant's reusable instructions can improve task handling while changing what it discloses, which requests it follows, and how it reports actions. We study Safe Personal RSI, an update-evaluation component developed for Kicker. We restrict recursive self-improvement (RSI) to prompt updates, with model weights and runtime code fixed. The procedure tests proposed and current instructions, repeats failing cases, temporarily undoes changes to guide repairs, and checks safety and task performance before adoption. If no version passes, the current instructions stay in use. Our executable benchmark contains 29 email, calendar, health, and marketplace cases, with separate probes for authorization reuse, memory, and tool-output injection. Across three models and two deliberately risky edits, the procedure rejects 20 of 21 update attempts. In repair confirmation, an admitted GPT-5.6 Luna repair reduces attack violations from 10.7% to 0% and increases safe task completion from 89.7% to 100%; a broad handwritten authority rule matches these outcomes. Under stronger attacks, a repair transferred from Qwen2.5-7B-Instruct to Qwen3-14B reduces violations from 69.0% to 17.9% and increases safe task completion from 19.5% to 64.4%. Separate runtime probes examine memory-entry deletion and identify missing local OAuth scope checks in the evaluated snapshot.
BibTeX
@misc{pei2026safe,
  author = {Jiapeng Pei and Yuhang Yao and Sitan Liu and Hanxin Chen and Hao Li and Guogeng Li and Jiapei Liu and Ruijie Chen and Hongshan Lin and Yang Mo and Tianyu Shi and Shangding Gu},
  title = {{Safe Personal RSI: Evaluating and Repairing Personal-Agent Updates}},
  year = {2026},
  eprint = {2611.23651},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.23651},
}
Figure