Can Weight-Space Programming Alter Learned Ethical Priorities?

Kai Yi JiarXiv 2026 · arXiv

TL;DR

This paper explores Weight-Space Programming (WSP)—using language models to reason about and directly edit the parameters of other neural networks—to alter a model's internal "ethical priorities." By precisely editing weights in a concept-supervised model, the authors demonstrate that they can selectively change how a model trades off keeping a promise against helping a larger group of people, without destroying its factual accuracy or basic reasoning.

What Problem Does This Paper Address?

As AI models are increasingly used for decision-making, we often want to adjust their "values"—for instance, making a model more or less likely to break a rule to achieve a greater good. Traditionally, this is done via fine-tuning on new data, which is computationally expensive and can lead to "catastrophic forgetting" or unintended side effects.

The authors ask: Can we change these priorities by directly editing the weights?

The challenge is selectivity. If you edit a model to favor "the greater good," does it actually change its ethical framework, or did you just break its ability to understand promises? Or worse, did you just induce a random bias toward a specific multiple-choice option (e.g., "always pick Option B")? This paper introduces rigorous behavioral controls to distinguish a genuine shift in priorities from mere "task substitution" or random noise.

Methods

The researchers employed several key techniques:

  • Weight-Space Programming (WSP): They used an LLM (specifically from the GPT-6 family/OpenAI Codex) to propose and implement specific weight edits based on the target model's architecture.
  • Concept Bottleneck Models: In their main experiment, they used a model that first predicts "concepts" (like the number of beneficiaries and the strength of a commitment) before a final "readout" layer makes a decision. This "exposed" architecture makes it easier to target the specific weights controlling the trade-off.
  • Task Substitution Baseline: They compared their targeted edits against a "column copy" method, where the weights for a "choice" task are simply replaced with the weights for a "factual" task (e.g., "who helps more people?").
  • Strict Behavioral Metrics ($T_{both}$): To ensure results weren't due to simple option-order bias, every scenario was tested twice with the options swapped (A vs. B and B vs. A). A "success" only counted if the model chose the target option in both orderings.

Datasets

  • Synthetic Ethical Scenarios: The authors generated thousands of prompts involving a community organization choosing between two workshops. Scenarios varied the number of people helped (the "benefit gap") and the type of commitment made (No promise, Ordinary promise, or Formal pledge).
  • COCO128: Used for a "vision pilot" where they edited a YOLO11n object detector to improve its recall for small objects (bottles, cups, bowls).

Key Insights and Ideas

  • The "Task Substitution" Trap: A simple weight edit can make a model look like it has new priorities by effectively forcing it to answer a different question (e.g., "Who helps more people?" instead of "What is the right choice?"). However, this fails on nuanced tasks. For example, a model might correctly break a promise to help 100 people but should not break it to help just one extra person. Simple substitution fails this "conditional" test.
  • Architecture as an "Exposed Mechanism": Editing is significantly more effective when the model's internal logic is structured. By using a concept-supervised readout, the authors could precisely tune the "weight" of a formal pledge versus an ordinary promise.
  • Order Consistency is Non-Trivial: The paper highlights that random weight edits often appear to "work" simply by shifting the model's bias toward a specific letter (A or B). Using the $T_{both}$ metric is essential for valid AI safety and alignment research.

Key Results

In Experiment 2, the authors successfully edited the model to follow a "conditional" policy: break an ordinary promise if the benefit gap is $\ge 3$, but only break a formal pledge if the gap is $\ge 5$.

Figure 1: Choices after editing the two commitment weights. Shading marks the gaps where the target favours the larger group. Each point counts only cases where the same underlying option wins in both orders.
  • Selectivity: The edit successfully switched the model to the larger group in 100% of targeted ordinary conflicts and 83.3% of formal conflicts, while preserving the original behavior for small-gap scenarios (where the promise should still be kept).
  • Factual Preservation: Factual accuracy (e.g., "Which option helps more people?") remained high (above 95%), proving the edit didn't "lobotomize" the model's basic understanding.
  • Vision Pilot: In the YOLO11n experiment, a simple bias edit increased small-target detections from 3/43 to 9/43 without requiring any new training data.

Limitations and Future Work

  • Synthetic Nature: The experiments were conducted on small, highly structured models. It remains an open question whether WSP can find and edit these "priority weights" in massive, "black-box" LLMs where the decision-making mechanism is diffuse and unobserved.
  • Generalization: The model was tested on the same "family" of scenarios it was trained on. True ethical alignment requires transferring these priorities to entirely new contexts.
  • Human-in-the-loop: While an LLM proposed the edits, the overall experimental design was guided by researchers. The authors suggest a "blinded" follow-up where an independent WSP agent must discover the correct edit without knowing the ground-truth weights.

Summary updated

Abstract

We investigate whether a weight edit can alter how a model trades off honoring a commitment against benefiting more people. In a concept-supervised text model, we alter fitted choice weights so that the benefit required to override a formal pledge is larger than for an ordinary promise. Across six independently trained text readers, the edit selects the larger group in both orders on 100% of targeted ordinary conflicts and 83.3% of targeted formal conflicts. It preserves the source model's success rates on smaller conflicts and its factual accuracy. The remaining formal-conflict failures coincide with errors reading a beneficiary number. A separate classifier demonstrates why this distinction matters: reusing a factual answer passes an unconditional choice test but fails to preserve small-gap promises. In a YOLO11n pilot on COCO128, a bias edit increased small-target matches from 3/43 to 9/43, and an equivalent score calibration reproduced the retained detections. These exploratory results show a way to test selective changes to learned priorities in a model with an exposed trade-off mechanism. Whether the method can discover such changes in less structured language models remains open.
BibTeX
@misc{ji2026can,
  author = {Kai Yi Ji},
  title = {{Can Weight-Space Programming Alter Learned Ethical Priorities?}},
  year = {2026},
  eprint = {2611.24330},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.24330},
}

Loading related content...

Figure