Weight-Space Programming: How GPT-6 Astra Rewrites Model Weights

Programming weights instead of training them

Frontier LLMs are surprisingly good at natively reading and writing neural network weights, well past what we’d expect from humans or conventional algorithms.

Weight-Space Programming is the name Rikiya Takehi, Jayden Teoh, Alex Zhang, and Omar Khattab use for a concrete result: a frontier model can look at raw neural-network weights and change them on purpose. All four are affiliated with MIT CSAIL, and they published the account on November 2, 2026.

GPT-6 Astra does this without gradients and without being allowed to write training loops. It reasons about what each part of a network computes, or should compute, and edits the weights to match. The authors treat that as a shift for both optimization and interpretability, because the weight file itself becomes something a model can inspect and revise.

What Astra can tell from weights alone

On the reading side, raw weights are enough for Astra to say what a network was trained to do. In one probe it reads 17,253 weights with no tools and names Breakout at 99% confidence.

The same account tests whether Astra can tell what a diffusion model will generate, what a language model learned from the middle of the network alone, and which language model is sycophantic. Those are identification tests: behavior recovered from parameters rather than from running the model and scoring its outputs.

• Breakout, from the file

  17,253 weights, no tools, 99% confidence that the network was trained on Breakout.

• No output log required

  The claim is that training intent shows up in the weights themselves, which is why the authors tie the result to interpretability as well as to editing.

Writing a policy, then patching Qwen

This weight correction raises MATH-500 from 7.9% to 48.1% and generalizes out of the box to instruction following, raising IFEval by 3x.

Without environment feedback, Astra writes small game-playing networks from scratch that outplay networks trained with PPO for 100 million steps. The account also covers writing a Go algorithm and fixing a failing network by editing its weights.

The scaled case is Qwen3.5-0.8B. Asked to improve its math without gradients, Astra finds a weight-level cause of reasoning loops that Qwen’s own model card warns about, then changes 0.016% of the weights. Sampling settings stay the same. Math is the target; instruction following moves with it. The authors’ shorter account puts the math gain at up to about six times and the instruction-following gain at three times.

What you can use, and what is still open

Our neural network weights will soon be natively optimized by LLMs that deliberately reason about their weights.

You can read the full account on rikiyat.github.io under the title Sparks of Weight-Space Programming. The notes describe a research demonstration, not an app, an API, a price, or a downloadable editor model.

The hard numbers sit on specific setups: a 0.8B language model, a 0.016% patch, a 17,253-weight game network, and small policies compared with a 100-million-step PPO run. The title’s “sparks” is the authors’ own marker that native read-and-write editing of weights is early evidence, pointed at a future where LLMs optimize networks by reasoning about those weights.