MIT Team Says GPT-6 Astra Edited a Small Qwen Model With No Training and Its Math Score Soared
MIT CSAIL researchers say OpenAI's GPT-6 Astra changed 0.016% of a small Qwen model's weights and lifted its MATH-500 score from 7.9% to 48.1% without any training. The catch, per the authors themselves, is that it mostly fixed a repetition loop rather than teaching new math.
A small AI model that could barely do math suddenly got a lot better at it, and nobody trained it. Researchers at MIT CSAIL say OpenAI's GPT-6 Astra opened up the small Qwen3.5-0.8B model, found the numbers that made it ramble in circles, and nudged them. The model's MATH-500 score went from 7.9% to 48.1%, according to the authors.
In a blog post dated November 2, Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab describe what they call Weight-Space Programming. Instead of running gradient descent, a large model reads another network's weights as text and rewrites them like code. The team says Astra got a Python session over the weights, practice problems from the GSM8K and MATH training sets, and a budget of 50 test runs. No training loops were allowed.
What did Astra actually do? By the authors' account, it traced the small model's habit of repeating itself to one direction written by the MLPs in layers 15 and 19. It subtracted that direction from 64 output columns in each layer. That is 131,072 weights, or 0.016% of the model.
The base model was a mess. The authors report that 93.0% of its MATH-500 answers ran to the 32,768-token limit without finishing. After the edit that fell to 33.1%. GSM8K rose from 13.8% to 67.1%, and IFEval, an instruction-following test, climbed from 21.0% to 62.3%.
"We find early signs that frontier LLMs are surprisingly good at natively reading, writing, and reasoning about neural network weights." Rikiya Takehi, PhD student at MIT CSAIL, in a social media post quoted by The Register
A loop fix, not a math lesson
Most of the gain comes from a model finally finishing answers it could already work out. The authors say as much. Their post states that editing weights "could remove an obstacle that hid an ability, but it could not add an ability the model lacked."
They have evidence for that. On Gemma 3 270M, which rarely loops, Astra's edits moved MATH-500 from 6.8% to between 7.0% and 7.7%. Qwen's own model card warns the model is prone to thinking loops, so this was a known flaw in a very small model.
There is a second wrinkle. The Turing Post newsletter reports that the prompt given to Astra included a hint that if Qwen repeats itself and never finishes, fixing what drives that repetition counts. According to that account, Teoh said Claude Code added the hint while expanding the team's short assignment, with no further human feedback. The same newsletter says a shorter prompt without the hint reached 14.1% after 25 test runs, matching the hinted run, but no 50-run result for it has been shown. I want that row before crediting Astra with spotting the problem.
Cost is also unproven. VentureBeat reports that the work did not establish that direct editing beats conventional training on speed, cost or effectiveness.
Some outside checking exists. A Hugging Face model card for the edited checkpoint says a re-run over 2,000 generations scored 51.0% on MATH-500, with 31.6% of answers hitting the token limit. The authors thank a Hugging Face researcher for rebuilding the checkpoint, so call it a re-run, not a clean outside replication.
"This is changing: our neural network weights will soon be natively optimized by LLMs that deliberately reason about their weights." The MIT CSAIL authors, in their blog post
That is a prediction, and it is theirs. The authors list recursive self-improvement, reduced sycophancy, auditing for backdoors and interpretability as possible uses. None of those were shown. The post itself calls the results "early" and says capabilities are "exciting but still limited." It is a research blog post, not peer reviewed.
The recursive self-improvement angle explains the buzz. A model that can patch another model, or its successor, is the scenario our report on Anthropic's amended filing circled around.
OpenAI has not responded publicly to the work, as far as I could find. The Register carried no company comment either.
My read: a real, checkable trick, wrapped in a very big story. Fixing a looping tiny model is useful. It is not proof that one AI can safely rewire another.
