Finding a new language model is not a trivial task. This process includes a long training plan and high computing power to fine-tune the LLM and ensure that it can process questions and solve them as effectively and efficiently as possible. But what if, instead of retraining a model to improve it, another AI could change it from the inside? That is more or less achieved.
achievement. Models are improved by training them to achieve greater capacity and more efficient processes. What scientists at MIT CSAIL have achieved is a shortcut that could lead to a significant breakthrough for more contained models: they’ve been able to improve a superior AI by editing a minimal portion of its weights.
process. The scientists used Qwen3.5-0.8B and tried to push the model up a notch in mathematical processing. To do this, they changed the weight of its cousin to solve some logic loops in the GPT-6 Astra. Following that process, Qwen increased its MATH-500 power from 7.9% to 48.1%.
Not only is the use of another language model different and the absence of training, the interesting thing is that Astra was able to solve mathematical problems better by changing its weights. Specifically, Astra worked on 0.016% of the peso in the currency. MIT calls this process Weight-Space Programming.
Weight-space programming: Program an AI directly by typing in its weights, millions of numbers that determine how the model behaves.
This is not the end of the training. The advance is important and opens a new line of research. It also suggests a path through which different companies can improve their cheap models in the same way: on the cheap. It sounds promising, but there are still many obstacles.
The first is that MIT research is not peer-reviewed, it is part of primary work. The second is that the method only works if there is something to remove. With the Gemma 3 270M, a Google model that rarely suffers loops, the improvement was almost nil.
The first step towards self-healing AI?. If an AI can already fix others from the inside, the next step, which worries companies like Anthropic, is to implement the improvements themselves. The MIT researchers have hypothesized this conclusion themselves, although it has yet to happen.
Another question that weight programming leaves us with is the current optimization of models. Because if researchers are able to make QUEN more deterministic in math after applying scissors, are we using LLMs that could be better if there was more optimization behind it?
Cover image
In Xataka | “They’re going to get a lot worse”: Jason Clinton, Anthropic’s deputy director of cybersecurity, warns of what’s to come
