Most people have seen the autoresearch pattern by now. Karpathy's repo is the clean version of it: you have an ML model, you give a coding agent a metric, and you ask it to modify the code until the error goes down. A hill climb over a model.
For well-posed coding tasks this works very well. But read the labels on the steps it kept: halve the total batch, add 5% warmup, change the unembedding LR, change the RoPE base frequency, change the random seed. Every one of them is a hyperparameter, or a knob that was already exposed in the code.
That is the shape of the thing. When the problem is well-posed, an agent will find the knobs and turn them competently. When the problem is open-ended and long-horizon, which is most scientific problems, the agent saturates.
Agents sprint, then plateau
The gap here is not implementation. Agents are extremely good at implementation: writing the code, changing the code, running experiments over a lot of data, parsing logs, presenting results. What they run out of is ideas, which is what people call research taste.
Good humans keep going. The top 1% keep improving over time. The difference is that good hypotheses keep coming up.
What actually needs fixing in the loop
Most of this cycle is a memory and implementation problem. Learning from an iteration, not repeating a mistake, keeping track of what was already tried. Those are real bottlenecks for coding agents, but I think they are largely solved by simply organizing patterns of activity: log what ran, log what was kept, prune it, summarize it.
Hypothesis is the difficult one. So the question I want to focus on is narrow: how do we get good hypotheses out of a coding agent, so that it keeps improving instead of saturating?
The problem: in silico PET
Here is what we are doing at Radicait, so the example is concrete. We build in silico PET: you have a CT, and we generate a PET scan from it.
A patient has a nodule in the lung and the question is whether it is cancerous. To answer it they do a PET scan, which is a difficult and very time-consuming process. Instead we do an ML image translation: change the modality, learn the structure of the body, and infer what the activity of the tissue would look like in a PET scan. Certain tissues absorb more radioactive tracer and light up, and tumours usually do.
That is one model. But the problem itself has many components, and this is true of any scientific task. A research programme that runs two years or ten years decomposes into steps, and each of those steps is fundamentally a goal and a loop: prepare the data, train the model, register the scans, build the metrics. About 80% of the work is bringing together a good dataset before any of the modelling matters.
Decompose the problem into /goal
/loop
↗ Prepare data
·
↘ Train MLWhere the climb saturates
You can see the shape of it. A range of possibilities gets explored, some become dead ends, some improve nothing, and at a certain point it saturates. And then you really need a good idea.
Here is the specific case. The initial model was 2.5D, meaning each CT slice was treated as 2D convolutions, stacked over the channel dimension. Give that setup to a coding agent with an objective and it will go through hyperparameters and play around with the problems it already knows. It will not make a radical change. It will not propose replacing channel-stacked 2D convolutions with true 3D convolutions, and it will not turn the problem upside down.
What I actually did was sit in the middle of the Codex loop and interject: what about this idea, what about that one, go read the papers out there and see what other people are trying. That works. But it is not autonomous, and it does not scale.
Induce a hierarchy of the problem
This is the trick I found works very efficiently, and it is very similar to chain of thought applied at the level of the problem instead of the answer: make the decomposition an explicit, separate action.
At the top is the problem exactly as described above: translate pulmonary-nodule CT patch stacks into equivalent PET/SUV patch predictions. Below it are the domains, which are the data and contracts, the learning core with the architecture and the training losses, the evidence and metrics, and the operational scripts that run the model. Below those are the components, and below those, the code.
You do not draw this by hand. A coding agent induces it from the codebase with a single prompt: read the code, then write a series of linked markdown documents, one layer of abstraction per level, continuing down until the last node is a link to the actual source file.
1 · the general problem statement
2 · all components, one paragraph each, in a single file
3 · a detailed page per component, with pseudo-code or short snippets
4 · continue down until the leaf is a link to the real code fileThe result reads as ordinary documentation: a README with the problem statement, a components page, then a page per component that works from a conceptual responsibility down to functions and files. When you open it as a graph, it is a tree with the problem at the root and source files at every leaf.
Generate hypotheses against the hierarchy
Now the task changes. Instead of here is our codebase, here is my objective, optimize this process, which saturates after a while, it becomes: given this hierarchy, come up with ideas that would improve this problem.
Because the scaffold forces the model to walk every component, the search becomes comprehensive rather than opportunistic. It has somewhere to attach each idea, and it cannot quietly skip the parts of the system it finds least interesting. Each hypothesis comes out with the change, why it may help, how to implement it, how to evaluate it, and what the risk is.
That is the radical change I previously had to supply by hand. It now falls out of the scaffold, along with ones I would not have thought of: two-stage coarse-then-residual models, teacher–student distillation from a larger offline 3D model, and a data-centric alternative that argues the bottleneck is label and domain coverage rather than architecture at all.
You can then review the plan adversarially or collaboratively with a second model, before any of it is implemented.
Anatomy of a loop
The loop itself is simple: codebase, output metrics, /goal achieved? No, hypothesize a required change, implement it, back to the codebase. What makes it go faster is not a tighter loop, but a better hypothesis at the one station that was weak.
It also means the loop should be able to call out of itself. For hypothesis generation and for critique I reach for a model with much larger test-time compute, gpt-5.5-pro, through Peter Steinberger's @steipete/oracle CLI, which packages the code and the data, sends them to the API, and brings the answer back into the loop.
Every pass asks the same thing: this is what was implemented, this is the outcome, does this make sense, and what would you do next. The critique matters as much as the generation.
The second loop: when the metric is a picture
Data preparation has the same structure, and it shows the other half of the problem. Take image registration.
At the beginning these two scans are not aligned. They were taken at different times, sometimes days or a week apart, on a different scanner, with different positioning, and the body is not in the same configuration because of respiration and because the patient moved. Registration is the optimization loop that brings them together.
It has a goal, and it has many components and many metrics. But a lot of those metrics are not quantitative. They are qualitative, and it is hard to tell which bug or which issue has come across.
So put that judgement in the loop as a skill. Part of the metrics step calls a model with better multimodal capability and asks for an opinion: does this look right, is this scan aligned, is the mask correct.
What is still missing
Implementation is largely solved, especially where the problem can be run in a box. Hypothesis generation is what this scaffold addresses. But the big thing in science is better observations, and that is where current multimodal models are genuinely weak.
No LLM today identifies these nodules well. They are simply not trained on scientific images and scientific data.
The tiny changes are what matter, such as the existence of that one nodule in that one slice. Until a model can observe the way a trained scientist observes, it cannot close the loop on its own. There is a lot of room here for the people who build these models to fine-tune or adapt them for scientific observation. I think that, more than reasoning, is the biggest bottleneck standing between us and a full scientist running in the data center.
This is a chain-of-thought-era trick
I want to be honest about what this is. Chain of thought sat on top of a base model like GPT-4o and bought much better reasoning, until post-training absorbed the idea and you stopped needing to ask for it. Hierarchical decomposition is the same kind of trick, applied to the structure of the problem rather than the structure of the answer.
Going forward, with models that are better post-trained to compartmentalize and break down problems, we will probably need less and less of this. But in the meantime, and in the absence of models that do it natively, you can do it explicitly: decompose the problem into a hierarchy, then go through those hierarchies for improvements in each component.
That gives you a structured way to scale test-time compute, a place to put the next hundred thousand tokens that is not just a longer context polluted with everything that already happened. Somewhere to attach each idea, and a reason to keep the loop going.
All demonstrations here come from real scientific problems solved in industrial and applied research settings. The pipeline shown is Radicait's CT-to-PET nodule model, and the hierarchy, the hypothesis backlog and the QC sheets are the actual generated artifacts.
This piece is adapted from the talk I gave at the AI Engineer (AIE) 2026 conference in San Francisco. The recording is at the top of this page.