The Loop Is the Product

One thing I keep noticing when I use AI tools is that the model matters less than I expected, and the loop around the model matters more.

By loop, I mean everything that happens between the user typing something and the model responding. How the input is shaped. What context is loaded. What tools the model can call. How its output is checked. What gets retried and what gets shown to the user. All of that is the loop.

When I switch between two products built on the same underlying model, the experience can feel completely different. Sometimes one feels smart and the other feels dumb, even though the raw capability is identical. The gap comes from the loop.

I think this is why loop engineering has quietly become one of the most interesting parts of building with AI. The model gives you a floor of intelligence. The loop is what determines how much of that intelligence actually reaches the user. A good loop can make a mediocre model feel great. A bad loop can waste an excellent one.

What I have not fully worked out is how to design a loop well. Most of what I have seen is intuition dressed up as engineering. People try things, some of them stick, and after a while a pattern emerges. There does not seem to be a strong theory yet. Or if there is, I have not found it.

A few rough ideas I keep coming back to. Loops that fail loudly are better than loops that fail silently. Loops that let the model ask for help usually beat loops that force it to guess. Loops that store some memory of past decisions tend to feel more coherent, even if the underlying model has no idea what happened yesterday.

There is also a subtle thing about verification. Every step in a loop is an opportunity to check the model. But checks cost latency, and too many of them make the whole system feel slow. Finding the right balance between trust and verification feels like one of the core skills of loop engineering, and I do not think I am close to being good at it yet.

If I had to guess where a lot of value in AI products is going to come from over the next few years, I would put it here. Not in bigger models, though those will keep coming. In the space around the models. In the loops. That is where the taste of a builder actually shows up.