← Blog··10 min read

Attraction Basins

Part 3 of Code in the AI-Primary Era. Previous: A Model Is Not a Calculator.

Anyone who has watched an agent debug for long enough has seen the loop: fix, run, fail, near-identical fix, run, fail, a third fix that reverts to the first. The trajectory bounces between two states, each step locally reasonable, going nowhere. It is tempting to read this as stupidity. It is more useful to read it as dynamics — because dynamics can be measured, predicted, and engineered around, and “stupidity” cannot.

The iterated map

At every level of granularity — token generation, a tool-use turn, a whole agentic session — a model-driven system is a fixed-point iteration:

Xn+1=F(Xn)X_{n+1} = F(X_n)

where XnX_n is the state (context, prompt, in-flight code, conversation history) and FF is one model step plus the state update it causes. Whether the iteration converges, cycles, or drifts is governed by the contraction properties of FF — and FF is not unconditionally contractive. It contracts on some inputs, in some regimes, and not in others.

Several things everyone has observed empirically fall out of this framing:

Termination is learned, not intrinsic. Frontier models rarely run away into unbounded generation — but that is a trained stopping policy, not a property of the underlying dynamics, which reassert themselves the moment the policy is weakened.

Cycles persist one level up. The fix–fail–fix loop is a limit cycle in the agentic iteration. Mid-tier models exhibit them constantly on code tasks; frontier models converge in a handful of iterations. The difference is not “smarter” in the abstract — it is a more contractive FF on that input class, which lands in a fixed point faster and more reliably.

The object of study is the basin, not the output. Since Xn+1=F(Xn)X_{n+1} = F(X_n) can express arbitrary computation, any model behavior is some trajectory settling somewhere. The informative question about a task is not “what did the model output” but “what is the basin of attraction of FF for this input, and how fast does it contract.”

A symbolic algorithm, in this language, is the degenerate case: a single fixed point, one fully analyzed trajectory, contraction engineered to be guaranteed — which is where reproducibility and verifiability come from. A model is the general case: many fixed points, irregular basins, perturbation-sensitive trajectories, and no a-priori map from an input to which fixed point it reaches. Crucially, the fixed points include correct answers, confident errors, and partial-answer cycles. “The model can do task T” translates to: the correct-answer basin is large and strongly contractive over T’s input distribution. Nothing more.

An archipelago, not a continent

For models, the coordinate system that matters is in-distribution (InD) versus out-of-distribution (OoD/OOD), and the geography of it is the load-bearing fact: InD support is fragmented. The set of inputs a model handles well is not a continent with a coastline you can map — it is a constellation of islands scattered across an out-of-distribution sea. Basins nucleate around training examples; two problems that look adjacent to a human may sit on different islands; a small rephrasing can teleport a query from one island to another, or off support entirely.

The InD archipelago. Islands are cores of dense training support; the dot at each center is a fixed point. The basins tile the whole space — the dotted watersheds are their shared boundaries, where an arbitrarily small perturbation changes which fixed point the iteration reaches. Open water is still basin territory, just weakly contractive. The green diamond is a correct answer in open water: it lies inside a basin whose fixed point is elsewhere, so an unforced iteration initialized near it slides down to the island and converges confidently to the wrong answer.

Two familiar behaviors are this geography seen from different sides.

No gibberish. Model output is essentially never incoherent, even when it is wrong. The iteration almost always lands in some InD basin, producing structured, fluent, confident output — even when the correct answer lies far from every island. Confidence, in this picture, is a property of being in a basin, not of being in the correct one. This is worth internalizing, because every intuition humans have about confidence — trained on other humans, whose confidence loosely tracks competence — misfires on models.

Low inventiveness. The same property with the sign flipped. The model maximizes time spent inside islands and avoids open water, because open water is where contraction is lost and accuracy collapses. For the typical workload this is adaptive: you want the system that stays near what it knows. For invention — an answer that lies in no island — it is exactly wrong.

Steering, and forcing

If the well-handled region is an archipelago, then someone has to navigate between islands, and the model will not do it alone. This reframes what the human in the loop is actually doing.

The human’s function is not only evaluating output. It is steering: perturbing the state — a hint, an example, a partial implementation, an error message, an intermediate reframing — to relocate where the iteration lands. Mechanically: a change in XX changes the dynamics, which changes the landing basin. Every diagnostic, every failing test, every fragment of context is a candidate perturbation. “Prompting” describes this about as well as “typing” describes programming.

When the target genuinely lies in open water — a design no one has written, a bug class no one has documented — steering escalates to forcing. Left unforced, the iteration relaxes to the nearest known pattern and fails by producing a familiar-shaped answer to an unfamiliar problem; anyone who has asked a model for a novel architecture and received last year’s standard one has watched this happen. Holding an iteration out in the chaotic region, stepwise, against its own contraction toward familiarity, is an active, adversarial process. Invention is categorically harder than refinement — not because the model lacks some ingredient, but because its core dynamics actively pull away from it.

Three geometries, one fixed point

The agentic workspace is shared by three kinds of agent, with characteristically different basin geometry. Models: wide, smooth, irregular basins — fast settling in-distribution, drift and cycles out of it, boundaries unaligned with human intuition. Humans: narrow, sharp basins bounded by working memory and attention — precise on a few patterns at a time, fatigue-limited on long iterations. Symbolic algorithms: degenerate basins — one fixed point, zero drift, zero fatigue, but covering only the regions someone bothered to construct.

A naive system lets each iterate independently and forces translation at every boundary; the agents’ updates displace one another’s trajectories and the joint state drifts. A well-designed system makes the three geometries interlock: the model’s contraction lands inside the region the symbolic checker can verify; the checker’s verdict lands in a representation the human can audit; the human’s judgment lands in a structured form the model can attend to. The composed map is more contractive than any component alone — which is the whole point of having three agents rather than one.

NAIVE — EACH AGENT ITERATES ALONE INTERLOCKED — ONE COMPOSED MAP three private fixed points — the joint state cycles model symbolic human one fixed point one shared fixed point — the composed map contracts
Three basin geometries in one workspace. Left, the naive composition: the model's wide soft basin, the human's narrow sharp one, and the checker's hard-edged domain each pull the joint state toward a private fixed point — every hand-off is a translation, and the trajectory orbits without settling. Right, the interlocked composition: the model's step lands inside the checker's domain, the checker's verdict lands inside the human's basin, the human's nudge re-enters the model's contraction — and the composed map converges to a single shared fixed point. Solid segments are deterministic snaps; dashed segments are stochastic steps.

This gives sharper definitions to some everyday objects. A diagnostic is a boundary object — one anchor that has to sit inside the basins of all three agents simultaneously, which is why “human error message” versus “machine payload” is a false choice. A tool is well-chosen when its output lands in the basin of every downstream reader, not just its producer. And a system failure is a basin-separation event — the agents stop co-converging — which is a more actionable description than “hallucination.”

The practical consequences for a platform: perturbation should be a first-class, cheap, structured operation, not an afterthought chat box. The provenance of an output — which training neighborhood it came from — is high-value and currently unobservable; that is an open tooling opportunity. And there should be an explicit way to say “this request is non-modal; suppress the regression to the nearest pattern.”

Above all: convergence rate is a platform variable, not just a model variable. The same model inside a better feedback loop converges in fewer iterations. That lever belongs to whoever builds the tools.

So far the model has been a dynamical system — trajectories, basins, landings. There is a complementary static question: what is actually stored in a trained model, and which operations does that store support? The answer explains the most expensive failure mode in AI-written code, and it is the subject of the next part.

Next: The Model Cannot List What It Knows — point queries, missing inverses, and why self-review cannot find what generation omitted.