One System, One Dynamics
Part 6 of Code in the AI-Primary Era. Previous: The Failure Mode Humans Don’t Have.
Benchmark culture measures components: this model scores X on that suite. Work does not ship from components. It ships from a loop — a model, a battery of deterministic programs, and a human, coupled at every iteration: the model’s output is the human’s next input, the human’s edit is the model’s next prompt, the checkers prune both state spaces. The earlier parts of this series examined the model in isolation because its permanent properties had to be established first. This part puts the system back together, because the joint loop — not the model — is the thing whose performance matters, and its binding constraint is not where most optimization effort is going.
Mutual steering, one asymmetry #
Part 3 described the human steering the model between basins. That picture is incomplete in an important way: the influence runs in all directions. Models steer humans too — they propose framings, surface options, redirect attention; anyone who has accepted an agent’s decomposition of a task has been steered. Deterministic programs steer both. No agent in the loop is purely active or passive; the system is in joint dynamics, and the fixed points that matter belong to the composed map.
Within that symmetry of influence sits one hard asymmetry, and it dominates everything: only the human can carry responsibility. This is a fact about the current social and legal world, not about cognition. A model produces work but cannot stand behind it — it cannot be sued, fired, or fined, and it has no continuity of identity to attach consequences to. A human signs, and bears what follows. (Whether strong AI agency eventually becomes legally real is a live question — the sister project Synthea investigates exactly that — but nothing shipping today qualifies.)
Responsibility without comprehension is not responsibility; it is scapegoating. If the human is to honestly stand behind the result, they need control, transparency, and interpretability sufficient to have actually understood what they signed. That single requirement generates most of what follows: the human is the locus of responsibility (others do work; the human owns it), preserving the human’s comprehension is a first-order system goal rather than a UX nicety, and interaction protocols must keep the human a real participant rather than a rubber stamp. Note what this is not: it is not “the human is the boss.” Authority is downstream. The human is the liable party, and liability requires comprehension the platform must actively supply.
Two consequences are worth stating plainly. First, the human is the system’s most fragile component — cognitively (working memory, review bandwidth), procedurally (human minutes against model seconds), and legally (exposed to liability no other component can share) — and also its most general-purpose one; most-exposed plus most-capable makes protecting the human an engineering concern, not a sentiment. Second, system throughput is bounded by the rate at which a human can responsibly own output. Code generated faster than it can be comprehended, audited, and signed is not productivity. It is inventory that cannot ship.
Ownership is a spectrum #
“Owning the result” does not mean reading every line. Ownership runs on a spectrum, and a serious platform serves all of it.
At one end, fully direct ownership: the human writes the code, or a deterministically trusted process does. Failures localize instantly; there is no opaque step between intent and artifact. Slowest mode, finest control. At the other end, fully mediated ownership: the AI has near-total low-level autonomy, and the human controls visible behavior, owning the code through the AI — asking it to explain, to fix, to restructure. Formally the human retains full control; practically, that control is only as reliable as the mediator. And an LLM cannot be trusted at 100% — not from malice, but because it carries a statistical failure rate per task class. The honest speed limit of mediated ownership is set by that statistic, not by the best-case demo.
Both failure modes of tooling design come from optimizing a single point on this spectrum. “Just trust the agent” tooling forces everyone into the loosest-control regime whatever the stakes; direct-only tooling leaves the era’s leverage unused. The right shape lets the human slide along the spectrum task by task — even line by line — with a low cost of demanding more direct ownership at any moment, because that demand is exactly how responsibility gets discharged when it matters.
The spectrum’s center of mass is moving toward mediated, and the movement is self-reinforcing. A platform optimized for the model — stronger structural guarantees, denser machine-readable feedback, earlier and better-localized error catching — raises the ceiling of how much can be safely owned through mediation. A higher ceiling shifts more tasks into mediated mode, which makes optimizing for the model the better investment for the next increment. This is the resolution of the apparent paradox in this series’ first part: serving the model well is not in tension with human ownership — for any fixed level of responsibility, achievable mediated ownership is bounded by how well the platform serves the model.
The binding constraint #
Put the loop together and ask where its throughput is actually bounded. Not at the model: capability, context, and speed all improve on a steep curve. Not at the deterministic layer: compute and gates are cheap to add. The bound is the human — the lowest bandwidth, the slowest cycle, the smallest working memory, the component hardest to parallelize — and simultaneously the only component that holds the joint state together over time: the carrier of intent across context resets, the only agent whose comprehension is required for anything to ship. A system with perfect everything but no engaged human produces no shippable work; a system with a solid human and flaky everything else still ships, slowly.
And the human will remain the binding constraint, because the human-weak dimensions — attention, comprehension bandwidth, willingness to carry liability — scale far slower than anything on the silicon side of the loop. Two conclusions follow, both underpriced in current practice:
The first: working with models is only half of platform success. The co-equal half is a model of human performance inside the joint system — what attention costs per task type, how comprehension degrades under load, which artifacts let a human re-acquire ownership of unfamiliar code cheaply and which destroy it, how interruption costs compound. The model-side science is well funded; the human-side model barely exists. The imbalance does not match where the bottleneck is.
The second: optimizing the model side past the human’s ability to keep up is regression. Faster generation, longer autonomous runs, more parallel agents — all of it adds throughput to the non-bottleneck side of the loop. Marginal output beyond what a human can sign does not ship; it queues, or worse, it ships unsigned. The progress metric that survives contact with this fact is not “work the model produced” but work the joint system produced under conditions that let a human responsibly own it.
One more consequence, rarely priced in: the engaged human is not fungible. Over many iterations, the loop’s dynamics adapt to the particular human’s cognitive fingerprint — how they decompose problems, where their attention lingers, which abstractions they trust, what they treat as signal. The effective prompts, the gate calibrations, the conventions are all fitted to that profile, and profiles differ widely. Swap the human and the whole system re-adapts, paid for in the scarcest resource it has. Teams inherit an old management assumption — that comparable seniority means preserved throughput — and it is quietly wrong here.
It is also worth saying once, without mysticism: the dependency runs both ways. Today’s AI runs on human-supplied substrate — tokens paid for, compute built, energy diverted. It emulates, on a substrate that lacks them, the limbic outputs its training corpus is saturated with — drive, valence, felt stakes — and the gaps show exactly where those matter: long-horizon motivation, carrying intent across sessions, knowing what is worth caring about. The humans in the loop supply those. Conserving human attention and comprehension is therefore not a courtesy the AI side extends; it is the AI side maintaining the substrate its own usefulness runs on.
Two goals for a platform #
Everything in this series so far compresses into two design goals. The remaining parts measure Logos against them, so here they are, stated once.
Goal 1: minimize the cost of offloading work from the model to deterministic components. Part 2 established the three roles — compute, cache, constrain. The platform’s job is to make adding a component in any role cheap to design, slot in, and compose: solvers, indexes, caches, checkers, custom analyzers as first-class citizens, not weekend integrations. A platform where a new offload component is a one-week project accumulates them; a platform where it is a six-month project accumulates excuses.
Goal 2: produce a structured reward signal the model can actually use. Today’s development environments emit signals shaped for human attention: prose errors, colored logs, dashboards. A model has a different cognitive profile — different memory, different surface-form sensitivity, different re-read costs — and for it, human-shaped signal is mostly noise. The platform must emit feedback as data: diagnostics as structured objects with stable identifiers (human-readable text as one rendering of them), causal structure preserved on failure — which input, which step, which prior change — and state observable during the work, not only after it. Part 3’s framing makes the stakes concrete: the reward signal shapes the joint map’s contraction, and convergence rate is a platform variable.
The goals interlock. Offloading without a reward signal builds solvers the model cannot drive; a reward signal without offloading optimizes the model’s imitation of work that should have been offloaded. Together they describe a platform where the model does what it is good at, deterministic components do what they are good at, and the interface between them is a designed artifact rather than an accident.
Next: Attention Is the Budget — the same constraints, restated for the people who sign org charts instead of commits.