A ladder for coding agents

Sat Jul 18, 2026 · 932 words

---

Most AI coding setups choose a model before the task produces any evidence.

A router takes a prompt and selects a model. For a straightforward task, it can choose something cheap. For a difficult task, it can send the work to an expensive model. That is better than sending every task to the same expensive model.

The choice still happens too soon. A task that looks routine can need a careful repair. Another task can look intimidating but collapse into one focused test failure. The useful signal arrives after the first attempt.

I built the Pareto ladder around that fact. It starts with a cheaper worker model, checks the work, and escalates after evaluation rejects the attempt. The model list sets the order. The loop decides whether to keep climbing.

A Pareto frontier compares model performance at each price point. It keeps the models that are top performers for their cost: no other available model costs no more and is at least as capable. The frontier becomes an ordered list, from cheaper workers to more capable workers with higher expected cost.

A worker is not a verdict

The ladder takes four inputs:

• a task
• an ordered list of worker models
• a disposable workspace
• an evaluator

The evaluator is pluggable. It can run an exact test command, call a model judge, or combine both. A worker can say “done.” The evaluator decides whether the run accepts the attempt.

Suppose the first worker changes a parser and runs the relevant regression. The regression still fails. The ladder records the worker output, the evaluator result, and a snapshot of the change. It then discards the change and restores the workspace to the baseline before it calls the next model.

The next worker gets a short account of what happened: which model tried, what it changed, and why the evaluator rejected it. It does not inherit a half-correct patch with unknown side effects.

What changes from a generic harness

A generic coding harness gives one model a long session. The model reads files, edits the workspace, runs tests, sees failures, and keeps going. That design gives the model continuity and time to build a detailed picture of the task.

The Pareto ladder changes the unit of work. A rung is a bounded experiment. One worker gets a shot. An evaluator decides whether the result is acceptable. On rejection, the ladder preserves the evidence, returns to a known state, and changes the worker.

That lets the system spend more money when the evidence calls for it. A cheap worker that passes the evaluator ends the run. The task does not need a more capable worker.

It also makes failure useful. Cheap models can surface a useful error. A rejected patch can show the next worker which assumption was wrong, which test still fails, or which part of the task was misunderstood. The goal is not to treat the cheap model as disposable. The goal is to turn a rejected attempt into a bounded artifact.

The caller supplies the model order. In the implementation, the ladder rejects duplicate model IDs and records each attempt. After a rejected attempt, it snapshots the change, resets the workspace, and passes the previous worker result and evaluator learning into the next attempt. It returns separate outcomes for success, exhaustion, worker failure, and evaluator failure.

Those outcomes need to stay separate. A task that exhausted the ladder did not pass. A task where the evaluator crashed did not pass either. If you later make a policy decision from the run, those distinctions matter.

The evaluator does the hard part

A model router can select a worker. It cannot tell you whether the worker solved the task.

The ladder can use a deterministic check as the evaluator. For a code repair, that might be the exact regression that describes the bug. It can also use a model judge when the task needs judgment that a test cannot express. A practical setup can require both: the test passes, and the judge finds the change consistent with the task.

Cost comparison

To see whether the harness actually saved on cost, I built a small benchmark of 13 coding-repair tasks and compared the ladder against fixed-model policies, including several frontier models. The result was more pronounced than I expected: depending on the fixed-model comparison, the ladder cut run cost by about 87% to 98%.

The full run used 13 failing repair fixtures. Every task-policy cell had the same prompt, bounded tools, fresh workspace, and focused node --test regression. Fixed models got one continuous session of up to 72 response/tool turns per task. The frozen Pareto configuration used nine rungs with up to eight turns each.

In that run, the Pareto ladder resolved all 13 fixtures for $0.106386 in provider-reported cost. A fixed GPT-5.6 Luna run resolved 12 for $0.797856. Grok 4.5 also resolved all 13, for $1.368134. Claude Opus 4.6 resolved all 13 for $5.971770. Qwen3 Coder resolved six for $1.164105.

Those numbers are not a model leaderboard. They come from one run on fixtures I own. They show that a resettable escalation loop can match a stronger fixed-worker result on this task set without paying for the strongest worker on every task.

The full report includes the task rows and provider-cost accounting.

Try it

Pareto Curve Agent Harness is a TypeScript library. If you already have a coding-agent harness, you can add the ladder around the worker you have: supply an ordered worker-model list, an evaluator, and a workspace adapter. The repository includes the implementation, examples, and the owned fixture suite used for the cost run.