What is the difference between model and harness?

Summary slides

Why conflating the model and the harness leads teams astray

A common, practical mistake is attributing an agent’s observed failures entirely to the underlying model, when a considerable share of production agent failures trace back to the harness, a tool description that’s ambiguous, a context-management strategy that’s dropped relevant information, an execution loop with no effective retry logic, and a team that reflexively reaches for a bigger or newer model every time an agent underperforms, without first examining whether the harness itself is the actual bottleneck, wastes considerable effort chasing model upgrades that a fixed harness issue would have addressed for a fraction of the cost.

This conflation runs in the other direction too, a team that credits a strong, capable model for an agent’s overall reliability sometimes fails to notice how much of that reliability depends on the harness’s careful engineering, and when that same, capable model is later deployed inside a different, more thinly built harness, the resulting agent’s performance can be considerably worse, revealing that the original system’s strength was never purely the model in isolation but the combination of that model and the harness built around it.

What the model contributes to an agent’s behavior

The model’s direct contribution is its raw reasoning and generation capability, given a context, how well it can infer what’s being asked, how reliably it can follow complex, multi-step instructions, how accurately it can reason about a novel situation it hasn’t seen an exact analog of before, and these intrinsic capabilities set a practical ceiling on what any harness, however well engineered, can get that model to reliably do, since no amount of clever scaffolding turns a weaker model into one that reasons as reliably as a stronger one.

But this ceiling is considerably higher, in practice, than what a poorly built harness lets a team reach, a capable model fed an incomplete, badly organized context, or tool descriptions that don’t clarify when to use which tool, performs considerably below its intrinsic ceiling, and recognizing that the model sets an upper bound while the harness determines how close an actual, deployed agent gets to that bound is the practical distinction this discussion is built around.

What the harness contributes that the model alone cannot provide

The harness supplies everything the raw model, on its own, lacks persistence across multiple, separate calls, since a model call is inherently stateless and has no actual memory of a prior, separate invocation without the harness reconstructing and re-supplying that context; access to the outside world, since a model can’t query a live database or send an actual message without the harness providing that concrete tool interface; and structural governance, since a model has no actual, built-in concept of which of its proposed actions should require human approval before proceeding, a necessary capability the control loop discussion covered elsewhere in this collection describes as belonging entirely to the harness.

Recognizing this division of responsibility clarifies where a team should invest when building a new agent, an underperforming agent whose model is already near the top of what’s available needs harness investment, better context curation, clearer tool interfaces, more effective error handling, since there’s often no meaningfully stronger model available to upgrade to, while a harness that’s already well built but still produces unreliable reasoning may need a stronger, underlying model, since no amount of additional scaffolding compensates for a model that can’t reason reliably about the task at hand.

How the boundary between model and harness has shifted over time

Earlier generations of language models needed considerably more harness scaffolding to behave reliably, explicit few-shot examples embedded directly in the prompt, heavily engineered prompt templates that walked the model through each, individual reasoning step, because the underlying model’s intrinsic ability to follow complex instructions without that scaffolding was more limited, while more recent models have absorbed considerably more of that capability directly into the model itself, following complex, multi-step instructions and reasoning through novel problems with meaningfully less explicit harness scaffolding than earlier models required.

This historical shift doesn’t mean the harness has become less important, it means the work the harness does has shifted, away from painstakingly, explicitly walking a weaker model through each, individual reasoning step, and toward the different work of tool integration, context management at scale, and structural governance, the categories of capability this discussion has described as belonging to the harness regardless of how capable the underlying model itself becomes, since no future model, however capable, will ever intrinsically know how to call an organization’s internal API without a harness providing that concrete interface.

How a team diagnoses whether a failure is a model problem or a harness problem

Given an observed agent failure, the practical, diagnostic question is whether the model, given the actual context and tools it was provided, could plausibly have reasoned its way to a correct outcome, and if the answer is yes, the model had everything it needed and still got it wrong, the failure points toward the model’s intrinsic limitations, while if the answer is no, the model was missing relevant context, or was given an ambiguous tool description, or had no mechanism for correcting course after an early mistake, the failure points toward the harness.

Running this diagnostic well often means a team reconstructing, as precisely as possible, exactly what context and tools the model had access to at the failing step, since a common, practical mistake is assuming the model saw and understood everything a human reviewer, looking at the full situation from the outside, can plainly see, when the model’s actual context at that moment was considerably narrower or more ambiguous than the full picture a human reviewer naturally assumes it had.

How context engineering sits at the actual boundary between model and harness

Context engineering, the discipline of deciding what information gets fed into the model at a step, sits precisely at the practical boundary between these two layers, the harness is entirely responsible for constructing that context, retrieving the relevant information, formatting it clearly, deciding what to include and what to leave out, but the model is entirely responsible for what it does with that context once it’s received it, and a well-engineered harness that constructs excellent context still depends on the underlying model to reason well over that context, meaning even the best possible harness work can’t fully substitute for underlying model capability.

This interdependency is why context engineering, covered in its dedicated discussion elsewhere in this collection, is harness work rather than model work, even though its entire purpose is to get the best possible performance out of the model, and understanding this distinction, that the harness prepares the ground but the model does the actual reasoning on top of it, is what keeps a team from either under-investing in context quality because “the model should just figure it out,” or over-investing in ever more elaborate context engineering when the underlying bottleneck was the model’s intrinsic reasoning limitations all along.

How model and harness upgrades need to be evaluated separately

Because an agent’s observed performance is a product of both layers together, a team evaluating a proposed change, whether to upgrade the underlying model or to revise the harness, benefits from isolating which layer a change is touching, and measuring that change’s effect independently rather than only observing the agent’s aggregate, end-to-end performance before and after, since an aggregate improvement after swapping in a newer model might be driven partly by an unrelated, simultaneous harness change a team made around the same, real time, muddying what caused the observed improvement.

Building this kind of isolated evaluation discipline connects directly to the broader discussion of LLM evaluation covered elsewhere in this collection, but applied specifically here to the practical question of attribution running a harness change against a fixed, unchanged model, or running a model upgrade against a fixed, unchanged harness, is what lets a team understand which layer drove an observed improvement, rather than bundling both changes together and having to guess, after the fact, which one mattered.

How the model-harness distinction shapes vendor and procurement decisions

A team evaluating a third-party agent product benefits directly from asking which parts of that product’s quality come from its underlying model versus its proprietary harness, since a vendor whose competitive advantage lies primarily in harness engineering excellent tool integrations well-tuned context management, offers a different kind of value than a vendor whose product is largely a thin wrapper around a publicly available, underlying model with comparatively little, additional harness investment on top.

Understanding this distinction helps a team accurately assess what it’s paying for, and what happens to that value if the underlying model a vendor’s harness was built around becomes outdated or is later replaced, a vendor with strong, independent harness engineering can often migrate to a newer, underlying model while preserving most of its value, while a vendor whose value was mostly borrowed from an underlying model’s raw capability offers considerably less durable value once a comparably capable model becomes generally available elsewhere.

How the model-harness distinction applies when building an agent in-house

A team building its agent in-house faces this same distinction as a concrete resource-allocation decision, given a fixed engineering budget, how much of that budget should go toward experimenting with different, underlying models versus how much should go toward harness engineering, better tool interfaces, more careful context management, more effective control-loop policy, and the correct answer depends on where that team’s current agent sits relative to its ceiling, a team whose harness is already well built gets more value from model experimentation, while a team whose harness is still immature gets more value from harness investment regardless of which model currently sits underneath it.

Making this allocation decision well means a team running the diagnostic described earlier in this discussion across its accumulated set of observed failures, and if that accumulated evidence shows failures clustering around missing context, ambiguous tools, or absent governance, harness investment is where the marginal return lies, while if failures cluster around the model failing to reason correctly even when given everything it needed, that’s the concrete signal that model investment is where a team’s next engineering effort should go.

How the model-harness distinction shapes an agent’s cost structure

A model swap and a harness change carry different cost implications, upgrading to a more capable, underlying model typically increases a per-call inference cost directly and predictably, while harness investment typically carries an upfront engineering cost that once built, doesn’t necessarily increase the ongoing, per-call cost of running the agent at all, and a team weighing these two paths toward improving an agent’s performance benefits from comparing these different cost profiles rather than treating “improve the agent” as a single, undifferentiated decision with one, uniform cost shape.

This distinction often tips the practical calculation toward harness investment for an agent running at high volume, since a harness improvement’s cost is largely fixed and one-time, while a model upgrade’s cost compounds with every, additional call the agent makes, meaning the same harness fix that looked marginally cheaper at low volume can become considerably, more decisively cheaper once an agent’s call volume scales up.

How the model-harness distinction applies to debugging a regression

When an agent that was working reliably starts producing worse outputs, the model-harness distinction gives a team its first diagnostic branch, did the underlying model change, a silent provider-side update, an automatic version bump, or did the harness change, a recent edit to a tool description, a context-window adjustment, a modified prompt template, and answering this first branching question narrows the investigation considerably before a team ever has to dig into the individual failure cases themselves.

Teams that skip this first branching question often waste considerable, real time investigating individual failures in isolation, when a simple check of what changed, on either layer, around the time the regression began would have pointed directly at the root cause, and building this first-branch discipline into an agent team’s standard incident-response process is what turns a potentially, hours-long investigation into a considerably faster, more targeted one.

How the model-harness distinction clarifies what “prompt engineering” is really doing

Prompt engineering, the practice of crafting the instructions and framing to a model, sits within the harness layer, it’s the harness’s mechanism for shaping how the model approaches a task, and recognizing this placement clarifies a common, practical confusion, a team that treats prompt engineering as a mysterious, model-side art rather than a concrete harness-engineering discipline tends to iterate on prompts through trial and error, rather than applying the same diagnostic rigor this discussion has described for harness work more broadly.

Treating prompt engineering as structurally part of the harness means testing prompt changes the same way a team would test any other, harness-level change, isolated from simultaneous model changes, measured against a concrete evaluation set, rather than treating a prompt as a soft, informal artifact that gets tweaked casually whenever an agent’s output happens to look wrong on a particular day.

How the model-harness distinction shapes how a team benchmarks agent performance

A benchmark that reports only an agent’s aggregate, end-to-end success rate conflates the two layers this discussion has described, offering no way to tell whether a low, observed score reflects model-level reasoning limitations or harness-level engineering gaps, and a team relying purely on this kind of aggregate benchmark to compare two, different agent products, or two, different versions of its agent, risks drawing a wrong conclusion about which layer needs improvement.

A considerably more useful benchmarking approach separates these two layers directly, testing the underlying model in isolation against a fixed, standardized harness, and separately testing a harness’s quality by holding the model fixed and varying only the harness, and this layered benchmarking discipline, while more work to set up than a single, aggregate score, gives a team the actionable clarity this discussion has argued matters directly for where engineering investment should go next.

How the model-harness distinction plays out differently across agent maturity stages

An early-stage, experimental agent typically benefits most from actual model exploration, trying several, different underlying models against a comparatively simple, minimal harness to establish which model’s intrinsic reasoning capability best fits the task at hand, since building elaborate harness infrastructure around a model that later turns out to be a poor fit wastes considerable engineering effort on scaffolding a team may end up discarding anyway.

Once a team has settled on an underlying model that reasons well enough for the task, the marginal value shifts decisively toward harness investment, context curation, tool-interface clarity, control-loop policy, since further model experimentation at that later stage tends to produce diminishing returns compared to the same engineering effort spent tightening the harness around a model whose intrinsic capability has already been validated as sufficient for the task.

How the model-harness distinction affects how teams should think about future-proofing an agent

Because underlying models continue to improve over real time, a team building an agent today benefits from designing its harness in a way that stays decoupled from any single model, avoiding harness logic that depends on a particular model’s idiosyncratic quirks, so that a future model upgrade can happen with minimal harness rework rather than forcing a team to rebuild significant portions of its scaffolding every time a newer, underlying model becomes available.

This decoupling discipline is what lets a team capture the ongoing benefit of model improvements over time without having to treat each, individual model upgrade as its disruptive harness migration project, and a team that instead builds its harness tightly, specifically around one, particular model’s current quirks finds itself facing a harder, more expensive upgrade path every single time a newer, more capable model becomes available to adopt.

How the model-harness distinction helps a team communicate agent limitations honestly

When a stakeholder asks why an agent occasionally produces a wrong answer, a team equipped with the model-harness distinction can give a considerably more precise, honest answer than a vague appeal to “AI isn’t perfect,” explaining whether a class of failure reflects a known limitation of the underlying model’s reasoning, or a fixable gap in the harness that a planned engineering change will address, and this precision matters directly for setting accurate expectations about what’s likely to improve and on what, realistic timeline.

A stakeholder told that a failure is a harness gap can reasonably expect that gap to close once the relevant engineering work is done, while a stakeholder told that a failure reflects a current model limitation understands that meaningful improvement may depend on the broader, external pace of model development rather than anything the team’s immediate engineering effort can directly control, and this honest distinction, communicated clearly, builds considerably more durable, warranted trust than treating every, individual limitation as an equally vague, equally uncertain “AI thing.”

Common mistakes teams make around the model-harness distinction

Several patterns recur often enough across teams building and evaluating agents that naming them directly is worth doing before they lead to misallocated engineering effort.

1. Attributing every agent failure to the underlying model without first checking whether the harness itself was the bottleneck.

2. Crediting a strong model for an agent’s reliability without recognizing how much of that reliability depends on the harness built around it.

3. Assuming the model saw and understood the full situation a human reviewer sees, rather than reconstructing what context the model had at the failing step.

4. Bundling a model upgrade and a harness change together, making it impossible to attribute an observed improvement to either layer.

5. Treating context engineering as something the model should simply handle on its own, under-investing in the harness work that prepares that context.

6. Over-investing in ever more elaborate context engineering when the underlying bottleneck was the model’s intrinsic reasoning limitations.

7. Evaluating a vendor’s agent product without asking how much of its value comes from harness engineering versus a borrowed, underlying model.

8. Allocating engineering budget toward model experimentation for a team whose harness is still immature, missing the larger return harness investment would have provided.

9. Treating prompt engineering as a mysterious, model-side art rather than a concrete harness-engineering discipline that deserves the same, rigorous testing.

10. Relying on a single, aggregate benchmark score that conflates model-level and harness-level quality, obscuring which layer needs improvement.

11. Building elaborate harness infrastructure around a model before that model’s intrinsic fit for the task has been validated.

12. Continuing to experiment with different, underlying models well past the point where harness investment would have produced considerably greater returns.

13. Building harness logic tightly, specifically around one model’s current quirks, making every future model upgrade a disruptive, expensive migration.

What connects all thirteen of these mistakes is a single underlying pattern: collapsing two distinct layers, the model’s intrinsic reasoning capability and the harness’s engineering quality, into a single, undifferentiated notion of “how good the agent is,” rather than diagnosing which layer is responsible for an observed strength or an observed failure.

The deeper principle underneath all of this is that an agent’s deployed quality is always the product of these two separate layers working together, and a team that learns to tell them apart, diagnosing failures correctly, evaluating upgrades in isolation, allocating engineering investment toward whichever layer holds the current bottleneck, builds meaningfully better agents than a team that treats “the model” as a single, undifferentiated stand-in for everything an agent is.