From Model Calls to Agent Systems

Since we began building Oracle in 2025, I have repeatedly encountered the same mismatch: the model can make a strong judgment while the product still cannot tell whether the work actually happened.

The first problem was turning a response into a task that could keep advancing. Then tasks began to outlive requests, which introduced waiting, interruption, recovery, and delivery. Media generation later placed timeouts, asynchronous callbacks, files, and settlement inside one lifecycle. Model fallback showed that a request can succeed for the user while the infrastructure underneath it is already degrading.

These problems did not arrive together as a finished architecture diagram. Each appeared first as a local loss of certainty: state disagreed, an action could not be confirmed, stale context changed a decision, or the model declared completion without evidence. Only in retrospect did the missing system between model calls and reliable execution become visible.

This essay is therefore a provisional system map. It places two objects in the same frame:

A model call produces an output or proposes an action from the current input.
An agent system advances a goal over time while constraining actions, preserving recoverable state, and verifying results.

The earlier essays record where these boundaries surfaced. This one asks how they fit together into an execution system that can be diagnosed, implemented, and maintained.

The boundary of a model call

The simplest LLM product resembles an approximate pure function:

output = model(instructions, user_input, context)

The input is assembled before the call and the output returns when the call ends. The model does not need to account for what happened in the external world or resume the same work several minutes later.

This shape is excellent for bounded tasks: rewriting text, explaining a concept, extracting fields from fixed material, or drafting something a human will continue. The success condition mostly lives in the output itself, and a person can judge it immediately.

Many real goals, however, are not sentences waiting to be completed.

“Fix this production issue” implies locating code, understanding a data flow, forming hypotheses, editing files, running tests, and checking for regressions. “Research a market and produce a usable recommendation” requires retrieval, comparison, evidence, gap detection, and a changing direction. When the model first sees such a goal, it usually does not possess enough information to generate the final result.

The unit of the system is no longer a response. It is an unfinished task.

When an output becomes a next action

With tools, a model can produce an action rather than an answer: read a file, search code, execute a command, call an API, or ask for more material. The result returns to the context and the model chooses again.

Goal
  ↓
Model → Action → Environment
  ↑                 ↓
  └──── Observation ┘

Written as a state transition, each step roughly becomes:

(state_t, observation_t) → decision_t
decision_t → action_t
(state_t, action_t, result_t) → state_t+1

This is the smallest useful shape of an agent loop. Instead of guessing the final result in one pass, the model acquires the information it needs through action.

ReAct demonstrated a pattern that interleaves reasoning traces and task-specific actions: reasoning updates plans and handles exceptions, while actions interact with an environment. It helps explain why “reason, act, observe, continue” is better suited to open-ended work than one-shot answering.

But ReAct is a reasoning-and-acting pattern, not a complete execution system. It does not automatically decide whether a tool is permitted, who persists state, how a timeout is recovered, or who proves that the goal is complete.

A loop lets the model continue. It does not yet make the system reliable.

Where a naive loop fails

The simplest agent implementation is often a while loop: execute tool calls as long as the model emits them, append each result to the message history, and stop when the model returns final text.

This can produce a compelling demo. It also hands the same probabilistic system the most dangerous decisions: what to do, how long to do it, which results to trust, and when the work is done.

Several failures follow.

Context grows without control

Tool outputs accumulate. One verbose log, a large file, or repeated search results can displace the goal and its constraints. The history becomes more complete while the model’s effective view of the current state becomes less clear.

Side effects cannot be casually replayed

A failed read can be retried. Sending an email, charging an account, or creating a remote resource cannot. After a timeout the system may only know that it did not receive a response, not whether the action happened. Repeating the call may duplicate the side effect.

State and conversation become indistinguishable

The message history can contain an old plan, new requirements, failed attempts, and obsolete conclusions at once. Without a separate representation of current facts, the model must infer where the task is from prose.

Errors become ordinary text

permission denied, timeout, and validation failed require different responses. If all three are merely strings in the next message, a model can easily treat a structural failure as a reason to try the same action again.

Completion is only a sentence

A model can say “fixed” without a passing test, produce a path to a file that does not exist, or state a conclusion without evidence that covers the goal. Generating a terminal utterance and proving completion are different capabilities.

Permission means hiding a tool from the prompt

Tool visibility is not a security boundary. External content can carry prompt injection, and tool arguments can exceed their intended scope even when the model follows most instructions. Executable capabilities require deterministic policy enforcement, not a hope that the model will always judge correctly.

These failures share a cause. They are not simply evidence that the model is insufficiently intelligent. They show that the system has not assigned clear responsibility for action.

What I mean by an agent harness

Harness does not yet have a single industry definition. It can refer to the execution framework around a coding agent, a testing facility, or a narrower orchestration layer.

I use the term with an explicit definition:

An agent harness is the execution system built around a model. It supplies selected context, tools, state, and feedback; constrains actions; preserves facts; and turns continued generation into work that is controllable, recoverable, and verifiable.

The harness is not another module beside runtime, context, or evaluation. It is the system boundary that contains those responsibilities.

Agent Harness
├── Runtime & lifecycle
├── Context construction
├── Task state
├── Memory
├── Tools & permissions
├── Verification
└── Observability & evaluation

This definition deliberately does not treat reasoning as an ordinary harness module. Reasoning primarily comes from the model and its decision policy. The harness can provide a plan representation, constraints, feedback, and a compute budget; it should not pretend to contain a second independent intelligence.

Responsibilities of the harness

Runtime: controlling the life of a task

A runtime drives the loop, but it does more than call the model repeatedly. It maintains a task lifecycle such as:

created → running → waiting → running → verifying → completed
                    ↘ failed / cancelled / blocked

Every action needs an identity, start time, input summary, execution result, and explicit status. A timeout cannot always mean failure because an external action may have occurred. A retry cannot simply repeat the request; it must consider idempotency keys, execution records, or compensation.

The runtime also owns control the model should not hold alone: step limits, cost budgets, cancellation, concurrency, retryable error classes, and mandatory human approval points.

The model may propose continuing. The runtime decides whether the system is still entitled to continue.

Context: constructing the current working set

Context engineering is not the practice of placing everything known into a context window. It constructs the smallest sufficient working set for the next judgment.

An effective context can be abstracted as:

C_t = instructions
    + goal_and_constraints
    + current_task_state
    + relevant_evidence
    + selected_memory
    + recent_interaction

There is no “entire history” term. Retrieval, trimming, deduplication, compaction, and progressive disclosure all answer the same question: which information could change the next decision?

Context must therefore remain distinct from state. Context is the view given to the model now. State contains facts that must remain consistent whether or not the model currently sees them. The former can be compressed; the latter must not disappear during compression.

State: preserving facts about what happened

Task state needs to express at least the goal version, current phase, completed actions, unresolved questions, artifact references, and external side effects. It should not exist only as a prose summary.

For example, payment_requested, payment_succeeded, and payment_status_unknown are three different states. Compressing them into “payment attempted” destroys the basis for reliable continuation.

Good state does not require turning every judgment into a database column. It does require every fact that affects recovery, idempotency, or a user outcome to have an authority more stable than the conversation.

Memory: carrying knowledge that remains useful

Memory is not about saving more chat. It asks which knowledge is worth reusing later. Project conventions, stable preferences, verified debugging conclusions, and durable methods may belong in memory. A task’s temporary hypothesis, obsolete plan, and giant tool output usually do not.

Therefore:

Context asks what this step needs to see.
State records what is currently true for this task.
Memory preserves what remains valuable across steps or tasks.

Collapsing all three into a transcript gives a system many tokens without giving it clear facts.

Tools and permissions: turning access into constrained capability

A tool should have more than a name, description, and JSON schema. The system also needs to know its side-effect class, scope, timeout policy, retry semantics, output limit, and approval requirements.

Tool capability
├── input contract
├── execution boundary
├── side-effect class
├── permission policy
├── timeout / cancellation
├── retry / idempotency
└── typed result

Only then do read_file and delete_resource stop looking equivalent merely because both use the same tool-call protocol.

Permission enforcement must also happen at execution time. A model may explain why it needs additional authority; a user or deterministic policy grants it; the executor accepts only a validated capability. Prompts guide. Policies constrain.

Verification: turning completion into evidence

The stopping rule of an agent cannot be model_says_done == true.

Each task type should map success conditions to observable evidence wherever possible. A coding task may require targeted tests, no new type errors, and a diff consistent with scope. A data task may require invariant checks. An external write may require an independent read from the system of record.

Verification lives inside the execution loop and asks whether this task is actually complete. Evaluation operates more at the system level and asks whether a strategy remains reliable across a set of tasks. They are related but not interchangeable. A strong offline eval does not prove that this payment succeeded; one passing test does not establish broad system reliability.

Observability: making failure explainable

If the system saves only the final transcript, it is difficult to distinguish a poor tool choice, a tool failure, a permission denial, missing context, and an incorrect verifier.

Observability should at least correlate model decisions, tool attempts, policy decisions, state transitions, verification results, and human interventions.

The goal is not to record more private reasoning. It is to establish an accountability chain for action. A system can avoid storing hidden reasoning while still recording which visible evidence led to an action and what result the action produced.

A fuller system map

Placed together, the responsibilities form something closer to this:

                         ┌────────────────────┐
                         │  Goal / User input │
                         └─────────┬──────────┘
                                   ↓
┌────────────────────────────────────────────────────────┐
│                     Agent Harness                      │
│                                                        │
│  Policy & Permissions ─────┐                            │
│                            ↓                            │
│  State → Context Builder → Model → Proposed Action     │
│    ↑                           │            │           │
│    │                           └── stop? ───┤           │
│    │                                        ↓           │
│  Event Log ← Runtime ← Typed Result ← Tool Executor     │
│    ↑                                        │           │
│    └──────── Verification ← Evidence ───────┘           │
│                                                        │
│  Memory supplies selected knowledge; observability     │
│  records decisions, attempts, transitions and results. │
└───────────────────────────┬────────────────────────────┘
                            ↓
                Filesystem / APIs / Humans / World

There are really three loops operating at different speeds:

The action loop selects an action and receives an observation.
The task loop checks progress, replans, waits, resumes, and verifies the goal.
The evaluation loop finds failure patterns across many tasks and changes the model, context, policy, tools, or runtime.

Optimizing only the innermost loop often produces a system that calls tools fluently while remaining unable to deliver dependable results.

What a harness cannot solve

Engineering can reduce uncertainty. It cannot eliminate it.

When a goal is contradictory, the harness can only expose the conflict or request clarification. When the environment offers no observable signal, a verifier cannot invent one. When the model cannot understand the problem, another state machine will not manufacture the right judgment. When a task takes months to reveal whether its outcome was good, the system cannot replace real evaluation with a convenient immediate metric.

The value of a harness is not that an agent never fails. It gives failure a boundary:

uncertainty is not disguised as certainty;
retries do not casually duplicate side effects;
state survives interruption;
permissions cannot be bypassed by a prompt;
completion requires evidence, not confidence.

This is how I judge the maturity of an agent system. I do not ask how many tools it can call in sequence. I ask whether it knows what has happened, what remains unproven, and when a decision must return to a person.

Using the map to diagnose a system

When a real task fails, this map offers a set of locating questions before it offers another vocabulary:

Did the model make a judgment from the wrong evidence?
Did the context builder omit a constraint that would have changed the decision?
Did the runtime replay an action that could not safely repeat?
Did state authority conflict with the conversation, interface, or external system?
Did policy allow a side effect beyond the scope or budget?
Did the verifier accept a completion claim as the result?
Did evaluation measure activity without measuring movement toward the goal?

A stronger model may improve the first item. It does not automatically repair the remaining gaps in responsibility. Whether “harness” becomes the industry’s preferred term matters less than giving each system responsibility a clear home and leaving evidence when it fails.

This essay provides the map. Coding Agents as Execution Systems follows it through a bug that crosses a frontend, an API, and persistence.