Notes on agent runtime design

In January 2026, we moved Oracle’s planner from generated XML to native tool calling.

It looked like an obvious modernization. Instead of writing a plan that another layer had to parse, the model could select a tool and submit structured arguments directly. One formatting convention disappeared, the intent of each call became clearer, and we expected fewer parsing failures and less unnecessary inference.

I kept returning to the same question: was this the standard shape of a modern agent? If the model supported tool calling natively, could we finally remove much of the planner machinery we had built around it?

The answer became clearer only after we finished the migration.

Tool calling is a better model interface. It is not an agent runtime.

From a format to a protocol

Asking a model to produce XML, JSON, or another structured text format is effectively inventing a temporary protocol through prompting. The model must remember the tags, fields, and nesting rules. The system then parses the text into an action it can execute.

This can work, but it concentrates complexity in the wrong place. A missing tag, an extra explanation, or an incomplete field can turn a sound decision into an unusable result. The system accumulates repairs, fallbacks, and parser branches to accommodate those variations.

Native tool calling creates a cleaner boundary. The system declares the available tools and their arguments. The model returns the call it selected. The model expresses intent; the protocol carries it.

That was the right change. It removed accidental complexity and made the division of responsibility between model and system easier to see.

Structured, however, does not mean valid. Arguments can still be empty. Fields can satisfy a schema while violating a real product constraint. The model can choose a syntactically perfect tool in the wrong task state. The protocol ensures that a decision can be expressed. It does not ensure that the decision deserves to be executed.

When the planner becomes a tool

We went further and made planning itself available as a tool.

The idea was natural. Planning should not happen only before the task begins. New material may overturn an assumption, an action may fail, and the user may change the requirement halfway through. An agent should be able to replan when the work demands it rather than remaining bound to its first proposal.

Once the planner becomes a tool, planning moves from a prescribed stage outside the system into a capability inside the agent loop. The model can decide when to decompose the goal, when to continue executing, and when the remaining work needs to be reorganized.

This made Oracle more flexible. It also raised another question immediately: who determines whether the planner is being called appropriately?

A model can keep planning without acting, or keep calling tools without finishing. One call may look reasonable while dozens of consecutive calls have already drifted away from the goal. Tool calling knows nothing about task budgets, stopping conditions, or which actions have already completed.

We had to establish safety limits around calls and distinguish when the task should continue, pause, retry, or stop. At that point, the problem was no longer whether the planner prompt was good enough. It was how a runtime should govern a sequence of model decisions.

The runtime begins where the tool call ends

A tool call is only a request for action. There is a considerable distance between that request and work that has actually been completed.

The runtime must answer questions such as:

• is this tool allowed in the current task state;
• do the arguments satisfy real product constraints rather than only a schema;
• which facts must be persisted before and after execution;
• after a timeout, should the system recover, retry, or admit that the result is unknown;
• when the same action is submitted twice, how are duplicate side effects prevented;
• when the task has drifted, exhausted its budget, or completed, what makes it stop?

None of these questions disappears because the model supports tools natively. The easier it becomes for a model to initiate action, the more important the runtime boundary becomes.

Tool calling makes decisions cheap. The runtime determines whether a cheap decision becomes an expensive mistake.

A follow-up is not another appended message

At the same time, we were working on follow-up requests issued from the middle of an Oracle task.

In ordinary chat, a follow-up is usually another message appended to the transcript. An agent may already have completed part of the work, created a plan, and produced files and intermediate results. When a user continues from one of those points, they may be correcting the original task or opening a new branch on top of what already exists.

Passing the entire history into the new task can repeat work that has already finished and preserve constraints that are no longer valid. Creating a completely independent task causes the system to forget the useful results it already earned.

We began representing the relationship between parent and child work explicitly. A new branch could have its own lifecycle while inheriting only the completed actions, materials, and requirements that remained relevant.

This changed how I understood context engineering. It is not merely the practice of placing more conversation into a prompt. It decides which parts of the work remain valid, which should be inherited, and which state branch owns the new judgment.

Context is not a block of text. It is the lineage of the work.

Context caching is not memory either

We were also studying context caching to reduce latency and token use across repeated planner runs.

Caching is valuable. A long shared prefix may not need to be processed from the beginning every time, making repeated inference faster and cheaper. But a cache preserves computation over an input. It does not preserve the system’s understanding of a task.

It cannot know that an artifact is obsolete, determine whether a follow-up belongs to an old task or a new branch, or decide which state is the current truth. Hitting a cache for more history only repeats that history more efficiently.

Memory asks what should be remembered. Context asks what should be used now. Caching asks whether repeated computation can cost less. They are related, but they are not the same layer of capability.

Treating caching as memory can produce lower latency while preserving the same confused state.

Native tool calling still mattered

None of these problems means the migration was unimportant.

Native tool calling gave the model and the system a cleaner interface. The planner no longer had to disguise executable structure through formatting tricks, and tools could have explicit arguments and results. It let us move our attention away from parsers and toward the runtime problems that actually mattered.

Its greatest value was not that it turned Oracle into an agent. It revealed what the agent was still missing.

Once formatting problems were removed, what remained was state authority, execution lifecycle, context inheritance, recovery, budgets, and stopping conditions. Those are runtime responsibilities.

An agent is not a model that supports tools

A model supporting tool calling means it can propose structured action. An agent system must also keep those actions coherent over time and remain accountable for their consequences.

The model makes judgments while the runtime preserves facts. The model can choose a next action while the runtime must know whether the previous one actually happened. The model can replan while the runtime must ensure that a new plan does not accidentally duplicate the old world.

A modern agent is not defined by a shorter prompt, a more native tool call, or the ability to invoke more tools in sequence. It is modern when responsibilities are placed in the right layer:

The model proposes decisions. The runtime turns them into reliable action.

Tool calling answers, “What does the model want to do?”

The agent runtime answers, “How does it actually happen, and after it happens, how does the system still know where it is?”