The Boundaries of Agent Autonomy

Discussions about agents often collapse into a false choice: either the system follows a predefined workflow or the process is handed entirely to the agent.

The first option is safe but rigid. The second appears intelligent but uncontrolled. Products then look for an abstract setting between them: how autonomous should this agent be—30 percent, or 80 percent?

The question itself is inaccurate.

An agent may search an entire repository but modify no files. It may refactor code inside a sandbox but never push. It may choose an execution path without expanding the budget. It may draft a message for a user but never send one on the user’s behalf.

The agent is not “60 percent autonomous.” It holds some decision rights and lacks others.

The useful question is not:

How autonomous is the agent?

It is:

Under which conditions has this agent earned the right to make which decisions?

Autonomy is not another name for model capability or a product personality. It is dynamically allocated authority.

Capability is not authority

A model’s ability to perform an action establishes only technical possibility. Whether it may execute that action is a different system problem.

A model may be capable of generating SQL, calling a payment endpoint, deleting cloud resources, and sending email. Capability alone does not answer:

whether the action matches the user’s actual request;
which resources and how many people it may affect;
whether failure is recoverable;
who bears the cost of a wrong outcome;
which evidence permits continued execution;
when the decision must return to a person.

Traditional software answers these questions in deterministic code before execution. An agent system allows some judgments to happen at runtime. That does not remove control. It changes control from a prescribed path into explicit decision boundaries.

I therefore distinguish three concepts:

Capability: whether the system can perform an action class.
Authority: whether this agent may decide and execute it under current conditions.
Accountability: whether the system can explain, verify, and assume responsibility for the consequences.

A mature product does not grant authority merely because a capability exists.

Autonomy is a set of decision rights

Instead of assigning one global autonomy level, decompose autonomy into rights that can be granted separately.

Decision right	What the agent may decide	Typical boundary
Observation	Which files, data, and external material to read	Data scope, sensitivity, network boundary
Planning	How to decompose a goal, explore branches, and replan	Time, tokens, candidates, task scope
Tool selection	Which permitted tool to use next	Allowlist, argument constraints, call frequency
Mutation	How to change files, drafts, records, or local state	Workspace, diff size, reversibility
External action	Whether to send, publish, or change a remote resource	Approval, recipient, environment, affected users
Resource spending	How much model, compute, API, or money to consume	Per-step and task budgets, time window
Completion	Whether the goal is complete and execution should stop	Verifier, success contract, human acceptance
Delegation	Which task and authority to give another agent or tool	Inheritance, maximum depth, accountability

These rights do not form a required progression.

A research agent may have broad observation and planning rights but no external-action authority. A coding agent may edit an isolated worktree while requiring approval to push or deploy. An operations agent may restart a known stateless service automatically while lacking authority to touch a database or declare an incident resolved.

This is why “human in the loop” remains too coarse. Human participation should not mean one approval somewhere in the task. It should sit at the boundary of a specific decision right.

An autonomy contract

Every authority grant should contain at least these dimensions:

Authority Grant =
  Actor
  × Action
  × Resource Scope
  × Preconditions
  × Budget
  × Evidence Requirement
  × Expiry

A concrete contract might look like:

Actor:       current coding agent
Action:      edit files and run tests
Scope:       current worktree, excluding Git push
Precondition: repository instructions read; worktree state recorded
Budget:      at most 20 commands; no network
Evidence:    targeted tests + typecheck + final diff
Expiry:      task completion or user revocation

This is much more precise than “allow the agent to operate automatically.”

Scope limits access to relevant resources. Preconditions define what must be true before the right exists. A budget bounds the loss even when judgment fails. Evidence requirements tie continuation and completion to feedback outside the model. Expiry prevents authority from leaking permanently into future tasks.

Authority should also be non-transitive unless delegation is explicit. File-write access granted to an agent does not imply that every script or subagent it calls inherits the same boundary. The deeper the call chain, the more important it becomes to state whether authority is inherited, narrowed, or approved again.

Invariants, policy, and model judgment

A reliable agent product needs three different forms of control.

Invariants cannot be violated regardless of model judgment

Examples include prohibitions on reading a sensitive data class, exceeding an account balance, messaging an unauthorized recipient, or modifying production resources outside the task scope.

Deterministic systems must enforce invariants. Placing them only in a prompt influences behavior but does not create enforcement.

Policy automatically grants or requests approval under known conditions

Reading within a workspace might be automatically allowed. Modifying more than ten files might require approval. Network access may be restricted to specific domains. Recoverable writes in a test environment may run automatically while production writes always return to a person.

Policy translates organizational risk preferences into runtime rules. It can also use identity, environment, amount, and historical evidence as inputs.

Model judgment selects a path inside the permitted space

The agent should decide which terms to search, which file to read first, how many candidates to generate, whether a failed approach deserves another attempt, and which hypothesis is supported by current evidence.

This is where autonomy produces value. The product preserves boundaries that cannot move while avoiding a fixed recipe for every step.

Hard invariants define the world that must remain true.
Policy decides which actions may cross a boundary.
The agent chooses a path inside the granted space.

Put all three in the prompt and the system becomes unreliable and difficult to debug. Put all judgment in a workflow and the agent retains little freedom beyond generating text.

Risk is more than probability of failure

When granting authority, it is not enough to ask how often an agent is correct.

A read-only search and an irreversible payment should not receive the same authority even at identical success rates. A non-rigorous but useful risk model is:

Operational Risk ≈
  Probability of error
  × Impact radius
  × Irreversibility
  × Time to detection

This is not a precise score. It is a reminder that observability, locality, and reversibility can matter more than average model accuracy.

Autonomy should therefore expand first around actions that are:

Observable: the system quickly learns what happened.
Reversible: a failure can restore the previous state.
Local: the error remains inside one user, workspace, or test environment.

This is also the value of a sandbox. It does not only prevent dangerous operations. It creates an environment in which the agent can safely receive more autonomy. The same edit has a radically different risk structure in an isolated worktree and on a production branch.

Irreversible or slowly detected actions require narrower scope, stronger preconditions, independent verification, or recommendation-only authority.

How an agent earns autonomy

Earning authority does not mean accumulating a global reputation score. Evidence must be specific to the decision right, task class, environment, and impact radius.

Reliability in frontend edits does not grant database-migration authority. Correct notifications in a test environment do not justify messaging every real user. Strong performance on ten-minute tasks does not establish sound judgment about outcomes that unfold over months.

A better expansion path is:

Shadow
  The agent observes and records what it would do
      ↓
Recommend
  The agent proposes an action; a person executes or rejects it
      ↓
Approve-before-act
  The agent prepares the concrete action and executes after approval
      ↓
Bounded autonomy
  The agent acts in a small, low-budget, recoverable scope
      ↓
Expanded autonomy
  Sustained evidence expands one dimension

Each level should measure decision quality rather than only final success:

Did the agent choose the right action class?
Did it recognize when approval was required?
Did it stop when evidence was insufficient?
Did it avoid duplicating side effects after failure?
Did it remain inside the original scope?
Did the verifier measure the user’s actual outcome?

Expansion should change one major variable at a time. Increase file scope before simultaneously increasing file scope, network access, and remote-write authority. Otherwise a failure cannot reveal which new boundary was unsound.

Evaluation delay sets an autonomy ceiling

Some tasks can be verified in seconds: whether code compiles, a file exists, or a schema is valid. Other tasks take days or months before anyone learns whether the original decision was good.

This is the central problem of long-horizon work. An agent can run continuously, replan, and preserve durable state while remaining unable to answer the most important question: months later, how will we know today’s action was right?

Products often shorten feedback with proxies such as clicks, simulations, model judges, or human ratings. These signals can be useful. They validate parts of the contract but do not automatically replace the real outcome.

When evaluation is delayed or the outcome is weakly observable, the honest response is to lower autonomy: shorten commitments, insert stage approvals at directional decisions, limit irreversible investment, and preserve exit options.

A system that cannot evaluate outcomes promptly should not receive longer autonomy merely because it can keep executing.

Completion authority is easy to underestimate

Tool permission is visible. Stopping permission is often ignored.

Letting an agent announce completion grants a consequential decision right. Stopping too early delivers an incomplete result. Refusing to stop consumes cost, expands the diff, and may repeat external actions.

Completion should therefore have its own contract:

Completion Authority
├── success criteria
├── required evidence
├── maximum unresolved risks
├── verifier identity
└── human acceptance boundary

For a low-risk task, the agent may stop automatically after tests pass. For a cross-system write, it may need an independent read from the system of record. For aesthetic, strategic, or high-stakes business judgment, the system may only state that execution is complete and present evidence; acceptance still belongs to a person.

An agent can own the execution path without owning the definition of good.

Delegation cannot dissolve responsibility

Multi-agent systems amplify the authority problem.

When a primary agent delegates research, it should pass the minimum authority required for that subtask rather than cloning every capability. After a subagent returns a recommendation, the executor must still check scope, evidence, and side effects.

Parent authority
  ├── delegated subset → research agent
  ├── delegated subset → coding agent
  └── retained rights  → approval / external action / completion

Delegation can distribute work. It cannot distribute away accountability. The system must still know who proposed an action, who approved it, who executed it, and which verifier accepted the result.

Otherwise every agent can assume that another agent performed the necessary check while the origin of authority disappears down the call tree.

Authority must be revocable

Autonomy does not grow forever after one successful grant.

An environment change, model upgrade, tool update, new attack, abnormal cost pattern, or severe failure can invalidate old evidence. The system must be able to:

expire grants automatically;
narrow scope when error rates or cost rise;
suspend a tool or external action class;
downgrade automatic execution to approve-before-act;
cancel actions that have not begun;
send unknown external side effects to human resolution.

Degradation is not failure. It is normal control in a dynamic system.

An authority mechanism that can expand but never revoke turns past success into future permission debt.

The interface should reveal control

Transparency does not require a timeline of every tool call, token, and internal activity. Users need to understand consequential control relationships:

the goal the agent is pursuing;
the decision rights it currently holds;
the budget it is consuming;
the actions that have produced external effects;
the evidence that will make it stop;
the decisions that still belong to the user.

A good approval should not show only a shell command. It should explain intent, impact scope, reversibility, and the alternative if approval is denied.

Users do not need to supervise every step. They do need to predict what “let it continue” means.

Autonomy is a governance system

Placed together, agent autonomy does not live inside the model. The harness and product enforce it jointly:

User / Organization Policy
              │
              ↓
      Authority Controller
      ├── invariants
      ├── grants & expiry
      ├── budgets
      └── approval rules
              │
              ↓
Agent Decision → Tool Executor → Environment
      ↑               │               │
      │               ↓               ↓
      └──── Context / State ← Evidence / Effects
                             │
                             ↓
                    Verification & Eval
                             │
                  expand / retain / revoke

From Model Calls to Agent Systems describes how a harness constrains action and produces evidence. Coding Agents as Execution Systems follows an engineering goal through execution. This essay asks the governance question that follows: once those mechanisms exist, which judgments should belong to the agent?

The answer is neither “all of them” nor “60 percent.”

A product should decompose autonomy into concrete decision rights, declare scope, budget, evidence, and expiry for each grant, expand authority inside observable and reversible environments, and revoke it when evidence no longer holds.

A mature agent is not one that never asks a person.

It knows which decisions are authorized, which evidence permits it to continue, which boundaries cannot be crossed, and when judgment must return to the person who bears the consequences.