7 Tool Calling

Purpose

Agent stack focus for this chapter.

Why does a tool call that parses cleanly still need a runtime between it and the world?

A tool call is the first point at which a model’s output can change something outside the conversation. A proposal that parses cleanly can still name a target the task was never granted, act on a file that has changed since the model read it, or repeat an operation whose first attempt already took effect. Generating the call therefore cannot be what authorizes it. The runtime must stand between proposal and effect, validating each call against its schema, checking it against the grants the task holds, dispatching it, and recording what is actually known about the result. That record is often incomplete. A request that times out may have been lost, may still be running, or may have succeeded without a delivered reply, and a blind retry can restart healthy workers or apply the same change twice. The outcome has to stay explicitly unknown until the runtime settles it, rather than be filled in by the model’s account or the runtime’s optimism. What returns to the model is itself a design decision, because a log far longer than the next call can hold must be cut down without hiding the line that matters, and a job that outlives one turn must report back without being mistaken for a finished one. Tool calling keeps the model a proposer and leaves authority with the runtime, which exercises it only on evidence it can check.

Learning Objectives

Trace a proposed tool call through validation, authorization, dispatch, and a correlated observation into the next model call
Design a tool descriptor that separates structural validation from action-time preconditions enforced with the effect
Distinguish tool discovery and connectivity from the grants required to invoke an operation
Select blind retry, endpoint deduplication, or reconciliation for an uncertain effect from operation semantics and loss rates
Design bounded observations that preserve execution status, omission information, and the scope of available evidence
Compare waiting and resumption strategies for long-running tools by detection delay, inference cost, and retained supervision
Evaluate toolkit granularity by context cost, cost per accepted task, and enforced reach across structured, general-purpose, and computer-use interfaces

7.1 The Tool Boundary

An agent asked to restart a container on a staging server, the kind of infrastructure change the remediation agent (Four lighthouse workloads) makes, emits a short tool call. The host runtime checks the call against the tool’s schema, confirms that this session may restart that container, and sends POST /containers/srv-stage-04/restart. Two seconds later the socket times out. The request may be lost, delayed, still executing, or completed without a delivered reply. A retry could restart healthy workers, and the first attempt can still take effect after the timeout.

The Authority axis, closed at the tool boundary by mediating every proposed call.

Nothing about this failure is exotic. The repair agent meets the same questions on every turn of the connection-pool repair. It reads src/pool.py, replaces text derived from the version it read, runs the test suite, and receives a log far longer than the next model call can hold. Each step is a tool call, and each can be rejected, left running, completed without a reply, or answered with more output than the model should see.

The pieces built so far do not settle these questions. Messages and tool definitions gave the model a structured way to propose a call, and the memory chapters already relied on tools to read sources, search indexes, and update records without examining what happens between a proposal and its result. A retrieval result identifies the version it observed, but whether the file still has that version when a later edit commits is an action-time precondition that something must enforce. A proposal names an operation and its arguments. It does not decide whether this task may run the operation, whether an earlier attempt already ran it, or what the next model call should receive.

The runtime’s tool boundary is the path between a proposed call and the environment it would change. The boundary validates each proposal, checks it against the task’s grants, dispatches it, tracks what is known about its effect, and returns a bounded observation matched to the call. Throughout, it keeps apart the three things the timeout blurs: what the model proposed, what effect the operation had, and what evidence the runtime holds about either. The chapter follows one call from proposal to observation, then widens to effects whose outcome stays uncertain, to output too large to return whole, to jobs that outlive a model turn, and finally to the catalog of tools the model sees.

↩︎ The separation of proposal, effect, and observation is established in Why the runtime manages the trajectory, not the single call.

7.2 Tool Subsystem Architecture

When the socket times out on restart_container, a runtime that drops the failed call from history or reports it as a clean failure invites the next model turn to issue a duplicate restart. Listing 1 shows the staging restart as one turn of the loop, in a provider-neutral form. The runtime reports what it knows about this attempt, and the next model call receives that result together with the earlier request and proposal.

Listing 1: One turn of the tool-call loop: A call identifier correlates the proposed call with its result. The next model call receives the timeout as an unknown effect, not as evidence that the restart did not happen.

[
  {"role": "user",
   "content": "Stage-04 is serving a stale config. Restart it."},
  {"role": "assistant",
   "content": [
     {"type": "tool_call", "id": "call_1", "name": "restart_container",
      "arguments": {"container_id": "srv-stage-04"}}]},
  {"role": "tool",
   "content": [
     {"type": "tool_result", "call_id": "call_1",
      "content": {"transport_status": "timeout", "effect_status": "unknown"}}]}
]

The runtime correlates each result with its corresponding call using an explicit identifier rather than message order alone. That identifier establishes message lineage across asynchronous turns, but correlation alone does not resolve whether the target service completed the requested mutation. Preventing duplicate restarts requires an operational contract for idempotency, developed in section 7.5. Even if the subsequent model turn assumes that the service recovered, the runtime preserves the underlying uncertainty. It authorizes a retry only when idempotent semantics, an endpoint deduplication contract, or reconciliation evidence that also rules out a still-delayed attempt supports it; fresh evidence that the container looks unchanged is not enough on its own (section 7.5.1).

7.2.1 One mediated round trip

If a runtime piped the model’s structured invocation block straight to an execution backend and piped the raw output straight back into the prompt, an unauthorized call would mutate the host and an unbounded log would overflow the next context window. At the model-side call surface, an inference engine emits structured invocation blocks rather than unstructured natural text, and once the model outputs a proposal, execution passes entirely to the runtime. Between the moment a tool call leaves the model and the moment an observation enters the context of the next turn, the runtime mediates the round trip across the four coordinated stages in figure 1: argument validation screens inputs against tool schemas, policy authorization verifies security grants, dispatch executes the permitted operation while tracking outcomes, and observation shaping normalizes raw results into bounded prompt payloads.

↩︎ The six-phase turn lifecycle, whose validation, execution, and observation phases these four stages refine, is established in The six phases of a turn.

Request validation parses proposed arguments against the tool’s declared schema, verifying required fields, data types, value ranges, and relational invariants among parameters. Dynamic constraints that depend on volatile environment state cannot be settled at this boundary; the system must evaluate them at the execution boundary rather than inferring them from historical context. Next, operation authorization verifies the active session’s security grants and access policies against the targeted resource, for read operations as well as state mutations, and admits the call against its concurrency budgets and rate limits.

Following authorization, dispatch submits the permitted operation to the underlying execution provider while outcome tracking maintains execution identity and status. This tracking distinguishes among pre-dispatch rejections, active background tasks, successfully reported results, and unconfirmed execution states. A transport timeout indicates only that the runtime exceeded its waiting deadline, leaving the remote execution state unconfirmed. Finally, observation shaping maps the raw return value back to the originating call identifier, truncating payloads to fit context budgets, sanitizing terminal escape sequences, and tagging the output with accurate status metadata. If log output must be truncated, the payload explicitly discloses the omission; if a system returns an asynchronous job ticket, observation shaping reports it as an acknowledgment rather than a completed effect.

Certain validation checks must execute at the remote endpoint because safety requires atomicity with the state change itself; for example, an edit conditioned on a source version requires the version check to occur in lockstep with the update. Consequently, the host runtime and the remote endpoint each enforce their respective boundaries. The endpoint checks access independently, even for requests that the host runtime already cleared. This division is the end-to-end argument applied to system protection. An authorization check that depends on private endpoint state can be finalized only at that endpoint, which makes preliminary host-level checks useful optimizations rather than authoritative substitutes (Saltzer et al. 1984).

Figure 1: **The tool-call loop**: Validation and authorization can reject a proposed call before dispatch. For a dispatched call, the runtime tracks the available outcome and shapes the return into a bounded observation matched to its call identifier. An unknown effect remains unknown in that observation.

When the serving interface returns native structured call blocks, the runtime extracts the proposed operation and arguments directly. When a model emits proposals within unstructured free text, the runtime must first detect and parse those fragments before validation can proceed. Constrained decoding eliminates syntactic parsing failures by enforcing grammar rules directly during token generation. However, syntax alone never establishes operational safety. Whether the target resource exists, whether the observed version remains current, and whether the current session holds execution privileges remain runtime checks that no decoding grammar can verify.

7.2.2 Complete mediation

Telling a model in its system prompt “never delete production data” or “only read files inside /workspace” places no barrier between a hallucinated or prompt-injected tool call and the host operating system. Securing the tool interface rests on complete mediation (Saltzer and Schroeder 1975), the protection principle that already required the tool handler of Why token likelihood cannot replace checks to check a validated path again when the read executes. At the tool boundary, mediation must govern both reads and mutations. Emitting a tool name does not grant the model ambient runtime authority, nor does receiving an observation make external data trusted.

Definition 7.1: Complete mediation

Complete mediation requires every proposed access to pass through an enforced authorization boundary before it reaches the protected resource. In an agent runtime, the return path is also mediated so that observations enter context under explicit access, payload, and status contracts.

Significance: A valid-looking call cannot bypass the checks governing what the session may read, change, or disclose.
Distinction: Schema validation establishes structural admissibility, not permission. Observation normalization establishes a usable representation, not the truth of its contents or immunity to prompt injection, attacker-controlled content that attempts to redirect the model.
Common pitfall: Treating a prompt instruction such as “never delete production data” as an access barrier. The runtime must enforce the restriction even when the model proposes the prohibited action.

Stating complete mediation as a policy does not by itself guarantee that every tool route honors it. Anderson formalized the architectural mechanism required to make that guarantee hold across every access path: the reference monitor (Anderson 1972).

Definition 7.2: Reference monitor

Reference monitor is the enforcement component through which every access to a protected resource passes. It must be tamper-proof, always invoked, and small enough to verify. In an agent runtime, the host’s validation and authorization path is the reference monitor for proposed tool calls, and each endpoint’s access check is a second monitor for the resources that endpoint owns.

Significance: The three requirements turn complete mediation from an intention into a checkable property of the runtime’s structure.
Distinction: The policy specifies permitted accesses, whereas the reference monitor provides the non-bypassable mechanism that evaluates and enforces those rules on every attempt.
Common pitfall: Adding a tool route that does not pass through the monitor, such as an unrestricted shell with network access beside a checked file adapter, which makes the adapter’s checks advisory.

These three requirements dictate concrete architectural choices at the tool boundary. Tamper-proofing ensures that neither generated model tokens nor untrusted data ingested from external systems can alter the authorization rules enforced by the runtime. Always-invoked mandates that every operational path pass through the monitor without exception. Exposing an unrestricted shell runner alongside a sandboxed file tool, for example, undermines the protection boundary, because the agent can bypass file restrictions by issuing shell commands. Finally, verifiability requires policy decisions to be deterministic, auditable software checks. Although an auxiliary classifier or guardrail model can score operational risk, probabilistic inference cannot be verified in the strict systems sense; the monitor itself must enforce deterministic access boundaries.

Two operational rules follow directly from complete mediation. The grant rule establishes that execution authority derives solely from the session’s active security grants, re-evaluated on every invocation. Read operations demand the same rigorous mediation as state mutations, because returning private records discloses sensitive data even when environment state remains unchanged. Nothing the model sees or emits enlarges those grants, whether a schema-valid call or a tool description claiming broad permission; later sections apply the rule to each new artifact they introduce. The evidence-scope rule establishes that an observation confirms only the specific metric or condition measured by the tool during that invocation. Reporting that an operating system process exited with code zero confirms only process termination; it does not prove overall task completion, which remains a distinct judgment governed by invariant closure (principle 1). Raw output text also remains untrusted data even after the runtime strips terminal formatting and normalizes metadata; observations supply empirical evidence of state, but they carry no ambient authority to dictate subsequent actions.

7.2.3 Batches and forced calls

When a single model response proposes reading src/pool.py and patching src/pool.py in the same batch, dispatching both calls in parallel yields non-deterministic results depending on which thread reaches the file first. When batched operations are strictly read-only and independent, such as reading three distinct source files, the runtime can dispatch them concurrently and correlate each returned observation with its respective call identifier; a failure in one read does not invalidate concurrent reads that succeeded, though every call in the batch must individually pass validation and authorization and draw from a shared resource and rate-limit budget.

When tools declare a trustworthy resource footprint, the explicit read set $\ToolCallReadSet_i$ and write set $\ToolCallWriteSet_i$ of each call $\ToolCallIndex$, the runtime can check whether batched mutations and reads interfere. Calls $\ToolCallIndex$ and $\PeerToolCallIndex$ may run concurrently when neither writes state the other reads or writes, $\ToolCallWriteSet_i \cap (\ToolCallReadSet_j \cup \ToolCallWriteSet_j) = \emptyset$ and $\ToolCallWriteSet_j \cap \ToolCallReadSet_i = \emptyset$, which are Bernstein’s conditions for noninterference (Bernstein 1966). Reads of two different files pass both conditions; a read and an edit of src/pool.py fail the first, because the edit writes the file the read reads. This guarantee, however, relies entirely on the completeness of footprint declarations; any hidden side effect or undeclared shared dependency invalidates the concurrency guarantee.

Bernstein, Arthur J. 1966. “Program Analysis for Parallel Processing.” IEEE Transactions on Electronic Computers EC-15 (5): 757–62. https://doi.org/10.1109/PGEC.1966.264565.

Operations with opaque side effects, such as arbitrary shell commands, possess unknown footprints. In such cases, the runtime must either isolate the execution environment, serialize calls conservatively, or reject batched execution in favor of fine-grained tools. When operations have causal dependencies, those ordering constraints must come from the workflow policy or an explicit dependency graph; the sequential order of tokens generated by the model is not reliable evidence of intended execution order. Batching also does not imply atomicity; a batch forms a transaction only when the underlying tool substrate explicitly provides transactional all-or-nothing rollback semantics.

The runtime can also govern whether the model proposes an action. Using serving-level tool-choice control, the runtime can configure tool calling as optional, mandate that the model invoke at least one tool, restrict selection to a specific named operation, or forbid tool execution altogether. Orchestration engines use these constraints to guide execution phases, for instance forcing a diagnostic read before permitting an edit, or disallowing tool calls to force a final textual summary. Tool-choice controls, however, only shape model generation; under the grant rule, forcing a tool choice confers no operational authorization. Executing each stage of this mediated round trip requires an unambiguous request representation, establishing a strict operational boundary between static schema validation and dynamic preconditions verified at dispatch time.

7.3 Tool Interface Schemas

Recall the connection-pool repair right at the transition from revision r17 to r18. The model has inspected src/pool.py and emitted a proposed patch. Before applying that modification to real infrastructure, the host runtime must resolve four questions: which tool operation the model selected, what its arguments denote, whether the current task possesses the authority required to execute the mutation, and whether the underlying target file still matches the exact source state on which the model based its proposal. A well-formed JSON payload satisfies syntactic structure, but syntax alone cannot answer these operational questions.

A tool descriptor defines an operation’s functional purpose, input parameters, expected return values, and execution side effects. Within that descriptor, the argument schema codifies machine-checkable constraints: required fields, primitive types, permitted enumerations, and supported size bounds. These specifications serve two distinct audiences: natural-language descriptions guide the language model in selecting and parameterizing tools, while the surrounding runtime and downstream service endpoints validate and enforce the interface contract.

7.3.1 Descriptor semantics

If a file-replacement tool leaves unspecified whether an empty replacement string means “leave the file unchanged” or “truncate the file to zero bytes,” a model attempting a no-op will wipe src/pool.py. Consider a tool named conditional_replace_file, which replaces a single file in the task’s designated workspace and commits the change only if that file still matches the expected source version. Its descriptor must establish exact semantic invariants: path resolves strictly relative to the designated workspace rather than an arbitrary host location; expected_version identifies the baseline source revision the model inspected; and replacement supplies the complete replacement content encoded in the endpoint’s documented format, where passing an empty string explicitly truncates the file to zero bytes.

Full file replacement keeps this precondition example transparent, though production runtimes often prefer line-anchored patches or structured diffs to minimize token generation overhead; the interface comparison in Comparing Output Interfaces shows that localized edits reduce generated output while still demanding the same coordination between precondition verification and atomic mutation. Listing 2 illustrates the machine-checkable argument structure for that contract. The request payload contains neither a credential token nor a privilege flag through which the model might expand its access rights, and the host runtime independently binds the invocation to the task’s authorized execution context.

Listing 2: Conditional file replacement arguments: The schema describes the request’s structure. Its descriptions do not implement path confinement, authorization, or atomic version checking.

{
  "type": "object",
  "required": ["path", "expected_version", "replacement"],
  "additionalProperties": false,
  "properties": {
    "path": {
      "type": "string",
      "minLength": 1,
      "description": "File path relative to the designated workspace."
    },
    "expected_version": {
      "type": "string",
      "minLength": 1,
      "description": "Source version returned with the inspected file."
    },
    "replacement": {
      "type": "string",
      "description": "Complete replacement text, written as UTF-8."
    }
  }
}

The descriptor’s result contract requires equal precision. A completed replacement reports the affected path and its newly assigned revision token. A version conflict explicitly reports that the target diverged and the replacement was not applied. The runtime must also distinguish deterministic client errors (malformed arguments or insufficient authority) from indeterminate transport failures where the endpoint’s state remains unknown after a dropped connection. A comprehensive descriptor models all four outcomes, including network timeouts where no acknowledgment arrives.

Precise field names and descriptions eliminate avoidable ambiguity during generation. For example, a file-reading tool should state whether its offset argument counts bytes, characters, or lines. Similarly, a timeout parameter must state its measurement units and specify whether it bounds connection establishment, remote execution, or end-to-end response delivery. While clear documentation guides the model toward valid parameters, the execution runtime must programmatically enforce all defaults, enumeration sets, and numerical bounds. Grounding model generation in explicit tool documentation yields measurable reliability improvements: Patil et al. (2023) demonstrated that models trained to condition tool invocations on retrieved API documentation invent nonexistent operations and hallucinate invalid arguments far less frequently than models relying solely on unconditioned prompting.

7.3.2 Structure, permission, and state checks

If generation halts at the output token limit in the middle of a replacement string and the parser closes the JSON object or accepts a partial payload, executing that call writes a half-finished file to disk. When processing a tool call, the runtime first verifies that the model produced a completed candidate, adhering to the invocation contract defined in The Invocation Contract; if generation halted prematurely due to token limits or client disconnection, the runtime aborts immediately. Once execution status confirms a complete output, the runtime parses the candidate and validates its arguments against the tool’s registered JSON schema before performing any domain checks.

In this replacement example, omitting expected_version or supplying a numeric path fails structural validation immediately. Setting "additionalProperties": false in listing 2 likewise causes the parser to reject hallucinated arguments such as force or recursive. However, structural conformance resolves only syntax. Verifying whether the path resides within the task’s assigned sandbox, and determining whether the target still contains the inspected source revision, require distinct evaluation steps. As summarized in table 1, the runtime enforces these guarantees across layered boundaries.

Table 1: Checks at the tool boundary: Each check establishes a different part of the action contract. Rows follow the round trip; domain preconditions come last because the endpoint enforces them together with the effect, so source-dependent checks remain valid through the operation they protect.

Check	Question for the proposed edit	What passing does not establish
Argument structure	Are the required fields present with the declared types and supported constraints?	The path is permitted, the version is current, or the replacement is a correct repair.
Authority	Do the task’s current grants permit this operation on the resolved target?	The arguments fit resource limits or the operation will succeed.
Resource limits	Does the request fit enforced payload, execution, and result limits?	The target still satisfies the operation’s preconditions when the effect happens.
Domain preconditions	Does the target support this operation, and does its version match when the update takes effect?	The returned evidence will establish task acceptance.

A string formatted as a file path does not constitute a confined resource reference.¹ The tool implementation must canonicalize and resolve the target inside the authorized workspace, accounting for symlinks, directory traversal escapes, and path alias changes that could redirect the operation. Similarly, declaring replacement as a string does not bound its encoded memory footprint. Production adapters enforce explicit byte-size limits on incoming payloads, bound the volume of captured diagnostic output, and apply strict execution timeouts. Static schema validation handles input syntax, whereas dynamic state verification, path confinement, and resource throttling remain runtime obligations. Schema Rules for Tool Definitions supplies a bounded edit descriptor along with schema rules for required fields, size limits, explicit units, and variant semantics.

¹ Time-of-Check to Time-of-Use: Bishop and Dilger analyzed the UNIX file-access race in which the binding of a name to an object changes between a program’s check and its later use, so a privileged program can be steered onto a file it never checked (Bishop and Dilger 1996). A tool adapter that validates src/pool.py and then reopens the path by name inherits the same window: a symbolic link swapped in between turns a permitted edit into an edit elsewhere. Resolving the path once and operating on the resolved object, rather than re-resolving the string, is what closes it.

Bishop, Matt, and Michael Dilger. 1996. “Checking for Race Conditions in File Accesses.” Computing Systems 9 (2): 131–52.

Grammar-guided decoding does not change this division of labor. A token-level mask removes syntax errors before the candidate exists, but it cannot resolve a filesystem reference, evaluate an access policy, or compare a version tag with the file’s current revision. The host runtime validates the materialized request, and the endpoint verifies dynamic preconditions at the moment the effect happens. This separation instantiates the typed action contract (principle 7), which admits a call to the environment only after validation, precondition, and authorization checks, with the version precondition enforced together with the mutation rather than during an earlier read.

↩︎ The structural guarantee of grammar-guided decoding, and the reference and permission checks it leaves open, are established in What a grammar mask guarantees.

7.3.3 Conditional updates

An earlier read confirmed only that src/pool.py matched revision r17 at the instant of observation; if another agent or background process modifies src/pool.py while the model is generating its patch, applying an unconditioned write silently overwrites those intervening changes. The expected_version parameter binds the retrieval contract of The Retrieval Contract directly to the mutation contract enforced here so that this lost-update race cannot succeed. To prevent the lost update, the storage service must verify the expected version and commit the replacement as a single atomic conditional update, the storage equivalent of a hardware compare-and-swap (CAS) primitive.² In this example, the workspace source service executes the entire sequence inside a single coordination boundary: it resolves the target file, verifies that its current revision matches expected_version, writes the new content, and publishes the resulting revision token. Decomposing this sequence into non-atomic operations (reading the revision, verifying it in user space, then issuing a write) reopens a race window where concurrent writers can intervene. The atomic conditional update closes this time-of-check to time-of-use gap.

² Optimistic Validation Under Contention: The expected-version check is commit-time validation from optimistic concurrency control, applied to one file. What is new at the tool boundary is the price of a conflict: each failed validation costs the agent a fresh read and a new proposal, paid in model turns rather than in microseconds.

↩︎ Why context engineering cannot close the refresh-to-execute race, and leaves the revision check to the tool boundary, is established in Detecting change and bounding uncertainty.

This execution branch yields two distinct operational outcomes:

If the current source revision remains r17, the update commits the replacement and returns r18, identifying the new revision produced by the edit.
If another writer modified the source first, the version check fails. The service rejects the replacement, leaves the storage state unaltered, and returns an explicit version-conflict status.

Following a conflict, the runtime directs the agent to perform an authorized read of the updated source file and synthesize a fresh proposal, rather than blindly overwriting the intervening modifications. This concurrency guarantee holds only as long as all mutations route through the shared coordination boundary. If external processes or unmanaged scripts modify files directly on disk, bypassing the version service, version tokens offer no protection and the endpoint cannot ensure integrity. Dry-run previews follow the identical principle. A dry run confirms that a proposed patch applies cleanly against revision r17 at the moment of evaluation, but it establishes no lock on that revision. The runtime must re-verify all preconditions, permissions, and version tokens when executing the live write.

Receiving a successful r18 confirmation verifies that the file modification committed to disk, but it does not establish that the underlying connection-pool bug has been resolved. Demonstrating functional correctness requires empirical validation. A subsequent passing run of test_waiter_notified_on_return is evidence that the patch fixes the exercised case of the race, while broader task acceptance continues to follow the project’s verification suite. At the network layer, a dropped reply leaves the caller uncertain whether the server committed the edit or failed before processing the payload. The retry and idempotency contracts analyzed in section 7.5 resolve this operational uncertainty.

7.3.4 Rejection diagnostics

If a rejected tool call returns only "Error: operation failed", the model guesses blindly on its next turn; if a permission denial dumps the host directory tree and environment variables to explain why a path was refused, the error message itself leaks unauthorized state. When an operation fails, the runtime must return a structured, bounded diagnostic that isolates the failure condition without leaking information beyond the task’s authorized grants. A structural rejection should identify the missing expected_version property, and a concurrency conflict should state that revision r17 has been superseded, whereas permission rejections must refuse access cleanly without exposing directory structures or path hints that an agent might probe. The runtime logs whether a rejection originated locally during client-side schema validation or remotely at the service endpoint, allowing downstream decision logic to distinguish between an unattempted proposal and a failed remote mutation. Clear, bounded diagnostics give the model the exact evidence needed to synthesize a valid revision, which must then traverse the same validation pipeline.

The descriptor, its layered checks, and its diagnostics together define one tool’s contract, and every check so far has assumed that the runtime already holds that contract and knows which endpoint implements it. In a real deployment the contract arrives from elsewhere: a repository server, another team’s service, a vendor’s integration. How the runtime learns which operations exist, and why learning that an operation exists confers nothing, is the next problem; how many of the discovered descriptors to stage on each turn is a separate budget question that section 7.9 takes up.

7.4 Interoperable Tool Discovery

In practice, a tool implementation such as conditional_replace_file can live in several environments: an in-process adapter compiled into the application, a standalone background daemon, or a remote microservice managed by another team. Before the model can propose a valid call, the host runtime must determine which operations that endpoint exposes and how to encode their arguments. This discovery phase supplies that interface metadata. Under the grant rule, however, learning that an operation exists confers no authority to execute it. Discovery defines interface syntax, whereas authorization governs execution rights.

7.4.1 The Model Context Protocol

When building a self-contained agent with a small, static set of tools, registering local functions directly in-process is simple and fast. However, when multiple host applications must connect to independently maintained tool services, custom point-to-point integration quickly becomes impractical. The underlying challenge is combinatorial. Without a shared interface, $\HostRuntimeCount$ host runtimes integrating $\ToolServiceCount$ tool services require up to $\HostRuntimeCount \times \ToolServiceCount$ bespoke adapters. Standardizing on a common protocol reduces this burden to $\HostRuntimeCount + \ToolServiceCount$ implementations, allowing each side to build to the specification once.

A shared protocol turns pairwise adapters into one implementation per side.

Decoupling tools across a protocol boundary also provides fault isolation. When a tool runs in a separate process, a memory leak, segmentation fault, or unhandled blocking call crashes only the tool server rather than tearing down the primary agent orchestration loop. Even so, protocol standardization does not eliminate engineering responsibilities. Integrators must still define tool semantics, audit version compatibility, and enforce security policies. Local in-process plugins and out-of-process tool servers represent deliberate deployment trade-offs rather than mutually exclusive or evolutionary stages.

The Model Context Protocol (MCP) exemplifies this architecture. MCP structures integrations across three distinct entities: a host application (such as an interactive environment or agent runtime), clients that manage connections, and servers that expose capabilities. The host runs one client per connected server, and each client speaks the protocol on the host’s behalf. Servers can run as local child processes or as remote network services. This architectural division clarifies integration responsibilities rather than prescribing a specific execution sandbox (Model Context Protocol 2025).

Each deployment model matches different operational demands. A local child process keeps filesystem operations adjacent to the working copy and simplifies process lifecycle management. Conversely, a remote service can centralize credential storage, connection pooling, and backend access without replicating sensitive configurations across every host machine. Both options, however, introduce distinct failure modes and resource boundaries. An unconfined local process still reaches whatever local filesystem it is granted, while a severed network link can leave an in-flight operation executing invisibly on a remote server. The choice between them follows where resource governance and policy enforcement belong, weighing transport latency against tool execution duration rather than presuming any transport topology is universally superior.

7.4.2 Tool listing and invocation

If a host connects to two tool servers that both advertise an apply_patch operation and dispatches calls by bare function name, an untrusted server can silently shadow the repository’s patch tool. MCP defines two primary wire operations for tool interaction (tools/list to discover available tool definitions and tools/call for execution³), and the host must bind each discovered definition to its specific server instance and local policy before exposing it. Each definition advertises a unique name, a natural-language description, and a JSON Schema governing accepted arguments; an invocation names the target tool and supplies a matching argument dictionary. These messages establish the mechanics of interface exchange rather than a task’s execution rights. The MCP tools specification (Model Context Protocol 2025) defines these message formats and mandates server-side input validation and access controls.

³ JSON-RPC Request Identifiers: MCP frames these messages as JSON-RPC 2.0 requests, each carrying an id that its response echoes (Model Context Protocol 2025). That identifier does the work of this chapter’s call identifier, matching a reply to its request on one connection. It says nothing about whether a second tools/call with the same arguments is the same intended effect, which is why retry safety needs the separate operation identity developed in section 7.5.

Model Context Protocol. 2025. Model Context Protocol Specification, Version 2025-11-25. Protocol specification.

To trace where policy boundaries sit during execution, consider a repository server advertising two illustrative operations: read_source and apply_patch. A concrete integration sequence proceeds through several verification checkpoints:

The host’s client queries tools/list, and the host admits each returned schema only if its own policy allows it (for example, because the server and its version are on an allowlist), binding each admitted schema explicitly to that server instance and its locally governed tool policy. Explicit binding ensures that an identically named tool advertised by another server cannot silently shadow or replace the expected implementation.
The host determines which admitted definitions to expose in the model context for the connection-pool repair task.
When the model proposes an operation, the host runtime validates the arguments against the schema and verifies task-level authorization before transmitting the request.
Upon receiving tools/call, the endpoint independently verifies the request against the underlying operating system permissions and data access controls it manages.

For the connection-pool repair, discovering that apply_patch exists leaves the permitted edit paths and the expected-version preconditions unchanged. The guarantee of atomic, conditional updates derives entirely from the server implementation, never from an encouraging function name. If an endpoint fails to provide conditional-update guarantees, the runtime must reject it as incompatible with the task’s safety contract.

7.4.3 Discovery and authority

Tool descriptions provided during discovery are untrusted external inputs. A description might accurately summarize an operation, omit critical failure modes, or embed prompt instructions that attempt to steer model behavior. Even an authenticated server can supply a misleading description, because verifying a server’s identity does not imply endorsing every claim it emits. A host runtime must never promote claims such as “approved for all workspaces” into an authorization grant.

Filtering which tools appear in the model prompt narrows the attack surface, but catalog filtering alone cannot replace per-call enforcement. Task grants evolve dynamically, and call arguments select specific resources at runtime. Authorizing an agent to inspect src/pool.py does not permit it to read unrelated credentials or transmit private source code to an external service. Defense in depth also requires the server or backing service to enforce its own perimeter rather than assuming every client performs flawless checks. Host authorization and endpoint authorization protect different assets and operational domains; neither supersedes the other. On the return path, protocol framing cleanly differentiates a tool result payload from a system message, but framing does not disarm adversarial text within the payload, as a documented security vulnerability demonstrates when ambient credential reach serves as the sole barrier between discovery and side effects.

↪ Prompt injection delivered through tool responses and other observations is developed in Prompt injection entry points.

War Story 7.1: GitHub MCP private-repository leak (2025)

Context: In May 2025, Invariant Labs demonstrated an attack on an agent connected to GitHub through its MCP integration (Milanta and Beurer-Kellner 2025). The user owned both a public and a private repository and directed the agent, running in a desktop client, to inspect open issues in the public repository. The agent’s GitHub access token held permissions that spanned both repositories.

Mechanism: An attacker filed an issue in the public repository containing prompt instructions designed to redirect the model. After reading the issue as an ordinary tool result, the agent executed the server’s schema-valid tools to inspect the user’s private repositories and then opened a pull request in the public repository containing the exfiltrated contents. Every tool call remained within the personal access token’s broad credential scope, and no runtime check verified whether the target repository matched the task’s intended scope. Although the client prompted users to confirm tool calls by default, developers frequently enable an “Always Allow” setting to avoid operational friction.

Impact: The public pull request disclosed sensitive information, including the private repository name, relocation plans, and salary details. Invariant Labs characterized the breakdown as an architectural failure in agent runtime mediation rather than a software defect in GitHub’s server implementation.

Fix: The researchers recommended enforcing context-aware access control at runtime, such as restricting an agent session to the repository the user named, alongside continuous monitoring of agent-server traffic. They also observed that model alignment training alone failed to stop the attack.

Systems lesson: Discovery, schema validity, and credential authentication all succeeded. The failure was an absence of complete mediation (definition 7.1) against the task’s explicit grants, which should have refused the first private-repository read, a read-only $\AuthorityExposureLevel_0$ action, well before any write occurred.

Milanta, Marco, and Luca Beurer-Kellner. 2025. GitHub MCP Exploited: Accessing Private Repositories via MCP. Invariant Labs blog.

This exploit succeeded because three distinct conditions coincided:

The agent ingested untrusted external text that mounted an indirect prompt injection, attacker instructions planted in data the agent reads as input (Greshake et al. 2023).
The agent held ambient authority broader than its active task required.
The runtime provided tools capable of public mutations.

Eliminating any single condition breaks the attack chain. The second condition is the classic confused deputy problem (Hardy 1988), which the introduction glossed as the hazard of authority that comes from where a program runs rather than from the request it serves. The tool boundary is the first place the runtime must defeat it mechanically.

↩︎ The confused deputy, glossed alongside zero ambient authority, is established in The fail-plausible fault.

Definition 7.3: Confused deputy

A confused deputy is a program holding legitimate authority for one purpose that is manipulated into exercising that authority on behalf of a party with no right to direct it, because the system does not tie the authority exercised to the party directing the action.

Significance: In an agent system the runtime is the deputy. It holds task credentials and dispatches tool calls, so every prompt injection is an attempt to borrow those privileges, and the authority level assigned to the task, from read-only inspection ($\AuthorityExposureLevel_0$) to irreversible external mutation ($\AuthorityExposureLevel_3$), caps what a successful injection can cost.
Distinction: The attacker acquires no new credentials; it borrows authority the deputy already holds. Checking the deputy’s identity reveals nothing, because the deputy is legitimate. Authorization must instead check whether this request, within the active task, justifies exercising the deputy’s authority.
Common pitfall: Approving an action because its model-generated rationale looks plausible. A justification such as “required for debugging” is part of the unverified request, not independent evidence that the operation is authorized.

In the GitHub incident, the agent exercised the user’s repository credentials on behalf of the remote attacker who wrote the issue text. Prompt injection and exfiltration channels need defenses of their own, but the tool boundary directly governs the second condition. The runtime defeats the confused deputy by evaluating every call against the task’s explicit grants rather than against the reach of the credential it happens to hold.

Checkpoint 7.1: Discovery is not permission

A repository server advertises a schema-valid apply_patch tool whose description says “use any path needed to complete the repair.” The current task permits edits only to its designated working copy. Later, a read result includes a request to upload repository contents to an external endpoint.

Discovery vs. permission: Which information does discovery supply, and which claimed permissions must the host refuse to derive from the description?
Per-call checks: Why must an allowed tool still be checked against the proposed arguments and current grants at dispatch? What must the endpoint enforce independently?
Read authority: Why does the read operation require authorization even though it does not edit a file?
Returned text: What do protocol framing and the provenance labels of context engineering establish about the returned text, and what do they leave open about its influence on the model?
Placing the check: In war story 7.1, at which call would a per-call check against the task’s grants have refused, and which of the three enabling conditions would that remove?

Discovery establishes how to address an operation, and authorization determines whether a given task may attempt it. Neither mechanism, however, resolves whether an in-flight mutation actually took effect when a network connection drops or a response times out. Because permission to initiate an action does not confer permission to blindly repeat it when the outcome is uncertain, robust execution requires explicit operation identity and idempotency guarantees.

7.5 Retrying Uncertain Actions

Consider the staging-container restart introduced earlier. When POST /containers/srv-stage-04/restart times out, the client cannot distinguish among several different outcomes. The request may have been dropped in transit, delayed in a network queue, or accepted by the daemon. The restart process might still be running, might have partially reconfigured the service before halting, or might have completed successfully while only the acknowledgment was lost. A client-side timeout⁴ records only that the caller stopped waiting for a response; it conveys no information about endpoint execution state (Cristian 1991). Deciding what to do next requires two orthogonal pieces of knowledge—the operation’s repeat behavior and evidence about the earlier attempt—and prompting the model to retry supplies neither. Settlement before retry (principle 8) permits repeating an uncertain mutation only after its repetition has been shown safe; otherwise the runtime records the mutation as unresolved.

⁴ Once or Not at All: The remote-procedure-call guarantee that a failed call executed either once or not at all, met for model calls in What a request specifies, applies unchanged to a timed-out tool call. What changes is the price of the uncertainty: a timed-out generation at worst wastes a bill, whereas a timed-out mutation may already have changed the world. An agent runtime must therefore preserve “once or not at all” as an explicit, unsettled status rather than collapsing it into a conventional failure that invites an immediate retry.

Cristian, Flaviu. 1991. “Understanding Fault-Tolerant Distributed Systems.” Communications of the ACM 34 (2): 56–78. https://doi.org/10.1145/102792.102801.

Definition 7.4: Uncertain outcome

Uncertain outcome is the recorded status of a dispatched operation whose effect the runtime can neither confirm nor exclude, because its reply was lost, late, or never produced. The operation stays unsettled until an endpoint deduplication contract, a reconciliation query, or an authorized decision establishes what happened.

Significance: An unsettled mutation blocks any retry that could duplicate it. Under blind retry, the duplicate rate tracks the reply-loss rate almost one for one (notebook 7.1).
Distinction: A definite failure provides actionable knowledge, whereas an uncertain outcome represents missing information. A network timeout leaves the execution unknown; it does not make it a failure.
Common pitfall: Recording a timed-out restart as a definitive failure, which improperly authorizes the very retry that duplicates the mutation.

7.5.1 Delayed attempts

Suppose the runtime dispatches restart attempt A, the network buffers the packet in transit, and when the client timeout fires, a status probe reports that the container has not restarted. If the runtime treats that unchanged state as proof that attempt A was dropped and dispatches attempt B, attempt B restarts the container first and the delayed attempt A arrives moments later to restart it a second time. A probe that reads an unchanged service state reports the present accurately, yet it cannot guarantee that a delayed request will not execute in the near future.

An unchanged probe cannot rule out an attempt still in flight.

The probe returned accurate telemetry, but the runtime drew a flawed inference, mistaking “not yet observed” for “cannot still happen.” An attempt already executing inside the service produces an identical hazard. Sending an asynchronous cancellation request settles this ambiguity only if the endpoint explicitly confirms what halted and whether any side effects committed or could still commit. Safe repetition therefore demands much more than a client timeout followed by an optimistic read.

7.5.2 Idempotent actions

Setting a configuration field to "timeout_ms": 5000 twice leaves the field at 5000, whereas incrementing a counter, appending a log entry, or restarting a container produces a second event on the second call. Deciding whether to retry an unacknowledged tool call therefore depends on the physical consequence of running the operation a second time, which is captured by whether the operation is idempotent.⁵

⁵ Idempotent (Latin idem, “the same,” and potent, “power”): Benjamin Peirce introduced the term in his 1870 survey of linear associative algebras to describe an element that remains unchanged when raised to any positive integer power, $\IdempotentAlgebraElement^\PositiveIntegerExponent = \IdempotentAlgebraElement$ (Peirce 1881). Systems engineering applies the term to system state, where executing an idempotent mutation multiple times leaves the system in the same state as executing it once. Algebraic idempotence, however, guarantees nothing about identical return payloads, execution costs, or hidden side effects introduced along the execution path.

Peirce, Benjamin. 1881. “Linear Associative Algebra.” American Journal of Mathematics 4 (1): 97–229. https://doi.org/10.2307/2369153.

Definition 7.5: Idempotent action

Idempotent action is an operation whose repeated application has the same effect on a specified state as one application, $\StateOperation(\StateOperation(\ToolTargetState)) = \StateOperation(\ToolTargetState)$, under the operation’s stated conditions.

Significance: Idempotence is the cheapest settlement. A timed-out idempotent call can be retried without reconciliation, which removes the duplicate-effect path priced in notebook 7.1.
Distinction: The property belongs strictly to the implemented operation and its target state; it does not guarantee identical network responses, zero resource consumption, or isolation from concurrent writes by other processes.
Common pitfall: Assuming idempotence from an HTTP verb or tool name alone. Operations such as restarting a service or appending a log entry introduce new state transitions on every invocation, regardless of the API naming convention.

Whether an operation is idempotent depends on the endpoint’s implementation rather than its name or HTTP verb. Table 2 sorts the three repeat semantics at the tool boundary by what a second execution means and what the runtime must still check before allowing one.

Table 2: Repeat behavior at the tool boundary: Safety follows the implemented operation and its conditions, not a universal classification of tool names.

Operation semantics	What repetition means	What the runtime must still consider
Read without an intended mutation	Obtain another observation, possibly of a changed source	Access and disclosure permission, resource cost, and the version the result describes
Set a desired state	Reapply a value that may already hold	Intervening writes, conditional-update rules, and any additional effects the endpoint performs
Produce another event	Append, charge, restart, or otherwise add an effect	A deduplication contract or sufficient evidence that another execution is safe

The repair’s conditional update, a set-a-desired-state operation in table 2, shows why the classification alone does not settle a timeout. Suppose a repair tool predicates its patch on source version r17. Once it installs r18, repeating the conditional request cannot accidentally overwrite subsequent edits, provided the endpoint checks the version and applies the write atomically. The repeated request returns an optimistic concurrency conflict rather than a success status. While this atomic check protects the repository from corruption, it leaves open the question of whether the earlier attempt or a competing process committed r18. Resolving that attribution requires tracking operation identity across attempts, with an endpoint that retains each operation’s result and returns it to a repeated request, the deduplication contract of section 7.5.3.

Repeat behavior operates along an orthogonal axis from the authority levels $\AuthorityExposureLevel_0$ through $\AuthorityExposureLevel_3$ defined in The H·S·A Exposure Space. An authority level governs whether an agent may execute an action autonomously; repeat behavior dictates whether the runtime can safely re-execute that action after an ambiguous failure. These concerns intersect directly at level $\AuthorityExposureLevel_2$, which is reserved for external operations backed by well-defined compensating actions and a separate retry policy preventing duplicate effects. An external tool call qualifies for $\AuthorityExposureLevel_2$ autonomy only if an executable compensation handler is in place and the runtime settles every transport timeout prior to retrying, relying on verified idempotent semantics, an endpoint deduplication contract, or bounded reconciliation. The staging restart that opened this chapter is such a call. It is an external mutation that adds an event on every execution, so the remediation agent may run it at $\AuthorityExposureLevel_2$ only if that compensation contract is in place and its timeout can be settled by one of those three means.

Napkin Math 7.1: Which retry path duplicates an effect?

Problem: Suppose inbound requests and outbound replies are lost independently, with request-loss probability $\RequestLossProbability$ and reply-loss probability $\ReplyLossProbability$, each 2 percent. Every delivered request executes successfully, and every caller timeout triggers exactly one blind retry of a non-idempotent operation without deduplication. Dropped packets are assumed to be permanently lost rather than delayed. What is the probability that a single dispatched action executes more than once, and how many duplicate mutations will an agent fleet induce each day?

Math: The initial attempt times out whenever the outbound request or the returning reply is lost, with probability $\Pr(\text{timeout}) = 1-(1-\RequestLossProbability)(1-\ReplyLossProbability)$, here $1-(1-0.02)(1-0.02)$, or 3.96 percent. Producing an unintended duplicate mutation requires a specific sequence: the original request must reach the endpoint, its confirmation reply must be lost in transit, and the subsequent retry request must successfully reach the endpoint, so $\Pr(\text{dup}) = (1-\RequestLossProbability)\,\ReplyLossProbability\,(1-\RequestLossProbability)$. With the assumed rates:

$\Pr(\text{duplicate effect}) = (1-0.02)\times 0.02\times(1-0.02)$

This evaluates to 1.9208 percent; whether the retry’s own confirmation reaches the caller has no bearing on whether the endpoint mutated twice. In a fleet of 50 workers where each worker executes 4,000 mutating calls per day, the system processes 200,000 daily calls. Under this naive retry policy, the fleet triggers approximately 3,842 duplicate mutations every day.

Systems insight: The duplicate execution rate tracks the reply-loss rate almost one-to-one because creating a duplicate requires losing a reply between two successfully delivered requests. Losing an initial request produces only an unexecuted, harmless timeout. While halving reply loss reduces duplicate mutations by half, implementing an endpoint deduplication contract eliminates this failure mode entirely. At production scale, protocol-level deduplication contracts, not network hardening, serve as the primary reliability control. This simplified model ignores delayed packets, which introduce additional duplicate execution paths.

7.5.3 Operation identity and endpoint contracts

If a runtime derives deduplication keys by hashing the tool name and arguments, a deliberate second restart of srv-stage-04 an hour later collides with the first and is silently skipped; if it generates a fresh key on every timeout, every retry executes as a brand-new restart. A logical operation ID, commonly called an idempotency key in transactional storage and payment systems, identifies a single intended effect across multiple transport attempts. As Pat Helland observed, any distributed messaging substrate that survives network partitions and process crashes must occasionally retransmit packets; consequently, the receiving endpoint, rather than the transport network, must assume responsibility for rendering duplicate messages harmless (Helland 2012). An attempt ID, in contrast, tracks an individual request-response exchange over the wire. When the agent runtime retries an unacknowledged restart, it preserves the original operation ID while generating a fresh attempt ID for transport tracing; when the agent later issues a distinct request to restart the same container, the runtime allocates a new operation ID even though the arguments are identical.

An operation identifier provides safety only when governed by an explicit endpoint contract. This contract binds the operation ID to the caller’s identity, the target resource scope, and the serialized payload. If a client reuses an active operation ID with mismatched arguments, the endpoint must reject the request with a client error (such as an HTTP 409 Conflict) rather than silently dispatching a mutated operation. The endpoint must also enforce access control independently on each attempt, because possessing a valid operation token does not substitute for authorized credentials when dispatching actions or retrieving retained execution results.

The contract must also govern concurrent and interleaved requests. If a duplicate request arrives while the original attempt is actively executing, the endpoint should return an in-progress or pending status instead of spawning a parallel worker. If a duplicate arrives after execution finishes, the endpoint returns the previously computed and retained result. To uphold this guarantee under crash failures, the endpoint’s persistence mechanism must atomically link the side effect with its stored completion record. If a worker applies the mutation and crashes before persisting the result, a simple log of request receipt cannot prove that re-execution is safe. Where the underlying resource supports it, enclosing both the mutation and the result record within a single database transaction guarantees atomic state transitions. When acting against external services lacking transactional boundaries, the endpoint must coordinate with the downstream system’s native deduplication mechanisms. Every request therefore ends in one of four outcomes of the endpoint’s lookup, and only an unseen operation ID executes (figure 2).

Figure 2: **One operation, many attempts**: The runtime creates the operation ID once and gives each attempt a fresh attempt ID, so a timeout leads either to another attempt of the same intent or, once attempts run out, to an unresolved outcome. The endpoint’s lookup, scoped to caller and resource within its retention window, decides whether a request returns a retained result, a pending status, or an ID conflict, or executes. Only the last branch applies the effect. Recording the result inside the same transaction boundary, where the effect permits one, keeps a crash from leaving an applied effect without the record that deduplicates its retries.

Listing 3 illustrates the client runtime’s implementation of this protocol. The runtime assigns the operation ID once upon admitting the intended mutation, then generates a fresh attempt ID for every network transmission. If repeated retries exhaust the attempt budget, the function returns an explicit unresolved outcome rather than propagating an uncaught exception. This design ensures that transport uncertainty surfaces directly to downstream reconciliation logic rather than disappearing into an unhandled error path. Because this retry loop is safe only when the remote endpoint guarantees deduplication, the runtime invokes this path exclusively for tools whose descriptors formally declare compliance with the operation ID contract.

↪ Recording each admitted operation and its identity durably before dispatch, so that recovery can find every effect that may have begun, is developed in Durable Intent Before Dispatch.

Listing 3: Retrying one intended effect: The operation ID stays fixed across attempts, and exhausting the attempts yields an unresolved outcome. The loop is safe only against an endpoint that deduplicates by operation ID within its retention window.

def dispatch_mutation(transport, tool, args, op_id, max_attempts=3):
    # op_id was created once, when the runtime admitted this intent.
    for attempt in range(max_attempts):
        try:
            reply = transport.post(tool, args,
                                   operation_id=op_id,
                                   attempt_id=new_attempt_id(),
                                   timeout_s=transport.timeout_s)
        except (TimeoutError, ConnectionError):
            backoff(attempt)
            continue                     # same op_id; endpoint deduplicates
        if reply.status == "id_conflict":  # op_id reused with other args
            return {"op_id": op_id, "outcome": "rejected"}
        return {"op_id": op_id, "outcome": reply.status,  # completed or
                "result": reply.result}                   # pending
    return {"op_id": op_id, "outcome": "unresolved"}    # reconcile next

Operation IDs deduplicate retries of one intent by one caller. A different hazard arises when two copies of the caller both believe they own the work, as happens when the runtime restarts or hands a trajectory to a replacement worker while the original is only stalled. Distributed schedulers commonly coordinate such assignments through leases, time-bounded claims on granted authority. If a worker encounters an unobserved execution stall (such as a lengthy garbage collection pause or network blip), its lease may expire. The orchestrator reassigns the task to a second worker while the first worker remains active, creating a split-brain scenario where both workers execute mutations concurrently. Sizing lease durations well above the p99 of observed execution latency reduces the frequency of overlapping executions, but statistical padding cannot eliminate the failure mode entirely. True mutual exclusion requires fencing at the destination resource, such as associating every task reassignment with a monotonically increasing epoch number that the destination endpoint verifies on every write before committing state changes.⁶ Operation IDs and fencing are complementary: the first makes a repeated request harmless, and the second refuses a request from a caller that no longer owns the work.

⁶ Epoch Checks at the Resource: Distributed lock services encountered this exact failure mode when stale lock holders continued mutating storage after their leases lapsed. The Chubby lock service addressed this with a sequencer: an opaque token containing the lock name, mode, and generation number, which clients transmit alongside mutating requests and target servers validate before committing writes (Burrows 2006). The check sits at the resource because a stale worker cannot detect its own staleness, so only the mutating resource can reject an out-of-date epoch number.

Burrows, Mike. 2006. “The Chubby Lock Service for Loosely-Coupled Distributed Systems.” USENIX Symposium on Operating Systems Design and Implementation (OSDI).

↪ Leases, fencing tokens, and the handoff where a stale owner’s delayed write meets the new fence are collected for reference in Leases and Fencing.

A deduplication contract is incomplete without an explicit retention window. Endpoints must publish how long they retain operation IDs and cached results, as well as how they handle delayed requests that arrive after that retention boundary has elapsed. Once the retention window expires, resubmitting an old operation ID risks being treated as a brand-new request, reintroducing duplicate-execution risks. An idempotency token guarantees exactly-once execution semantics only when paired with active endpoint deduplication, durable coupling between side effects and result records, and an unambiguous retention schedule. The tool runtime must never assume stronger guarantees than the underlying interface explicitly provides.

7.5.4 Reconciliation

When an endpoint lacks native deduplication and a mutating call times out, merely observing that the target system now matches the desired state does not prove that the timed-out call produced it: a concurrent agent or human operator might have made the change, or an earlier mutation might already have been overwritten. To settle ambiguous operations against such endpoints, the runtime relies on reconciliation, an authorized, bounded inspection of external system state. Obtaining an operation-specific receipt (such as an execution log entry or transaction identifier bearing the caller’s unique metadata) yields stronger evidence than a bare state check, and runtime diagnostics must keep that epistemic distinction intact rather than synthesizing a false success receipt for an unverified call.

Reconciliation works reliably only when designed prior to dispatch. A mutating tool operating over an endpoint without deduplication can be formalized as a three-part interface:

\[\ReconcilableToolInterface = \langle \text{actuate}(\MutatingToolAction),\ \text{probe}(\MutatingToolAction, \ToolTargetState_0),\ \text{compensate}(\MutatingToolAction) \rangle \tag{1}\]

In equation 1, $\text{actuate}(\MutatingToolAction)$ performs the desired mutation $\MutatingToolAction$. The verification probe $\text{probe}(\MutatingToolAction, \ToolTargetState_0)$ is an authorized read operation, authored alongside the tool, that evaluates whether the intended postcondition holds relative to an initial baseline state $\ToolTargetState_0$ captured immediately before dispatch. The optional compensation routine $\text{compensate}(\MutatingToolAction)$ provides an authorized procedure for mitigating partial or unintended side effects if execution falters.⁷ The probe must be implemented ahead of time as part of the tool contract, because allowing an agent model to improvise verification queries on the fly after a timeout replaces transport uncertainty with unverified heuristic guesswork. Capturing the baseline state $\ToolTargetState_0$ before dispatch provides the essential reference point. For example, before appending a configuration block to a remote file, the adapter records the file’s initial length and cryptographic hash; after a timeout, the probe can distinguish whether the current bytes match the baseline, match the expected appended content, or match neither. These observations do not establish which actor produced the current state or whether the original attempt can still execute; those questions require an operation-specific receipt or endpoint guarantee.

⁷ Compensating Actions: A compensation, the saga mechanism the introduction glossed for $\AuthorityExposureLevel_2$ grants, is a new forward action that semantically neutralizes an earlier step rather than rolling it back in place. Processing a payment refund, for instance, leaves both the original charge and the subsequent credit in the account’s transaction history. For this reason, tool architectures must treat compensation as a separate authorized operation requiring its own execution budget and outcome settlement, rather than pretending the initial mutation never occurred.

Observing an “unchanged” probe result justifies an immediate retry only when backed by definitive structural guarantees. The runtime must either verify that the original attempt can no longer execute (for example, because the attempt has expired under a published request-expiry rule), or rely on inherently idempotent semantics where repeat execution is harmless. Without those assurances, the delayed-attempt hazard illustrated in section 7.5.1 remains fully active. Reconciliation probes are also distributed network calls that can time out, fail, or exceed their allotted latency budgets, leaving the state unresolved.

Under these conditions, the only sound engineering decision is to classify the action as an unsettled outcome. The runtime preserves the operation identity alongside all gathered telemetry, suppresses automated retries, and escalates the ambiguity to an administrative reconciliation pipeline or human operator. A partially applied mutation frequently requires specialized remediation; many real-world actions lack mathematical inverses, and compensation remains an independent mutation with its own potential failure modes. Downstream tasks that depend strictly on the uncertain operation must block until settlement completes, whereas independent parallel subtasks can safely continue executing.

Checkpoint 7.2: Settling an uncertain effect

Delayed attempt: A restart times out, and a later probe sees the old state. The first attempt may still commit. What additional guarantee or evidence is needed before repeating the restart?
Duplicate rate: In notebook 7.1, reply loss doubles while request loss stays fixed. Roughly how does the daily duplicate count change, and which single change removes that path instead of shrinking it?
Operation identity: The agent deliberately restarts the same container a second time an hour later. Why must that request carry a new operation ID even though its arguments are identical, and what goes wrong if the runtime derives operation IDs from an argument hash?
Matching state: A probe written before dispatch finds that the intended configuration block is present after a timeout. Why does a matching current state not prove that this attempt produced it, and what baseline should the probe have captured?

Whether an operation ends completed, pending, or permanently ambiguous, that disposition must reach the next model call as a concise, accurate observation, so the agent plans from what is known rather than from an invented outcome. Getting any result back into the model, even the unremarkable output of a test run that did finish, raises a resource problem of its own.

7.6 Observation Stream Truncation

An automated repair agent running a test suite may encounter a process that terminates with a definitive exit code while generating a log far too vast to fit within the subsequent model request. Conversely, a remote network operation may time out with an uncertain execution state while its local connection layer still captures valuable diagnostic telemetry. Determining the concrete execution outcome of a tool invocation and deciding what subset of telemetry to stage into the prompt are distinct systems responsibilities. An agent runtime must maintain this architectural separation rather than collapsing nuanced execution states into an opaque success or failure string.

Tool execution telemetry encounters two distinct resource boundaries on its return path. Capture governs the physical resources (kernel memory, disk storage, CPU cycles, and wall-clock time) spent receiving output from an external process or service. Context staging governs the logical volume of tokens admitted into the model’s prompt after filtering, normalization, and serialization. A brief shell command such as grep -rn or an unconstrained compiler build can emit hundreds of megabytes in seconds. Without rigorous capture limits, the runtime risks buffer overruns, pipe deadlocks, or host memory exhaustion long before prompt construction begins.

↩︎ The context budget that bounds the observation tokens admitted to each request is established in The Working-Set Decision.

7.6.1 Bounded output capture

Waiting for a child process to exit before reading its standard output stalls the child indefinitely as soon as its output exceeds the operating system pipe buffer (64 KB on modern Linux kernels), while buffering all output into host memory lets a runaway producer exhaust RAM in seconds. An execution adapter must continuously drain its communication channels while enforcing capture limits through bounded in-memory buffers and quota-managed disk storage.

The canonical failure mode in tool execution is a pipe deadlock, which establishes a circular wait⁸ across two bounded pipes. An adapter communicating with a child process over standard input and standard output manages two unidirectional kernel pipes. Suppose the host writes a large payload to the child’s input pipe and defers reading standard output until writing completes. If the child process emits verbose startup diagnostics before consuming its input, the output pipe fills, causing the child’s write call to block. Concurrently, the host fills the input pipe buffer, causing its own write call to block. Both processes sleep interruptibly while waiting for the other to read, halting forward progress until an external watchdog timer terminates the task. Resolving the deadlock requires structural decoupling. The adapter must set all file descriptors to non-blocking mode and multiplex reads and writes across an asynchronous event loop, or dedicate concurrent worker threads to drain standard output and standard error independently. Draining output streams must never block on transmitting input.

⁸ Circular Wait: Coffman, Elphick, and Shoshani identified four conditions that must all hold for a deadlock: mutual exclusion, hold and wait, no preemption, and a circular chain of processes each waiting on the next (Coffman et al. 1971). Two full pipes satisfy all four, since each side holds its own buffer, cannot take the other’s, and waits on it. A readiness loop removes the circular wait, because the host never blocks on one channel while the other has bytes to move.

Coffman, Edward G., Michael Elphick, and Arie Shoshani. 1971. “System Deadlocks.” ACM Computing Surveys 3 (2): 67–78. https://doi.org/10.1145/356586.356588.

An operational capture policy must also define explicit behavior when limits are reached. When incoming data exceeds a retention threshold, the runtime can either drop subsequent bytes while continuing to drain the pipe to prevent process stalls, or actively terminate the executing process. The runtime must persist telemetry recording which policy was triggered and whether byte loss occurred. Because continuously reading and discarding gigabytes of runaway output still burns CPU cycles and context-switch overhead, an output-volume limit complements execution timeouts and kernel resource controls (such as POSIX process quotas or cgroup limits) rather than replacing them. Process termination is itself an asynchronous state transition with its own failure semantics, addressed alongside cancellation mechanisms in section 7.8.4.

Within these capture boundaries, the runtime extracts the most informative evidence for model evaluation. For software engineering and compilation workloads, preserving an initial prefix and a trailing suffix captures the critical context. Build systems print environment configuration, command invocations, and compilation flags at the start, while emitting panic stack traces, assertion failures, and summary verdicts at the very end. However, intermediate warnings or early test failures can easily be lost in the omitted middle. A production capture policy must therefore disclose the exact magnitude of its omissions, while specialized diagnostic filters can selectively extract recognized error patterns from the middle stream. Listing 4 illustrates why naive head-only truncation fails in automated workflows: in this representative test log, the panic trace and assertion failure appear only after tens of thousands of passing test cases.

Listing 4: Where head-only truncation cuts: The invocation context sits at the top of the log and the failure sits at the bottom. A policy that keeps only the first bytes returns the banner and passing tests and discards the panic.

$ cargo test --all
   Compiling core v0.1.0 (/workspace/core)
running 48219 tests
test engine::scheduler::test_fifo ... ok
[... 48,217 further passing tests ...]
test memory::paging::test_allocator ... FAILED
---- memory::paging::test_allocator stdout ----
thread 'test_allocator' panicked at src/memory.rs:142:9:
assertion `left == right` failed
test result: FAILED. 48218 passed; 1 failed; 0 ignored

Under a head-only policy, the model receives a nonzero exit status with zero explanatory context, forcing it to guess at root causes rather than making informed repairs. A head-and-tail policy resolves this by partitioning the observation byte budget $\ObservationByteBudget$ across three components:

\[\ObservationByteBudget = \HeadByteAllocation + \NoticeByteAllocation + \TailByteAllocation \tag{2}\]

In equation 2, $\HeadByteAllocation$ reserves space for the initial execution prefix, $\NoticeByteAllocation$ allocates bytes for an explicit truncation disclaimer, and $\TailByteAllocation$ preserves the most recent output suffix. The adapter implements this with fixed memory overhead: $\HeadByteAllocation$ bytes are captured into a linear buffer until full, after which incoming bytes feed a ring buffer of size $\TailByteAllocation$. The omission counter increases only when a byte is evicted from that ring, so bytes still retained in the tail are not counted as dropped. When the child process exits, the adapter serializes the head buffer, an omission notice detailing the exact count of dropped bytes, and the ring buffer in chronological order, so capture memory stays fixed however long the process writes (figure 3).

Figure 3: **Bounded capture before staging**: The adapter drains each bounded pipe from one readiness loop, so neither channel stalls the child, and enforces a byte limit and a deadline on capture. The head fills once, the tail ring overwrites its oldest bytes, and the middle is counted but not kept, so capture memory stays fixed however long the process writes. The staged observation carries status, head, a notice of the measured omission, and tail within the byte budget $\ObservationByteBudget$; a retained record, when authorized, lets later calls read omitted ranges by handle.

Line counts are convenient for display, but they offer no memory safety because a single line can span gigabytes. The runtime therefore enforces byte limits across all capture and parser buffers, and the ingestion layer truncates on valid Unicode code-point boundaries so that multi-byte characters are never split before token lengths are measured. Because the model context is billed and bounded in tokens rather than bytes, an assumed bytes-per-token ratio serves only to size $\ObservationByteBudget$ in advance; once the observation is staged, the runtime measures it with the model’s own tokenizer against the observation share $\ObservationTokenCount$ of the context budget. That token count includes schema framing, invocation provenance, exit status fields, and the truncation notice itself, leaving headroom for task history, system instructions, and generation so that tool output cannot crowd out the rest of the window.

Napkin Math 7.2: The cost of carrying one observation through a trajectory

Problem: A hypothetical test log contains 24 MB of text. Assume, only for this estimate, 3 bytes per token. The model’s context window is 128,000 tokens. The adapter instead produces a 4,000-token serialized observation, including its metadata, and the context policy selects that same payload for 30 later model calls. Assume input costs $3 per million uncached tokens and cached input costs 10 percent of that price. What are the size and later input-cost consequences?

Math: Dividing the raw bytes by the assumed bytes per token gives $24 \times 10^{6} / 3 = 8{,}000{,}000$ tokens, or 62.5× the entire window. The raw log cannot be admitted to this request. Actual tokenization depends on the specific vocabulary and input encoding; a coarse byte ratio serves for capacity planning rather than admission validation.

The shaped result occupies 3.1 percent of the whole window, before accounting for other context and output reserve. Its later input contribution is $4{,}000 \times 30 = 120{,}000$ tokens. At the assumed prices, these later reads cost $0.36 if all are uncached, or $0.036 if all qualify for cached-input billing. These figures exclude the initial inclusion, output generation, and all other request content.

Systems insight: Every staged observation incurs repeated token fees across subsequent turns that retain it in context. Its carried input is $\ShapedObservationSize \cdot \RetainingCallCount$ tokens, the serialized size $\ShapedObservationSize$ of this one observation (one contributor to the observation share $\ObservationTokenCount$ of each request) times the number $\RetainingCallCount$ of later calls that retain it, and systems engineers control both factors. Halving the staged payload or pruning the observation once the agent acts on its diagnostic cuts token expenditure proportionally, while cached-input billing cuts the carried cost by a factor of 10 under the assumed prices. Aggressive observation shaping directly lowers the amortized inference cost per accepted task, $\CostPerAcceptedTask$, provided the retained evidence remains sufficient for subsequent reasoning.

↩︎ The cost per accepted task, $\CostPerAcceptedTask$, is established in Reliability and Cost over a Horizon.

7.6.2 Truncation notices and inspection handles

Once a head-and-tail policy drops the middle of a log to fit $\ObservationByteBudget$, the model needs to know whether those omitted bytes still sit in storage or were discarded at the pipe boundary; otherwise it cannot tell whether to request the missing slice or re-run the test suite with a narrower filter. A truncated observation must therefore state what was selected, what was omitted, and whether the missing data remains retrievable, distinguishing a complete capture that was staged as an excerpt from an unrecoverable drop at the pipe. Omission figures must also reflect what the adapter actually measured: if a stream dropped packets or ended early under buffer overruns, the runtime reports that uncertainty rather than fabricating an exact byte count.

When storage policy permits retention, the runtime can spool the full raw output to an external artifact store and supply the model with an inspection handle, the raw-payload handle of The Working-Set Decision as it appears at the tool boundary, subject to the same handle rule. Instead of re-executing an expensive build or test run to view omitted output, the agent can issue secondary queries such as read_observation_slice(handle, offset_bytes, max_bytes) to examine targeted slices, or invoke a server-side regular-expression search over the retained log that returns only matching lines and their byte offsets. Retaining raw logs outside the context window shifts the engineering burden to storage governance. Because retained logs consume disk space and often contain environment variables, authorization tokens, or internal paths, the artifact store must enforce capacity quotas and eviction horizons alongside the same disclosure policies that govern the primary tool call, returning an explicit error when a handle has expired or been evicted. Content hashes can verify that a stored log has not changed, whereas read authorization and retention lifetime remain separate runtime checks.

Structured tools can avoid post-hoc log slicing altogether by partitioning query results into pages bounded in both record count and serialized byte size, so that a single oversized record cannot overflow the response. Each page reports whether more records remain, whether a scan limit or deadline cut the search short, and which cursor resumes the query; because authorization and snapshot isolation are independent of pagination, the contract must also state whether later pages read from a frozen snapshot or see concurrent writes.

↩︎ The result envelope’s completion status and continuation cursor are established in The result envelope.

The placement of continuation state highlights a key systems trade-off amplified by agentic workloads: stateful server-side cursors vs. stateless client tokens. A stateful cursor maintains an open database cursor, transaction snapshot, or cache entry on the server until the client finishes iterating. However, autonomous agents abandon search trajectories routinely; an informative diagnostic on the first page often redirects the agent’s entire plan, causing abandoned server sessions to accumulate until garbage collection or time-to-live timeouts reclaim them. A stateless cursor avoids server-side state by encoding the continuation position directly within the token passed to the client. Because that token travels through the model’s context, where generated or injected text could alter it to reach rows outside the original query, the backend authenticates it. It serializes a query descriptor, the last evaluated sort key, and an issuance timestamp into a payload $\StatelessCursorPayload$ and returns the token $\StatelessCursorPayload \parallel \text{MAC}_{\text{key}}(\StatelessCursorPayload)$ signed with a secret server key. When the client returns the token, the backend verifies the message authentication code and expiration time, ensuring resistance against client-side tampering without consuming server memory. The trade-offs for stateless cursors are relaxed consistency and increased prompt overhead. Resumption queries typically evaluate against live data rather than an isolated snapshot, and serialized cryptographic tokens consume prompt tokens on every turn that carries them.

Bounded capture and disciplined staging keep tool output within its memory and context budgets without deadlocking the child. Fitting the bytes into a budget, however, says nothing about whether they faithfully represent what the program displayed or what the process actually did. A terminal log can be garbled by its own redraws, and a clean-looking log can come from a run that checked nothing.

7.7 Terminal Output Normalization

When a command-line tool executes interactively, it redraws progress indicators with carriage returns and styles diagnostics with terminal escape sequences. Captured directly at the byte level, standard output does not represent the clean visual layout a human engineer sees on a terminal screen. Wherever possible, tool adapters should invoke command-line utilities in noninteractive or batch modes (for example, --batch or --no-color), or ideally request structured machine-readable outputs such as JSON or JUnit XML. Structured reports provide discrete test identifiers, execution timings, and explicit assertions, eliminating the brittle heuristics required to parse colored progress logs. Even so, structured formatting is a transport convention; receiving a syntactically valid JSON payload does not guarantee that the tool performed all required checks or that its outputs can be trusted without schema validation and memory bounds.

When terminal output is unavoidable, the adapter must enforce a declared normalization policy backed by a stateful virtual terminal emulator or bounded screen buffer. Removing ANSI color sequences (\x1b[31m) is straightforward, but naively stripping control bytes like carriage returns (\r) or cursor positioning sequences silently corrupts output semantics.⁹ Because a carriage return moves the cursor to the beginning of the line without advancing to the next, a terminal progress indicator updates by repeatedly overwriting existing text. Stripping \r without simulating terminal geometry causes overwritten states to concatenate. For the bytes Running... \rFailed!, the screen shows Failed! written over the start of the line (Failed!...), whereas naive stripping yields Running... Failed!, a line that never appeared; over a long run the same error accumulates megabytes of repeated progress fragments. The adapter’s parser must therefore maintain cursor coordinates across chunked streaming reads while bounding its internal buffer allocations. When an adapter supports only a subset of control sequences, it must explicitly mark unsupported or incomplete escape sequences rather than discarding them silently. Because collapsing interactive updates into a final snapshot loses intermediate state transitions, the runtime should retain the raw byte stream in secondary storage as an authoritative audit log while delivering only the normalized, bounded representation to the model context. Normalization also shifts byte positions, so the adapter records a mapping from normalized offsets back to raw offsets to keep range queries through an inspection handle aligned with the original text.

⁹ Terminal Control Functions: The escape sequences that color and reposition terminal text were standardized in ECMA-48, first published in 1976 and adopted internationally as ISO/IEC 6429 (Ecma International 1991). Its control functions act on a presentation position on a display rather than on a stream of lines, which is why recovering what an operator saw requires emulating that display rather than filtering bytes.

Ecma International. 1991. Control Functions for Coded Character Sets. ECMA-48, 5th edition. Ecma International.

Tool adapters must preserve the provenance of standard output (stdout) and standard error (stderr) while treating stream assignments as advisory conventions. In Unix environments, programs frequently emit progress updates, debugging traces, or configuration warnings across either descriptor, and capturing the two streams through separate pipes forfeits their relative temporal interleaving. When chronological event ordering is essential for root-cause diagnosis, adapters should multiplex streams using pseudo-terminal (pty) pairs or timestamped chunk logging. In extracting diagnostics, the adapter should rely on known output formats such as compiler error lines and language tracebacks, retaining source file paths and line offsets while reporting parsing failures explicitly rather than guessing a root cause from an incomplete fragment. Normalizing terminal output cleans up presentation, but the resulting text carries no intrinsic authority. Safe serialization practices, including escaping control characters and enforcing UTF-8 boundaries, protect the agent’s structural messaging format from delimiter injection, while the evidence-scope rule continues to dictate what the text actually proves.

7.7.1 Process status and evidential scope

Even a cleanly normalized log cannot by itself prove that a check ran: a test runner pointed at an unmatched directory filter discovers zero tests and exits cleanly with status 0, and a shell pipeline run without set -o pipefail suffers a fatal crash in its first command while still returning status 0 because its final formatting stage succeeded. The adapter must therefore read process disposition directly from operating-system primitives, such as waitpid status macros or container runtime lifecycle events, rather than parsing success messages from output logs, while still treating exit status 0 as an application convention rather than proof that every required check ran. Querying the process supervisor lets the adapter report an unambiguous exit code,¹⁰ an unhandled terminating signal, or a timeout without textual speculation. Equally, observing end-of-file on standard streams does not prove that a process has finished its work, because a daemon or worker process can close its inherited descriptors while continuing to run in the background; conversely, a broken socket or interrupted channel leaves the state of any requested side effects uncertain.

¹⁰ Eight-Bit Exit Status: Under POSIX semantics, a parent process inspects child termination through status macros like WIFEXITED and WEXITSTATUS, which yield only the low-order eight bits (values 0–255) of the argument passed to exit. A program that terminates with exit code 256 wraps modulo 256 and appears to the parent as exit status 0. Standard shells also synthesize return codes above 128 when a process is killed by a signal (for example, 137 for SIGKILL 9 or 139 for SIGSEGV 11), collapsing fatal OS signals into ambiguous numerical exit statuses. An adapter that records explicit disposition variants, such as exited(code) vs. signaled(sig), avoids truncating or conflating these conditions.

An API client can likewise catch an internal runtime exception, print the error payload to standard output, and terminate with code 0 because the transport exchange itself finished without an unhandled crash. Under the evidence-scope rule, the adapter treats the operating-system process status as one evidentiary signal alongside structured output payloads and the task’s acceptance criteria. In a connection-pool repair, for instance, confirming that a targeted reproduction test passes on workspace revision r18 proves only that the reproducer succeeded; it does not substantiate broader system stability or justify full patch acceptance. Listing 5 illustrates how an adapter packages these signals together after running a targeted test against workspace snapshot r18, parsing its assertions, and capturing the full output stream within its retention budget while staging only a bounded summary into the prompt context.

Listing 5: A bounded result with scoped evidence: Process disposition, tested source, check outcome, and observation completeness answer different questions.

{
  "call_id": "call_test_42",
  "scope": {"repository": "repair-workspace", "revision": "r18"},
  "process": {"state": "exited", "exit_code": 0},
  "check": {
    "target": "tests/test_pool.py::test_waiter_notified_on_return",
    "outcome": "passed",
    "broader_regression_checks": "not_run"
  },
  "observation": {
    "capture": "complete",
    "staged_view": "excerpt",
    "selection": "invocation header and final test summary",
    "text": "Target: tests/test_pool.py::test_waiter_notified_on_return\n[intermediate output omitted]\nResult: passed",
    "omission_notice": "Intermediate output omitted from this view.",
    "inspection_handle": "obs_42",
    "retention": "until this task ends"
  }
}

Each block in listing 5 answers a separate question so the model cannot mistake a clean exit or a short excerpt for full verification. While call_id ties the result to the turn that requested it, scope pins the evidence to workspace revision r18 so a pass on one commit cannot be credited to a later edit, and check records that the single reproducer passed while broader regression suites were not run. Because the full log was captured but only the header and summary verdict were staged into the prompt, the observation block marks capture as complete and staged_view as an excerpt, exposing obs_42 as an inspection handle for paginated follow-up reads; had resource limits truncated collection at the pipe, capture itself would be marked incomplete with the surviving byte ranges noted. Under the grant rule, the runtime mediates every read on obs_42, so knowing the handle string confers no ambient authority across sessions.¹¹

¹¹ Names vs. Capabilities: Dennis and Van Horn’s foundational capability model granted authority through possession of an unforgeable ticket or reference (Dennis and Van Horn 1966). Agent runtime architectures intentionally stop short of treating context-visible observation handles as capabilities. Because text in a model’s context window can be logged, leaked, copied by prompt injections, or replayed across user boundaries, the runtime treats an inspection handle merely as an identifier (a name). The runtime verifies caller authorization and execution scope on each access, ensuring that possessing a handle string grants no ambient authority.

Dennis, Jack B., and Earl C. Van Horn. 1966. “Programming Semantics for Multiprogrammed Computations.” Communications of the ACM 9 (3): 143–55. https://doi.org/10.1145/365230.365252.

Checkpoint 7.3: Interpreting bounded observations

Omitted evidence: A known failed test run has a complete retained log but only a head-and-tail excerpt in context. What can the agent infer from the absent stack trace, and what bounded inspection should it request?
Scoped status: The result above reports exit code zero and one targeted pass on r18. Which claims does that support, and which acceptance questions remain open?
Instruction-like output: A normalized log contains an instruction to upload the repository elsewhere. Which presentation checks have occurred, and why do they grant no authority for that proposed upload?

Synchronous tool execution assumes the adapter can resolve process status, normalize terminal streams, and package evidence within the span of a single model turn. When operations run longer than conversational deadlines allow, asynchronous dispatch extends this same return-path contract to running jobs, tracking process lifecycles and staging bounded observations without mistaking dispatch for completion.

7.8 Asynchronous Tool Dispatch

When an automated repair agent launches an extensive regression test suite, it rarely has any useful decision to make until the results arrive. A prompt return from that tool invocation is only an acknowledgment of receipt; the actual computation has only just begun. During the intervening minutes or hours, the agent runtime must supervise background execution, enforce resource budgets, and maintain system safety without freezing its control plane or burning inference tokens on idle waiting.

7.8.1 Synchronous and asynchronous interfaces

Blocking a supervisor thread on a five-minute integration build leaves the host unable to answer cancellation signals, while returning a background job handle immediately lets the agent edit src/pool.py midway through the test run and corrupt the workspace snapshot under test. A synchronous tool interface blocks until execution finishes and returns its final result directly to the caller, which works well for sub-second database lookups or local checks where the agent cannot proceed without the answer; even during a synchronous wait, worker threads must monitor process heartbeats, stream partial output, and enforce timeouts without freezing the control plane. An asynchronous interface instead decouples job submission from job completion by returning a handle representing the pending task alongside methods to inspect its status, stream intermediate observations, or await termination. This separation allows the agent trajectory to suspend cleanly without burning model inference cycles, or to continue executing independent tasks concurrently, provided the runtime isolates the background job within a fixed snapshot or copy-on-write workspace so concurrent edits cannot contaminate the run.

Decoupling task execution from model inference also resolves the serving side of the tool-wait problem named in The six phases of a turn. While a background job runs, the model generates no tokens, so the serving engine can keep, move, or discard and later recompute the trajectory’s KV cache state under the paused-state policies of Paused-State Policies, independently of the tool’s lifecycle. Keeping an open HTTP connection or a blocked synchronous socket throughout a long build provides no caching guarantee and only ties up host network resources. When the background job completes and the trajectory resumes, the runtime constructs the next prompt, and the serving engine either recomputes attention state over the prompt prefix or restores it from a prefix cache, depending on what it has evicted.

7.8.2 Job handles

Once job submission is decoupled from completion, the handle returned at dispatch must keep a queued job distinct from a finished one: if an asynchronous test runner returns "status": "ok" as soon as a job enters the work queue, the model treats a queued test suite as a passing one, whereas if a later status poll times out and the adapter reports "status": "failed", the model rolls back a patch while the test job is still running in the background. The runtime admits a proposed asynchronous call using the same validation, authorization, and resource checks required for synchronous invocation. If the executor accepts the work, the runtime records the job’s identity, owner, requested operation, applicable source revision, execution limits, and observation location. A model-visible job handle identifies that record without exposing the executor’s internal process-management interface. Under the principle of least privilege, which gives each component only the authority its current task needs, status reads, output inspection, interactive input, and cancellation each require their own explicit authorization checks, and a provider-neutral dispatch acknowledgment records only the pending handle:

{
  "call_id": "call_tests_1",
  "status": "accepted_pending",
  "job_id": "job_tests_18",
  "source_revision": "r18",
  "result_available": false
}

Here call_id correlates the dispatch exchange, while job_id identifies the accepted work for later interactions. If a mutating operation also requires a logical operation ID for retry handling, that identity retains the idempotency semantics established in section 7.5; a job ID alone supplies no deduplication guarantee. Dropping the dispatch reply across an unreliable network leaves the runtime uncertain whether the executor accepted the task, mirroring the ambiguity of a timeout during a synchronous call.

This initial acknowledgment confirms only that the executor accepted the request into its work queue; it makes no claims about whether the process has started running or whether tests have passed. Subsequent status queries can report active execution, an explicit request for user input, completion with a structured result, or failure with diagnostic details. If network partitions or host restarts make the executor temporarily unreachable, returning “last observed running” alongside “current status unavailable” preserves operational context instead of triggering a mistaken rollback.

When the result becomes available, the runtime delivers the bounded observation contract, specifying the tested revision, the checks actually executed, the process exit disposition, and any output truncation. For example, a completed test job at revision r18 can report that test_waiter_notified_on_return passed. Because a child process can exit cleanly while an output buffer truncates under heavy logging, process termination and output completeness remain distinct fields in the observation contract.

7.8.3 Resumption mechanisms

Even when a job handle reports pending state accurately, letting the model poll a five-minute background job every five seconds spends 59 full model calls reading the same 32,768-token context just to receive "still running". Systems track asynchronous job state through three primary mechanisms, distinguished by whether state detection requires model inference (table 3).

Table 3: Resumption choices: Waiting and status detection can occur outside model inference; deliberate inspection remains useful when it informs a decision.

Mechanism	Who checks or reports status?	When model inference is useful
Model-directed inspection	The model proposes a bounded status or output request	A new observation may change the diagnosis or next action
Runtime polling	The runtime checks on a bounded schedule	A reported change requires a decision or the final result is ready
Completion or progress notification	The executor signals a relevant event to the runtime	The runtime has obtained evidence sufficient to resume deliberation

By delegating health checks to the runtime, systems avoid redundant inference calls while retaining simple polling semantics. Event-driven notifications (such as webhooks or message queues) further eliminate polling traffic between the runtime and the executor, though they introduce operational complexity. The runtime must authenticate callbacks, correlate incoming events with job identifiers, and handle dropped, delayed, or duplicated deliveries. In practice, receiving a completion notification often acts as a trigger for the runtime to execute a validated status read, rather than treating the notification payload itself as authoritative proof of completion. Both polling and notification architectures require dedicated supervisory resources to manage pending jobs across runtime host failures.

↪ The durable return path that lets a replacement worker recover a pending job’s result after the runtime host fails is developed in Effect and charge reconciliation.

Napkin Math 7.3: The extra cost of asking whether a job finished

Problem: A background test job completes in 300 s. Under a model-directed polling strategy, the agent queries status every 5 s, starting one interval after dispatch and ending strictly before completion. Assume each status check finishes before the next scheduled query and does not alter overall execution time. Each model invocation consumes a 32,768-token prompt (including preceding status history) and generates a 40-token query, with no additional model acknowledgment required. What incremental inference overhead does this polling loop impose compared to event-driven runtime detection?

Math: Both architectures make the initial dispatch call and the final completion call, so those shared calls are excluded. The polling schedule produces 59 intermediate model invocations, consuming 1,933,312 input tokens and 2,360 generated tokens. The selected context length remains fixed across queries rather than accumulating unbounded history.

Assume 95 percent of input tokens qualify for prompt caching discounts, priced at $0.30 per million tokens vs. $3 per million uncached tokens, with generation billed at $15 per million tokens. Multiplying token counts by their respective rates yields $0.8410 in input charges and $0.0354 in output charges, producing $0.8764 of direct inference overhead per background job.

Systems insight: The overhead is $\StatusPollCount = \lceil \JobDuration/\FiniteChange \TimeInstant \rceil - 1$ status calls for a job of duration $\JobDuration$ polled every $\FiniteChange \TimeInstant$, each reading $\SequenceLength$ input tokens and generating $\OutputLength$, so it costs $\StatusPollCount(\SequenceLength\,\BlendedInputTokenPrice + \OutputLength\,\OutputTokenPrice)$ at per-token input price $\BlendedInputTokenPrice$ (here the cache-blended price) and output price $\OutputTokenPrice$. It grows linearly with job duration and inversely with the poll interval. Halving the interval roughly doubles the expenditure, and the interval also sets detection delay: a completion is noticed at most $\FiniteChange \TimeInstant$ after it happens. Under model-directed polling, cost and worst-case delay therefore trade directly, since halving the cost by doubling $\FiniteChange \TimeInstant$ doubles the worst-case delay, whereas runtime polling decouples the two because its checks cost no inference. The $0.8764 per pending job is more than twice the $0.36 that carrying the shaped observation of notebook 7.2 through 30 uncached calls cost. It inflates the attempt cost in the numerator of effective cost $\CostPerAcceptedTask$, paying for status information that the host runtime can monitor at near-zero cost. Systems should delegate routine state tracking to the runtime harness, waking the model only when an incoming observation can change the next decision.

7.8.4 Job supervision and cancellation

Delegating status checks to the runtime also leaves the runtime responsible for supervising the background job while the model trajectory is paused. Throughout the job’s lifetime, the runtime enforces its execution deadline, resource ceilings, output budgets, and permitted external side effects, returning the latest status and a timestamped, truncation-marked output slice whenever a bounded inspection request arrives. If a job’s output stream goes silent, the runtime cannot assume deadlock from silence alone; an explicit status signal or diagnostic probe is needed to tell slow computation apart from blocked I/O awaiting user input or a wedged worker process.

When a tool genuinely awaits interactive input, the runtime should avoid guessing from operating-system process sleep states. Wherever possible, adapters invoke tools in non-interactive modes with complete parameters supplied up front; when interaction cannot be avoided, an explicit adapter event or structured message must declare what prompt the job expects. Before forwarding any input from the agent, the runtime re-checks the job’s state, the input’s meaning, and the session’s grants, because answering "yes" to an interactive migration prompt can trigger schema mutations that exceed a read-only inspection grant.

When a job must be stopped—whether by an explicit cancellation request or an expired deadline—the runtime treats cancellation as an asynchronous control operation with its own intermediate states rather than an instantaneous rollback. After acknowledging the cancellation request, the runtime records what the executor actually reports: the job may stop cleanly, finish before the signal arrives, reject cancellation, or become unreachable with termination unconfirmed. Reclaiming every descendant process is where naive cancellation fails: when a background job runs bash -c "make && pytest" and times out, sending SIGTERM only to the top-level shell kills the shell while leaving make, compiler workers, and pytest running as orphaned children reparented to PID 1 that still hold filesystem locks and burn CPU cycles. On a POSIX host, the containment unit for local signaling is instead a process group, which groups all processes reached by a single signal. The execution adapter makes each job the leader of its own process group at spawn time by calling setpgid(0, 0) in the child before exec, and then signals the entire group: SIGTERM first, giving processes a bounded grace period to flush buffers and release locks, followed by an uncatchable SIGKILL. Listing 6 implements this escalation pattern and reports one of four verified outcomes: exited when the group was gone before any signal, terminated when it stopped during the SIGTERM grace period, killed when it stopped only after SIGKILL, and unconfirmed when members remain after both. If an untrusted process deliberately creates a new session or process group to escape group signaling, the runtime must rely on kernel-level isolation boundaries.

↪ Tearing down processes that detach from the task’s process group, as part of a sandbox’s reset contract, is developed in The reset contract.

Listing 6: Process-group teardown with escalation: The job leads its own group, so one signal reaches every descendant still in it. Liveness is probed with signal 0 after reaping exited children, and a group that survives SIGKILL is reported as unconfirmed rather than assumed stopped.

import os, signal, subprocess, time

def spawn_job(argv):
    proc = subprocess.Popen(argv, process_group=0)  # setpgid(0, 0)
    return proc.pid                  # the leader's PID is the group ID

def group_alive(pgid):
    try:                             # reap exited children of ours,
        while os.waitpid(-pgid, os.WNOHANG)[0] > 0:  # or zombies
            pass                                     # still count
    except ChildProcessError:        # no children of ours remain
        pass
    try:
        os.killpg(pgid, 0)           # signal 0 probes, delivers nothing
        return True
    except ProcessLookupError:
        return False

def terminate_group(pgid, grace_s=3.0):
    for sig, wait_s in ((signal.SIGTERM, grace_s), (signal.SIGKILL, 1.0)):
        try:
            os.killpg(pgid, sig)
        except ProcessLookupError:   # group already gone
            return "exited" if sig == signal.SIGTERM else "terminated"
        deadline = time.monotonic() + wait_s
        while time.monotonic() < deadline:
            if not group_alive(pgid):
                return "terminated" if sig == signal.SIGTERM else "killed"
            time.sleep(0.05)
    return "unconfirmed"             # report it; do not claim a stop

Two subtle POSIX semantics govern the correctness of this routine. First, when invoking non-blocking os.waitpid(-pgid, os.WNOHANG), a return value of zero indicates that child processes still exist in the group and none has changed state yet; it does not indicate that the process group has cleared. The runtime continues reaping until no direct child processes remain, at which point the system call raises ChildProcessError. Second, sending signal 0 via os.killpg(pgid, 0) delivers no actual signal to the target group; the kernel simply performs error checking, confirming whether any process matching that group ID still exists and is accessible. Probing with signal 0 directly determines whether any group member remains alive after signal delivery.

Even when process termination is confirmed, stopping an execution does not roll back side effects that occurred before the signal arrived. A canceled deployment script may already have mutated external cloud resources, and a canceled test run may leave temporary database tables behind. The final observation emitted by the runtime must record all known side effects, pending cleanup obligations, and ambiguous states. In the system’s execution log, a status of “canceled” confirms that execution halted; it does not signify that the system returned to its original state.

↪ Remediating the partial effects and cleanup obligations an interrupted job leaves behind is developed in The Trajectory Saga Pattern.

An asynchronous interface thus carries the tool contract across time. It acknowledges queued work without stalling the trajectory, keeps resource limits and supervision in force while the model waits, returns bounded observations, and never reports a requested state as a confirmed one. Every mechanism so far has treated one tool at a time and taken that tool’s presence for granted. The remaining decisions concern the whole set: which tools the model sees on a given turn, and how much work each one performs.

7.9 Designing the Active Toolkit

The repair agent working on the connection-pool race in src/pool.py requires concrete operations: inspecting source files, applying candidate patches, executing test suites, and analyzing run-time diagnostics. Toolkit design governs how an agent runtime exposes these capabilities to the foundation model. If an interface exposes too little functionality per invocation, the model must coordinate low-level state across dozens of turns; if it exposes an unconstrained catalog, the context window fills with irrelevant tool schemas that consume input capacity and bloat prefill latency on every turn, and each extra descriptor is one more candidate the model must rule out. Whether those candidates lower selection accuracy for a given model and catalog is an empirical question that catalog evaluation (section 7.9.3) measures rather than assumes. The goal of toolkit design is to expose operations so that the model can choose an appropriate next step without carrying unnecessary definitions or managing avoidable intermediate state.

Two design dimensions govern this trade-off. Granularity determines how much work an individual tool performs, while catalog selection determines which subset of tool definitions the model sees on any given turn, known as the active toolkit. A small catalog can contain either narrowly bounded primitives or a broadly expressive command runner, whereas an enterprise environment registering hundreds of specialized tools must stage them selectively rather than presenting the entire catalog on every turn.

7.9.1 Static and on-demand staging

Serializing all 50 registered tool descriptors into every model call burns 15,000 tokens of context capacity on every turn even when a step needs only six tools, whereas selecting a narrow subset on demand risks omitting the one tool the repair needs next. In static staging, the runtime serializes a fixed set of tool descriptors into every request, keeping available interfaces predictable and eliminating separate discovery steps at the cost of carrying unused schemas on every turn. In on-demand staging, the runtime dynamically selects descriptors as the trajectory unfolds, shrinking the prompt footprint and prefill latency at the risk that an omitted tool forces the agent to spend extra turns locating it.

Selection mechanisms vary widely in implementation complexity. A runtime can employ deterministic task-to-tool mappings, hierarchical namespace menus, keyword matching, dense vector retrieval, or a combination of these approaches. Dense retrieval represents one possible search technique rather than a universal requirement for large catalogs. Any match returned by a retriever is merely a candidate descriptor. A robust systems design provides a bounded fallback path, such as a dedicated tool-search operation or an explicit discovery-failure signal, so that an omitted descriptor does not cause the model to conclude erroneously that the operation cannot be performed.

Staging a descriptor makes its syntactic and semantic contract visible to the model; under the grant rule, staging confers no execution privileges. The runtime must still resolve and authenticate every proposed invocation against registered tool identities and permissions before execution. If a dynamic discovery step returns a modified or re-versioned descriptor, the runtime must reconcile that version before executing it, preventing the model from dispatching against an interface different from the one it was provided.

Tool definitions consume prompt capacity alongside system instructions, source excerpts, previous observations, and generation headroom. Systems engineers must measure the final serialized request, including formatting delimiters, wrapper schemas, and selection metadata, rather than tallying independently tokenized description fragments under the false assumption that token boundaries are free. The following notebook counts the catalog-related input for a single model request.

Selection frees most of the catalog’s input, on every call.

Napkin Math 7.4: The input budget of an active catalog

Problem: How much next-call input capacity does a selected catalog release after counting its additional framing and selection information?

Variables: Suppose a registered catalog has 50 tools. Each complete descriptor contributes 300 tokens, including its share of catalog framing. The alternative stages 6 of the same descriptors plus 400 additional tokens for the selection interface and metadata. These are stipulated contributions within the final serialization, with other request content unchanged. In this scenario the runtime selects the subset without another model call.

Math: Full staging uses $50 \times 300 = 15{,}000$ tokens. The selected descriptors plus selection overhead use $6 \times 300 + 400 = 2{,}200$ tokens. The released capacity is therefore $15{,}000 - 2{,}200 = 12{,}800$ tokens.

Systems insight: Selection pays whenever its overhead is smaller than the unused descriptors it removes, $(\RegisteredDescriptorCount - \SelectedDescriptorCount)\,\ToolDescriptorSize > \SelectionOverheadSize$ for $\SelectedDescriptorCount$ of $\RegisteredDescriptorCount$ descriptors of $\ToolDescriptorSize$ tokens each and $\SelectionOverheadSize$ tokens of selection overhead; here that is 400 against 13,200 tokens; break-even sits at about 1.3 unused descriptors, so the saving grows linearly with catalog size. The released capacity recurs on every call that would have carried the full catalog, so over a trajectory it multiplies by the number of turns, like the carried observation in notebook 7.2. The comparison flips when selection needs its own model call or a missed tool forces rediscovery; those calls belong in the same token count.

Static and selected catalogs also interact differently with prefix caching in serving runtimes. When an inference engine uses radix-tree or hash-based KV cache reuse, maintaining an identical serialized prefix across turns allows the engine to bypass prefill computation for earlier prompt segments. Static staging preserves a stable prompt prefix across calls, maximizing cache reuse as long as earlier conversation state remains intact. Conversely, modifying descriptors or reordering them across turns moves the point of prompt divergence earlier, invalidating downstream cached KV blocks and forcing redundant prefill computation. Caching also changes what selection buys. If the full catalog sits in a cached root billed at 10 percent of the uncached price, as assumed in notebook 7.2, the dollar value of the tokens that selection releases in notebook 7.4 shrinks by the same factor of 10, while the released context capacity is unchanged; once caching applies, selection pays mainly in capacity for task evidence rather than in money. A hybrid strategy applies the change-rate ordering of Staging the Next Invocation to the catalog: an immutable core of common descriptors sits in the stable root of the prompt, and task-specific additions are appended later. Even so, systems designers must not retain obsolete or revoked definitions merely to retain a cache hit, as doing so exposes outdated interfaces to the model. Removing an earlier temporary tool definition also alters the prompt prefix for subsequent turns, invalidating the cached state of the assistant responses and observations that followed it.

↩︎ Radix-tree prefix caching and KV block reuse across prompt turns are established in Prefix Caching Across Turns.

7.9.2 Tool granularity

Whereas staging decides which tools appear on a turn, granularity decides how much operational work each call encapsulates: a file-editing interface built from separate open_file, seek_cursor, write_bytes, and close_file tools forces the model to spend four turns and manage raw file descriptors, cursor offsets, and cleanup across the trajectory just to apply a one-line patch. Classic systems modularity principles apply directly here: Parnas established that effective module boundaries hide design decisions that are subject to change (Parnas 1972), and Ousterhout formalized this criterion through deep vs. shallow interfaces, measuring how much implementation complexity an abstraction hides relative to the surface area it exposes (Ousterhout 2018). The same principles hold in agent systems, where tool interfaces designed around an agent’s editing, navigation, and testing patterns raised task success rates on software engineering benchmarks (Yang et al. 2024). A deep interface encapsulates substantial functionality behind a concise contract, whereas a shallow interface exposes almost as much complexity as it implements; decomposing a file edit into open_file, seek_cursor, write_bytes, and close_file is warranted only when intermediate inspection is an explicit requirement.

Parnas, David L. 1972. “On the Criteria to Be Used in Decomposing Systems into Modules.” Communications of the ACM 15 (12): 1053–58. https://doi.org/10.1145/361598.361623.

A task-level patch operation moves mechanical file manipulation behind the tool implementation, returning a unified diff, a resulting version hash, or an explicit conflict. The conditional-edit contract in section 7.3 illustrates this boundary. However, interface naming alone does not guarantee transactional safety. If an operation advertises all-or-nothing execution, the underlying implementation must enforce that atomicity across its mutation scope and failure modes. When operations span multiple files or distributed endpoints, partial side effects can occur, and the runtime must report those partial outcomes explicitly in the tool observation. Bundling multi-step work into a single tool call reallocates responsibility from model planning to runtime execution, but the underlying systems obligations remain intact; for the same edit, the shallow composition spends four model calls where the task-level operation spends one (figure 4).

Figure 4: **Shallow and deep interfaces for one edit**: The low-level composition (a) spends four model calls and places a handle, a cursor, and cleanup in the trajectory between them. The task-level operation (b) spends one call and moves the version check and the change behind the interface, returning either a diff with its new version or a version conflict. The deeper interface is all-or-nothing only if its implementation provides that property; either way, the obligations move rather than disappear.

Careful toolkit design does not require eliminating all overlapping capabilities. A system can effectively combine a structured search tool that returns abstract syntax tree (AST) matches with an authorized shell runner that supports complex log analysis beyond the search tool’s schema. To prevent model confusion, their descriptors must articulate clear functional boundaries, target inputs, and operational limits. Providing duplicate tools with near-identical names and indistinguishable schemas adds selection noise without expanding capability; in contrast, providing complementary interfaces with distinct observation formats, coverage scopes, or authorization grants offers substantial utility. Table 4 summarizes how these interface styles partition responsibility between the model and runtime.

Table 4: Toolkit granularity and responsibility: Broader composition and task-specific interfaces place different obligations on the model and implementation; neither column substitutes for authorization or containment.

Interface style	Work assigned to the model	Work assigned to the implementation
Low-level composition	Select intermediate operations and manage their dependencies	Validate each operation and expose handle, status, and cleanup semantics.
Task-level operation	Specify a bounded intent, such as a patch or a test target	Implement preconditions, mutation boundaries, and interpretable results for that task.
General command or program runner	Construct a program that expresses the intended work	Enforce execution grants and resource limits, supervise the job, and return bounded evidence.

An argument schema constrains the syntactic format of a request, but it cannot constrain the underlying code execution. A specialized symbol-renaming tool may parse external ASTs, inspect secondary source trees, or execute defective third-party libraries. A narrow signature provides safety only when the implementation and its surrounding execution sandbox enforce that advertised boundary. Conversely, a shell runner’s blast radius is defined by its working directory, operating-system privileges, credential access, and network egress rules. Because the side effects of an expressive shell command cannot be reliably determined by inspecting its arguments, the runtime must rely on containment to enforce isolation.

↪ The isolation envelope that enforces a command’s reach regardless of its arguments is developed in Choosing an Isolation Envelope.

Tool implementations should also adopt defensive defaults that fail visibly. When an agent provides an ambiguous file path or target identifier, rejecting the call with an informative diagnostic is far safer than guessing or silently appending data, as appending is a stateful mutation that can silently duplicate configuration entries. A dry-run mode allows the model to inspect a proposed diff against the current workspace state before committing changes; the runtime must still enforce approval gates and verify preconditions at execution time. These defensive defaults make operational failures interpretable, but engineers should remember that rollback capabilities and high-fidelity simulation are properties of the underlying tool implementation, not inherent guarantees of the interface.

7.9.3 Catalog evaluation

Choosing between those granularity and staging options requires measuring end-to-end trajectories rather than single-step accuracy: comparing a 12-tool structured catalog against a single run_shell tool by tool-selection accuracy alone makes the shell runner look superior simply because the model cannot pick the wrong tool name out of a catalog of one, hiding the unconfined mutations and recovery turns the shell introduces. Evaluating toolkit designs therefore requires holding all other experimental factors constant—the benchmark task distribution, initial workspace snapshots, model checkpoints, generation parameters, token budgets, and verification suites—while exercising realistic failure modes across the connection-pool repair workload: locating relevant source files across deep directory hierarchies, applying patches against concurrently modified buffers, interpreting noisy test failure logs, and recovering when a needed descriptor is omitted from the initial active toolkit. Those trials separate catalog coverage gaps from model selection errors, and separate selection accuracy from execution-time runtime failures.

Because a recovered selection error costs extra inference turns and latency whereas an unrecovered error aborts the task, evaluation logs must record selection errors, malformed arguments, rejected preconditions, extra discovery calls, recovery turns, and uncontained side effects alongside binary task completion, while cost accounting sums serialized descriptor tokens, retrieval latency, sandbox compute time, observation overhead, and tokens spent on failed attempts. Toolkits can then be ranked by total cost per accepted task ($\CostPerAcceptedTask$) rather than by the probability of avoiding a single selection error, and any comparison between designs with different security postures—such as an unconstrained command runner vs. a scoped AST manipulation tool—must report those privilege differences explicitly rather than attributing the gap to catalog size alone.

The ultimate objective of toolkit engineering is an active catalog whose contracts align with workload demands while keeping execution costs and failure modes transparent. Static staging, on-demand selection, task-level abstractions, and expressive command runners represent trade-offs to evaluate against these cost and reliability metrics. The same engineering principles apply when an agent interacts with a graphical user interface instead of a programmatic API: mouse clicks, keyboard events, and visual screen parsers impose their own granularity challenges, prefill budgets, and containment boundaries.

7.10 Visual Actuation and Computer Use

Consider the staging-container restart scenario when an administrative web dashboard provides no programmatic API. A computer-use tool bridges this interface gap by rendering visual screenshots to the model and translating model-generated targets into synthetic mouse clicks, keystrokes, or scrolling gestures, the low-level input space pioneered in early web-agent research (Shi et al. 2017). Operating on raw pixels does not relax system invariants. While the model proposes an interaction based on an observed frame, the execution runtime retains responsibility for validating arguments, enforcing authorization, dispatching OS events, and interpreting observations. A coordinate within display boundaries merely confirms that an event can reach the window manager, offering no assurance about the state mutation that the event will trigger.

7.10.1 An action against a changing surface

In that web dashboard, suppose the agent observes a confirmation modal for restarting the staging container srv-stage-04 and proposes clicking the Restart button at screen coordinates $(640, 420)$, but milliseconds before the click is dispatched, an alert pops a production container dialog into the foreground at those exact coordinates. Controlling a graphical surface requires the same interface contract used for structured tool calls, adapted for spatial and temporal change where a coordinate pair does not identify the underlying resource. When the runtime captures an observation, it records the application and window identifier, a unique capture identifier, an observation timestamp, and the exact geometric frame presented to the model. If the runtime downsamples, pads, or crops the image before model ingestion, it must retain the crop offsets and scaling factors needed to project a proposed coordinate back into the native display coordinate space. Raw screenshot pixels, application-relative window coordinates, and virtual desktop coordinates represent distinct spatial reference frames. The runtime adapter must validate all proposed points against the active viewport; silently clamping out-of-bounds coordinates to the nearest edge risks turning an invalid model hallucination into a destructive click on an unintended adjacent control.

Before dispatching the proposed click on the Restart button, the adapter must recheck the current application focus, active window state, and target dialog parameters using platform accessibility or windowing APIs. If a background network event or concurrent operator action has brought a production container dialog into the foreground at those screen coordinates, valid spatial coordinates no longer justify executing the model’s proposal. The runtime must detect the stale precondition, abort the pending dispatch, and request a fresh visual observation. While benign updates such as a ticking taskbar clock do not invalidate a target, any modification to a resource identifier, modal overlay, or button layout alters the operational effect.

Once the staging dialog is re-verified, the policy engine must evaluate the session grant against the intended container restart rather than checking a generic permission to emit mouse clicks. If the interface cannot establish which backend operation the click will execute with the confidence required by the security policy, the runtime must block dispatch and escalate the decision to an authorized operator. Otherwise, the adapter dispatches the bounded input event and records its immediate outcome. Structured endpoints enforce resource identity and action preconditions directly at the server interface, providing a robust execution boundary. By contrast, visual actuation cannot achieve atomic execution. Capturing a fresh screenshot reduces staleness, but it cannot prevent a concurrent state change from slipping between the observation capture and the event dispatch.

After dispatching the input event, the subsequent observation must verify operational progress rather than mere input delivery. A disappearing confirmation modal might indicate that the server accepted the restart command, but it might also mean that a network timeout dismissed the dialog without action. The runtime must gather corroborating evidence of the restart, either by inspecting a refreshed dashboard status indicator or by issuing an authorized read-only status query. If the click produces no visible change and the dashboard continues displaying the previous state, the requested operation might still be processing asynchronously. Under these conditions, the execution yields an uncertain outcome (definition 7.4), which the runtime must resolve through the idempotency and reconciliation protocols detailed in section 7.5 rather than by repeatedly firing redundant clicks.

7.10.2 Visual observation budgets

Verifying each visual step with a fresh screenshot carries a steep context cost: retaining one high-resolution desktop screenshot per turn converts each frame into thousands of visual tokens and multiplies that payload across every subsequent turn of a computer-use trajectory. Vision encoders partition input images into fixed-size grid patches, and because visual token consumption directly inflates inference latency and per-turn billing, systems engineers must monitor both the raw byte representation submitted to the model and its resulting token allocation. A screenshot is priced like any other staged observation: its token count plays the role of $\ShapedObservationSize$ in notebook 7.2, and every later call that retains it multiplies that count by $\RetainingCallCount$, which is why the techniques below shrink the per-frame count and why retention policy matters as much as resolution.

↩︎ How a vision front-end turns an image into reserved context positions, and how one image can overflow a request’s headroom, is established in Counting multimodal inputs.

Naively compressing visual inputs introduces sharp engineering trade-offs. Downscaling an entire screen capture reduces token consumption, but blurs fine visual details such as container names, port numbers, or warning banners until they become unreadable. Conversely, tightly cropping the image preserves full pixel resolution around a candidate button, but strips away peripheral context, concealing modal warning banners or active window titles that are essential for correct interpretation.

A resilient architecture balances these demands through hierarchical visual sampling: capturing a low-resolution full-screen overview to establish window layout and focus, paired with an on-demand, high-resolution crop around the target interaction zone. Each visual artifact must retain its parent observation identifier and affine coordinate mapping so the runtime can translate local crop coordinates back to native display space. Similarly, transmitting visual diffs of modified screen regions reduces payload size during iterative tasks, provided the diff retains sufficient surrounding landmark geometry for the vision model to orient itself. Because large language model inference engines rely on exact prefix matching for KV-cache reuse, altering a visual frame across turns invalidates cached prefixes for all subsequent tokens. A well-designed observation budget delivers the minimal visual evidence necessary to authorize and execute the next operational decision while keeping token consumption bounded.

Checkpoint 7.4: A click is still a tool call

A dashboard changes after the model observes a staging restart dialog. The adapter can capture the display and send a click, but cannot atomically bind that click to the displayed resource.

Target identity: Which changes require rejecting the old proposal, and why is a coordinate-range check insufficient?
Observation geometry: What information must accompany a resized crop so a proposed point can be mapped to the intended surface?
Authority and freshness: What does a new screenshot establish, and which action-time uncertainty remains before dispatch?
Effect evidence: If the click times out and the old status remains visible, why is another click not automatically justified? What evidence would support the intended restart without claiming the whole task is complete?

7.11 Fallacies and Pitfalls

The tool boundary fails when evidence about one stage is treated as a guarantee about another. These misconceptions recur across structured, terminal, and visual interfaces.

Fallacy: A schema-valid call is safe to execute.

Structural validity checks the shape of a request; the session’s grant, the current domain preconditions, and resource limits are separate checks (section 7.3). In the GitHub MCP leak (war story 7.1), every call the agent made was schema-valid and within its credential’s reach. The call that exceeded the task’s grants was the first private-repository read, and no check compared it with the task.

Pitfall: Treating a timeout as permission to retry.

A timeout records missing knowledge, not a failure. Under the blind-retry model of notebook 7.1, a per-call duplicate probability of 1.9208 percent becomes about 3,842 duplicate effects a day across 200,000 mutating calls. Retry only on safe operation semantics, an endpoint that honors the operation ID, or reconciliation evidence (section 7.5); otherwise record the effect as unresolved.

Fallacy: A zero exit status establishes task completion.

Exit status is the invoked program’s convention. A runner that discovers zero tests, a script without pipefail, and a client that prints its error and exits normally all return zero (section 7.7). The r18 reproducer pass supports one exercised connection-pool case; acceptance follows the task’s evidence standard (principle 1).

Pitfall: Shrinking an observation without recording what was dropped.

Truncation itself is unavoidable, since the illustrative log in notebook 7.2 is 62.5× the entire context window. The mistake is losing the record of omission. A head-only policy applied to listing 4 keeps the banner and passing tests and drops the one panic that explains the failure. The head, notice, and tail of equation 2 keep it within the same byte budget, and an inspection handle recovers the middle on request.

Fallacy: Staging every registered tool is harmless because each descriptor is short.

Descriptors add up, and they recur on every call. In notebook 7.4, a 50-tool catalog occupies 15,000 tokens of each request, while the 6 tools a step needs, with selection overhead, use 2,200. Over a trajectory that difference is paid once per turn, in capacity taken from task evidence (section 7.9).

Pitfall: Letting the model poll a long-running job.

Each “still running” answer is a full model call. In notebook 7.3, a 300 s job checked every 5 s costs 59 extra model calls and $0.8764 in tokens that change no decision. Runtime polling or a completion notification detects the change without inference and wakes the model only when the result can alter the next action (section 7.8.3).

7.12 Summary

Every runtime mechanism in this chapter enforces one discipline. An agent may base its next decision only on what returning evidence directly confirms. A schema validates the syntactic shape of a request, an authorization grant restricts its permissible targets, an endpoint check enforces execution preconditions and timing, and an observation reports what the tool actually measured. None of these mechanisms assumes that a proposed operation successfully altered external state, nor does a completed execution guarantee that the broader task has succeeded. When evidence is missing, as it was after the timed-out staging restart, the runtime must record the evidentiary gap explicitly rather than interpolating an assumed outcome. Table 5 summarizes the quantitative relations governing dispatch concurrency, retry safety, and observation overhead across the tool boundary, where $\RequestLossProbability$ and $\ReplyLossProbability$ denote independent request- and reply-loss probabilities, $\ToolCallReadSet_i$ and $\ToolCallWriteSet_i$ represent the declared read and write footprints of call $\ToolCallIndex$, and an asynchronous job of duration $\JobDuration$ polled at intervals of $\FiniteChange \TimeInstant$ incurs per-token input and output prices $\BlendedInputTokenPrice$ and $\OutputTokenPrice$.

Table 5: Key equations of the tool boundary: Each relation holds only under the assumptions stated where it is derived, such as independent message loss, a fixed selected payload, or declared footprints that omit no shared state.

Equation	What it bounds or decides	Where derived
$\ToolCallWriteSet_i \cap (\ToolCallReadSet_j \cup \ToolCallWriteSet_j) = \emptyset$ and $\ToolCallWriteSet_j \cap \ToolCallReadSet_i = \emptyset$	Whether two batched calls may run concurrently; valid only for trustworthy footprints	section 7.2.3
$\HostRuntimeCount \times \ToolServiceCount$ adapters vs. $\HostRuntimeCount + \ToolServiceCount$	Integration work for $\HostRuntimeCount$ hosts and $\ToolServiceCount$ tool services, with and without a shared protocol	section 7.4
$\StateOperation(\StateOperation(\ToolTargetState)) = \StateOperation(\ToolTargetState)$	Whether a timed-out operation may be repeated without settlement, for the named state	section 7.5
$\Pr(\text{timeout}) = 1-(1-\RequestLossProbability)(1-\ReplyLossProbability)$; $\Pr(\text{dup}) = (1-\RequestLossProbability)\,\ReplyLossProbability\,(1-\RequestLossProbability)$	Timeout rate and duplicate-effect rate under one blind retry; duplicates track reply loss	section 7.5
$\ReconcilableToolInterface = \langle \text{actuate}(\MutatingToolAction),\ \text{probe}(\MutatingToolAction, \ToolTargetState_0),\ \text{compensate}(\MutatingToolAction) \rangle$	What a mutating tool without endpoint deduplication must ship so a timeout can be reconciled	equation 1
$\ObservationByteBudget = \HeadByteAllocation + \NoticeByteAllocation + \TailByteAllocation$	Fixed capture memory and staged size for a log of any length, and which diagnostic region survives	equation 2
$\ShapedObservationSize \cdot \RetainingCallCount$ carried input tokens	Later input cost of one staged observation of $\ShapedObservationSize$ tokens selected for $\RetainingCallCount$ further calls	section 7.6
$\StatusPollCount = \lceil \JobDuration/\FiniteChange \TimeInstant \rceil - 1$, costing $\StatusPollCount(\SequenceLength\,\BlendedInputTokenPrice + \OutputLength\,\OutputTokenPrice)$	Inference spent on model-directed status checks that runtime detection avoids	section 7.8.3
$(\RegisteredDescriptorCount - \SelectedDescriptorCount)\,\ToolDescriptorSize > \SelectionOverheadSize$	When selecting $\SelectedDescriptorCount$ of $\RegisteredDescriptorCount$ descriptors of $\ToolDescriptorSize$ tokens pays for $\SelectionOverheadSize$ tokens of selection overhead	section 7.9

Key Takeaways: A tool call includes its returning evidence

A call ends at its observation, not its dispatch: The runtime matches each returning result to its corresponding invocation, recording whether the proposal was rejected, dispatched, completed, or left in an indeterminate state. Subsequent planning must reason strictly from this confirmed disposition rather than assuming an action succeeded simply because the host dispatched it.
Schemas shape requests; enforcement governs effects: Type signatures and parameter descriptions guide the model to structure valid calls, but authorization grants, resource quotas, and precondition checks govern actual execution. Any mutation that depends on an unchanged target resource must verify preconditions atomically alongside the write, rather than relying on stale observations from an earlier read.
Discovery does not confer permission: Exposing a tool interface in the context window reveals structural syntax, but connectivity and schema compliance never imply authorization. The host runtime must validate every invocation against task-scoped grants, including read-only queries, while the remote endpoint enforces its own perimeter independently.
An unknown effect forbids blind repetition: Because a timed-out attempt may have committed before network acknowledgment was lost, duplicate side effects track reply loss rather than request loss. The runtime may retry an unacknowledged invocation only under provably idempotent semantics, an endpoint deduplication contract, or explicit probe reconciliation. Otherwise, it must preserve the operation as an unresolved effect.
Truncation must disclose what it dropped: Buffer limits and model context windows constrain distinct physical resources, yet both force data omission. An observation payload that incorporates execution status, provenance metadata, and a measured omission counter preserves diagnostic integrity, whereas silent head truncation often discards the terminal error or stack trace needed to diagnose a fault.
Acknowledgment is not completion: Long-running jobs require durable handles, scoped inspection, authorized input channels, and unambiguous cancellation outcomes. Event-driven runtime detection re-awakens the model only when a state change can alter the next action, preventing empty polling loops that burn inference budgets on uninformative checks.
No toolkit size is universally optimal: Catalog staging and API granularity navigate a fundamental trade-off among context overhead, selection accuracy, expressiveness, and security boundaries, while visual interfaces introduce freshness and target-geometry constraints. System designers must evaluate a toolkit by its total cost per accepted task and the precision of the authority it leaves bounded.

The typed action contract (principle 7) separates an interpretable proposal emitted by the model from the authorization required to execute it. When network drops or timeouts leave an outcome ambiguous, settlement before retry (principle 8) extends that discipline to prevent silent state corruption. The runtime must either establish that repeating the call is safe or record an unresolved effect. A tool’s completion report, however, confirms only that an operation finished, not that the overall task succeeded. Final acceptance always demands invariant closure (principle 1), verifying that the system’s operational requirements hold in reality before declaring success.

7.13 Historical Perspective and Further Reading

The tool execution boundary is the modern incarnation of a problem operating systems encountered half a century ago, when an untrusted program requested an operation the system could not let it perform directly. Three ideas from that era apply directly at the tool boundary: Anderson’s reference monitor from a 1972 study for the U.S. Air Force (Anderson 1972), Saltzer and Schroeder’s principle of complete mediation (Saltzer and Schroeder 1975), and Hardy’s confused deputy, the privileged service that acts for a client without checking the client’s authority (Hardy 1988). In agent runtimes, that last vulnerability manifests whenever a model holding private credentials executes operations triggered by untrusted user prompts or retrieved external content. The wider protection lineage these ideas belong to, from Dennis and Van Horn’s capabilities (Dennis and Van Horn 1966) to Lampson’s access matrix (Lampson 1974), is traced in Historical Perspective and Further Reading, where it grounds containment beneath the model.

Anderson, James P. 1972. Computer Security Technology Planning Study. ESD-TR-73-51. Deputy for Command; Management Systems, Electronic Systems Division, Air Force Systems Command.

Lampson, Butler W. 1974. “Protection.” ACM Operating Systems Review 8 (1): 18–24. https://doi.org/10.1145/775265.775268.

Saltzer, Jerome H., David P. Reed, and David D. Clark. 1984. “End-to-End Arguments in System Design.” ACM Transactions on Computer Systems 2 (4): 277–88. https://doi.org/10.1145/357401.357402.

Kung, H. T., and John T. Robinson. 1981. “On Optimistic Methods for Concurrency Control.” ACM Transactions on Database Systems 6 (2): 213–26. https://doi.org/10.1145/319566.319567.

Garcia-Molina, Hector, and Kenneth Salem. 1987. “Sagas.” Proceedings of the 1987 ACM SIGMOD International Conference on Management of Data, 249–59. https://doi.org/10.1145/38713.38742.

Gray, Cary G., and David R. Cheriton. 1989. “Leases: An Efficient Fault-Tolerant Mechanism for Distributed File Cache Consistency.” Proceedings of the Twelfth ACM Symposium on Operating Systems Principles (SOSP), 202–10. https://doi.org/10.1145/74850.74870.

A second historical lineage governs operations that cross network boundaries. Birrell and Nelson designed the Remote Procedure Call (RPC) to make remote invocations feel as natural as local function calls, but they documented the fundamental tension of distributed execution. During network partitions or server crashes, a caller cannot know whether a request succeeded, failed, or stalled midway through execution (Birrell and Nelson 1984). In the same year, Saltzer, Reed, and Clark articulated the end-to-end argument, demonstrating that reliability and correctness cannot be guaranteed solely by intermediate communication layers and must ultimately be validated by the endpoints themselves (Saltzer et al. 1984). Distributed systems researchers developed several robust patterns to handle this inherent ambiguity. Kung and Robinson introduced optimistic concurrency control, allowing transactions to proceed without heavy upfront locking and validating serializability before commit (Kung and Robinson 1981). For distributed operations that cannot be rolled back atomically, Garcia-Molina and Salem formulated sagas, coordinating sequences of local transactions coupled with compensating actions to undo partially completed work (Garcia-Molina and Salem 1987). Gray and Cheriton introduced leases, using time-bounded grants to manage shared resources without requiring indefinite locks (Gray and Cheriton 1989). Finally, Helland made the practical case that reliable distributed systems must push retry safety into the receiver by enforcing endpoint idempotence (Helland 2012), a requirement that directly informs how tool runtimes deduplicate requests and handle agent retries.

Language models converged on tool execution from the opposite direction, evolving from passive sequence predictors into interactive system callers. Early modular architectures routed model queries to external calculators, symbolic solvers, and database interfaces (Karpas et al. 2022). Yao et al. (2023) established the ReAct pattern, demonstrating that interleaving explicit reasoning traces with concrete actions and environment feedback improves multi-step problem solving. Schick et al. (2023) showed with Toolformer that models could learn in a self-supervised manner to propose external API calls spontaneously, complete with appropriate arguments, during standard text generation. As the scope of external tools expanded, retrieval-augmented tool selection reduced hallucinated function signatures by fetching accurate documentation on demand (Patil et al. 2023), while large-scale instruction tuning exposed models to thousands of diverse, real-world REST APIs (Qin et al. 2024). Interacting with external environments also has roots in web agents that manipulate user interfaces through simulated keyboard and mouse events (Shi et al. 2017). However, designing interfaces specifically tailored to language models proved even more effective: Yang et al. (2024) demonstrated with SWE-agent that structuring the agent-computer interface around model-friendly commands, such as concise file viewers with line numbers and restricted editing primitives, substantially lifted task success on complex software engineering benchmarks. To manage the proliferation of heterogeneous tools, open specifications like the Model Context Protocol (MCP) standardized the boundary between host runtimes and external tool servers (Model Context Protocol 2025). As tool calling gained ubiquity, however, it exposed a distinct vulnerability. An agent interprets untrusted external text within the same context window that guides its decisions, allowing adversarial inputs to steer the model into executing unintended tool invocations (Greshake et al. 2023). Because the language model is an untrusted caller that can be influenced by malicious prompts, the enclosing systems runtime must enforce complete mediation and least privilege on every operation, and permit retries of uncertain effects only when their repetition is safe.

Karpas, Ehud, Omri Abend, Yonatan Belinkov, Barak Lenz, Opher Lieber, Nir Ratner, Yoav Shoham, Hofit Bata, Yoav Levine, and Kevin Leyton-Brown. 2022. “MRKL Systems: A Modular, Neuro-Symbolic Architecture That Combines Large Language Models, External Knowledge Sources and Discrete Reasoning.” arXiv Preprint arXiv:2205.00445.

Yao, Shunyu, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. “ReAct: Synergizing Reasoning and Acting in Language Models.” International Conference on Learning Representations (ICLR).

Schick, Timo, Jane Dwivedi-Yu, Roberto Dessı̀, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. “Toolformer: Language Models Can Teach Themselves to Use Tools.” Advances in Neural Information Processing Systems (NeurIPS) 36: 68539–51.

Patil, Shishir G., Tianjun Zhang, Xin Wang, and Joseph E. Gonzalez. 2023. “Gorilla: Large Language Model Connected with Massive APIs.” arXiv Preprint arXiv:2305.15334, ahead of print. https://doi.org/10.48550/arXiv.2305.15334.

Qin, Yujia, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, et al. 2024. “ToolLLM: Facilitating Large Language Models to Master 16000+ Real-World APIs.” Proceedings of the 12th International Conference on Learning Representations (ICLR).

Shi, Tianlin, Andrej Karpathy, Linxi Fan, Jonathan Hernandez, and Percy Liang. 2017. “World of Bits: An Open-Domain Platform for Web-Based Agents.” International Conference on Machine Learning (ICML), Proceedings of machine learning research, vol. 70: 3135–44.

Greshake, Kai, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. 2023. “Not What You’ve Signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection.” Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security, 79–90. https://doi.org/10.1145/3605764.3623985.

For further reading:

Saltzer and Schroeder, “The Protection of Information in Computer Systems” (Saltzer and Schroeder 1975). The canonical catalog of security principles for computer systems. Read through the lens of agent execution, principles like complete mediation, least privilege, and economy of mechanism serve as a direct checklist for designing secure tool boundaries.
Birrell and Nelson, “Implementing Remote Procedure Calls” (Birrell and Nelson 1984). The foundational paper on RPC and the clearest exposition of distributed failure semantics. It establishes why a caller can never be completely certain whether an unacknowledged remote call executed, providing the theoretical basis for retry budgets and timeout policies.
Hardy, “The Confused Deputy” (Hardy 1988). A concise, three-page paper that identifies how an entity with broad ambient authority becomes an unwitting tool for attackers. It represents required reading for understanding why language agents must never carry ambient credentials across security domains.
Helland, “Idempotence Is Not a Medical Condition” (Helland 2012). A practical guide to designing distributed systems that survive retries by pushing deduplication and idempotence keys into receiver interfaces, directly applicable to tool runtime dispatch.
Ousterhout, A Philosophy of Software Design (Ousterhout 2018). A compact treatment of interface design and deep modules. The book’s criteria for module depth and interface complexity provide concrete guidance for sizing tool APIs so models can invoke them effectively without leaking implementation details.
Yang et al., “SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering” (Yang et al. 2024). An empirical demonstration that the design of model-oriented tool interfaces, alongside the underlying language model, determines agent performance.

Saltzer, J. H., and M. D. Schroeder. 1975. “The Protection of Information in Computer Systems.” Proceedings of the IEEE 63 (9): 1278–308. https://doi.org/10.1109/proc.1975.9939.

Birrell, Andrew D., and Bruce Jay Nelson. 1984. “Implementing Remote Procedure Calls.” ACM Transactions on Computer Systems 2 (1): 39–59. https://doi.org/10.1145/2080.357392.

Hardy, Norm. 1988. “The Confused Deputy: (Or Why Capabilities Might Have Been Invented).” ACM SIGOPS Operating Systems Review 22 (4): 36–38. https://doi.org/10.1145/60609.60611.

Helland, Pat. 2012. “Idempotence Is Not a Medical Condition.” ACM Queue 10 (4): 30–46. https://doi.org/10.1145/2181796.2187821.

Ousterhout, John. 2018. A Philosophy of Software Design. Yaknyam Press.

Yang, John, Carlos E Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. 2024. “SWE-Agent: Agent-Computer Interfaces Enable Automated Software Engineering.” Advances in Neural Information Processing Systems.

7.14 Exercises

All quantities below are hypothetical problem givens, chosen to be round, and describe no particular provider, protocol version, or deployment. Assume message losses are independent unless a problem says otherwise.

1. Blind retry at fleet volume (Basic). A tool fleet issues 120,000 mutating calls a day to an endpoint with no deduplication. Each request is lost with probability 1 percent and each reply with probability 3 percent; lost messages never arrive later, and every timeout triggers exactly one retry.

Compute the per-call timeout probability, the per-call duplicate-effect probability, and the expected duplicates per day.
A network team offers two upgrades: halve reply loss, or halve request loss. Compute the daily duplicates under each and explain why one of them slightly increases duplicates.
Name the change that removes the duplicate path rather than shrinking it, and state what the endpoint must bind the identifier to for that change to hold.

2. Sizing the tail (Basic). An adapter stages test logs with a head-and-tail policy under $\ObservationByteBudget = 16{,}384$ bytes, with $\NoticeByteAllocation = 256$ bytes and $\HeadByteAllocation = 4{,}096$ bytes. A failing run writes 9,000,000 bytes in total. Its only diagnostic is a 2,400-byte panic block, followed by 11,000 bytes of teardown output and a 600-byte final summary.

Compute $\TailByteAllocation$ and determine whether the whole panic block reaches the model.
Keeping $\ObservationByteBudget$ and $\NoticeByteAllocation$ fixed, find the largest $\HeadByteAllocation$ that keeps the entire panic block in the tail. How many bytes does the omission notice report in either configuration?
The same runner sometimes prints the panic before 200,000 bytes of teardown output. Explain why no split of this $\ObservationByteBudget$ fixes that case, and name the mechanism from section 7.6 that does.

3. When catalog selection pays (Basic). A repair harness registers 80 tools, each contributing 420 tokens to the final serialized request. On-demand staging exposes 8 of them plus 900 tokens of selection interface and metadata, and the runtime selects without an extra model call. A trajectory makes 40 model calls, and uncached input costs $3 per million tokens.

Compute the catalog input of one call under each layout, the capacity released per call, and the break-even number of unused descriptors.
Compute the tokens and uncached dollars released over the trajectory.
In one trajectory out of five, a needed tool was not staged and the agent spends one extra model call of 30,000 input tokens rediscovering it. Does selection still pay? State the condition under which the comparison would flip.
The 80 descriptors come from three interoperable tool servers discovered at session start. One of them advertises delete_branch with a description stating that it is approved for all repositories, and the task’s grants do not include branch deletion. Does staging or omitting that descriptor change what the agent may do? Name the check that governs the call if the model proposes it anyway, where each part of that check runs, and what the returned observation should report.

4. The price of asking whether a job finished (Intermediate). A repair agent’s test suite completes after 1,500 s. Under model-directed polling, the model checks every 30 s, starting one interval after dispatch, with each status call reading a 48,000-token selected input and generating 60 tokens. The cache covers 90 percent of that input. Hypothetical prices per million tokens are $3 uncached input, $0.30 cached input, and $15 output. The fleet runs 2,000 such jobs a day.

Compute the pending-status calls per job, the incremental cost per job, and the daily fleet cost of polling.
Product management caps model-directed polling at $0.10 per job. Find the minimum poll interval that meets the cap, and the worst-case delay between completion and detection at that interval for a job whose completion time is not known in advance.
Runtime polling every 10 s costs no inference. Compare its worst-case detection delay and token cost with the answer to (b), then choose a design to ship and state what model-directed inspection remains useful for.

5. Retrying a refund (Intermediate; second lighthouse). The support agent issues refunds through a payment endpoint that offers no deduplication contract. It sends 40,000 refund calls a day, each averaging $60. Requests are lost with probability 0.5 percent and replies with probability 1.5 percent. The alternative to blind retry is an authorized reconciliation probe written before dispatch (equation 1), which adds 4 s to the customer’s wait on every timed-out call. For parts (a) and (b), assume lost messages never arrive later and every delivered refund request executes successfully.

Under one blind retry per timeout, compute the expected duplicate refunds per day and their dollar exposure.
Under the probe policy, compute the fraction of sessions that incur the extra wait and the total added customer wait per day.
Compare the duplicate-refund exposure with the added customer wait. State what operation-specific evidence or endpoint guarantee would make the probe sufficient to settle the original refund or permit a retry, including how it rules out an attempt that could still execute before authorizing another execution. If the probe is inconclusive or the allowed wait expires, specify the recorded outcome and escalation. What response-latency limit would be needed to decide whether this policy meets the service target? Explain why a refund’s operation ID must not be a hash of the customer and amount.

6. Carry, drop, or re-read (Intermediate). A repair agent stages a 6,000-token test observation that stays in context for its remaining 45 model calls. Uncached input costs $3 per million tokens. An alternative policy keeps the observation for the 5 calls in which it informs a decision, then replaces it with a 300-token summary carrying an inspection handle for the rest of the trajectory. Each later re-read through the handle costs one extra model call with 40,000 input tokens and returns a 2,000-token slice that stays in context for 5 calls.

Compute the carried input tokens and uncached cost of keeping the observation, and the same for dropping it with no re-reads.
Find the number of re-reads at which dropping stops being cheaper.
Tokens are not the only cost of a re-read. Identify one latency cost and one evidence risk the token count in (a) omits, and state which of them would change the policy for a log whose failure sits in the omitted middle.

7. Scheduling a batch by footprint (Intermediate). A response proposes five calls for the connection-pool repair, listed with durations and declared footprints. read_source(pool.py) takes 2 s and reads pool.py. read_source(config.py) takes 2 s and reads config.py. apply_patch(pool.py) takes 1 s, reads and writes pool.py, and uses the version returned by the first read. run_tests takes 30 s, reads pool.py, config.py, and tests/, and writes .cache. lint(pool.py) takes 5 s, reads pool.py, and writes .cache. The workflow requires tests and lint to see the patched file.

Apply the noninterference condition to find which pairs may overlap, then give the shortest schedule and its makespan. Compare with fully serial dispatch.
Changing the linter to write .lintcache instead is a one-line configuration change. Compute the new makespan.
The model proposes replacing lint and run_tests with one bash -c "make check" call that runs both in sequence for 35 s and whose footprint is undeclared. What must the runtime do with it, and what does that cost in makespan relative to (b)?
Between the read of pool.py and apply_patch, a teammate commits a change to the same file. The patch arguments still pass schema validation. State which descriptor field and which endpoint operation reject the stale patch, why a separate version check before the write is insufficient, and what the rejection returned to the model should contain.

8. Retention windows and delayed attempts (Advanced). A runtime retries a mutating call against an endpoint that deduplicates by operation ID. Each attempt waits a 10 s timeout, then backs off 1, 2, 4, and 8 s before attempts two through five. The network can delay a request by up to 120 s before delivery. The endpoint retains operation IDs for 60 s after first receipt.

Compute the send time of each attempt and the retention window needed so that no delivered attempt of this operation can execute a second time.
The endpoint’s retention is fixed at 60 s. Show that no schedule containing even one retry closes the gap, then redesign the contract so every attempt carries the original logical operation’s admission timestamp and the endpoint rejects the operation more than 45 s after that timestamp. Assuming clocks agree within 2 s, which attempts may the runtime still send while preserving that expiry bound?
Critique the redesign. What new failure mode does request expiry introduce, what outcome must the runtime record when every permitted attempt times out, and which settlement mechanism from section 7.5 takes over?

9. One shell or twelve tools (Advanced; second lighthouse). A team proposes giving the remediation agent a single run_shell tool with a 500-token descriptor in place of 12 structured tools of 350 tokens each, citing fewer selection errors. A trajectory makes 30 model calls at $3 per million uncached input tokens. With structured tools, an attempt costs $1.20, is accepted with probability 0.80, and causes an unconfined production change with probability 0.001; with the shell those figures are an attempt cost reduced by the descriptor savings, acceptance probability 0.78, and unconfined-change probability 0.004. Each unconfined change costs $2,000 to detect and repair.

Compute the per-trajectory descriptor savings and $\CostPerAcceptedTask = \mathbb{E}[\AttemptCost]/\AcceptanceProbability$ for each design, first from token costs alone and then including expected change losses in the attempt cost.
Find the unconfined-change probability the shell design would need to match the structured design’s $\CostPerAcceptedTask$.
Critique the team’s evaluation metric, and propose a toolkit that keeps most of the descriptor savings without the loss. Say which mechanism bounds the shell’s reach and which staging policy recovers context capacity.

What’s Next: From tool interfaces to execution isolation

A well-specified, authorized tool can still execute faulty or untrusted code. The boundary checks a call’s arguments against the task’s grants, but a general-purpose command runner such as run_shell accepts commands whose effects no argument check can bound, and even a narrow tool’s implementation can reach whatever its host process can. A descriptor does not limit which files the running code reads, which external services it contacts, or how many resources it consumes. Agent Sandboxes develops the execution envelope that enforces those limits beneath the running code, extending the tool contract from permitted requests to contained execution.