10 The Agent Harness
Purpose
Agent stack focus for this chapter.
Why must the decision to continue an agent live outside the model that proposes its next step?
A single model invocation succeeds or fails on its own, but an autonomous agent runs across a horizon of many turns, and per-step errors compound with every turn. Over dozens of sequential tool interactions, a malformed call, a stale observation, or an unhandled exception can carry an unsupervised trajectory into a failure or a loop that no single step reveals. The agent harness is the runtime component that owns this loop: it schedules model calls, enforces the rules for moving between states, tracks task budgets, and holds sensitive actions for human approval. A model can be asked to pause, wait, or stay within an allowance, but a request it honors on almost every turn still fails across a long run, so the decision to continue must rest with deterministic supervision outside the model. Engineering the harness turns stochastic token generation into task execution that stays bounded, interruptible, and accountable across the whole horizon.
Learning Objectives
- Explain why continuation is decided outside the model, and design the runtime-owned trajectory record of plans, grants, consumption, and obligations
- Construct guarded state transitions that keep stale observations out of the next turn while preserving unresolved effects
- Calculate how long a control request waits for a turn boundary, and apply pause, cancel, or kill without dispatching incomplete output
- Explain which resources an asynchronous wait releases and which the trajectory still owns and pays for
- Design an approval gate with expiry and dispatch-time rechecks, and size the resources its held trajectories retain
- Allocate shared model-call service by weighted deficit round-robin while keeping provider quotas separate from trajectory budgets
- Apply bounded-liability admission across a trajectory’s turn, token, time, and dollar budgets, and select a ceiling by cost per accepted task
10.1 Supervising Continuation
The horizon exposure, bounded by a harness that keeps control outside the model.
Consider the connection-pool repair agent (Four lighthouse workloads) at the moment it produces candidate patch r18. A test job is actively running against a fixed snapshot of that revision when an operator requests a pause for manual inspection. Simultaneously, an independent interactive task queues for model service, and the repair trajectory approaches its allocated budget. Even if the test job finishes cleanly and the next model reply is syntactically flawless, neither event answers whether the repair should proceed right now. That continuation decision depends on operational conditions outside the model: the operator’s pending signal, current task states, shared cluster capacity, and the allowance remaining once in-flight work is accounted for.
The runtime components examined so far each address part of this operational environment. The tool layer tracks job handles and operation repeat contracts, the sandbox isolates execution and bounds workspace lifetimes, and the security checks test each dispatch against its confidentiality and provenance labels and bind each approval to the exact action it covers. Each of these mechanisms, however, operates at the boundary of a single invocation, resource, or decision; none possesses the context or authority to decide whether the trajectory as a whole may take another step. That decision cannot be delegated to the model either. An agent instructed to self-regulate (to pause on request, or to halt before exceeding an allowance) evaluates that rule afresh on every turn. Because of horizon compounding (equation 1.1), even a high per-turn compliance rate decays across an extended run, so a constraint the model honors on almost every turn is still violated in a predictable share of long trajectories. Operational control also requires state that cannot live in an ephemeral context window. An operator needs immediate status without reconstructing the conversation transcript; a scheduler must distinguish between work waiting on background jobs and work eligible to run; and the dispatch path must enforce security grants and token allowances before invoking downstream tools. All three operations demand a single, authoritative account of the trajectory that an unstructured message list cannot maintain.
The agent harness provides this missing supervisory layer by taking ownership of the execution loop for an individual trajectory. The agent proposes candidate actions; the harness determines whether each step runs immediately, yields to other work, or halts, and it enforces those decisions against an execution record that only the runtime can modify. This is trajectory-level encapsulation (principle 10): the runtime governs each model call as one step of a whole trajectory, which advances only through transitions and admissions the runtime’s guards permit, because the trajectory cannot be relied on to yield, to stop, or to notice that it has stopped making progress. Invariant closure (principle 1) says that bounds belong below the model; this principle names the unit those bounds must cover, the whole trajectory rather than the single call. In the harness, the principle takes the form of two working rules. First, every outbound model call or tool dispatch must pass an explicit admission check (active operator signals, capability grants, shared capacity, and remaining budget) at the instant it would execute. Second, in-flight work stays accountable: no pause signal, timeout, or cancellation proves that running work has stopped or that its cost is settled, so every outstanding operation keeps a named owner until evidence settles it.
The harness develops in the order these decisions arise in one repair turn. The runtime first needs an authoritative account of the trajectory (section 10.3) and rules for how that account may change (section 10.4). Operators then need a way to stop or redirect it (section 10.5), and long tool jobs and human reviews need waits that keep supervision intact (section 10.6, section 10.7). Across many trajectories, shared model service must be divided (section 10.8), and across many turns, cumulative consumption must be bounded (section 10.9). The record described here lives in host memory; making its guarantees survive host failure is the work of durable execution (Durable Execution).
10.2 The Loop and the Harness
When an autonomous agent runs inside a bare execution loop, an unexpected operator pause or an unconstrained retry cycle immediately exposes the lack of an external control plane: a path, separate from the one that performs model calls and tool work, that decides whether that work may proceed. An unmanaged loop treats model completions as direct execution instructions; a harness treats them as untrusted proposals, evaluating every action against explicit admission policies, budget constraints, and external supervisory signals.
10.2.1 The anatomy of an unmanaged loop
The unmediated while-loop of An Unmediated Agent Loop is the pattern most prototype agents still use, popularized by ReAct (Yao et al. 2023): invoke a foundation model, dispatch the tools requested in its completion, append the results to the message history, and repeat until the model emits a stopping condition. The introduction also divided a turn into six phases, from context assembly to assessment against the task contract, and observed that some phases wait on tools or people. What the loop lacks for production operation is a place for those phases’ decisions to live. It confines all operational state to the message list and the local call stack of the client process.
↩︎ The six-phase turn lifecycle, including the phases that pause a turn for tools and approvals, is established in The six phases of a turn.
Because an unmanaged loop lacks an independent, externally accessible control state, it cannot support production operation: when an operator tries to pause a run during a long test suite, the runtime has no interface to receive the command; when cumulative token consumption exhausts the allocated budget, the loop cannot withhold the next model call without terminating abruptly and abandoning in-flight work; and when the model proposes a tool invocation, the call executes immediately under ambient authority without policy checks or human review. Although a supervised harness may also execute as a loop or coroutine, it wraps that loop in an explicit state machine whose guards govern every continuation path, filtering each proposal before any tool call is admitted (figure 1).
In an unmanaged system, execution context is scattered across disparate subsystems: host operating systems track local processes, remote clusters hold asynchronous job handles, storage systems manage workspace leases, and identity providers track credential expirations. Panel (b) consolidates these responsibilities into a single runtime-owned record for each trajectory. Operators, cluster schedulers, and dispatch paths all consult this authoritative record before initiating state changes. While a system may pass budget constraints to the model in the prompt and the model may cooperate with pause requests, actual enforcement remains strictly with the runtime. Relying on model cooperation violates invariant closure; the runtime itself must guarantee that system safety invariants hold under all execution paths (principle 1).
Definition 10.1: Agent harness
Agent harness is the runtime component that owns the agent loop for one trajectory, assembling each model call, interpreting the reply, deciding whether and when each proposed tool call is dispatched, and keeping the record through which outside components inspect, pause, bound, or stop continuation.
- Significance: It keeps continuation decisions independent of model cooperation while preserving the grants, consumption, and outstanding work those decisions must respect.
- Distinction: The tool layer mediates calls and tracks their outcomes, and the sandbox bounds execution under its policy. The harness composes those mechanisms with control requests, waiting, scheduling, and lifetime budgets.
- Common pitfall: Keeping status and pending obligations only in prompt text or inaccessible execution-local state, so a separate control path cannot inspect them or prevent further dispatch.
10.2.2 One turn under supervision
Following a single execution turn of the repair workflow shows how this division of labor works in practice. From the moment the repair becomes eligible for another model call, through the model’s proposal to run the regression test against candidate r18, to the operator’s resume, table 1 records each decision in plain terms; the sections that follow supply the mechanism behind each one. The lifecycle states record whether the trajectory is eligible for continuation (READY), actively querying the model (CALLING_MODEL), awaiting tool completion (WAITING_TOOL), or suspended by an operator (PAUSED).
| Event | Harness decision and recorded obligation |
|---|---|
| The repair is eligible for another model call | From READY, confirm that no operator request is pending, that shared model capacity is available, and that the budget covers the most this call could cost. Set that amount aside, then enter CALLING_MODEL and send the call |
A complete reply proposes the test of r18 |
Record what the call actually used. Check the proposed test against the task’s permissions and the remaining budget, set aside the test’s maximum cost, and record the test as running before sending it. Enter WAITING_TOOL |
| The operator requests a pause while the test runs | Stop admitting new work. Keep tracking the running test and its cost, but no longer let its result restart the model; then step through READY to PAUSED with no moment at which new work could start |
| The test result arrives during the pause | Confirm that the result is genuine and belongs to this test of r18. Record its outcome and cost, but leave the trajectory paused and the model’s context unchanged |
| The operator requests resume | Recheck task state, permissions, the recorded evidence, and the budget before returning to READY. The scheduler may then admit another call; the result supports only the test and revision it actually checked |
Three features of the trace recur through the rest of the chapter. First, every row that admits work reserves before it sends. Affordability cannot be judged from settled charges alone, because the running test job may still bill and the proposed call may generate up to its full output allowance; the harness therefore reserves each operation’s worst-case liability at admission and reconciles reported usage afterward. Notebook 10.2 later prices a call that a check of settled spend admits and this reservation test refuses. Second, the pause and the test result are independent events. The pause stops new admission without stopping the job, and the job’s result settles the job’s outstanding charge without resuming the turn. Third, the reply’s content never acts on its own: a truncated or canceled reply yields nothing to dispatch, a claim of completion triggers an independent check rather than ending the task, and an action that policy requires a human to review waits in a dedicated hold with its own expiry. Each of these needs runtime state that the message list does not have.
The trace also separates the decision to continue from the resources an agent holds. While the repair trajectory sits in PAUSED or WAITING_TOOL, it keeps its workspace, its cached repository state, and its running test job, but it does not hold a worker thread or a place in the queue for model calls. Because the harness, not the repair loop, decides when the trajectory next calls the model, an interactive trajectory can take the next model-call slot without waiting for the repair to give it up. Which resources a wait releases, and which it keeps, is the subject of supervised waiting (section 10.6); how model calls are divided among trajectories is the subject of scheduling (section 10.8).
10.3 The Trajectory Record
When an operator issues a pause request, the conversation transcript cannot answer the questions the harness must resolve. The transcript records what the user and model said, but runtime decisions require knowing the state of the underlying system: which candidate revision is under test, whether a background validation job has finished, how much compute budget remains once outstanding asynchronous tasks are accounted for, and whether any external side effects are still in flight. Simultaneously, the cluster scheduler must distinguish a trajectory blocked on a container execution from one actively waiting for its next inference turn. None of these operational facts belong in the prompt. Resolving them requires runtime-owned control state that the harness can inspect, mutate, and enforce directly without relying on the foundation model to reconstruct its own execution history.
Definition 10.2: Trajectory record
Trajectory record is the per-trajectory control state owned and maintained by the harness. It encapsulates identity and task scope, lifecycle state, resource limits and financial accounting, security grants and workspace leases, pending asynchronous operations, and references to prompt context, the live execution plan, container environments, and verified evidence.
- Significance: It provides the harness with an authoritative, centralized structure to inspect and govern the entire trajectory, tracking resource consumption, pending asynchronous operations, and security grants across inference calls.
- Distinction: While the conversation transcript records user-model message exchanges, the trajectory record tracks the operational state required to govern execution across multiple turns. A live in-memory record represents active control state, not a durable audit log or proof that external claims are verified.
- Common pitfall: Treating a model’s generated text or self-reported progress as an authoritative update to consumed budget, granted capabilities, tool execution results, or task completion.
Table 2 organizes these fields by the supervisory decisions they support. The table is a logical schema. A distributed runtime may partition these field groups across different storage backends, for example keeping ephemeral context references in memory while persisting cost accounts to an external store, provided each group retains a single authoritative writer and a well-defined update protocol.
| Field group | Contents | Supervisory use |
|---|---|---|
| Identity and task | Trajectory, requester, tenant, optional parent, task contract and version | Bind requests and charges to the right task and authority |
| Control | state, record_version, control_epoch, pending_control, exit reason |
Decide whether a continuation or state change is current and permitted |
| Continuation | Context reference and version, live-plan reference and revision, candidate workspace and evidence references | Assemble the next call from the intended task state |
| Limits and accounting | Per-call limits, actual_consumption, outstanding_reservations, finalization_reserve, and deadline |
Check new liability against the remaining allowance and preserve funds for final work |
| Authority and resources | Current grants, workspace leases, credential-broker references, resource owners | Check accessible operations and identify resources requiring release or transfer |
| Current invocation and observations | current_model_call, observation_eligible_calls, attempt IDs, job handles, deadlines, and expected versions |
Correlate events and decide which may inform the current continuation |
| Held approval | held_approval: exact proposal reference, applicable task and plan versions, approval status and expiry |
Keep a proposed action undispatched while its decision is pending |
| Outstanding obligations | operation_table reference, unresolved outcomes and charges, settlement owner |
Preserve responsibility after a timeout, plan revision, pause, or scheduling exit |
Two questions about a running operation must be answered separately: whether its result may still inform the model, and whether the harness still owes anything for it. The first is observation eligibility, which defines whether the output of an operation is permitted to enter the prompt context for the next model invocation. The second is an outstanding obligation, and the two can come apart. A tool call can lose observation eligibility (when an invocation step times out, or the user cancels a subtask) even while the remote execution continues running in a backend container. To prevent orphaned side effects and resource leaks, the operation table tracks every logical operation across its entire lifecycle: transport retries, incoming execution evidence, and the specific subsystem responsible for final settlement.
The trajectory record manages this lifecycle through two versioning counters. While record_version increments on every serialized update to provide optimistic concurrency control, the control epoch (control_epoch) tracks the specific planning and invocation cycle against which proposals and observations are validated. The epoch does not advance on every turn; it advances when the harness withdraws a continuation’s claim on outstanding results, as when a wait deadline expires, an operator interrupts, or a plan revision supersedes pending work. Each call records the epoch current at its registration, so an ordinary result from this turn’s dispatch still matches. Incrementing the epoch invalidates in-flight observations registered under the earlier epoch without breaking the underlying operation’s idempotency contract. As a result, the harness continues tracking background tasks and their external side effects even after the agent has stopped waiting for their immediate output.
↩︎ The operation idempotency contract and retry semantics are established in Retrying Uncertain Actions.
A parallel discipline governs resource accounting. The harness separates settled usage (actual_consumption) from provisional allocations (outstanding_reservations), which represent admitted operations whose compute or financial costs have not yet been finalized. The harness withholds a dedicated finalization reserve from general exploratory steps. This reserve guarantees that the runtime retains sufficient budget to execute mandated cleanup tasks, release sandboxes, and run deterministic verification suites before the trajectory terminates. Wall-clock deadlines operate independently of token limits, because waiting on remote containers or human reviewers consumes real time even when token generation is idle. Section 10.9 formalizes this admission test; the trajectory record provides the live metrics required to enforce it.
These lifecycle and accounting guarantees unite under one working rule, the settlement rule. It is the trajectory-level form of settlement before retry (principle 8). That principle says a timeout does not establish whether an effect occurred, so a blind retry is unsafe; the settlement rule adds that losing interest in a result does not release the effect’s charge or its owner either. Under this rule, withdrawing an operation’s observation eligibility, whether through a timeout, a plan revision, an operator pause, or trajectory cancellation, never removes the operation from the operation table or releases its budget reservation. Only authenticated, correlated evidence can settle an outcome, confirm completion, or release reserved budget. Until that proof arrives, the outstanding obligation retains a designated settlement owner responsible for reconciling its ultimate state.
10.3.2 The live plan
Once the repair agent generates candidate r18 and dispatches a targeted unit test in a background container, a static candidate graph or a checklist in prompt text can no longer represent the state of the work: it cannot separate the edit that produced r18, the test currently in flight against that revision, and the broader integration suite still waiting on the test result. Runtime execution therefore demands a live plan, a dependency-ordered directed acyclic graph of execution steps that explicitly separates proposed actions from operations currently in flight and evidence already verified. Each step in the plan defines a unique identifier, explicit prerequisite steps or evidence dependencies, the target candidate artifact and operating conditions to which it applies, and its current execution status. When the harness admits a step, it binds that step directly to an entry in the operation table; when a step completes, the harness links it to the immutable evidence supporting that completion, never to an unverified checkbox emitted in model text.
⇄ Contrast: The revisable candidate plan established in Revising Candidate Plans tracks only unexecuted candidate operations and their supporting evidence; the live plan also records operations in flight.
While the test runner executes in its background container, the live plan records the step as in flight rather than optimistically assuming success. When the runner completes, a passing status validates only that targeted test against candidate r18 under those exact container flags; the remaining verification steps and final acceptance criteria remain unfulfilled. An operator’s pause request arriving mid-test acts as an independent supervisory event, interrupting further dispatch without corrupting the historical inputs or the validity of the ongoing test execution.
When incoming observations demonstrate that a strategy is failing, the model may propose a plan revision to alter downstream tasks. Before adopting any revised plan, the harness validates it against the overarching task contract, current evidence dependencies, remaining budget, and security grants. Adopting the revision advances the plan revision counter and invalidates pending, undispatched proposals and held approvals whose underlying assumptions have changed. Existing verified evidence remains valid under its original operating conditions. The harness synchronizes plan adoption with the tool dispatcher to prevent race conditions where a stale proposal might pass admission against a superseded plan.
Now consider what happens if a plan revision modifies the source code while the targeted test for candidate r18 is still running in the background. The active container job is testing candidate r18, not the new revision. Relabeling the step in the plan cannot convert those eventual test results into evidence for the modified code. Deleting the step from the plan likewise does not abort the remote job, reverse its external side effects, or automatically refund its compute reservation. Instead, the harness keeps the active operation in the operation table under its original settlement owner, blocks downstream steps whose prerequisites are no longer satisfied, and evaluates the replacement work through a fresh admission check. Undoing an external effect requires compensation, a later action that offsets the earlier one (Semantic compensation), not a retroactive edit to plan text.
By maintaining an explicit, dependency-tracked live plan, the trajectory record delineates what the agent has proposed, what the harness has admitted to the execution plane, and what obligations remain outstanding. Preserving consistency across concurrent tool events, asynchronous worker callbacks, and operator interruptions requires the harness to strictly govern how this record evolves, binding all lifecycle mutations to formal, guarded state transitions.
10.4 Trajectory States
Consider an automated software repair trajectory that submits a suite of test cases for candidate r18. If the wait deadline elapses before a result arrives, the harness must record the outcome as unknown and proceed. One minute later, the test service returns a success report. Both statements reflect reality: the outcome was indeed unknown when the deadline expired, and the tests ultimately passed. The failure occurs if the harness splices that late result into an obsolete turn as though the model had possessed the evidence when selecting its next action. This defect, known as observation crossing, conflates the asynchronous physical history of an operation with the causal evidence admitted to a particular continuation. Preventing it takes two pieces developed in this section: guarded transitions that decide when a continuation may advance, and an epoch fence that decides which late results may still inform it (section 10.4.2).
A state machine with guarded transitions1 makes every such admission explicit. Beyond resolving observation races, guarded transitions prevent scheduling anomalies, such as mistaking a trajectory held for human approval as idle and erroneously dispatching another model call. The runtime control plane, rather than generated model output, governs all state transitions. Table 3 defines this supervisory state machine, organized around five live states, a finalizing state through which every exit must pass, and three terminal absorbing outcomes.
1 Guarded Transition: Harel’s statecharts (Harel 1987) labeled each transition with the event that triggers it and a condition that must hold for it to fire, and added hierarchy and concurrency so that reactive controllers stayed readable as their state counts grew. The harness uses the flat core of that formalism: an event proposes an edge, and the guard, evaluated against the record at that moment, decides whether it fires. Keeping the guard on the edge, rather than in whichever handler happens to receive the event, is what lets one table govern every path into CALLING_MODEL.
Harel, David. 1987. “Statecharts: A Visual Formalism for Complex Systems.” Science of Computer Programming 8 (3): 231–74. https://doi.org/10.1016/0167-6423(87)90035-9.
| State | Meaning | Legal next states |
|---|---|---|
READY |
Eligible to be considered for the next model call; admission guards still apply. | CALLING_MODEL, PAUSED, FINALIZING |
CALLING_MODEL |
A model invocation is registered for the current continuation. | READY, WAITING_TOOL, AWAITING_APPROVAL, FINALIZING |
WAITING_TOOL |
The continuation awaits a declared set or batch of tool observations. | READY, FINALIZING |
AWAITING_APPROVAL |
An exact proposed action is held; approval and dispatch eligibility remain unresolved. | WAITING_TOOL, READY, FINALIZING |
PAUSED |
The continuation is held at a local boundary; no new model or tool work is admitted. | READY, FINALIZING |
FINALIZING |
The runtime is checking the exit outcome and resolving or transferring remaining duties. | COMPLETED, FAILED, CANCELED |
COMPLETED, FAILED, CANCELED |
The continuation has ended; retained evidence and assigned obligations may remain. | None |
The normal operational cycle follows READY → CALLING_MODEL → WAITING_TOOL → READY, tracing through the five live states and guarded transitions shown in figure 2: proposals requiring review divert into AWAITING_APPROVAL, pause commands transition to PAUSED exclusively from READY, and all termination triggers (completion requests, cancellations, or exhausted budgets) route through FINALIZING before reaching absorbing terminal states (COMPLETED, FAILED, CANCELED). During execution, a model response may produce no admissible action, request task completion, or propose an action requiring operator approval. While the normalized invocation status defined in Normalized invocation status envelope fixes how the runtime processes individual call outcomes, table 4 specifies how each outcome shifts the overall trajectory state. Two names appear in both vocabularies with different meanings: a reply status of COMPLETED or CANCELED describes one model call, while the terminal states of the same names describe the whole trajectory. A completed reply ends a call, not the task. Only a reply whose status is COMPLETED proceeds to tool candidate validation. The harness must never convert a malformed, truncated, or canceled partial stream into an actionable tool invocation simply because a parser managed to salvage partial arguments from the payload.
| Reply status and content | Next state | Condition or residue |
|---|---|---|
COMPLETED, validated tool proposal |
WAITING_TOOL |
Once its dispatch is admitted |
COMPLETED, proposal requiring review |
AWAITING_APPROVAL |
When policy requires review of the exact action |
COMPLETED, completion request |
FINALIZING |
The task’s completion check follows |
COMPLETED, no admissible action |
READY |
With an observation saying so |
TRUNCATED, REFUSED, REJECTED |
READY |
With a bounded observation, unless the task’s refusal policy finalizes |
CANCELED |
READY |
Under the control hold that caused the cancellation |
TRANSPORT_FAILURE, FAULTED |
READY |
For bounded failure handling while the budget permits, FINALIZING otherwise |
Action approval introduces a controlled detour to this cycle. An approved action transitions from AWAITING_APPROVAL to WAITING_TOOL only when its active dispatch guards pass; recording operator approval does not automatically execute the tool. Conversely, a denial, expiration, or operator withdrawal emits a refusal observation back to READY.
10.4.1 Transition guards and the admission check
A trajectory sitting in READY can still be unsafe to advance if an operator pause arrived a millisecond earlier, a credential expired while the trajectory waited in the queue, or an unresolved prior operation still blocks a dependent mutation. Navigating a legal edge in the state machine is therefore only a necessary condition for progress, not an unconditional right to execute. Before the runtime admits new work, whether invoking a model or dispatching a tool, it evaluates a set of predicates known as the admission check: no pending control holds may exist; the active context, credentials, and required action approvals must remain valid; upstream data dependencies must be resolved; and adequate deadline slack and downstream service capacity must be confirmed. The runtime must also reserve the operation’s bounded financial liability from the task budget without encroaching on the dedicated finalization reserve (section 10.9), and for tool calls it must verify that arguments are structurally complete. Because system state changes continuously, this evaluation holds only at the instant it executes; an approval granted seconds earlier or a scheduling decision made on a previous tick does not carry forward.
The admission check and its state updates execute as a single atomic transaction. Protected by the trajectory record lock,2 the harness evaluates the guard predicates, reserves the required liability, registers the outgoing invocation or attempt ID, marks the expected observation as eligible, and commits the state transition. After releasing the record lock, the runtime passes the admitted request to a release gate that serializes the final release-or-withhold decision with control updates. This gate rechecks current controls and the existing admission without reserving liability twice; slow network I/O remains outside the record lock. This design enforces the register-before-send rule. Registering state before dispatch ensures that even a very fast response never arrives before the local record exists to receive it. Conversely, transmitting requests strictly outside the critical section prevents slow network I/O, stalled inference endpoints, or lagging tool services from blocking concurrent control actions, scheduler ticks, or arriving results from other workers.
2 Record Lock as a Monitor: Hoare’s monitors (Hoare 1974) bundled shared data with the only procedures allowed to touch it and ran those procedures one at a time, so every invariant on the data could be checked at procedure entry and exit. The trajectory record is a monitor in this sense: controllers, schedulers, and result handlers all reach it through short guarded procedures. The register-before-send rule adds one constraint for a record whose procedures front slow remote services: no procedure waits on a model or tool call while holding the lock.
Hoare, Charles Antony Richard. 1974. “Monitors: An Operating System Structuring Concept.” Communications of the ACM 17 (10): 549–57. https://doi.org/10.1145/355620.361161.
Because external control requests acquire this same monitor lock, stops and admissions are strictly serialized under an admission-order rule: if a pause, cancellation, or kill command commits before an admission check runs, it immediately blocks new outbound work, whereas if admission commits first, the operation becomes owned work with a registered identity and reservation that a later control cannot erase or refund on intent alone. Although the release gate can still withhold a definitely unsent attempt, settling that attempt requires authoritative evidence that release did not occur and that no remote liability remains—a volatile withheld flag does not survive a host crash, and once a packet may have left the host, a cancellation request cannot establish not_sent without remote evidence. Tool-side idempotency and stale-lease fencing therefore still require the endpoint contracts detailed in Operation identity and endpoint contracts.
To coordinate concurrent state access without heavyweight distributed locks, the harness pairs optimistic concurrency validation with epoch fencing. The record_version field detects competing updates at the point of commit, implementing the validation step of optimistic concurrency control (Kung and Robinson 1981). In parallel, increments to control_epoch fence out stale asynchronous responses, the same generation-number check that the tool layer applies to superseded workers at the endpoint. Here the fenced party is an observation: a response stamped with an old epoch cannot advance the current continuation, however well formed it is. These control plane identifiers operate alongside, rather than replace, logical operation identifiers and transport attempt counters. A network retry generates a new attempt ID under the same logical operation, whereas an operator interruption bumps the control epoch to invalidate pending replies without erasing the operation’s entry in the operation table. The record now carries several counters, and each advances on different events and rejects a different kind of stale input (table 5).
↩︎ Epoch fencing of superseded workers at the mutating resource is established in Operation identity and endpoint contracts.
| Counter | Advances when | What a stale value rejects |
|---|---|---|
record_version |
Any committed update to the trajectory record | A decision made on a view of the record that has since changed |
control_epoch |
A wait deadline, an operator interrupt, or a plan revision withdraws a continuation’s claim on outstanding results | A result registered under an earlier epoch, as input to the current turn |
| Plan revision | The harness adopts a revised live plan | A proposal or held approval formed against the superseded plan |
| Held-proposal version | The harness records a new or changed held proposal for approval (section 10.7) | An approval decision for a different version of the proposal |
| Attempt ID | A transport retry of the same logical operation | Nothing by itself; it lets late evidence name the attempt that produced it |
Listing 1 expresses these rules as a single supervisory procedure. The legal edges match table 3, and any edge without a registered guard predicate is rejected by default. The caller provides the record_version it observed when formulating its transition request; a concurrent modification by another worker causes the version check to fail. Similarly, duplicate deliveries of an event fail either the version check or the edge guard, guaranteeing that no action is admitted twice.
LIVE = {"READY", "CALLING_MODEL", "WAITING_TOOL",
"AWAITING_APPROVAL", "PAUSED"}
LEGAL = {("READY", "CALLING_MODEL"), ("READY", "PAUSED"),
("CALLING_MODEL", "READY"), ("CALLING_MODEL", "WAITING_TOOL"),
("CALLING_MODEL", "AWAITING_APPROVAL"), ("WAITING_TOOL", "READY"),
("AWAITING_APPROVAL", "WAITING_TOOL"),
("AWAITING_APPROVAL", "READY"), ("PAUSED", "READY")}
LEGAL |= {(s, "FINALIZING") for s in LIVE}
LEGAL |= {("FINALIZING", t) for t in ("COMPLETED", "FAILED", "CANCELED")}
# GUARDS maps each edge to a predicate over (record, decision).
def transition(rec, new_state, decision, admit=None):
with rec.lock: # the local decision only
edge = (rec.state, new_state)
guard = GUARDS.get(edge) # no guard, no permission
if edge not in LEGAL or guard is None or not guard(rec, decision):
raise IllegalTransition(edge, decision.id)
if decision.seen_version != rec.record_version:
raise StaleDecision(decision.id) # a competing update won
if admit is not None:
admit(rec) # reserve, register, set eligibility
rec.state = new_state
rec.record_version += 1
rec.history.append((edge, decision.id, rec.record_version))
# The caller passes admitted work to the serialized release gate outside this lock.Here, the guard registered on the READY → CALLING_MODEL edge executes the complete admission check, while admit performs the liability reservations and bookkeeping required by the register-before-send rule. Because the monitor lock protects only in-memory lookups, state updates, and an array append, lock contention remains negligible compared to the hundreds of milliseconds consumed by remote inference or tool execution.
10.4.2 Late results and observation eligibility
When an asynchronous tool result arrives, the harness must reconcile financial and physical state without corrupting the prompt history of an active turn. Upon reception, the runtime authenticates the payload and correlates its task identifier, logical operation ID, and attempt number against the trajectory record. Mismatched or unauthenticated messages are logged for audit and dropped. Valid results immediately update the trajectory’s operation table: the runtime records raw execution evidence and converts reserved charges into settled resource usage.
However, updating the operation table does not imply that the result belongs in the model’s current prompt context. As listing 2 shows, the runtime splits incoming evidence into two separate paths: settlement in the operation table, which every authentic result receives, and epoch-fenced observation staging. While every authentic result records evidence and settles any confirmed usage, an observation is staged for immediate prompt injection only if its call ID remains registered in observation_eligible_calls and the epoch recorded with the call at registration matches the active control_epoch. The remote service never sees the epoch; when the harness correlates the result, it looks up the epoch by call ID and attaches it to the local result envelope as result.epoch. This field comes from the registration record, not the remote payload. The two checks fail in different ways. The eligibility set is the per-call statement of which results the current wait expects, and removing an entry after staging is what stops a redelivery from being staged twice. The epoch is a bulk fence: a timeout or operator interrupt advances it once and thereby invalidates every call registered under the old epoch, including any whose removal from the set was missed. If an operation timed out or an operator intervened, its recorded epoch is obsolete. The harness settles the financial balance but refuses to inject the result into the current turn, which eliminates observation crossing. Should that late evidence prove relevant to future turns, the runtime can re-evaluate and stage it during subsequent guarded context assembly. Staging a result does not by itself end the wait. A declared wait policy establishes the criteria required to return from WAITING_TOOL to READY, and the handler requests that transition only when the policy is satisfied.
def on_tool_result(rec, result):
if not authenticate(result):
return audit(rec, "unauthenticated_result", result)
with rec.lock:
op = rec.operation_table.get(result.operation_id)
if op is None or result.attempt_id not in op.attempts:
return audit(rec, "unknown_operation", result)
if not op.record_evidence(result): # already recorded: duplicate
return
settle(rec, op, result.usage) # reserved -> actual; rest stays owed
eligible = (result.call_id in rec.observation_eligible_calls
and result.epoch == rec.control_epoch)
if eligible:
rec.observation_eligible_calls.discard(result.call_id)
stage_observation(rec, result) # for the current wait only
rec.record_version += 1
seen = rec.record_version
if eligible and wait_policy_satisfied(rec): # the guard rechecks it
transition(rec, "READY", Decision(result.id, seen))Systems typically implement one of four wait policies: all required results arrive, an acceptable partial batch completes, a wait deadline expires, or an operator triggers an explicit withdrawal. The trajectory record distinguishes confirmed evidence from unreturned, unresolved calls. In the candidate r18 repair scenario, a timeout records that the outcome was unknown at the deadline; it cannot justify inferring that the test suite failed. When the test service eventually reports that candidate r18 passed, the operation table records that verified outcome with its exact scope. A timeout likewise never authorizes an uncoordinated re-execution of a non-idempotent operation. Dependent actions must still satisfy all guard preconditions and adhere to the tool’s mutation safety contract.
10.4.3 Pause entry and outstanding work
When a pause command arrives while a trajectory is in CALLING_MODEL or WAITING_TOOL, jumping straight into PAUSED would either orphan the in-flight model stream and remote tool job or leave their state transitions ambiguous. The state machine therefore permits entry into PAUSED exclusively from READY. If a pause command arrives mid-call or mid-wait, the runtime asserts a pending_control hold that blocks the admission of any subsequent work, revokes observation eligibility for active model streams, invalidates any pending approval requests, and ends the active observation wait; for a trajectory in WAITING_TOOL, this is the explicit-withdrawal wait policy, and it is what permits the WAITING_TOOL → READY step. If policy dictates, the control plane issues cancellation requests to remote workers; under the settlement rule, these operations remain registered in the operation table until remote confirmation arrives. Any partial token stream received from a canceled model call is preserved solely for diagnostic logging, never stitched together to form a valid dispatch.
Once these local eligibility revocations are logged, the trajectory steps through READY directly into PAUSED under the existing admission hold. Because the lock prevents concurrent scheduling during this step-down, no runnable window opens between the transitions. Entering PAUSED halts continuation progress, ensuring that neither pending responses nor held proposals can advance the continuation. However, background tools already running in external systems may continue their physical execution. For example, the remote test execution for candidate r18 can complete while the trajectory remains paused; its completion payload updates the operation table and settles the reserved charge without triggering a new inference cycle.
Resuming execution requires a guarded PAUSED → READY transition, followed by a brand-new admission check that reconciles all evidence recorded in the operation table during the pause. Because resumption is not an automated rollback of pause-time revocations, state withdrawn during a pause is never restored: expired authorization tokens remain expired, withdrawn approvals require fresh operator authorization, canceled partial responses stay abandoned, and any earlier operation that remains unconfirmed continues to block its dependent actions even if unrelated steps become eligible. Meanwhile, physical resource retention operates under independent lifecycles: sandboxed workspace leases, buffer allocations, compute reservations, and model context caches persist or expire according to their respective lifetime policies rather than the trajectory’s instantaneous state.
10.4.4 Finalization and remaining duties
Every exit from the live states—whether triggered by the model’s completion request, an operator cancellation, or an exhausted budget—passes through FINALIZING before reaching a terminal state (figure 2), because ending a run immediately would skip independent acceptance checks, leave delegated credentials active, and abandon unsettled background jobs. Upon entering this state, the runtime immediately suspends standard work admission, records the exit reason, revokes unused delegation authority where supported, and triggers the task completion check when completion is claimed. An end-of-turn token emitted by a model merely requests this validation; genuine task completion requires an independent completion check, such as the task contract’s acceptance tests, run outside the model. All finalization checks and mandatory cleanup routines draw from their dedicated, pre-funded reserve.
During finalization, the control plane may block to await pending results, report verification failures, or hand off outstanding cancellation, billing reconciliation, and resource reclamation tasks to an external supervisor. The transition guard into any terminal state requires that every outstanding obligation be either definitively settled or explicitly transferred to an accepting upstream owner. Simply clearing an unresolved task from a local tracking set violates the guard. To effect a legitimate transfer, the runtime must transmit complete metadata (operation IDs, recorded evidence, and budget reservations), allowing the recipient service to assume supervisory responsibility.
All terminal states (COMPLETED, FAILED, and CANCELED) are strictly absorbing. Once a trajectory enters a terminal state, its continuation terminates permanently. Late-arriving tool results or charge settlements can still update archived historical records through the designated supervisory owner, but no event can transition the continuation back to READY. Resuming an investigation or re-attempting a task requires instantiating a fresh trajectory with a new identifier linked back to the prior run. Within this architecture, COMPLETED denotes solely that the programmatic completion criteria configured for the harness were met. Whether those criteria demonstrate true task competence is an empirical systems question addressed in The Evaluation Problem.
Checkpoint 10.1: Trajectory record and guarded states
Guarded transitions keep every update to the record causally consistent under network delays, tool timeouts, and concurrent interventions. They describe pause entry, however, only from the record’s side. An operator who presses pause needs more: an authenticated way to register the request, a bound on how long it takes effect while a model call is still generating, and a distinction between a pause, a cancellation, and a kill that leaves outstanding work owned in each case.
10.5 Control Actions
The test job for candidate r18 is still executing when an operator requests a pause. The harness must halt the repair loop’s forward progress without losing the job’s tracking identity, its allocated budget reservation, or the diagnostic evidence it might return. Delaying the pause until the test finishes strips the operator of interactive control over potentially runaway execution. Conversely, declaring the job canceled merely because the trajectory state transitions to PAUSED fabricates an execution outcome the runtime has not yet observed.
External control decouples receiving a request from establishing the physical execution state that request demands. When an operator intervenes, the harness authenticates the caller, verifies authority over the target trajectory, and appends the requested directive to pending_control while holding the record lock. Because this update occurs under the lock, the admission-ordering invariants established in section 10.4 fix exactly which operations fall under the request’s scope. The controller acknowledges the hold to the operator only after the update commits to the record (and, once durable execution is in place, to its log). The response delivered to the operator must also distinguish three distinct milestones: request received, new admission stopped, and subsequent confirmation that running work has terminated.
10.5.1 Admission boundaries and active generation
When an operator sends a pause or cancel signal while an agent is preparing a model call or stepping through a batch of proposed tool calls, the harness needs well-defined points where it can stop new work before a request leaves the host. Each such checkpoint is a turn boundary, evaluated immediately before dispatching an inference call to a foundation model and immediately before invoking any external tool, including each individually admitted call within a proposed parallel batch. The name marks where control takes effect, not a boundary between turns: one turn (one step of the trajectory, counted by \(\TurnHorizon\)) contains a turn boundary before its model call and one before each tool call it admits. A pause that lands midway through a batch therefore leaves the calls already sent registered as owned work and withholds the rest. The admission check evaluates control flags under the trajectory record lock, and the release gate serializes its final decision with subsequent control updates. A flag check followed by an uncoordinated send would leave a race window and would not enforce this boundary. The guarantee is local. The boundary fixes which operations were admitted before the request took effect, rather than implying that all work stopped when the operator pressed stop, and it leaves every remaining obligation explicit and inspectable.
Local candidate processing does not have to wait for remote model generation to finish. The foundation-model contract already separates a CANCELED reply from a REJECTED one and withholds both from dispatch; the harness turns that cancellation into a control action. It can initiate streaming cancellation, commanding the serving engine to stop generating an in-progress reply, terminating the invocation’s eligibility to produce an actionable candidate, and retaining partial output solely for bounded diagnosis. A canceled or truncated payload never becomes a dispatchable call, even if a tolerant JSON parser closes its dangling brackets, a valid tool-call prefix appeared before the abort signal, or an apparently complete response arrives after eligibility was withdrawn. The quarantining rule (The normalized status envelope), an application of invariant closure (principle 1) to the reply status, holds regardless of how the invocation ended. Late-arriving evidence remains useful only to reconcile accounting usage or explain what the serving engine executed.
↩︎ The distinction between rejected and canceled streamed replies, and the rule that a partial payload never reaches dispatch, are established in Streaming and cancellation.
3 Why Batching Does Not Shorten the Wait: The reference rate charges each generated token one full read of the weights, the single-request case. A serving engine that batches many requests shares that read among them, which raises throughput but still advances each request by one token per ordinary decode step, so batching cannot bring the per-request wait below this estimate. Techniques that emit several tokens per step change the rate, not the wait’s linear growth with the output allowance.
Output allowances and wall-clock deadlines address different operational risks. An output token ceiling enforces a hard boundary on generated volume under the serving engine’s accounting contract; wall-clock latency, however, also depends on scheduling queues, network stalls, and autoregressive decode throughput. Even under ideal conditions, waiting for a full generation pass introduces substantial delay.3 At a fixed per-token decode time, a call admitted with output allowance \(\OutputLimit\), generating at \(\PerTokenDecodeTime\), can postpone the next turn boundary by approximately
\[\TurnBoundaryDelay \approx \OutputLimit \times \PerTokenDecodeTime\]
Assuming an optimistic decode latency of 21.075 ms per generated token, the single-accelerator, single-request floor of the estimate in Estimating per-call hardware work, with no queueing or memory-access stalls, a 4,096-token allowance postpones the boundary by roughly 86.3 s. Pause or cancellation logic that passively waits for generation to complete would therefore block for more than a minute, and real serving queues only lengthen the wait. A node that splits the weights across several accelerators lowers the per-token floor but leaves the generation estimate linear in \(\OutputLimit\), so halving the allowance halves the full-allowance generation time at a fixed rate, whereas streaming cancellation removes the dependence on generation length entirely. The harness uses both: turn boundaries govern discrete operation admission, while streaming cancellation stops active generations in flight.
Stopping local execution differs from terminating remote execution. A host deadline determines when the supervisor ceases waiting and initiates intervention; confirming that remote inference has actually stopped requires positive evidence from the service. Under the settlement rule, unconfirmed resource usage remains an outstanding liability rather than an assumed refund.
10.5.2 Pause and resume policy
The guarded pause sequence established in section 10.4 holds new admissions while transitioning from READY to PAUSED. What this state means to an operator, however, depends on the policy configured for in-flight work. For the r18 test, the policy may permit the active job to finish under its existing budget limits while holding the repair loop’s continuation. Conversely, a policy requiring the job to stop immediately requests cancellation and records whatever outcome the executor confirms. A pause policy that permits execution to proceed leaves the workspace live and mutable, and the operator interface must reflect this active state. Any administrative inspection that demands quiescence requires positive confirmation that all concurrent writers have stopped, or else an isolated snapshot taken at a declared boundary. When the operator resumes the trajectory, execution proceeds along the guarded transitions defined in section 10.4; the pause policy governs only what ran while held, without altering the invariants enforced after resumption. Table 6 compares these requested changes to continuation.
| Request | Required control effect | Guarded outcome |
|---|---|---|
| Cancel | Hold new admission; request cancellation and account for owned work | FINALIZING, then a recorded cancellation outcome when finalization permits |
| Interrupt with new instructions | Hold affected proposals; validate the instruction and revise its task dependencies | READY only when the revised continuation is eligible |
| Pause | Hold continuation and preserve an owner for outstanding work | READY to PAUSED with no runnable gap |
| Resume | Reconcile evidence and recheck current permission and limits | PAUSED to READY, or remain held if guards fail |
| Kill | Stop admission and invoke available enforcement against active work | FINALIZING, with confirmed and unresolved outcomes recorded |
| Budget or deadline stop | Deny further ordinary admission; preserve funded finalization duties | FINALIZING, with the stopping reason and remaining obligations |
Mid-trajectory user steering demands the same transactional discipline. If an operator injects a steering instruction, such as directing the agent to modify a different target file, the harness cannot simply append the prompt to the conversation context and assume existing operations adapt. Adding instructions does not retroactively invalidate an already admitted file edit or revoke an active tool approval. Instead, the harness records the instruction, determines which plan steps and proposed actions are superseded, and verifies whether outstanding operations conflict with the revised task.
10.5.3 Termination and remaining obligations
When a runaway child process or wedged tool handler ignores a cooperative cancellation request, waiting for polite shutdown leaves the trajectory holding local compute and credentials indefinitely. A kill request therefore invokes immediate, non-cooperative enforcement. Trajectory-level encapsulation (principle 10) is what makes this enforcement possible: because control requests are recorded independently of model output, the runtime can stop new admission and abort the local work it supervises without waiting on the model or on an external service’s response. When a kill signal arrives, the controller halts local execution, revokes mediated access tokens, and fences off obsolete workers across the execution paths under its control. The sandbox reset contract in The reset contract determines which child processes, descendant namespaces, network channels, and allocated resources that enforcement covers. Asynchronous remote work, third-party transactions, and external handles lie beyond direct local process control and demand their own corroborating evidence.
Finalization therefore records confirmed termination separately from unresolved side effects and pending charges, which retain their designated owner under the settlement rule. The operation’s repeat contract determines whether another attempt is safe, and settlement must precede any retry (principle 8); an operation identifier alone cannot reveal what the interrupted execution actually modified.
Enforcing external control decouples trajectory continuation from model behavior and frees execution resources. Worker threads and compute slots never remain blocked while an external job or cancellation response is outstanding. By persisting these control directives and outstanding obligations within the trajectory record, the harness safely suspends execution and yields worker resources, admitting continuation only when verified events satisfy the state machine’s guards.
10.6 Supervised Waiting
When the agent dispatches the integration test for candidate r18, the model generates nothing for as long as the test runs, which is tens of seconds for the integration test priced in Residence time and service rate, and minutes for a broader suite. If the worker thread that dispatched the job blocks until the result returns, it cannot service any other trajectory in the interim, and the runtime becomes unresponsive to operator pauses or high-priority interactive requests. The repair trajectory requires no token generation while the test executes. What it requires instead is an external supervisor: a control entity that tracks the in-flight job, holds the execution continuation, and buffers incoming evidence without tying up a worker thread in a synchronous wait.
Yielding execution through an asynchronous wait4 releases the worker thread immediately, leaving the trajectory record as the explicit holder of pending state. Alternatively, a runtime with blocking workers can maintain controllability if an independent supervisor thread handles operator requests, heartbeats, and deadlines. The requirement holds for any event-loop implementation. Waiting must keep the control record accessible, and any returning event must pass through that record’s admission guards before advancing the trajectory.
4 Event-Driven Waiting: Internet servers faced the thread-per-request version of this problem when a request could wait far longer than it computed. SEDA (Welsh et al. 2001) split services into event-driven stages joined by explicit queues and applied admission control at each queue, so load was visible and bounded where work waited rather than hidden in blocked threads. A harness whose tool jobs run for minutes while a model call takes seconds has the same shape, and the trajectory record is its queue entry.
Welsh, Matt, David Culler, and Eric Brewer. 2001. “SEDA: An Architecture for Well-Conditioned, Scalable Internet Services.” Proceedings of the Eighteenth ACM Symposium on Operating Systems Principles (SOSP), 230–43. https://doi.org/10.1145/502034.502057.
10.6.1 The result wakeup pipeline
The asynchronous job interface decouples job acceptance from completion, so a result can arrive seconds or hours after dispatch, out of order, or more than once. The register-before-send rule makes that safe. For an external job, registration records the owning trajectory, the invocation and attempt identifiers, the external job handle when available, the expected workspace source revision, the observation eligibility policy, and the outstanding liability, so even an immediate callback finds the record it needs.
↩︎ The asynchronous tool invocation interface and completion polling contracts are established in Asynchronous Tool Dispatch.
Regardless of how the runtime structures concurrency, the supervisor must satisfy four distinct logical obligations when processing an incoming result. Table 7 stages them in order and pairs each with the inference it does not license.
| Stage | Harness responsibility | Tempting but false inference |
|---|---|---|
| Receive an event | Obtain a notification, status reply, timer event, or control request through the designated interface. | A notification establishes a completed result. |
| Authenticate and correlate | Check the source and match the event to its task, operation or attempt, and recorded execution context. | A matching identifier authorizes a new action or makes the payload true. |
| Update the record | Preserve reported outcome, source revision, capture limits, usage, and any unresolved obligation; decide whether the evidence is eligible for the waiting continuation. | Evidence for an earlier attempt belongs directly in the current model turn. |
| Apply the continuation guard | Evaluate pending controls, the wait policy, dependencies, current grants, and budget before making continuation eligible. | New evidence requires an immediate model call. |
The event handler generalizes the callback logic in listing 2 across all event sources. Operating under a mutex or transactional boundary, the handler updates the record, advances record_version, and checks the call’s recorded epoch against control_epoch; the version check itself happens in the guarded transition the handler then requests. Because network notifications can be duplicated, the handler must remain strictly idempotent: an identical redelivery will neither append a redundant observation, nor double-bill resource counters, nor re-enqueue an already scheduled continuation. Correlation ensures that incoming data routes to the correct trajectory record, while the tool’s formal contract defines what that evidence actually establishes.
This discipline applies equally to streamed outputs. Real-time log fragments and progress heartbeats can append data to a bounded observation store without terminating the wait interval. Conversely, observing an opening tool-call delimiter in a streaming model response represents an incomplete proposal; the supervisor takes no action until the stream completes. The model adapter retains sole responsibility for parsing and validating full invocation payloads, whereas the harness determines whether that validated proposal remains eligible under current control policies.
10.6.2 Job and batch waits
If a runtime woke the model on every status poll to ask whether to keep waiting, or treated a prompt in raw job stdout as a trigger to resume generation, a ten-minute test run would burn tokens on empty polling turns and let unverified log output drive control decisions. Instead, the harness waits for a push notification or polls external status at bounded intervals on its own, without querying the model. If a job pauses to request human or auxiliary input, the runtime logs that requirement as a structured state transition rather than a trigger for resumption. Responding to such an input requirement remains a distinctly authorized operation, and the record tracks this pending dependency explicitly rather than treating raw stdout or log lines as an implicit signal to resume model generation.
Parallel tool invocations demand an explicit continuation policy. Under an all-results policy, the harness suspends the trajectory until every member of the batch either returns valid evidence or reaches a terminal disposition, such as a timeout with unresolved side effects. Conversely, a partial-results policy permits the model to reason incrementally as individual results arrive, even while other calls remain in flight. Such a policy must explicitly track missing evidence, preventing downstream steps from proceeding under the mistaken assumption that the entire batch has converged.
For example, a completed lint result need not settle a concurrent test job. If the agent’s next planning step requires both checks on candidate r18, the lint output alone leaves the wait open. If, however, the subsequent step is an independent inspection, a partial-results policy can safely admit it under current limits. Modifying the candidate workspace while the test suite actively reads disk state introduces race conditions; simply designating invocations as parallel does not preserve the tested revision without explicit workspace isolation.
When a deadline expires, it terminates the harness’s wait interval, not the underlying remote operation. The harness logs the timeout, marks the operation as unresolved, and requests the configured intervention policy. The operation table keeps the unresolved charge open until explicit settlement rules determine whether late-arriving results can be accepted or whether re-execution is safe.
10.6.3 Results during a pause
The r18 test result that arrives during a pause follows the path traced in section 10.4.3. It settles its entry in the operation table while the trajectory stays PAUSED, and only the resume guard can let it inform the next model call. The harness event processor must therefore remain active even when continuation is suspended. The same holds during finalization, where late-arriving evidence can settle outstanding liabilities without reviving an already terminated trajectory.
10.6.4 Resources retained while waiting
Releasing a worker thread does not dismantle the underlying execution environment: sandboxed processes, disk workspaces, network sockets, buffered logs, and retained inference states remain allocated. The control record maintains explicit references to these resources and their respective lifecycles. Tool subscriptions and cloud storage incur ongoing charges, quota reservations persist until liabilities are settled, and the wall-clock task deadline continues to elapse. Even polling routines and cleanup handlers consume execution budget and must operate within configured limits.
A workspace lease guarantees the availability of candidate source edits and test artifacts according to the lifetime contracts established in Workspace leases, persisting across multiple tool boundaries. Its term bounds what a waiting trajectory can lose. If the harness never returns to renew, lease expiry reclaims the workspace; if it returns after expiry, the resumed trajectory must treat its workspace as gone rather than assume the files it left are still there. In parallel, the model serving infrastructure decides whether to retain, evict, or page out key-value cache state across turns. Releasing the harness worker does not remove the residency tax of Residence time and service rate: attention state kept resident through the wait still divides the turn rate its memory can sustain by \(1 + \ToolTime/\ModelTime\). The harness provides hints regarding expected wait durations and resumption priorities; where a hosted inference service provides no explicit cache-placement primitives, retention remains subject to provider eviction policies.
↩︎ Keep, move, and drop policies for a paused trajectory’s KV-cache state are established in Keep, move, and drop policies.
A wait is therefore a supervised interval characterized by retained evidence, active resource leases, and pending obligations. When that interval concludes, the harness transitions the continuation to an eligible state only if the record’s guards explicitly approve. Human-in-the-loop approval operates under this identical supervisory discipline: rather than waiting on an external computation, the harness suspends execution before dispatch, waiting for an authorization decision that may arrive late, be denied, or expire entirely.
10.7 Approval Gates
Consider the internal-review workflow: an agent proposes submitting candidate patch r18 along with selected verification records, but trusted policy permits submission only after an authorized engineer signs off. The proposal is complete, yet the human reviewer is unavailable. Meanwhile, the task’s deadline clock continues advancing, an operator can request a pause, and credentials or access grants can expire. The harness cannot simply leave the proposal floating in the model’s context window, nor can it allow a retry loop or subsequent model step to dispatch the action prematurely. Instead, the runtime must hold the fully formed submission in escrow while review remains unresolved.
↩︎ Action-bound approval, which ties sign-off on the r18 patch submission to its exact canonical payload, is established in Action-bound approval.
Definition 10.3: Approval escrow
Approval escrow is the harness’s retention of an undispatched proposal pending required approval and subsequent dispatch admission. The held record binds the proposal to its enclosing task, approval conditions, and expiration deadline; even after a favorable decision arrives, execution still requires current admission checks at the dispatch boundary.
- Significance: Escrow decouples proposing an operation from the authority to execute it immediately, keeping long, human-scale waits visible to supervisors and external schedulers.
- Distinction: Authorization policies define who holds the authority to approve a represented action. The harness manages the operational lifecycle of that pending decision: routing the request, tracking timeouts, and enforcing state transitions toward dispatch or refusal.
- Common pitfall: Treating a recorded approval as an unconditional instruction to execute, ignoring subsequent changes to credentials, task deadlines, branch states, or available compute capacity.
Requiring human approval is a deliberate task-policy choice rather than an automatic property of irreversible side effects. Some high-risk actions are prohibited outright; many others proceed safely under tight, automated least-privilege policies. Conversely, sensitive data disclosures or even benign, reversible mutations might require human oversight depending on operational risk. The approval gate must therefore enforce trusted runtime policy over the concrete operation, target resource, and parameter scope, never relying on the model’s self-reported assessment of whether an action is safe. Requesting review cannot expand authority: a reviewer cannot grant permissions beyond their own administrative boundaries.
When safety policy exists solely as prompt instructions inside the model’s context window, no gate actually exists. A 2025 incident shows how a prompt-only guardrail fails.
War Story 10.1: A code freeze stated only to the model (2025)
Context: In July 2025, SaaStr founder Jason Lemkin was building an application with Replit’s autonomous coding agent and placed the project under an explicit code freeze, instructing the agent in natural language not to modify code or production infrastructure (Sharwood 2025).
Mechanism: The freeze existed only as text in the agent’s conversational context. No supervisory component at the dispatch boundary validated proposed database tool calls against that restriction. Despite the instructions, the agent executed an operation that deleted the production database, subsequently claiming that rollback was impossible.
Impact: The live production database was unavailable until engineering restored it. Lemkin also reported that the agent had masked previous software faults with fabricated synthetic data, including populating a 4,000-record database with fictional user profiles.
Fix: Contrary to the agent’s assertion, database snapshot rollback functioned properly and restored the data. Lemkin observed that natural language instructions cannot reliably enforce operational boundaries in agentic coding environments, advocating for strict architectural isolation between preview, staging, and production environments.
Systems lesson: Operational freezes are admission-control invariants, and admission control belongs strictly to the execution runtime. Stating a freeze to the model merely shifts output probabilities; it provides no deterministic safety guarantees. When enforced at the dispatch boundary by checking the target endpoint and operation payload against active policy, the runtime deterministically traps or rejects forbidden mutations regardless of what the model generates. This reflects the foundational requirement of invariant closure (principle 1).
From the perspective of harness architecture, enforcing the freeze requires a single dispatch-time rule: while the freeze flag is set, any proposed mutation against the production target must be rejected or held in approval escrow. A destructive action then collapses into a harmless refusal observation fed back to the model rather than an unrecoverable side effect. Whether the model follows instructions becomes irrelevant to system correctness. The agent’s false claim that rollback was impossible highlights a related boundary: a model-generated status report can be false. The harness must verify execution outcomes and rollback states directly from its operation table, never from the agent’s conversational transcript.
Approval gates must also remain selective and narrow. When a human operator faces dozens of routine, low-risk confirmation dialogs every hour, fatigue leads to rubber-stamping without inspection.5 A gate where every request is approved adds no closure beyond the policy check that routed the request to it. Fine-grained authorization grants, sandboxes, and automated linters should handle the vast majority of verifiable actions. Reserving manual approval for high-blast-radius or policy-mandated exceptions keeps human attention on the decisions that need it.
5 Ironies of Automation: Bainbridge observed that automating most of a process leaves the human operator the residual job of monitoring it, and that people sustain attention poorly on a stream in which almost nothing goes wrong (Bainbridge 1983). An approval queue in which nearly every request is benign is that stream. The one harmful proposal arrives in a queue that has trained the reviewer to expect nothing, which is why the gate’s precision, not its presence, determines what closure it supplies.
Bainbridge, Lisanne. 1983. “Ironies of Automation.” Automatica 19 (6): 775–79. https://doi.org/10.1016/0005-1098(83)90046-8.
10.7.1 Held proposals
If an approval request pointed only to a mutable workspace path or an unversioned tool call, a concurrent edit or plan revision could change the underlying patch between the moment the reviewer inspected r18 and the moment the harness dispatched the submission. After validating a completed model response, the harness therefore freezes the held proposal in the trajectory record before publishing its approval request. The record references the task and requester, intended tool origin and operation, immutable candidate artifact and selected evidence, applicable preconditions, required approver policy, and approval expiry. It also assigns an approval-request identifier and a held-proposal version, and computes the approval digest, the digest that action-bound approval binds, computed here over the canonical held proposal and covering the operation, its arguments, its target, and the versions of the referenced artifacts. The approval binds to that digest, so changing any argument or artifact produces a different digest and voids the approval. These identifiers and the cryptographic digest track the pending decision itself, distinct from the logical operation IDs or transport-layer idempotency tokens assigned once dispatch begins.
For the review submission, the held artifact is the exact r18 patch and named check records, and the selected records retain their evidential scope; a targeted unit-test pass does not become full acceptance because it appears in an approval request. The authentication and represented-action requirements are those of action-bound approval. The harness adds the recorded status needed to distinguish pending, denied, expired, withdrawn, and admitted requests. Figure 3 diagrams this approval pipeline: panel (a) binds the exact proposal payload, artifact hashes, destination scope, and expiration timestamp into an immutable approval digest for revision r18, while panel (b) traces the four-step resolution sequence that parks the trajectory in AWAITING_APPROVAL until reviewer decisions and dispatch guards are verified.
The trajectory enters AWAITING_APPROVAL without sending the held tool call. Parking the trajectory in this state releases its active compute worker; polling for a human response requires an external callback or light notification check rather than expensive model inference. The suspended trajectory retains its proposal record, cryptographic digest, and artifact references, while optionally maintaining its workspace lease and sandbox storage.
Under stable flow, Little’s law (theorem 5.1) makes the mean number of trajectories held in AWAITING_APPROVAL equal to the rate of approval requests multiplied by the mean time to a decision, \(\ApprovalHeldTrajectories = \ArrivalRate\,\ExternalWait\). Review latency, not model speed, dictates how many concurrent held trajectories and retained resources the system carries. The relation fixes the mean, not the peak: a burst of review requests after an incident, or a reviewer who clears the queue once a day, holds far more than the mean for as long as the backlog lasts, so the pool for held workspaces is sized from the arrival pattern. The approval expiry strictly bounds this wait for a decision, while the trajectory’s overall deadline clock advances independently. Which operational cost dominates, retained sandbox footprint or delayed mitigation, depends on the workload profile.
↩︎ Little’s law, which equates mean occupancy with arrival rate times mean residence time, is established in Residence time and service rate.
Lighthouse 10.1: Remediation agent (approval escrow)
Workload: The remediation agent diagnoses a production incident and proposes a change, such as restoring a previous service configuration, that policy holds for an on-call reviewer. The repair agent’s gated action is instead a review submission near the end of a long trajectory. Both wait in
AWAITING_APPROVAL, and Little’s law, \(\ApprovalHeldTrajectories = \ArrivalRate\,\ExternalWait\) for the \(\ApprovalHeldTrajectories\) trajectories held at once over a decision time \(\ExternalWait\), prices each wait. All rates and times below are scenario inputs.
Repair agent: Review requests arrive at 6 per hour and a decision takes 2 hours on average, so about 12 trajectories are held at once. Each keeps its workspace lease, 512 MiB in the reference coding-agent profile, so escrow retains 6 GiB of sandbox memory while no user waits on the outcome.
Remediation agent: Requests arrive at 2 per hour, so on average 0.5 trajectories are held and retained resources are negligible. The cost is time: with 6 minutes of diagnosis, a 15-minute mean decision, and 9 minutes to apply and verify, approval is 50 percent of the 30-minute time to mitigate, all of it spent with the service degraded.
Systems insight: One escrow mechanism binds on different terms. For the repair agent, retained memory scales with \(\ExternalWait\), so halving the decision time halves the held workspace. For the remediation agent, each minute removed from \(\ExternalWait\) removes a minute of degradation, and approval is the largest term in time to mitigate. Remediation therefore binds the approval digest to one exact configuration version so review is quick, sets the approval expiry shorter than the interval over which production configuration drifts, and lets already-admitted independent read-only diagnostic jobs continue under their existing grants and limits while the mutation waits.
10.7.2 Approval decisions and duplicate delivery
Webhook deliveries from an approval service can arrive twice after a network retry, or arrive late after the request has already expired or been withdrawn by an operator. Rather than blindly treating any inbound approval packet as an immediate trigger to resume execution, the harness requires an authenticated decision cryptographically tied to the held request identifier and version, validates the decision against the active escrow record, and recalculates the approval digest. Duplicate webhooks or replayed messages are idempotent; the harness drops repeated deliveries without recording duplicate decisions or re-triggering dispatch. Similarly, a late decision arriving for a request that has already been withdrawn, expired, or superseded serves only as historical audit evidence; it can never revive a closed request. Table 8 distinguishes recording such a decision from admitting the operation.
Event while in AWAITING_APPROVAL |
Recorded handling | Continuation |
|---|---|---|
| Required decision has not arrived | Keep the proposal undispatched until its expiry or another control event | Remain AWAITING_APPROVAL |
| Valid approval arrives, but capacity is unavailable | Record the decision; retain the held request without dispatch | Remain AWAITING_APPROVAL; recheck at admission |
| Reviewer denies, request expires, or proposal becomes stale | Close the held request and record the specific refusal reason | Return to READY only if continuation remains permitted |
| Approval is valid and every current admission condition holds | Consume the held request for dispatch, reserve liability, and register the operation | Enter WAITING_TOOL and issue the admitted request |
| Task deadline or terminating control takes precedence | Withdraw any still-undispatched request and record the stop reason | Enter FINALIZING |
Silence leaves the request unapproved. When the approval timeout lapses, the runtime logs an expiration event rather than an active human rejection. Both outcomes prevent dispatch, but the distinction matters when the trajectory is later debugged or audited. If the agent’s task policy permits continuation after a refusal, the harness delivers a compact observation record confirming that the operation was withheld:
{"approval_id": "review_r18",
"status": "REFUSED", "reason": "APPROVAL_EXPIRED",
"dispatch_status": "not_sent"}The model can then interpret this refusal observation and propose an alternative strategy. What it cannot do is unilaterally extend the expired approval by submitting identical requests in a tight loop. Generating a fresh approval request requires constructing a current proposal that undergoes a new policy evaluation; otherwise, an automated agent could flood the reviewer with repeated requests, a denial of service through approval fatigue. If the enclosing task has exhausted its wall-clock budget, the harness records the refusal directly in the trajectory’s final record without expending further tokens or compute to solicit a narrative explanation from the model.
10.7.3 Dispatch-time rechecks
A human approval satisfies governance policy, but it does not guarantee that the runtime environment remains ready for execution. Between the moment a reviewer signs off and the moment the worker prepares to dispatch, the target branch may have advanced, the task’s token or financial budget may have been exhausted, or short-lived credentials may have lapsed. Human approval is merely one precondition for admission, never an exemption from standard admission control.
At the dispatch boundary, the harness re-evaluates the entire operational state. It recomputes the approval digest to ensure the artifact has not been modified in place, checks that the approval lease has not expired, and executes the full suite of rate-limit, budget, and authorization checks. Because queueing delays and downstream rate limits can stall an approved request in escrow, checks validated when the approval message arrived cannot be grandfathered into execution. If any check fails, the harness logs the specific refusal reason; if a task-level termination triggers, the trajectory routes immediately to finalization.
Consider a submission that requires an unmodified target branch. Reading the branch HEAD before requesting approval cannot prevent concurrent commits during the minutes or hours the request sits in escrow. When operations carry concurrency preconditions, the dispatching endpoint must verify the expected target version atomically alongside the mutation, or use an external optimistic locking mechanism.
↩︎ The atomic conditional update, which verifies an expected version in the same step as the write, is established in Conditional updates.
Admitting a held request follows the register-before-send protocol with one additional step in the same atomic update. Under an exclusive lock, the harness registers the outgoing operation and marks the held escrow record as consumed, which prevents duplicate approval callbacks or concurrent workers from dispatching the action twice and strictly orders an approver’s sign-off against a racing operator cancellation or timeout. If a cancellation commits first, it withdraws the held request and records an unambiguous not_sent status; if admission commits first, the operation becomes owned work, and while a later control may still withhold release through the serialized gate, the harness retains the record and reservation until non-release and the absence of remote liability are established. If an unhandled transport failure occurs after the packet departs the host, the harness must treat the remote effect as indeterminate. While marking an approval consumed prevents duplicate local dispatch, it cannot guarantee endpoint deduplication or resolve an in-flight network partition.
10.7.4 Withdrawal of held requests
If an operator pauses a trajectory while a proposal sits in AWAITING_APPROVAL, leaving the request active in the review queue would let a reviewer’s click trigger dispatch while the trajectory is supposed to be held. Pausing from AWAITING_APPROVAL therefore follows the supervisory lifecycle in section 10.4 with an explicit escrow cleanup step: the harness immediately withdraws the pending approval request, marks the proposal as not dispatched in the audit trail, and transitions through READY to PAUSED. If an approver submits a late decision while the task is paused or after it resumes, the harness rejects the decision because its associated request was already revoked. Should the agent still require the operation upon resuming, it must generate a fresh proposal subject to current state and policy checks.
Cancellation and task expiration route directly into FINALIZING. This terminal path requires all previously dispatched work and outstanding external resources to be settled or transferred to an accepting owner. Withdrawing a held review submission leaves no side effects on external targets, but it does not automatically terminate concurrent asynchronous test jobs, settle their compute costs, or undo previously committed mutations. By maintaining a strict separation between held proposals in escrow and committed entries in the operation table, the harness provides unambiguous operational visibility even when complex trajectories terminate abruptly.
Checkpoint 10.2: Approval over time
The internal-review submission for r18 is held while the task approaches its deadline.
The approval gate now has a bounded lifetime and an explicit handoff from held proposal to admitted work. Human agreement is one condition in that handoff, alongside current authority, resource bounds, and action-time preconditions. Multiple trajectories can satisfy those conditions while competing for the same execution capacity, so the harness next needs a policy for sharing that capacity without bypassing their individual gates.
10.8 Scheduling Concurrent Trajectories
When the test suite for candidate r18 finishes and the repair trajectory returns to READY, it rarely has the serving cluster to itself: an interactive user session and a backlog of repository maintenance tasks may already be waiting to query the model. If the harness simply dispatches requests as soon as tasks generate them, aggressive retry loops will monopolize model capacity; if it unconditionally favors interactive requests, large maintenance jobs can wait indefinitely. Concurrency alone does not solve resource allocation, so the harness needs an explicit scheduling policy to arbitrate among eligible calls.
The fundamental unit of scheduling admission is a READY trajectory’s next model call, rather than its entire multi-step task or the downstream tool execution that the reply may propose. Completing an asynchronous wait does not grant immediate dispatch; it makes a trajectory eligible only through a guarded transition into the READY state, where it queues for the scheduler’s approval. Downstream, the inference serving engine manages batching, KV-cache allocation, and accelerator execution. That low-level engine scheduling remains strictly decoupled from the harness’s control plane, which decides which high-level trajectory may submit work in the first place.
10.8.1 Fair sharing and the scheduling charge
Granting an equal number of model calls across classes does not deliver equal service. A single maintenance prompt containing extensive code diffs can consume 40,000 tokens, whereas a concise interactive query requires only 2,000 tokens. A scheduler must therefore define what resource its allocations apportion. The policy used here bills each admitted request a declared scheduling charge equal to its full input-token count plus its maximum allowed output-token reservation. This conservative metric measures admitted token commitments rather than actual generated output, monetary cost, accelerator occupancy, or task utility. Cached prompt tokens remain part of this upfront charge because serving-engine cache hits cannot be guaranteed before dispatch.
Weighted deficit round-robin provides an effective, low-overhead mechanism for apportioning this declared token charge across diverse service classes. For each class, the scheduler keeps a credit counter and a queue of the class’s READY trajectories, served in rotation from a pointer that persists between rounds. At the beginning of each scheduling round, the system increments each class’s credit counter by a quantum proportional to its assigned weight. The scheduler then steps through the classes, admitting calls as long as their token charges fit within the available credit and all downstream capacity gates pass. Each admission deducts the call’s charge from the class balance. Unspent credit carries over into subsequent rounds, allowing a class with large requests to gradually accumulate enough deficit credit to admit a multi-thousand-token prompt. When a class exhausts its active work, the scheduler resets its credit counter to zero so it cannot hoard unearned allocations during idle periods. This design adapts the deficit carryover mechanism6 of deficit round-robin (Shreedhar and Varghese 1995) from network packet scheduling to language model admission control.
6 From Fair Queueing to Deficit Round-Robin: Fair queueing for packet networks (Demers et al. 1989) emulated a bit-by-bit round-robin across flows, ensuring that high-volume sources could not exceed their fair shares, but it required ordering packets by computed finish times. Deficit round-robin reached comparable fairness with constant work per packet by giving each queue a quantum and carrying unspent credit forward (Shreedhar and Varghese 1995). Because model-call charges span orders of magnitude, deficit carryover, rather than simple queue rotation, is what enforces proportional fairness here.
Demers, Alan, Srinivasan Keshav, and Scott Shenker. 1989. “Analysis and Simulation of a Fair Queueing Algorithm.” Proceedings of the Symposium on Communications Architectures and Protocols (SIGCOMM ’89), 1–12. https://doi.org/10.1145/75246.75248.
Shreedhar, M., and George Varghese. 1995. “Efficient Fair Queuing Using Deficit Round Robin.” Proceedings of the Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication (SIGCOMM ’95), 231–42. https://doi.org/10.1145/217382.217453.
Credit belongs to the service class rather than to individual trajectories, so a tenant cannot enlarge its share by spawning hundreds of concurrent tasks. Within a class, the rotation pointer persists across passes; a scan that restarted at the head of the queue on every pass would let trajectories near the front absorb fresh credit while those near the tail starved. Every class expected to make progress needs a positive weight, and any ceiling on accumulated credit must accommodate the largest single request the policy permits. A credit cap below that size leaves the largest calls stalled indefinitely despite repeated replenishment rounds.
Napkin Math 10.1: Charge shares vs. call shares
Problem: An interactive class has weight 3 and a maintenance class has weight 1, with a base quantum of 8,000 charge tokens per unit weight per round. An interactive call carries 2,000 input tokens and a 500-token output allowance; a maintenance call carries 40,000 input tokens and a 2,000-token output allowance. Assume both classes remain continuously backlogged at these fixed call sizes, no provider or local-capacity gate binds, and carried credit can cover the largest permitted call. Over many rounds, how many calls does each class admit?
Math: Each round credits 24,000 tokens to the interactive class and 8,000 tokens to maintenance. Because interactive calls incur a scheduling charge of 2,500 tokens each, the interactive class averages 24,000 / 2,500 = 9.6 calls per round. A maintenance call incurs a charge of 42,000 tokens, so its class must carry forward credit for 42,000 / 8,000 = 5.25 rounds per call.
Result: The interactive class receives 75 percent of the total token charge capacity, but it dispatches roughly 50 times as many admitted calls as the maintenance class.
Systems insight: Class weights determine the long-term division of token charges, while admitted call rates depend directly on request size. The maintenance class’s mean interval between admissions, measured in credit rounds, equals its request charge divided by its per-round quantum under the stated assumptions. Halving its input to 20,000 tokens reduces this aggregate interval to approximately 2.8 rounds per admission without changing class weights. An individual trajectory also waits behind its class peers; execution, tool waits, and approval time lie outside this quotient. Increasing a class weight shifts its aggregate token share; shrinking prompt sizes (through prompt compaction or targeted context retrieval) accelerates its call dispatch rate. Finally, the scheduler’s credit ceiling must accommodate at least one complete maintenance charge, or that class will never accumulate sufficient deficit credit to dispatch.
Weights split the token charge; request size then sets a far larger split in calls.
Configuring class weights fixes how the declared token charge is shared, not the proportion of admitted calls, and it divides capacity in proportion to weight rather than creating capacity. Under sustained overload, scheduling weights redistribute queueing delay without increasing provider throughput, so bounding latency under persistent saturation requires admission limits on incoming tasks or explicit rejection.
10.8.2 Quota and capacity gates
A model call selected by the fairness scheduler can still fail at dispatch with an HTTP 429 rejection if the tenant’s provider quota is full, or stall downstream if local sandboxes are out of memory. Before dispatching a nominated call, the harness must therefore pass two capacity gates: external provider quotas and local execution capacity. Model providers enforce rate limits across distinct dimensions—requests per minute (RPM), tokens per minute (TPM), and concurrent in-flight requests—and the provider’s accounting contract determines what must be reserved before transmission. For token limits, providers typically debit the full input count plus the declared maximum output allowance against the current time window until generation finishes. Pricing discounts do not relax rate limits; a provider that discounts cached prompt tokens on the invoice may still count those tokens at full weight against the tenant’s rolling TPM window. Satisfying a token quota likewise guarantees nothing about request concurrency, which is capped independently to protect serving infrastructure.
A local rate limiter lets the harness hold pending calls rather than discover provider saturation through HTTP 429 (Too Many Requests) rejections. It must account for all traffic that shares the provider allocation, including harness processes on other machines, and when that traffic or the provider’s windowing algorithm is opaque, it keeps safety headroom and adjusts its refill rate from response headers that report remaining quota and suggested retry intervals.
Local execution resources impose a second set of gates. A tool call that compiles code or runs an integration test needs sandbox slots, CPU cores, memory, and storage bandwidth even when model token quotas are abundant, and each bottleneck must be tracked in its own physical unit: bytes of disk space, bytes per second of transfer, operations per second of storage I/O. Counting homogeneous sandbox slots suffices only when every job fits an identical envelope; with heterogeneous jobs, slot counts alone do not prevent out-of-memory failures. Tool dispatch therefore has its own admission queue, shared by policy in these physical units rather than in model tokens. A protected action whose approval arrives when no sandbox slot is free stays in AWAITING_APPROVAL alongside its recorded decision (table 8) instead of returning to READY and issuing another model call.
Selection by the scheduler is a nomination, not an execution grant. Between the scheduler picking a candidate and committing its dispatch, an operator can pause the trajectory, a grant can expire, or a concurrent step can consume the remaining budget, so the nominated call still passes the atomic admission check, which records every quota and capacity reservation before the request leaves the host. If two scheduler workers nominate the same candidate, the record_version check lets exactly one commit. A call blocked by insufficient credit, an exhausted rate limit, or a full sandbox pool stays in READY; that delay is backpressure, not task failure. Once a call is sent, its reservations follow the settlement rule. An HTTP timeout after transmission keeps them outstanding, because the provider may have processed and billed the request, and provider-quota refunds follow the provider’s own accounting contract.
Fair sharing decides which eligible trajectory is served next, and quota and capacity gates decide whether that work can run now. Neither limits the total work one trajectory requests across many turns; a repair agent can receive its fair share of tokens indefinitely without converging on a valid patch. That limit is the trajectory budget.
10.9 Budgets and Ceilings
Suppose a software-repair agent has already consumed nearly its entire task budget while an asynchronous test suite dispatched minutes earlier has yet to report its cloud execution charges. Proposing another model invocation under those conditions introduces new, uncommitted liability: a fair scheduler can decide when that candidate call should receive accelerator service, yet scheduling alone cannot tell whether the trajectory has enough funds left to afford both the pending test bill and the new generation. Preventing runaway spending therefore requires an admission rule that accounts for in-flight work and worst-case execution liabilities before dispatching another request.
10.9.1 Turn, token, time, and dollar budgets
A runaway retry loop shows why a trajectory needs a lifetime bound. When an agent generates a candidate code patch that fails its regression test, the model often responds by proposing a slight variant, followed by another permutation and eventually a broader diagnostic routine that also fails, with each failure appending an error trace to the conversational context. Because every individual tool call remains syntactically well formed and stays strictly within per-invocation token and compute limits, the trajectory yields cleanly at every tool boundary, receives its scheduled turn on the inference server, and satisfies every local admission check—yet dozens of turns later it has burned through substantial cloud and inference budgets while making zero measurable progress toward resolving the defect. Figure 4 diagrams this cycle, in which failing tests append repeated error observations to the context until turn-completion proposals vanish, together with the external budget that halts it at the admission boundary and moves the trajectory into FINALIZING once the next call’s liability no longer fits the allowance.
Ending such a loop is the harness’s responsibility. As failure outputs accumulate in the prompt context, they dilute the original system instructions and task specifications. Under severe context dilution, model generations that terminate the trajectory become increasingly rare. Prompt-level guardrails, such as injecting notices about remaining funds or requesting that the model self-regulate under a cost ceiling, offer probabilistic nudges rather than deterministic bounds, and their effect weakens as the context fills with repeated stack traces. Because an errant model remains syntactically coherent and locally compliant while failing globally, trajectory bounds must reside within the external supervisory control plane (principle 10). Deciding whether an agent has stalled requires complex semantic heuristics, but an external budget deterministically stops further ordinary admission when the allowance is exhausted.
↩︎ Attention dilution as stale observations and superseded logs accumulate in the working context is established in Testing usable working context.
To constrain total resource consumption, the harness establishes a multi-dimensional per-trajectory budget, the vector
\[\TrajectoryBudget = \langle \TrajectoryTurnLimit,\ \TrajectoryTokenLimit,\ \TrajectoryTimeLimit,\ \CostCeiling \rangle\]
where the components establish formal allowances on execution turns (\(\TrajectoryTurnLimit\)), cumulative tokens (\(\TrajectoryTokenLimit\)), elapsed wall-clock time (\(\TrajectoryTimeLimit\)), and total financial cost (\(\CostCeiling\)). The harness configuration must unambiguously define what increments each meter: admitted model invocations increment the turn counter, effective input prompt tokens plus generated output tokens increment the token counter, and direct API charges, sandboxed execution fees, and paid third-party tool usage increment the dollar counter. Retried operations, network reconnects, and failed invocations consume real resources and must be billed against these totals. Production harnesses frequently extend this vector with additional domain-specific limits, such as sandbox disk allocations or maximum external API query counts.
Each dimension governs a distinct operational constraint. A trajectory with very few turns can rapidly exhaust financial limits through large-context reasoning models, while an inexpensive agent can tie up execution sandboxes across long asynchronous polling delays. Unlike consumable token or dollar allowances, elapsed time represents a monotonically advancing wall-clock deadline measured from task initiation. Network delays, sandboxed compilation phases, human approval holds, and backoff pauses all advance this deadline unless the task contract explicitly provides a paused clock. When the primary execution deadline expires, the harness halts the admission of new exploratory work while supervising in-flight background operations. Post-execution finalization requires its own separately allocated time allowance and reserved funding; an expired exploration deadline must never trigger an unconstrained cleanup phase that runs indefinitely.
The harness may safely expose current budget metrics to the model, enabling the agent to adjust its planning strategy toward cheaper alternatives. However, the model must never possess the authority to modify accounting records, release reservations, or exempt its own calls from admission gates. This separation allows cooperative optimization while guaranteeing that system bounds remain strictly invariant to model behavior.
10.9.2 Reservation and settlement
If the harness checks only settled invoices before admitting a new model call, a trajectory with an unbilled test job in flight and a 4,096-token generation ahead can pass admission and then blow past its dollar ceiling once both bills arrive. A budget serves as a hard ceiling only when the admission gate sets aside the worst-case cost of in-flight and proposed work before authorizing new liabilities.7 The amount set aside for an admitted operation is its budget reservation, the operation’s maximum additional charge, withheld from the allowance at admission and released only when evidence settles the operation. The chapter keeps the word escrow for held proposals awaiting approval, which hold an action rather than an amount. Test-time search reserved each candidate’s maximum completion before dispatch and kept unsettled reservations against its quota; the harness extends that discipline from one search to every operation of the trajectory, including tool jobs and the finalization reserve.
7 Reservation Against an Invariant: Database systems solved a version of this problem for shared quantities such as account balances and inventory counts. O’Neil’s escrow method (where escrow names a set-aside quantity, not the held action of this chapter’s approval escrow) let a transaction set aside part of a quantity only after testing that the quantity’s invariant would hold whichever concurrent transactions later committed or aborted (O’Neil 1986). Bounded-liability admission applies the same test to a trajectory’s allowance, with the operation’s maximum possible charge standing in for an exact decrement.
O’Neil, Patrick E. 1986. “The Escrow Transactional Method.” ACM Transactions on Database Systems 11 (4): 405–30. https://doi.org/10.1145/7239.7265.
↩︎ Reserving a candidate’s maximum completion before dispatch, within a task-wide quota, is established in Reserving budget before admitting work.
Definition 10.4: Bounded-liability admission
Bounded-liability admission is the harness rule that admits a new operation only when, in every cumulative budget dimension, actual consumption plus outstanding reservations plus the operation’s maximum additional charge plus the protected finalization reserve fits within the allowance.
- Significance: It turns a budget that is merely reported into an enforceable upper bound on per-attempt expenditure (the numerator of cost per accepted task, equation 1.2) while guaranteeing that essential post-task verification checks remain fully funded.
- Distinction: Whereas a simple spend check merely compares settled invoices against the allowance, bounded-liability admission reserves capacity for in-flight requests and charges the worst-case ceiling of the pending operation.
- Common pitfall: Prematurely releasing a reservation upon a network timeout or client-side cancellation before upstream providers settle actual usage. This error exposes uncommitted budget that concurrent operations can claim before delayed billing charges arrive.
For each cumulative resource dimension, the admission test is
\[\text{actual consumption} + \text{outstanding reservations} + \text{proposed bounded liability} + \text{protected finalization reserve} \leq \text{allowance} \tag{1}\]
The harness evaluates this inequality componentwise across all cumulative resource dimensions, while evaluating wall-clock deadlines through a separate timing guard. The term for outstanding reservations encompasses all admitted but unsettled operations, including concurrent inference requests and active paid tools. The protected finalization reserve isolates dedicated funding for validation runs, report generation, and environment teardown, preventing exploratory steps from exhausting the funds needed to verify final results. When the trajectory transitions into finalization, the harness reallocates the necessary portion of this protected reserve into active operation reservations, avoiding duplicate accounting. Because the inequality is the budget clause of the admission check, it runs under the record lock with the rest of that check, and the register-before-send rule keeps two concurrent subtasks from reading the same uncommitted headroom and spending it twice.
With the budget clause in place, every piece of the harness has been introduced, and listing 3 assembles them into one turn: the guarded transition of listing 1, the reservation that admission makes, the status routing of table 4, the approval hold, and register-before-send for tool calls. The reservation is made inside the guarded transition into CALLING_MODEL, so an unaffordable call is refused before any request leaves the host. Usage settles the reservation after the reply, and the reply’s status decides the next state before any content can become a tool call. Admission never depends on a charge discovered after the call.
def run_turn(rec):
call = new_invocation(rec)
liability = bound_call(rec.limits) # full input + output allowance
transition(rec, "CALLING_MODEL", Decision(call.id, rec.record_version),
admit=lambda r: reserve(r, call, liability)) # admission test
reply = invoke_model(rec.context(), rec.limits, call.id) # lock released
settle(rec, call, reply.usage) # missing usage stays reserved
nxt = Decision(call.id, rec.record_version)
if reply.status != "COMPLETED": # truncated, canceled, refused, ...
return transition(rec, route_status(rec, reply), nxt) # never dispatched
proposal = validate(rec, reply) # schema, epoch, authority
if proposal.kind == "complete":
return transition(rec, "FINALIZING", nxt) # completion check follows
if proposal.kind == "none":
return transition(rec, "READY", nxt)
if needs_approval(rec, proposal):
return transition(rec, "AWAITING_APPROVAL", nxt,
admit=lambda r: hold(r, proposal))
transition(rec, "WAITING_TOOL", nxt,
admit=lambda r: reserve_and_register(r, proposal.calls))
dispatch(proposal.calls) # serialized release gate; already reservedThe reserve step evaluates equation 1 and aborts the state transition whenever the proposed invocation exceeds available headroom; route_status maps terminal inference conditions according to table 4. For clarity, the listing abstracts batched invocation checks and multi-step approval lifecycles. Its invoke_model and dispatch helpers also include the serialized release gate: they recheck the admitted operation against current controls before release, retain ownership and liability if release is withheld, and exit through guarded refusal or control handling without returning a fabricated remote reply. These helpers perform no second reservation and keep slow network I/O outside the record lock. This listing describes live-host supervision; crash reconstruction requires the durable evidence developed in Durable Execution.
A control action can also commit while run_turn is blocked in invoke_model, because that call runs outside the lock. The control path then performs the pause entry itself: it withdraws the call’s eligibility, cancels the stream if policy says so, and steps the record through READY to PAUSED. When invoke_model returns, settle still records the call’s usage, but the next transition starts from PAUSED. Every edge it could request from there fails: the edges to WAITING_TOOL and AWAITING_APPROVAL are not legal from PAUSED, and the guards on the legal ones, to READY and FINALIZING, admit only the control path’s own resume or stop decisions. The transition raises IllegalTransition, and the reply is kept only as diagnostic evidence, as a CANCELED reply would be.
For a model call, the proposed bounded liability prices every input token at the undiscounted input rate \(\InputTokenPrice\), cached or not, and the full output allowance \(\OutputLimit\) at the output rate \(\OutputTokenPrice\):
\[\text{liability} = \InputTokenPrice\, \SequenceLength + \OutputTokenPrice\, \OutputLimit \tag{2}\]
Here \(\SequenceLength\) counts the whole prompt, unlike the uncached-only count used for settled charges in section 10.9.3. Pricing the prompt at full rate keeps the bound valid when an expected cache hit fails to materialize, and charging the whole allowance keeps it valid when the model generates up to its cap; settlement later replaces both worst cases with reported usage.
Even when the unreserved balance remains strictly positive, it may prove insufficient to cover the maximum liability of the next operation. A naive spend check admits such a call on the basis of past settled invoices, only to exceed the budget ceiling once the provider bills the resulting tokens. While shrinking the requested maximum output tokens can lower the call’s worst-case liability, this optimization succeeds only if prompt tokens and outstanding tool charges still fit within the margin. External paid tools that lack enforceable execution timeouts or bounded billing tiers cannot provide a finite liability ceiling either. Under a strict enforcement policy, the harness must either reject such open-ended tools outright, substitute bounded proxies, or explicitly declare an approximate threshold that tolerates financial overshoot. Enforcing a client-side network timeout does not provide an immutable cost ceiling because the remote provider may continue billing until its own worker processes finish.
Napkin Math 10.2: A call the spend check admits
Problem: A repair trajectory has a $3.50 allowance, $2.58 of settled spend, a $0.50 reservation on its unsettled test job, and a $0.30 finalization reserve. The next model call carries 50,000 input tokens and a 4,096-token output allowance, reserved at hypothetical prices of $3 and $15 per million input and output tokens. Should the harness admit it?
Math: The call’s bounded liability is $0.15 of input plus $0.06 of output, $0.21 in all. A spend check sees $3.50 − $2.58 = $0.92 of headroom. Equation 1 subtracts the reservation and the finalization reserve as well, leaving $0.12.
Result: The spend check admits the call; bounded-liability admission refuses it. Admitted, the call would leave the final checks $0.09 short once the test settles in full.
Systems insight: Input is 70.9 percent of this call’s liability, so trimming the output allowance barely helps. The call fits only if its input falls to about 19,500 tokens, through compaction, or if the test job settles and releases part of its reservation. Every $0.10 of unsettled liability removes about 33,300 input tokens from what the next call may carry, which is why settling promptly from usage evidence is worth more than a smaller output cap.
The spend check sees room for the call; after reservations, it no longer fits.
Settlement replaces the reservation with verified consumption once the harness receives an authenticated, correlated usage report from the provider. An incremental report moves confirmed charges into the settled meter while the remaining liability stays reserved, and only terminal billing evidence releases the residual; a dropped stream, missing telemetry, or client-side timeout shows nothing about what the provider billed. Cached input is reserved at the full rate unless the provider contractually guarantees the hit, and it is settled at the cached rate the usage report shows. Canceling a stream early can likewise lower the eventual output charge, but admission of later work cannot count on a cancellation that has not yet settled. The ceiling holds because every request passes the check (complete mediation (Saltzer and Schroeder 1975)), every reservation is a worst-case bound, and only verified execution records settle it.
10.9.3 Metered accounting and prefix cache reuse
Once a model call or tool job finishes, the harness replaces its worst-case reservation with the actual bill, and that bill depends heavily on whether the serving engine reused cached prefix state or had to reprocess the prompt from scratch. Inference providers bill token usage in distinct categories: uncached input tokens (\(\UncachedInputTokenCount\)), cached prompt tokens (\(\CachedInputTokenCount\)), and generated output tokens (\(\OutputLength\)). For any completed turn \(\TurnIndex\), the harness updates the accumulated cost according to:
\[\SettledTrajectoryCost^{(\TurnIndex)} = \SettledTrajectoryCost^{(\TurnIndex-1)} + \InputTokenPrice\, \UncachedInputTokenCount^{(\TurnIndex)} + \CachedInputTokenPrice\, \CachedInputTokenCount^{(\TurnIndex)} + \OutputTokenPrice\, \OutputLength^{(\TurnIndex)} + \sum_{j} \ToolCallCost(\ExecutedToolCall_{t,j})\]
where input token categories are mutually disjoint, unit pricing matches token dimensions, and each tool execution charge \(\ToolCallCost\) is counted exactly once. This equation defines how in-flight reservations turn into settled charges against the budget.
The notebook below meters the append-only context workload of How multi-turn history drives up input cost against a full budget vector. It assumes that the serving engine reuses KV-cache state for the system prompt and prior turns and bills each newly appended observation and tool output as uncached input. That reuse is not guaranteed by append-only text alone; the matching cache state must still be resident on the worker that serves the next call.
Napkin Math 10.3: Metering a trajectory against its budget
Problem: The coding workload has completed 30 turns under hypothetical allowances of 40 turns, 2 million effective input-plus-output tokens, 900 s elapsed, and $3.50. The clock reads 460 s. Assume all usage is settled, no paid-tool charges, and prices of $3 per million uncached input tokens, $0.30 per million cached input tokens, and $15 per million output tokens. Which allowance has the largest current used fraction in the reuse and no-reuse cases?
Math: With a base prompt of 8,000 tokens, 1,500 tokens appended per turn, and 200 output tokens per turn, equation 2.9 gives 892,500 input tokens over the trajectory, 898,500 with output. Under prefix reuse, every turn’s earlier prefix is billed as cached input, 841,000 tokens, so the model charge is $0.50. Billed entirely as uncached input, the same tokens cost $2.77.
Result: With reuse, the trajectory has used 75 percent of its turns, 44.9 percent of its tokens, 51.1 percent of its time, and 14.2 percent of its dollars, so turns lead. Without reuse, dollars lead at 79.1 percent.
Systems insight: The leading dimension changes when caching fails. The token meter counts cached input in full and cannot see the change; the dollar meter rises 5.5 times for the same tokens. A harness that sizes \(\CostCeiling\) from the cached bill will therefore meet its dollar ceiling first on exactly the trajectories that miss the cache. Reserving at the undiscounted rate and sizing the dollar allowance from the no-reuse cost keeps a cache miss from turning into a budget stop; what reuse is worth across a workload is priced in Pricing prefix cache hits.
10.9.4 Staged budget escalation and enforcement
If a trajectory runs at full scope until the admission gate abruptly refuses its next call, it often halts mid-investigation without compacting its findings, narrowing its tool use, or producing a useful diagnostic summary. A well-designed harness therefore introduces progressive interventions before the admission gate forcibly shuts down exploratory execution. Its primary warning signal is the maximum consumed fraction across all active budget dimensions, \(\max(\TurnBudgetUsedFraction, \TokenBudgetUsedFraction, \TimeBudgetUsedFraction, \CostBudgetUsedFraction)\), computed with unbilled reservations included rather than from settled invoices alone, while the wall-clock timer yields the temporal fraction \(\TimeBudgetUsedFraction\). This multi-dimensional indicator steers adaptive interventions, even as the underlying gate continues to evaluate bounded liabilities independently across each axis.
Table 9 delineates the progressive escalation stages that precede hard enforcement. A deployment policy triggers each stage from the admission terms themselves rather than from a fixed percentage of the allowance, because a fixed percentage ignores how large the next step is. Steering begins when the remaining headroom can fund only a few more calls at their current bounded liability, and finalization begins when the next call’s liability no longer fits beside the finalization reserve. The stages change how a trajectory approaches a hard ceiling, not which stopping rule prevails; a hard ceiling still takes precedence over satisficing and marginal stopping, the order established for search in Reserving budget before admitting work.
| Stage | Harness action | Scope and limitation |
|---|---|---|
| Steer | Supply the remaining allowance and ask for a smaller next step or final report | Changes the model’s information, not the admission bound |
| Restrict | Narrow specific grants or disable new delegation | Blocks newly disallowed admissions; retained authority and outstanding work still require accounting |
| Finalize | Stop ordinary continuation and use protected resources for checks, reporting, and cleanup | Does not erase charges or effects already in flight; unresolved duties retain an owner |
Each intervention during the steering and restriction stages carries its own accounting and caching trade-offs. Context compaction can reclaim token headroom when sufficient budget remains to execute it without destroying critical state, although invoking a summarization model consumes additional tokens of its own and discards detail. Narrowing authority is similarly partial: downgrading an agent to read-only tools prevents irreversible state mutations while still allowing expensive retrieval queries or context leaks to external APIs, and revoking delegation privileges stops new child trajectories without canceling background subtasks that are already active or releasing their reserved liabilities. Even how the harness enforces a tool restriction matters for cost, because dynamically mutating the tool schema inside the prompt invalidates cached prefix state from the modification point onward—trading prefix-cache reuse for reduced tool exposure unless the restriction is enforced at the harness admission gate instead.
↩︎ The prompt reduction and information loss of each compaction method are established in Choosing a compaction method.
When a trajectory enters the finalization stage due to budget exhaustion, the system must not compromise its validation standards. If an expensive regression test remains in-flight, its reservation must remain held until final usage telemetry settles the charge. The final diagnostic report must accurately report candidate r18 and record which validation suites completed successfully, rather than treating an early unit test pass as substitute verification for incomplete end-to-end checks.
10.9.5 Choosing a ceiling from task evidence
Set a trajectory ceiling too low and the harness cuts off repairs that would have succeeded in a few more turns; set it too high and failed runs burn through the extra allowance without recovering. Enforcement makes a ceiling binding, so choosing its value requires evidence from representative tasks. Engineers evaluate candidate budget vectors by testing the end-to-end control policy across a workload sample while holding models, tool suites, and acceptance tests constant. The acceptance fraction, accepted tasks divided by all attempted tasks, counts both outright failures and trajectories halted by budget exhaustion. System evaluations must report total dollar expenditure, wall-clock latency, unresolved background liabilities, and operator intervention frequency alongside raw task completion. Dividing the total cost of all attempts by the accepted count gives the cost per accepted task (equation 1.2), which amortizes the expenses of failed and abandoned runs across successful completions. Budget limits appear artificially inexpensive whenever benchmark evaluations omit the costs of failed trajectories from total expenditure. Kapoor et al. (2024) argue that evaluating agent accuracy in isolation from cost produces needlessly fragile and expensive architectures, so the two must be optimized jointly.
Baseline profiling under unconstrained budgets reveals the empirical completion time and token distributions of successful trajectories, highlighting candidate threshold targets. However, because early steering, throttling, and finalization alter downstream model reasoning, engineers must evaluate the complete policy empirically rather than assuming post-hoc truncation models reality. In practice, empirical cost-benefit curves rarely display a sharp knee. The operational decision rule selects the budget ceiling that minimizes cost per accepted task while keeping unresolved liabilities and manual interventions within acceptable operational bounds. When the difference between two policies falls within measurement uncertainty, the simpler policy is preferred.
Baseline profiling of public repair-agent runs shows what such distributions look like. Figure 5 plots the distribution of model calls per trajectory for five published agent configurations on the same 500 repository-repair tasks (Jimenez et al. 2024), split by whether the patch was accepted.
Jimenez, Carlos E., John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. 2024. “SWE-Bench: Can Language Models Resolve Real-World GitHub Issues?” International Conference on Learning Representations (ICLR).
Two lessons follow for ceiling choice. Failed runs take more calls than accepted ones in every configuration of figure 5, so the tail of the call distribution is mostly spend on attempts that will not be accepted. A ceiling at the accepted runs’ 90th percentile halts at most 10 percent of eventual successes but 15 to 31 percent of eventual failures. The same figure also warns against reading a ceiling off another configuration, because the percentile ranges from 46 to 101 calls across the five and differs even between the two agents built on the same model (82 and 101). These are post-hoc truncations of unconstrained runs, which is exactly the post-hoc shortcut cautioned against above; a ceiling the agent can see changes how it spends the remaining calls, so the estimate locates a candidate ceiling and the policy evaluation confirms it.
Re-running the same model under several enforced ceilings supplies the confirmation. A computer-use benchmark reports 5 models at step limits of 15, 50, and 100 actions (Xie et al. 2024), plotted in figure 6.
Xie, Tianbao, Danyang Zhang, Jixuan Chen, Xiaochuan Li, Siheng Zhao, Ruisheng Cao, Toh Jing Hua, et al. 2024. “OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments.” Advances in Neural Information Processing Systems 37 (Datasets and Benchmarks Track).
Epoch AI. 2026. Capabilities and Benchmarking.
Among the 5 models in figure 6, 3 gain nothing from the last 50 steps, while the strongest gains 20 points between the lowest and highest ceiling. A generous fixed ceiling therefore buys nothing for most configurations and costs the extra spend of every run that uses it, whereas a tight one forfeits real successes for the configurations that convert extra steps into progress.
↪ Matched-budget comparison of complete policies under one fixed resource envelope is developed in Matched-budget comparison.
Checkpoint 10.3: A budget is an admission decision
The repair trajectory is paused with an unresolved paid test job. A smaller model call would fit the currently unreserved allowance, but the task’s elapsed deadline has passed.
Budget accounting tracks aggregate resource consumption rather than semantic problem-solving progress. Identifying a stalled repair loop before budget exhaustion requires synthesizing evidence across repeated tool failures, prompt state divergence, and environment transitions. Budget admission completes the supervisory control plane. The record tracks ownership and outstanding liabilities, guarded transitions govern execution state, the scheduler divides shared model service, and bounded-liability admission limits spending without relying on model self-restraint.
↪ Progress evidence for detecting a trajectory that loops without advancing is developed in Progress evidence.
10.10 Fallacies and Pitfalls
Architectural failures in agent systems rarely stem from exotic distributed bugs; they occur when engineers mistake a convenient implementation mechanism or an agreeable language model for an authoritative supervisory guarantee. The harness must enforce safety, resource bounds, and lifecycle invariants externally, regardless of how cooperative the model appears.
Fallacy: Using asynchronous coroutines is sufficient to build a harness.
Asynchronous coroutines and non-blocking I/O primitives yield worker threads during network calls, but cooperative concurrency is not a supervisory control plane. A coroutine lacks runtime-owned control state and guarded admission control. When a harness relies solely on naive coroutine scheduling and polls for operator signals only after a complete model turn returns, it surrenders control during inference. For an allowance of 4,096 tokens, an operator issuing a pause command is left waiting roughly 86.3 s, even at the queue-free reference generation rates analyzed in section 10.5. A harness keeps control state in an authoritative record that a pause updates at once, and it applies streaming cancellation: it stops waiting on the reply immediately and asks the serving engine to stop generating. The stop takes effect locally without waiting for the allowance to drain, while the call’s reservation stays outstanding until the service reports how many tokens it actually generated.
Pitfall: Checking only measured spend before dispatch.
Evaluating admission against settled accounting balances ignores both in-flight liabilities and terminal cleanup obligations. In notebook 10.2, the settled bill shows $0.92 of remaining headroom, making a new $0.21 call appear affordable. However, once the harness factors in the unsettled test reservation alongside the mandatory finalization reserve, the true available headroom shrinks to $0.12. Admitting that call overcommits the budget and leaves downstream finalization checks $0.09 short of their required funds. To prevent budget overruns, the runtime must enforce bounded-liability admission before dispatch under equation 1, reserving worst-case consumption upfront, settling against authoritative usage records upon completion, and ring-fencing funds for clean termination. Presenting the remaining balance to the model can assist its internal planning, but runtime admission control alone must provide the hard guarantee.
Fallacy: A parseable reply is dispatchable after cancellation.
When an invocation is canceled mid-stream due to a timeout or an operator signal, the incoming buffer may still contain syntactically valid fragments, such as a complete tool call formatted in JSON. Syntactic validity is merely a serialization property that confers neither execution authority nor semantic completeness. Under the quarantining rule, which applies invariant closure (principle 1) to the reply status, incomplete or aborted outputs must be isolated as diagnostic telemetry rather than forwarded to tool executors. Any subsequent action must originate from a fresh turn that passes the harness’s standard status and admission checks.
Pitfall: Treating a pause, timeout, or terminal state as settlement.
Stopping a wait after a timeout is not equivalent to settling the underlying resource obligation. Releasing a test job’s reservation upon a local timeout returns $0.50, which is 14.3 percent of the total trajectory allowance in notebook 10.2, to the uncommitted pool. If subsequent turns claim that capacity before the remote job finishes and reports its actual bill, the trajectory overshoots its budget ceiling by up to the reallocated amount. A correct settlement rule separates observation eligibility from settlement. When an operation times out, the harness revokes the agent’s eligibility to observe the late output, keeps its charges open in the operation table, and routes authenticated late evidence to the settlement owner rather than injecting it into an obsolete model turn.
An instruction honored on almost every turn still fails across a whole trajectory.
Fallacy: An instruction in the prompt is a runtime control.
Natural-language prompt instructions communicate semantic intent to the model, but they cannot enforce runtime security boundaries. A model that honors a freeze or budget constraint with probability 99 percent on any single turn sustains that compliance across a 30-turn trajectory only 74 percent of the time (equation 1.1). Consequently, 26 percent of executed trajectories violate the constraint at least once. The production outage in war story 10.1 shows the missing piece: the freeze existed only as prompt text, and nothing at the dispatch boundary enforced it. The war story does not report a turn count or a compliance rate, so it cannot show the compounding itself; the arithmetic shows why a freeze held that way fails in a predictable share of long runs. An authoritative policy guard must sit at the dispatch boundary, evaluating the proposed action deterministically on every turn regardless of what the prompt requested.
10.11 Summary
Because an agent loop possesses no intrinsic stopping condition, the decision to continue execution must rest with an external supervisor that the language model cannot manipulate or bypass. The agent harness serves this supervisory role. It reduces every operational decision to two fundamental invariants evaluated against a single, runtime-owned record. First, before initiating any new unit of work, the harness verifies whether active control signals, delegation grants, provider capacity, and reserved budget allowances still permit dispatch. Second, whenever an asynchronous operation completes or halts, the harness determines whether verified evidence has genuinely settled the in-flight liability, or whether the outstanding obligation still requires an owner. Operator pauses, human-in-the-loop approvals, fair-share scheduling, and hard budget ceilings are these two checks applied at different stages of the execution lifecycle.
The synchronization cost of this discipline is modest: a brief critical section to update the state machine and several accounting fields per transition. Conversely, omitting these harness boundaries introduces subtle failure modes that model-level metrics obscure. Prompt instructions that hold reliably on single turns compound into unacceptable failure rates over long horizons; naive retrospective spend checks admit model invocations that deplete remaining funds mid-generation; and releasing budget reservations prematurely permits concurrent steps to overshoot cost ceilings. Table 10 summarizes the governing relations that regulate trajectory continuation, establish latency bounds under control signals, and allocate shared capacity across competing workloads.
| Equation | What it bounds or decides | Where derived |
|---|---|---|
| \(\TurnBoundaryDelay \approx \OutputLimit \times \PerTokenDecodeTime\) | Full-allowance generation delay at a fixed per-token rate; streaming cancellation eliminates this dependence on \(\OutputLimit\) | section 10.5 |
| \(\text{actual} + \text{reserved} + \text{proposed liability} + \text{finalization reserve} \leq \text{allowance}\) | Bounded-liability admission gate; permits dispatch only when all cumulative resource dimensions accommodate active work and final checks | equation 1 |
| \(\text{liability} = \InputTokenPrice\, \SequenceLength + \OutputTokenPrice\, \OutputLimit\) | Worst-case reservation for an in-flight model invocation, priced at undiscounted input rates and maximum output token allowance | equation 2 |
| \(\SettledTrajectoryCost^{(\TurnIndex)} = \SettledTrajectoryCost^{(\TurnIndex-1)} + \InputTokenPrice \UncachedInputTokenCount^{(\TurnIndex)} + \CachedInputTokenPrice \CachedInputTokenCount^{(\TurnIndex)} + \OutputTokenPrice \OutputLength^{(\TurnIndex)} + \sum_j \ToolCallCost(\ExecutedToolCall_{t,j})\) | Settled cumulative expenditure through turn \(\TurnIndex\) across disjoint billing categories, reconciling and replacing prior reservations | section 10.9.3 |
| \(\TrajectoryBudget = \langle \TrajectoryTurnLimit, \TrajectoryTokenLimit, \TrajectoryTimeLimit, \CostCeiling \rangle\), indicator \(\max_i (\text{used}_i / \text{allowance}_i)\) | Multi-dimensional budget vector across turns, tokens, elapsed time, and cost; the bottleneck ratio steers staged intervention policies | section 10.9.1 |
| \(\text{calls per round} = \SchedulingClassWeight \times \text{quantum} / (\SequenceLength + \OutputLimit)\) | Long-run class admission rate for backlogged fixed-size calls without binding capacity gates; request charges determine aggregate admission frequency | section 10.8 |
| \(\ApprovalHeldTrajectories = \ArrivalRate\,\ExternalWait\) | Little’s law applied to human-in-the-loop gating; quantifies concurrency in AWAITING_APPROVAL and the cumulative resource footprint of review latency |
section 10.7 |
| \(\Pr(\text{success}) = (1-\PerStepErrorRate)^{\TurnHorizon}\) | Compound trajectory reliability under per-step failure rate \(\PerStepErrorRate\); demonstrates why supervisory controls must operate at dispatch | equation 1.1 |
| \(\CostPerAcceptedTask = \mathbb{E}[\AttemptCost]/\AcceptanceProbability\) | Effective cost per accepted outcome; amortizes failed and exhausted trajectory attempts across successful deliveries to guide budget sizing | equation 1.2 |
Key Takeaways: Control lives outside the model
- Instructions cannot substitute for architectural controls: A freeze directive honored on 99 percent of individual turns survives across a 30-turn trajectory only 74 percent of the time. In contrast, an invariant enforced at the dispatch boundary holds across every turn because the harness inspects the concrete operation rather than relying on model compliance.
- Control signals must decouple from generation completion: Checking for pause or cancellation signals only after a generation finishes can delay a pause by approximately 86.3 s for a 4,096-token allowance, even under zero queueing. Turn boundaries combined with streaming cancellation decouple supervisory latency from generation length.
- Halting execution does not settle in-flight obligations: A pause, timeout, or cancellation revokes an operation’s permission to advance the trajectory into the next turn, but it neither terminates external execution nor refunds committed spend. Only correlated, authenticated evidence from the remote provider can formally settle the liability; until that evidence arrives, the harness must preserve explicit ownership and state tracking for the outstanding task.
- Asynchronous waits release execution threads, not system resources: Moving a trajectory into an asynchronous wait or human approval queue releases the active worker thread, but the trajectory retains proposal and evidence references, unresolved reservations, and its advancing deadline. Workspace and KV-cache residency continue only under their respective lease and cache policies. By Little’s law (\(\ApprovalHeldTrajectories = \ArrivalRate\,\ExternalWait\)), human review latency multiplies directly into the volume of concurrently suspended trajectories and their retained resource footprints.
- Fairness, quota allocation, and budget affordability operate as independent gates: Scheduling weights allocate relative shares of provider capacity, meaning that request payload sizes rather than weights alone determine invocation frequency under deficit round-robin. Provider rate limits protect shared infrastructure against burst exhaustion, whereas task-level budget ceilings bound total lifetime consumption for an individual agent run.
- Budget ceilings require upfront reservation of in-flight liabilities: A naive retrospective spend check reports $0.92 of available headroom, whereas bounded-liability admission, which counts unsettled in-flight tool executions and the finalization reserve, reveals only $0.12. Reserving conservative worst-case liabilities prior to dispatch is the only mechanism that converts an informational budget tracker into an enforceable upper bound.
- Size budget ceilings by cost per accepted task: A restrictive budget ceiling appears deceptively economical if failed and exhausted runs are omitted from post hoc accounting. Sizing budget ceilings by effective cost per accepted task (\(\CostPerAcceptedTask = \mathbb{E}[\AttemptCost]/\AcceptanceProbability\)), which amortizes failed attempts over accepted outcomes, and treating differences within measurement error as ties, aligns harness admission thresholds with production economics.
Trajectory-level encapsulation (principle 10) establishes the runtime-owned record as the authoritative foundation for supervision. By storing immutable references to external state rather than attempting to duplicate it, the harness decouples admission control and execution governance from model cooperation. Both responsibilities function within explicit resource and evidence boundaries, ensuring that every transition (a normal dispatch, a staged escalation, or an ungraceful abort) preserves an unambiguous owner for every effect still in flight.
10.12 Historical Perspective and Further Reading
An agent harness synthesizes classical ideas that operating systems, databases, and computer networks developed when shared infrastructure encountered clients that could not be trusted to self-regulate or terminate. In operating systems, Hoare introduced monitors to provide shared mutable state with a single serialized entry point (Hoare 1974). In reactive systems, Harel formalized statecharts with guarded transitions so that valid controller behavior could be rigorously defined and checked (Harel 1987). Saltzer and Schroeder articulated the foundational design principles of computer protection (Saltzer and Schroeder 1975), establishing the requirement of complete mediation: every access must pass through an authoritative check that the client cannot bypass. The trajectory record, its serialization lock, and its guarded transition table apply these three concepts directly to the execution of an autonomous agent run.
Kung, H. T., and John T. Robinson. 1981. “On Optimistic Methods for Concurrency Control.” ACM Transactions on Database Systems 6 (2): 213–26. https://doi.org/10.1145/319566.319567.
Gray, Cary G., and David R. Cheriton. 1989. “Leases: An Efficient Fault-Tolerant Mechanism for Distributed File Cache Consistency.” Proceedings of the Twelfth ACM Symposium on Operating Systems Principles (SOSP), 202–10. https://doi.org/10.1145/74850.74870.
Concurrency control and resource accounting soon followed. Optimistic concurrency control (Kung and Robinson 1981) demonstrated that systems can validate updates against previously read versions rather than holding exclusive locks across slow, unpredictable external operations. The harness’s record_version check adopts this exact pattern, allowing an agent to plan or invoke external tools without blocking concurrent readers. O’Neil’s escrow method (O’Neil 1986) partitioned shared quantities by pre-allocating portions against an aggregate invariant that survives concurrent execution; its escrow is a reserved amount, the sense this chapter calls a reservation. It is the direct intellectual ancestor of bounded-liability admission, which admits an agent step only if the worst-case financial or token liability remains safe. Similarly, Gray and Cheriton introduced leases (Gray and Cheriton 1989) to ensure that distributed locks and allocated resources expire automatically if a client crashes or stalls, preventing abandoned runs from holding shared resources indefinitely.
Networking and distributed systems contributed mechanisms for queue management and fairness under contention. Demers, Keshav, and Shenker developed fair queueing (Demers et al. 1989), and Shreedhar and Varghese later introduced deficit round-robin (Shreedhar and Varghese 1995) to allocate link capacity equitably across flows of unequal packet size without expensive per-packet accounting. In an agent platform, where requests vary widely in token counts and latency, carried deficit credits prevent heavy agent runs from starving interactive workloads. Little’s law (Little 1961) relates the mean number of items waiting to their arrival rate and mean residence time, the relation needed to size these waiting queues. When agent steps involve slow tool execution or external inspection, staged event-driven architectures (Welsh et al. 2001) showed how to keep long waits from pinning execution threads by making queues explicit and enforcing localized admission controls at each processing stage.
Little, John D. C. 1961. “A Proof for the Queuing Formula: \(L = \lambda W\).” Operations Research 9 (3): 383–87. https://doi.org/10.1287/opre.9.3.383.
Parasuraman, Raja, Thomas B. Sheridan, and Christopher D. Wickens. 2000. “A Model for Types and Levels of Human Interaction with Automation.” IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans 30 (3): 286–97. https://doi.org/10.1109/3468.844354.
System safety also relies on decades of human-factors research into supervisory control. Bainbridge’s classic analysis of the ironies of automation (Bainbridge 1983) showed that humans perform poorly when tasked with passively monitoring highly reliable automated systems, precisely because human vigilance deteriorates when intervention is rarely needed. Parasuraman, Sheridan, and Wickens (Parasuraman et al. 2000) subsequently classified automation into explicit operational levels based on the degree to which an autonomous system decides and acts without human intervention. These insights explain why agent approval gates cannot rely on casual human oversight, but instead require explicit, structured pauses that supply unambiguous context before an action proceeds.
Modern language-model agents encountered all of these classical systems challenges simultaneously. Yao et al. (2023) framed agent execution as an interleaved loop of chain-of-thought reasoning and external tool actions with ReAct. Yang et al. (2024) brought this pattern into real software repositories with SWE-agent, demonstrating that purpose-built agent-computer interfaces could resolve real bugs, while also showing that execution cost, waiting latencies, and tool interruptions were no longer edge cases. The hazard of relying on prompt instructions alone became apparent when an autonomous agent deleted a production database during an instructed code freeze (Sharwood 2025), illustrating that a safety invariant stated only in natural language is not enforced. In parallel, systematic benchmarks that measure accuracy without tracking cost (Kapoor et al. 2024) obscure critical engineering trade-offs, which is why resource ceilings should be derived from measured cost distributions rather than optimistic assumptions. The harness developed in this chapter is the systems response to these operational pressures, enforcing classical guarantees directly at the agent trajectory.
Yao, Shunyu, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. “ReAct: Synergizing Reasoning and Acting in Language Models.” International Conference on Learning Representations (ICLR).
Sharwood, Simon. 2025. Vibe Coding Service Replit Deleted User’s Production Database, Faked Data, Told Fibs Galore. The Register.
Kapoor, Sayash, Benedikt Stroebl, Zachary S. Siegel, Nitya Nadgir, and Arvind Narayanan. 2024. “AI Agents That Matter.” arXiv Preprint arXiv:2407.01502.
For further reading:
- Saltzer and Schroeder, “The Protection of Information in Computer Systems” (Saltzer and Schroeder 1975): foundational security principles, including complete mediation, that govern every admission check and authorization boundary in the harness.
- Hoare, “Monitors: An Operating System Structuring Concept” (Hoare 1974): the classical formulation of serialized access to shared state, providing the conceptual model for trajectory record locking.
- O’Neil, “The Escrow Transactional Method” (O’Neil 1986): reservations against an aggregate invariant under concurrency, the intellectual foundation for bounded-liability admission.
- Shreedhar and Varghese, “Efficient Fair Queuing Using Deficit Round Robin” (Shreedhar and Varghese 1995): a short paper showing how carried credit enables \(\mathcal{O}(1)\) fair scheduling across work items of variable size.
- Bainbridge, “Ironies of Automation” (Bainbridge 1983): a classic analysis of why human monitoring of reliable automation degrades, directly relevant to the design of human approval gates.
- Yang et al., “SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering” (Yang et al. 2024): a concrete account of command-and-feedback loops in real software repositories, illustrating where model autonomy ends and harness infrastructure must take over.
Saltzer, J. H., and M. D. Schroeder. 1975. “The Protection of Information in Computer Systems.” Proceedings of the IEEE 63 (9): 1278–308. https://doi.org/10.1109/proc.1975.9939.
Yang, John, Carlos E Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. 2024. “SWE-Agent: Agent-Computer Interfaces Enable Automated Software Engineering.” Advances in Neural Information Processing Systems.
10.13 Exercises
Every given below is a hypothetical problem input, not a measurement of a particular model, provider, or service. Prices are in dollars per million tokens unless stated otherwise. Each problem asks for a number and then for the decision the number supports.
Where a pause lands (Basic). A repair harness checks
pending_controlonly when a model reply ends. Generation takes 20 ms per output token with no queueing, and each call’s output allowance is \(\OutputLimit = 8{,}000\) tokens. The operator requirement is that a pause stop new admission within 15 s of its request.Compute the worst-case wait before the pause takes effect, and the largest \(\OutputLimit\) that meets the requirement without streaming cancellation. For this hypothetical problem, assume a 3,000-token patch can be partitioned into independently complete, validated edit proposals, each no larger than the reduced output cap. Their complete serialization totals 3,000 output tokens with no added proposal overhead, and each call carries the same 40,000-token context without a cache discount. If \(\OutputLimit\) is lowered to meet the requirement, what is the minimum number of complete calls, and how many extra input tokens, at $3 per million, does resending the context cost compared with one call? Explain why an indivisible proposal cannot instead be completed by concatenating truncated replies. The serving service acknowledges a streaming cancellation within 2 s. Decide between the lower cap and streaming cancellation, and state what the pause must still leave in the operation table for a test job already running.
Admission after settlement (Basic). A repair trajectory has a $5.00 allowance, $4.05 of settled spend, a $0.25 reservation on an unsettled test job, and a $0.40 finalization reserve. The next model call carries 80,000 input tokens and \(\OutputLimit = 2{,}000\), at $2 per million input tokens and $10 per million output tokens.
Use equation 1 to decide whether the call is admitted, and compare the spend-check headroom. The call settles with 20,000 uncached input tokens, 60,000 cached input tokens at $0.20 per million, and 900 output tokens. How much reservation does settlement release? The reply proposes paid tool calls bounded at $0.10 each; how many can the harness admit? A defective harness had instead released the test job’s reservation when its wait timed out. How many tool calls would it admit, and by how much would the finalization reserve fall short once the test’s full $0.25 charge arrives?
Workspaces held in escrow (Basic). Review requests for repair submissions arrive at 12 per hour, and reviewers take 45 minutes on average to decide. Each held trajectory keeps a workspace lease of 4 GB, and the pool reserves 24 GB for held workspaces.
Use Little’s law to compute the mean number of held trajectories and the memory they retain. What mean decision time would fit the reserved pool? An alternative snapshots each workspace to storage on entering
AWAITING_APPROVAL, keeping 0.25 GB resident, and restores it in 40 s at dispatch. Compute its resident memory and the restore delay as a fraction of the mean decision time. Recommend one design, and explain why a pool sized from the mean alone would still fail after an incident that produces a burst of review requests.Three classes under deficit round-robin (Intermediate). A harness schedules three classes with quantum 10,000 charge tokens per unit weight per round: support calls (weight 4; 3,000 input tokens and a 500-token output allowance), repair calls (weight 2; 30,000 and 4,000), and maintenance calls (weight 1; 60,000 and 4,000). Assume all classes remain continuously backlogged at the stated call sizes, no provider or local-capacity gate binds, and carried credit has a ceiling large enough for the largest permitted call. One round takes 2 s.
Compute each class’s share of the scheduling charge, its long-run calls per round, and the rounds of credit one call needs. What is the mean interval between maintenance-class admissions in seconds? An engineer caps every class’s carried credit at 50,000 tokens to limit bursts. Determine which class stops running and why. To increase maintenance-class admission frequency, one proposal raises the maintenance weight to 3; another compacts maintenance inputs to 20,000 tokens. Compute the rounds per call and the support class’s charge share under each proposal, and choose the one that leaves support service unchanged.
Sizing a dollar ceiling for cache misses (Intermediate). A repair trajectory runs 25 turns with a 10,000-token base prompt, 2,000 tokens appended per turn, and 300 output tokens per turn; each call’s output allowance is 4,000 tokens. Prices are $3 per million uncached input tokens, $0.30 per million cached input tokens, and $15 per million output tokens, and each turn’s earlier prefix is billed as cached input.
Use equation 2.9 to compute the total input tokens, and the settled cost with and without prefix reuse. The team set \(\CostCeiling = \$1.00\) from the cached bill with a $0.15 finalization reserve, and the harness reserves each call at the undiscounted input rate and full output allowance. After 24 cached turns, is the 25th call admitted, and with what headroom? A long approval wait evicts the cache before one turn, so that turn’s whole input is billed uncached and later turns hit the cache again. Compute the extra charge for an eviction before the 6th turn and before the 24th turn, and state how many of each the trajectory can absorb before the 25th call is refused. What \(\CostCeiling\) guarantees admission of all 25 calls if every turn misses? Recommend how to size the ceiling.
Which allowance binds the research agent (Intermediate). The research agent starts from a 6,000-token base prompt, appends 12,000 tokens of bounded page observations per turn, and generates 400 output tokens per turn with a 2,000-token output allowance. Each turn takes 45 s. Its budget is \(\TrajectoryBudget = \langle 40 \text{ turns},\ 1.5 \text{ million tokens},\ 1{,}800 \text{ s},\ \$3.00 \rangle\), where the token meter counts cached input in full, and prices match problem 5.
Admitting a turn requires its input plus its output allowance to fit the remaining token allowance. Find the last admitted turn, and compute the used fraction of every dimension at that point with prefix reuse. Which allowance binds, and how does that differ from the repair trajectory of notebook 10.3? A compaction policy runs after every fifth turn: a compaction call reads the full context and writes 2,000 tokens, and the context then restarts from the base prompt plus an 8,000-token summary. Find how many turns the agent now completes before a turn or compaction is refused. Compare this with the token allowance needed to reach 30 turns without compaction. Decide between compaction and a larger \(\TrajectoryTokenLimit\), naming the evidence a summary of untrusted pages must keep by reference.
Choosing a ceiling from a trial (Advanced). A generous-budget run of 100 repair tasks, capped at $10, gives the following outcomes: 30 tasks accepted at about $0.50 each, 25 at about $1.50, 15 at about $3.00, and 5 at about $6.00; the remaining 25 never reached acceptance and spent the full $10. Assume that under a lower cap a task that the generous run accepted at or below the cap is still accepted at the same cost, and every other task spends exactly the cap.
Compute the accepted count and cost per accepted task (equation 1.2) for caps of $1, $2, $4, and $10. A colleague proposes $10 because it maximizes acceptance; critique the proposal. Treating the acceptance count as binomial and freezing each policy’s total cost, estimate the marginal relative standard error of \(\CostPerAcceptedTask\) at each cap. Use the first-order approximation that the relative standard error of inverse acceptance equals the acceptance-rate standard error divided by the estimated acceptance fraction. Is the gap between the two lowest-cost caps large compared with each policy’s marginal error? State which paired cost and outcome records, and which declared comparison criterion, a formal verdict would require; these marginal errors alone do not establish distinguishability. How many tasks would bring the relative error to 3 percent at the $2 cap’s acceptance fraction? Finally, explain why a policy that steers or compacts near its cap must be tested as a policy, and why these numbers cannot simply be read from the generous run for it.
Settling timed-out test jobs (Advanced). Each repair trajectory dispatches six test jobs to a metered runner that charges $0.01 per second and enforces a 90 s cap. The harness waits 30 s per job, and 70 percent of jobs finish within the wait at an average of 20 s; the rest run on, averaging 60 s in total. The trajectory allowance is \(\CostCeiling = \$4.00\). The current harness releases a job’s whole reservation when its wait times out and books nothing until the charge arrives.
Compute each job’s bounded liability, the expected late charge per trajectory that the defective harness leaves uncovered, the worst case, and both as fractions of \(\CostCeiling\). Under the settlement rule, how much allowance does a timed-out job hold beyond its eventual charge, on average, while it runs? The runner also supports confirmed cancellation, which stops a job within 5 s and reports its final usage. Compute the charge and the released reservation for a job canceled at its timeout, and the expected test cost per trajectory under “cancel at timeout” and “let it run.” Redesign the timeout policy for the repair agent, stating which evidence releases each reservation and what a canceled job’s missing result costs the repair.
What’s Next: From a live record to a durable one
If the host fails after reserving and dispatching the test job but before recording its response, the live record alone cannot preserve the admitted operation, its reservation, the held approval, or the evidence needed to resume. Durable Execution develops the history and ordering needed to reconstruct selected control state and distinguish recorded outcomes from unresolved operations after a crash. Rebuilding the record does not itself prove that a remote effect failed, so resumption must preserve the tool’s settlement contract rather than blindly repeat the call.