StreamCancelScenario
Agents scenarios · Implementation: mlsysim/engine/scenario_definitions/agents.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
A streamed reply with output_allowance tokens, rejected and canceled at reject_step.
Notes
Equation: Use the dimensional products, ratios, or selection rules documented on the calculated properties below.
Assumptions: Cancellation prevents all remaining allowed decode steps immediately, each priced at the fixed memory-time floor.
Excludes: Compute-bound decode, KV growth, cancellation latency, and already-generated buffered tokens are absent.
References
Derivation: the calculated properties below state their defining arithmetic and units. They are scenario accounting or input-selection rules, not independently measured relationships.
Examples
>>> from mlsysim import Scenarios
>>> case = Scenarios.CanceledStreamDecodeSavings
>>> round(case.skipped_steps, 4)
4000Attributes
| Name | Description |
|---|---|
| allowance_decode_time | Weight-read-floor time for the full output allowance. Equation: output_allowance * | | [decode_time_avoided](#mlsysim.engine.scenario_types.StreamCancelScenario.decode_time_avoided) | Weight-read-floor time attributed to canceled decode steps. Equation:skipped_steps * |
| floors | |
| output_allowance | |
| reject_step | |
| skipped_steps | Decode steps avoided by cancellation at the stated rejection step. Equation: |
floors
Type: DecodeScenario.
Declared default: required.
output_allowance
Type: int.
Declared default: required.
reject_step
Type: int.
Declared default: required.
skipped_steps
engine.scenario_types.StreamCancelScenario.skipped_steps
Decode steps avoided by cancellation at the stated rejection step. Equation: output_allowance - reject_step. Units: tokens/steps; cancellation is assumed immediate.
decode_time_avoided
engine.scenario_types.StreamCancelScenario.decode_time_avoided
Weight-read-floor time attributed to canceled decode steps. Equation: skipped_steps * floors.result.memory_time. Units: seconds; other decode work and cancellation overhead are excluded.
allowance_decode_time
engine.scenario_types.StreamCancelScenario.allowance_decode_time
Weight-read-floor time for the full output allowance. Equation: output_allowance * floors.result.memory_time. Units: seconds; context growth and serving overhead are excluded.
Decode time of the whole allowance (added 2026-10-20): K_max t_dec, the longest one admitted call postpones the next turn boundary when control waits for its reply.