ReasoningLatencyScenario
Serving scenarios · Implementation: mlsysim/engine/scenario_definitions/serving.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
Serial reasoning tokens before the answer, at a fixed time per token.
Notes
Equation: Reasoning duration=reasoning_tokens*per_token_time.
Assumptions: Serial decoding at a constant supplied token time.
Excludes: Prefill, final answer tokens, and any prediction of reasoning quality.
References
Derivation: A serial sequence of identical-duration token steps takes token count times duration per token. The expansion value uses one token as its reference unit.
Examples
The named registry scenario supplies the stated inputs; this is a modeled calculation, not a new measurement.
>>> from mlsysim import Scenarios
>>> scenario = Scenarios.ReasoningTokenLatency
>>> round((scenario.reasoning_time).to('s').magnitude, 6)
12.8Attributes
| Name | Description |
|---|---|
| expansion | Reasoning token count, interpreted as expansion relative to a one-token reference; dimensionless. |
| per_token_time | |
| reasoning_time | Serial reasoning latency in seconds: fixed time per token times token count; excludes prefill and answer generation. |
| reasoning_tokens |
per_token_time
Type: Quantity.
Declared default: required.
reasoning_tokens
Type: int.
Declared default: required.
reasoning_time
engine.scenario_types.ReasoningLatencyScenario.reasoning_time
Serial reasoning latency in seconds: fixed time per token times token count; excludes prefill and answer generation.
Equation: (self.per_token_time * self.reasoning_tokens).to(‘second’)
expansion
engine.scenario_types.ReasoningLatencyScenario.expansion
Reasoning token count, interpreted as expansion relative to a one-token reference; dimensionless.
Equation: self.reasoning_tokens