ChunkedPrefillScenario
Serving scenarios · Implementation: mlsysim/engine/scenario_definitions/serving.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
A long prompt advanced chunk_tokens per iteration beside decode.
Notes
Equation: Iterations=ceil(prompt_tokens/chunk_tokens); the optional result composes PrefixRebuildModel with ChunkedPrefillModel.
Assumptions: Positive chunk size; prefill and decodes must be supplied to request the solver result.
Excludes: A runtime scheduler or a measured gap distribution.
References
Implementation reference: mlsysim.solvers.ChunkedPrefillModel.solve documents the composed equation and its sources.
Examples
The named registry scenario supplies the stated inputs; this is a modeled calculation, not a new measurement.
>>> from mlsysim import Scenarios
>>> scenario = Scenarios.LongPromptChunkedPrefill
>>> scenario.iterations
20Attributes
| Name | Description |
|---|---|
| chunk_tokens | |
| decodes | |
| iterations | Integer iterations required to consume the prompt: ceiling of prompt_tokens / chunk_tokens; chunk size must be positive. |
| prefill | |
| prompt_tokens | |
| result | Return mlsysim.solvers.ChunkedPrefillModel.solve for this scenario; output fields retain the units and feasibility semantics specified by that solver. |
prompt_tokens
Type: int.
Declared default: required.
chunk_tokens
Type: int.
Declared default: required.
prefill
Type: Optional[PrefixRebuildScenario].
Declared default: None.
decodes
Type: Optional[int].
Declared default: None.
iterations
engine.scenario_types.ChunkedPrefillScenario.iterations
Integer iterations required to consume the prompt: ceiling of prompt_tokens / chunk_tokens; chunk size must be positive.
Equation: _math.ceil(self.prompt_tokens / self.chunk_tokens)
result
engine.scenario_types.ChunkedPrefillScenario.result
Return mlsysim.solvers.ChunkedPrefillModel.solve for this scenario; output fields retain the units and feasibility semantics specified by that solver.
Equation: mlsysim.solvers.ChunkedPrefillModel.solve (scenario fields).