ContextGrowthScenario
Serving scenarios · Implementation: mlsysim/engine/scenario_definitions/serving.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
A context window grown from base_tokens to tokens: a kept cache grows with the ratio, attention’s score work and stored scores with its square.
Notes
Equation: Linear cache growth is tokens/base_tokens; full score growth is its square.
Assumptions: Model geometry and storage precision remain fixed; base_tokens is positive.
Excludes: Windowed/sparse attention, cache eviction, and measured latency scaling.
References
Derivation: Retained KV state adds a fixed amount per token, so its ratio follows the context-length ratio. A full query-key matrix has two context axes, so its work/storage ratio is the square of that growth factor.
Examples
>>> from mlsysim import Scenarios
>>> s = Scenarios.BertWindowToFourK
>>> s.score_ratio == s.ratio**2
TrueAttributes
| Name | Description |
|---|---|
| base_tokens | |
| ratio | Dimensionless context-length growth factor; also the linear cache-growth factor. |
| score_ratio | Dimensionless full attention-score growth factor, the square of context growth. |
| tokens |
base_tokens
Type: int.
Declared default: required.
tokens
Type: int.
Declared default: required.
ratio
engine.scenario_types.ContextGrowthScenario.ratio
Dimensionless context-length growth factor; also the linear cache-growth factor.
Equation: tokens / base_tokens.
score_ratio
engine.scenario_types.ContextGrowthScenario.score_ratio
Dimensionless full attention-score growth factor, the square of context growth.
Equation: ratio ** 2.