KvAttentionDesignScenario
Serving scenarios · Implementation: mlsysim/engine/scenario_definitions/serving.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
Bytes per token across registry models, with one MHA, one GQA, and one MLA reference.
Notes
Equation: Result = mlsysim.solvers.KvAttentionDesignModel.solve with the named scenario fields; no extra fitting or postprocessing is applied.
Assumptions: The supplied inputs use the units and validity regime documented by KvAttentionDesignModel.solve; this class binds one reproducible configuration.
Excludes: Any behavior excluded by KvAttentionDesignModel.solve, and measurement or calibration of the supplied inputs.
References
Implementation reference: mlsysim.solvers.KvAttentionDesignModel.solve documents the composed equation and its sources.
Examples
The named registry scenario supplies the stated inputs; this is a modeled calculation, not a new measurement.
>>> from mlsysim import Scenarios
>>> scenario = Scenarios.KvBytesPerTokenByAttentionDesign
>>> round(scenario.result.mha_over_mla, 6)
67.147541Attributes
| Name | Description |
|---|---|
| gqa | |
| mha | |
| mla | |
| models | |
| precision | |
| result | Return mlsysim.solvers.KvAttentionDesignModel.solve for this scenario; output fields retain the units and feasibility semantics specified by that solver. |
| session_tokens |
models
Type: List[Workload].
Declared default: required.
mha
Type: Workload.
Declared default: required.
gqa
Type: Workload.
Declared default: required.
mla
Type: Workload.
Declared default: required.
precision
Type: str.
Declared default: 'fp16'.
session_tokens
Type: int.
Declared default: required.
result
engine.scenario_types.KvAttentionDesignScenario.result
Return mlsysim.solvers.KvAttentionDesignModel.solve for this scenario; output fields retain the units and feasibility semantics specified by that solver.
Equation: mlsysim.solvers.KvAttentionDesignModel.solve (scenario fields).