PrecisionDividendScenario
Serving scenarios · Implementation: mlsysim/engine/scenario_definitions/serving.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
Maximum batch before and after lower-precision weights and KV (PrecisionDividendModel).
Notes
Equation: Result = mlsysim.solvers.PrecisionDividendModel.solve with the named scenario fields; no extra fitting or postprocessing is applied.
Assumptions: The supplied inputs use the units and validity regime documented by PrecisionDividendModel.solve; this class binds one reproducible configuration.
Excludes: Any behavior excluded by PrecisionDividendModel.solve, and measurement or calibration of the supplied inputs.
References
Implementation reference: mlsysim.solvers.PrecisionDividendModel.solve documents the composed equation and its sources.
Examples
The named registry scenario supplies the stated inputs; this is a modeled calculation, not a new measurement.
>>> from mlsysim import Scenarios
>>> scenario = Scenarios.Llama2_70BPrecisionDividendGqaTp8
>>> round((scenario.result.weights_per_device).to('B').magnitude, 6)
17500000000.0Attributes
| Name | Description |
|---|---|
| context_tokens | |
| hbm_per_accelerator | |
| kv_bytes | |
| kv_bytes_low | |
| kv_heads | |
| result | Return mlsysim.solvers.PrecisionDividendModel.solve for this scenario; output fields retain the units and feasibility semantics specified by that solver. |
| tp_size | |
| weight_bytes | |
| weight_bytes_low | |
| workload |
workload
Type: Workload.
Declared default: required.
tp_size
Type: int.
Declared default: required.
hbm_per_accelerator
Type: Quantity.
Declared default: required.
context_tokens
Type: int.
Declared default: required.
kv_heads
Type: int.
Declared default: required.
weight_bytes
Type: Quantity.
Declared default: required.
weight_bytes_low
Type: Quantity.
Declared default: required.
kv_bytes
Type: Quantity.
Declared default: required.
kv_bytes_low
Type: Quantity.
Declared default: required.
result
engine.scenario_types.PrecisionDividendScenario.result
Return mlsysim.solvers.PrecisionDividendModel.solve for this scenario; output fields retain the units and feasibility semantics specified by that solver.
Equation: mlsysim.solvers.PrecisionDividendModel.solve (scenario fields).