ElementwiseOpScenario
Optimization scenarios · Implementation: mlsysim/engine/scenario_definitions/optimization.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
An element-wise or normalization kernel over elements values, run unfused.
Each element is read reads times and written writes times from device memory; parameter_values (a LayerNorm’s scale and shift) are read once. flops_per_element counts the arithmetic on each element.
Notes
Equation: intensity is the worked quantity; each property below states its equation and units.
Assumptions: An unfused kernel performs fixed per-element work and reads parameters once.
Excludes: Cache reuse, kernel fusion, vectorization, and measured execution latency.
References
Derivation: The equations below follow the stated counts, geometric relations, or accounting definitions; scenario inputs are supplied assumptions or observations, not fitted predictions.
Examples
Execute the registered worked inputs; this is a modeled calculation.
>>> from mlsysim import Scenarios
>>> case = Scenarios.Gpt2XlLayerNormPrefill
>>> value = case.intensity
>>> round(value, 3)
2.0Attributes
| Name | Description |
|---|---|
| bytes_per_value | |
| elements | |
| flops | Arithmetic work in FLOPs, element count times FLOPs per element. |
| flops_per_element | |
| input_bytes | Input traffic in bytes, element footprint times read count. |
| intensity | Arithmetic intensity in FLOPs per byte of modeled unfused traffic. |
| output_bytes | Output traffic in bytes, element footprint times write count. |
| parameter_bytes | Parameter traffic in bytes, stored parameter values read once. |
| parameter_values | |
| reads | |
| traffic_bytes | Total unfused traffic in bytes across inputs, outputs, and parameters. |
| writes |
elements
Type: int.
Declared default: required.
flops_per_element
Type: float.
Declared default: required.
bytes_per_value
Type: Quantity.
Declared default: required.
parameter_values
Type: int.
Declared default: 0.
reads
Type: int.
Declared default: 1.
writes
Type: int.
Declared default: 1.
flops
engine.scenario_types.ElementwiseOpScenario.flops
Arithmetic work in FLOPs, element count times FLOPs per element.
Equation: return Q_(self.elements * self.flops_per_element, ‘flop’)
input_bytes
engine.scenario_types.ElementwiseOpScenario.input_bytes
Input traffic in bytes, element footprint times read count.
Equation: return (self.reads * self.elements * self.bytes_per_value).to(‘byte’)
output_bytes
engine.scenario_types.ElementwiseOpScenario.output_bytes
Output traffic in bytes, element footprint times write count.
Equation: return (self.writes * self.elements * self.bytes_per_value).to(‘byte’)
parameter_bytes
engine.scenario_types.ElementwiseOpScenario.parameter_bytes
Parameter traffic in bytes, stored parameter values read once.
Equation: return (self.parameter_values * self.bytes_per_value).to(‘byte’)
traffic_bytes
engine.scenario_types.ElementwiseOpScenario.traffic_bytes
Total unfused traffic in bytes across inputs, outputs, and parameters.
Equation: return self.input_bytes + self.output_bytes + self.parameter_bytes
intensity
engine.scenario_types.ElementwiseOpScenario.intensity
Arithmetic intensity in FLOPs per byte of modeled unfused traffic.
FLOP per byte of the unfused traffic.
Equation: return (self.flops / self.traffic_bytes).to(‘flop/byte’).magnitude