ElementwiseIntensityScenario
Training scenarios · Implementation: mlsysim/engine/scenario_definitions/training.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
An unfused elementwise operation: flops_per_element FLOPs while operands_moved values of bytes_per_value each are read or written.
Notes
Equation: Traffic per element is moved_values*bytes_per_value; intensity divides per-element FLOPs by that traffic.
Assumptions: Every declared operand is transferred once and transfer width is positive.
Excludes: Fusion, caching, and extra address or indexing traffic.
References
Derivation: One element moves operands_moved values of the declared byte width. Arithmetic intensity is defined as its useful FLOP count divided by those transferred bytes; no hardware efficiency estimate enters this ratio.
Examples
>>> from mlsysim import Scenarios
>>> s = Scenarios.ResidualAddFp32
>>> s.bytes_per_element.to("byte").magnitude > 0
TrueAttributes
| Name | Description |
|---|---|
| bytes_per_element | Traffic in bytes per output element, counting each declared read or write. |
| bytes_per_value | |
| flops_per_element | |
| intensity | FLOPs per byte moved. |
| operands_moved |
flops_per_element
Type: int.
Declared default: required.
operands_moved
Type: int.
Declared default: required.
bytes_per_value
Type: Quantity.
Declared default: required.
bytes_per_element
engine.scenario_types.ElementwiseIntensityScenario.bytes_per_element
Traffic in bytes per output element, counting each declared read or write.
Equation: (operands_moved * bytes_per_value).to('byte').
intensity
engine.scenario_types.ElementwiseIntensityScenario.intensity
FLOPs per byte moved.
Equation: flops_per_element / bytes_per_element.to('byte').magnitude.