KernelFusionScenario
Optimization scenarios · Implementation: mlsysim/engine/scenario_definitions/optimization.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
A chain of chain_length element-wise operations, run as one kernel each or fused into one.
Unfused, every operation reads its input from device memory and writes its output back (two trips each) and pays dispatch plus launch; fused, the chain reads once, keeps intermediates on chip, writes once, and pays one launch. For N elements of b bytes the traffic is 2kNb against 2Nb.
Notes
Equation: Unfused traffic makes 2*k whole-array trips; fused traffic makes two while launch count falls from k to one.
Assumptions: All intermediates fit on chip and the complete chain can be fused.
Excludes: Register pressure, extra arithmetic, occupancy changes, and end-to-end speedup prediction.
References
Derivation: Separate kernels write each intermediate to memory and read it for the next operation, giving two whole-array transfers per operation. A fused chain retains intermediates on chip and transfers only its external input/output, while paying one dispatch/launch pair.
Examples
>>> from mlsysim import Scenarios
>>> s = Scenarios.ElementwisePairFusion
>>> s.traffic_ratio == s.chain_length
TrueAttributes
| Name | Description |
|---|---|
| chain_length | |
| dispatch | |
| fused_launches | Kernel launch count after fusion into one kernel. |
| fused_overhead | Dispatch-plus-launch overhead in microseconds for the fused kernel. |
| fused_trips | Whole-array read/write trips for one fused kernel retaining intermediates on chip. |
| launch | |
| overhead_speedup | Dimensionless launch-overhead ratio; it is not an end-to-end execution speedup. |
| traffic_ratio | Dimensionless unfused-to-fused whole-array traffic ratio. |
| unfused_launches | Kernel launch count before fusion, one per elementwise operation. |
| unfused_overhead | Total dispatch-plus-launch overhead in microseconds before fusion. |
| unfused_trips | Number of whole-array read/write trips when each operation is a separate kernel. |
chain_length
Type: int.
Declared default: required.
dispatch
Type: Quantity.
Declared default: required.
launch
Type: Quantity.
Declared default: required.
unfused_trips
engine.scenario_types.KernelFusionScenario.unfused_trips
Number of whole-array read/write trips when each operation is a separate kernel.
Equation: 2 * chain_length.
fused_trips
engine.scenario_types.KernelFusionScenario.fused_trips
Whole-array read/write trips for one fused kernel retaining intermediates on chip.
Equation: 2.
unfused_launches
engine.scenario_types.KernelFusionScenario.unfused_launches
Kernel launch count before fusion, one per elementwise operation.
Equation: chain_length.
fused_launches
engine.scenario_types.KernelFusionScenario.fused_launches
Kernel launch count after fusion into one kernel.
Equation: 1.
traffic_ratio
engine.scenario_types.KernelFusionScenario.traffic_ratio
Dimensionless unfused-to-fused whole-array traffic ratio.
Equation: unfused_trips / fused_trips.
unfused_overhead
engine.scenario_types.KernelFusionScenario.unfused_overhead
Total dispatch-plus-launch overhead in microseconds before fusion.
Equation: (unfused_launches * (dispatch + launch)).to('microsecond').
fused_overhead
engine.scenario_types.KernelFusionScenario.fused_overhead
Dispatch-plus-launch overhead in microseconds for the fused kernel.
Equation: (fused_launches * (dispatch + launch)).to('microsecond').
overhead_speedup
engine.scenario_types.KernelFusionScenario.overhead_speedup
Dimensionless launch-overhead ratio; it is not an end-to-end execution speedup.
Equation: (unfused_overhead / fused_overhead).to('').magnitude.