HostAcceleratorBoardScenario
Statistics scenarios · Implementation: mlsysim/engine/scenario_definitions/statistics.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
One board, a host CPU and an attached accelerator: model-only and end-to-end latency, active energy.
The end-to-end times add host preprocessing (and, on the accelerator path, the copy across the host link) to the model time; active energy is the component’s active power times its model-only latency, so it leaves out preprocessing and idle power.
Notes
Equation: The calculation members below define frame_budget, int8_speedup, isolated_speedup, end_to_end_speedup, power_ratio, host_energy, accelerator_energy, energy_ratio.
Assumptions: Active energy is power times model-only latency; end-to-end times are supplied separately.
Excludes: Idle energy, host/accelerator overlap, and energy of the surrounding response path.
References
Derivation: One board, a host CPU and an attached accelerator: model-only and end-to-end latency, active energy. Each member records the corresponding algebra or named physics function.
Examples
Evaluate the named configuration and display a modeled result.
>>> from mlsysim import Scenarios
>>> case = Scenarios.MobileNetV2CameraTrapBoard
>>> value = case.end_to_end_speedup
>>> print(f"{value:.3g}")
3Attributes
| Name | Description |
|---|---|
| accelerator_end_to_end | |
| accelerator_energy | Accelerator active energy per inference in mJ, power times model latency. |
| accelerator_latency | |
| accelerator_power | |
| end_to_end_speedup | Dimensionless host-to-accelerator full-path latency ratio. |
| energy_ratio | Host-to-accelerator modeled active energy ratio, dimensionless. |
| frame_budget | Per-frame latency allowance in ms, reciprocal of frame rate. |
| frame_rate | |
| host_end_to_end | |
| host_energy | Host active energy per inference in mJ, power times INT8 model latency. |
| host_fp32_latency | |
| host_int8_latency | |
| host_power | |
| int8_speedup | Dimensionless host FP32-to-INT8 latency ratio. |
| isolated_speedup | Dimensionless host INT8-to-accelerator model-only latency ratio. |
| power_ratio | Accelerator active power divided by host active power, dimensionless. |
host_fp32_latency
Type: Quantity.
Declared default: required.
host_int8_latency
Type: Quantity.
Declared default: required.
accelerator_latency
Type: Quantity.
Declared default: required.
host_end_to_end
Type: Quantity.
Declared default: required.
accelerator_end_to_end
Type: Quantity.
Declared default: required.
host_power
Type: Quantity.
Declared default: required.
accelerator_power
Type: Quantity.
Declared default: required.
frame_rate
Type: Optional[Quantity].
Declared default: None.
frame_budget
engine.scenario_types.HostAcceleratorBoardScenario.frame_budget
Per-frame latency allowance in ms, reciprocal of frame rate.
Time per frame at frame_rate: the latency each inference must meet.
Equation: (1 / frame_rate).to('ms').
int8_speedup
engine.scenario_types.HostAcceleratorBoardScenario.int8_speedup
Dimensionless host FP32-to-INT8 latency ratio.
Equation: (host_fp32_latency / host_int8_latency).to('').magnitude.
isolated_speedup
engine.scenario_types.HostAcceleratorBoardScenario.isolated_speedup
Dimensionless host INT8-to-accelerator model-only latency ratio.
Equation: (host_int8_latency / accelerator_latency).to('').magnitude.
end_to_end_speedup
engine.scenario_types.HostAcceleratorBoardScenario.end_to_end_speedup
Dimensionless host-to-accelerator full-path latency ratio.
Equation: (host_end_to_end / accelerator_end_to_end).to('').magnitude.
power_ratio
engine.scenario_types.HostAcceleratorBoardScenario.power_ratio
Accelerator active power divided by host active power, dimensionless.
Equation: (accelerator_power / host_power).to('').magnitude.
host_energy
engine.scenario_types.HostAcceleratorBoardScenario.host_energy
Host active energy per inference in mJ, power times INT8 model latency.
Equation: (host_power * host_int8_latency).to('mJ').
accelerator_energy
engine.scenario_types.HostAcceleratorBoardScenario.accelerator_energy
Accelerator active energy per inference in mJ, power times model latency.
Equation: (accelerator_power * accelerator_latency).to('mJ').
energy_ratio
engine.scenario_types.HostAcceleratorBoardScenario.energy_ratio
Host-to-accelerator modeled active energy ratio, dimensionless.
Equation: (host_energy / accelerator_energy).to('').magnitude.