QuantizedDeploymentProfileScenario
Optimization scenarios · Implementation: mlsysim/engine/scenario_definitions/optimization.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
A float model and its INT8 version on one accelerator, judged against acceptance limits.
Accuracy and confidence-calibration error are stated inputs (percent); latency and throughput come from profile (an engine profile with tensorrt(precision, batch) and large_batch_throughput(precision)); energy per image assumes full board power.
Notes
Equation: The calculation members below define top1_drop, float_latency, int_latency, latency_speedup, float_throughput, int_throughput, throughput_gain, float_size, int_size, size_reduction, float_energy, int_energy, energy_improvement, passes_top1, passes_calibration.
Assumptions: Latency/throughput are supplied by the selected profile; weight storage scales with bytes per parameter and energy uses TDP.
Excludes: A fresh hardware measurement, KV/workspace memory, and measured per-request power.
References
Derivation: A float model and its INT8 version on one accelerator, judged against acceptance limits. Each member records the corresponding algebra or named physics function.
Examples
Evaluate the named configuration and display a modeled result.
>>> from mlsysim import Scenarios
>>> case = Scenarios.ResNet50Int8ProfileH100
>>> value = case.latency_speedup
>>> print(f"{value:.3g}")
1.25Attributes
| Name | Description |
|---|---|
| calibration_error_limit_pct | |
| energy_improvement | Dimensionless FP32-to-INT8 estimated energy ratio. |
| float_bytes | |
| float_calibration_error_pct | |
| float_energy | TDP-divided-by-throughput estimate in mJ per FP32 request, not measured energy. |
| float_latency | Profiled FP32 TensorRT batch-one latency; uses the profile’s latency quantity. |
| float_size | Float weight footprint from Workload.size_in_bytes at the stated bytes per parameter. |
| float_throughput | Profile’s FP32 large-batch throughput in requests per second. |
| float_top1_pct | |
| hardware | |
| int_bytes | |
| int_calibration_error_pct | |
| int_energy | TDP-divided-by-throughput estimate in mJ per INT8 request, not measured energy. |
| int_latency | Profiled INT8 TensorRT batch-one latency; uses the profile’s latency quantity. |
| int_size | Quantized weight footprint from Workload.size_in_bytes at the stated bytes per parameter. |
| int_throughput | Profile’s INT8 large-batch throughput in requests per second. |
| int_top1_pct | |
| latency_speedup | Dimensionless profiled FP32-to-INT8 batch-one latency ratio. |
| passes_calibration | Whether INT8 calibration error stays below the preset percentage limit. |
| passes_top1 | Whether INT8 top-1 accuracy meets the preset percentage threshold. |
| profile | |
| size_reduction | Dimensionless float-to-quantized weight-memory ratio. |
| throughput_gain | Dimensionless INT8-to-FP32 large-batch throughput ratio. |
| top1_drop | Float-minus-INT8 top-1 accuracy in percentage points. |
| top1_limit_pct | |
| workload |
workload
Type: Workload.
Declared default: required.
hardware
Type: HardwareNode.
Declared default: required.
profile
Type: Any.
Declared default: required.
float_bytes
Type: Quantity.
Declared default: required.
int_bytes
Type: Quantity.
Declared default: required.
float_top1_pct
Type: float.
Declared default: required.
int_top1_pct
Type: float.
Declared default: required.
float_calibration_error_pct
Type: float.
Declared default: required.
int_calibration_error_pct
Type: float.
Declared default: required.
top1_limit_pct
Type: float.
Declared default: required.
calibration_error_limit_pct
Type: float.
Declared default: required.
top1_drop
engine.scenario_types.QuantizedDeploymentProfileScenario.top1_drop
Float-minus-INT8 top-1 accuracy in percentage points.
Equation: float_top1_pct - int_top1_pct.
float_latency
engine.scenario_types.QuantizedDeploymentProfileScenario.float_latency
Profiled FP32 TensorRT batch-one latency; uses the profile’s latency quantity.
Equation: profile.tensorrt('fp32', batch=1).
int_latency
engine.scenario_types.QuantizedDeploymentProfileScenario.int_latency
Profiled INT8 TensorRT batch-one latency; uses the profile’s latency quantity.
Equation: profile.tensorrt('int8', batch=1).
latency_speedup
engine.scenario_types.QuantizedDeploymentProfileScenario.latency_speedup
Dimensionless profiled FP32-to-INT8 batch-one latency ratio.
Equation: (float_latency / int_latency).to('').magnitude.
float_throughput
engine.scenario_types.QuantizedDeploymentProfileScenario.float_throughput
Profile’s FP32 large-batch throughput in requests per second.
Equation: profile.large_batch_throughput('fp32').
int_throughput
engine.scenario_types.QuantizedDeploymentProfileScenario.int_throughput
Profile’s INT8 large-batch throughput in requests per second.
Equation: profile.large_batch_throughput('int8').
throughput_gain
engine.scenario_types.QuantizedDeploymentProfileScenario.throughput_gain
Dimensionless INT8-to-FP32 large-batch throughput ratio.
Equation: int_throughput / float_throughput.
float_size
engine.scenario_types.QuantizedDeploymentProfileScenario.float_size
Float weight footprint from Workload.size_in_bytes at the stated bytes per parameter.
Equation: workload.size_in_bytes(float_bytes).
int_size
engine.scenario_types.QuantizedDeploymentProfileScenario.int_size
Quantized weight footprint from Workload.size_in_bytes at the stated bytes per parameter.
Equation: workload.size_in_bytes(int_bytes).
size_reduction
engine.scenario_types.QuantizedDeploymentProfileScenario.size_reduction
Dimensionless float-to-quantized weight-memory ratio.
Equation: (float_size / int_size).to('').magnitude.
float_energy
engine.scenario_types.QuantizedDeploymentProfileScenario.float_energy
TDP-divided-by-throughput estimate in mJ per FP32 request, not measured energy.
Equation: (hardware.tdp / (float_throughput / Q_(1, 'second'))).to('mJ').
int_energy
engine.scenario_types.QuantizedDeploymentProfileScenario.int_energy
TDP-divided-by-throughput estimate in mJ per INT8 request, not measured energy.
Equation: (hardware.tdp / (int_throughput / Q_(1, 'second'))).to('mJ').
energy_improvement
engine.scenario_types.QuantizedDeploymentProfileScenario.energy_improvement
Dimensionless FP32-to-INT8 estimated energy ratio.
Equation: (float_energy / int_energy).to('').magnitude.
passes_top1
engine.scenario_types.QuantizedDeploymentProfileScenario.passes_top1
Whether INT8 top-1 accuracy meets the preset percentage threshold.
Equation: int_top1_pct >= top1_limit_pct.
passes_calibration
engine.scenario_types.QuantizedDeploymentProfileScenario.passes_calibration
Whether INT8 calibration error stays below the preset percentage limit.
Equation: int_calibration_error_pct <= calibration_error_limit_pct.