QuantizedAcceptanceGatesScenario
Statistics scenarios · Implementation: mlsysim/engine/scenario_definitions/statistics.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
A float model and its quantized copy against acceptance gates set before compression.
Accuracies are percent; ECE is a fraction. Edge cases are edge_share of inputs, so a test_images test set holds edge_share * test_images of them, and the float edge-case accuracy is known only to the binomial half-width at confidence on that count.
Notes
Equation: The calculation members below define top1_loss, top5_loss, edge_drop, ece_ratio, edge_images, edge_half_width.
Assumptions: Accuracy inputs use percentage points; subgroup precision uses an independent-binomial normal approximation.
Excludes: Multiple-testing correction, paired disagreement, and certification from point estimates.
References
Derivation: A float model and its quantized copy against acceptance gates set before compression. Each member records the corresponding algebra or named physics function.
Examples
Evaluate the named configuration and display a modeled result.
>>> from mlsysim import Scenarios
>>> case = Scenarios.MobileNetV2Int8CameraTrapGates
>>> value = case.edge_half_width
>>> print(f"{value:.3g}")
12.9Attributes
| Name | Description |
|---|---|
| accuracy_loss_limit | |
| confidence | |
| confidence_threshold | |
| ece_ratio | Quantized calibration error divided by float calibration error, dimensionless. |
| edge_drop | Float-minus-quantized edge-subgroup score in percentage points. |
| edge_half_width | Normal-approximation float edge-accuracy half-width in percentage points, via calc_proportion_half_width. |
| edge_illuminance | |
| edge_images | Rounded count of edge-subgroup test images from subgroup share times test count. |
| edge_occlusion | |
| edge_rotation_deg | |
| edge_share | |
| float_ece | |
| float_edge | |
| float_top1 | |
| float_top5 | |
| int_ece | |
| int_edge | |
| int_top1 | |
| int_top5 | |
| test_images | |
| top1_loss | Float-minus-quantized top-1 score in the supplied percentage-point convention. |
| top5_loss | Float-minus-quantized top-5 score in percentage points. |
| workload |
workload
Type: Workload.
Declared default: required.
float_top1
Type: float.
Declared default: required.
int_top1
Type: float.
Declared default: required.
float_top5
Type: float.
Declared default: required.
int_top5
Type: float.
Declared default: required.
float_ece
Type: float.
Declared default: required.
int_ece
Type: float.
Declared default: required.
float_edge
Type: float.
Declared default: required.
int_edge
Type: float.
Declared default: required.
accuracy_loss_limit
Type: float.
Declared default: required.
edge_occlusion
Type: float.
Declared default: required.
edge_illuminance
Type: Quantity.
Declared default: required.
edge_rotation_deg
Type: float.
Declared default: required.
confidence_threshold
Type: float.
Declared default: required.
test_images
Type: int.
Declared default: required.
confidence
Type: float.
Declared default: required.
top1_loss
engine.scenario_types.QuantizedAcceptanceGatesScenario.top1_loss
Float-minus-quantized top-1 score in the supplied percentage-point convention.
Equation: float_top1 - int_top1.
top5_loss
engine.scenario_types.QuantizedAcceptanceGatesScenario.top5_loss
Float-minus-quantized top-5 score in percentage points.
Equation: float_top5 - int_top5.
edge_drop
engine.scenario_types.QuantizedAcceptanceGatesScenario.edge_drop
Float-minus-quantized edge-subgroup score in percentage points.
Equation: float_edge - int_edge.
ece_ratio
engine.scenario_types.QuantizedAcceptanceGatesScenario.ece_ratio
Quantized calibration error divided by float calibration error, dimensionless.
Equation: int_ece / float_ece.
edge_images
engine.scenario_types.QuantizedAcceptanceGatesScenario.edge_images
Rounded count of edge-subgroup test images from subgroup share times test count.
Equation: round(edge_share * test_images).
edge_half_width
engine.scenario_types.QuantizedAcceptanceGatesScenario.edge_half_width
Normal-approximation float edge-accuracy half-width in percentage points, via calc_proportion_half_width.
Binomial half-width of the float edge-case accuracy on edge_images, in points.
Calculation: mlsysim.physics.statistics.calc_proportion_half_width.