ProxyMetricLatencyScenario
Statistics scenarios · Implementation: mlsysim/engine/scenario_definitions/statistics.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
A search-width change that buys an offline metric gain with latency, against a serving budget.
Notes
Equation: The calculation members below define metric_gain, width_ratio, slowdown, budget_ratio.
Assumptions: Metric values, widths, and latencies are supplied for the same comparison.
Excludes: Treating offline score improvement as improved online task outcomes.
References
Derivation: A search-width change that buys an offline metric gain with latency, against a serving budget. Each member records the corresponding algebra or named physics function.
Examples
Evaluate the named configuration and display a modeled result.
>>> from mlsysim import Scenarios
>>> case = Scenarios.WiderBeamSearchTranslation
>>> value = case.budget_ratio
>>> print(f"{value:.3g}")
2Attributes
| Name | Description |
|---|---|
| budget_ratio | Optimized latency divided by serving budget; above one exceeds the budget. |
| latency_budget | |
| metric_gain | Optimized-minus-original offline score in the supplied score units. |
| optimized_latency | |
| optimized_metric | |
| optimized_width | |
| original_latency | |
| original_metric | |
| original_width | |
| slowdown | Optimized latency divided by original latency, dimensionless. |
| width_ratio | Optimized search width divided by original width, dimensionless. |
original_metric
Type: float.
Declared default: required.
optimized_metric
Type: float.
Declared default: required.
original_width
Type: int.
Declared default: required.
optimized_width
Type: int.
Declared default: required.
original_latency
Type: Quantity.
Declared default: required.
optimized_latency
Type: Quantity.
Declared default: required.
latency_budget
Type: Quantity.
Declared default: required.
metric_gain
engine.scenario_types.ProxyMetricLatencyScenario.metric_gain
Optimized-minus-original offline score in the supplied score units.
Equation: optimized_metric - original_metric.
width_ratio
engine.scenario_types.ProxyMetricLatencyScenario.width_ratio
Optimized search width divided by original width, dimensionless.
Equation: optimized_width / original_width.
slowdown
engine.scenario_types.ProxyMetricLatencyScenario.slowdown
Optimized latency divided by original latency, dimensionless.
Equation: (optimized_latency / original_latency).to('').magnitude.
budget_ratio
engine.scenario_types.ProxyMetricLatencyScenario.budget_ratio
Optimized latency divided by serving budget; above one exceeds the budget.
Optimized latency over the budget (above 1: the budget is broken).
Equation: (optimized_latency / latency_budget).to('').magnitude.