SkewCostScenario
Reliability scenarios · Implementation: mlsysim/engine/scenario_definitions/reliability.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
Training-serving skew that makes skew_rate of query_rate wrong, each wrong prediction costing cost_per_error.
Notes
Equation: Expected error cost rate is skew_rate*query_rate*cost_per_error; accumulated cost multiplies by duration.
Assumptions: Skew share, query rate, and per-error cost remain constant over the period.
Excludes: Measuring skew, attributing causality, heterogeneous error severity, and recovery effects.
References
Derivation: the scenario composes mlsysim.physics.economics.calc_skew_cost. Their contracts supply the underlying equations; property equations show the remaining unit conversions and accounting.
Examples
>>> from mlsysim import Scenarios
>>> scenario = Scenarios.RankingSkewOnePercent
>>> value = scenario.errors_per_day
>>> round(float(value), 3)
10000.0Attributes
| Name | Description |
|---|---|
| cost_per_error | |
| cost_rate | USD per day from error share, query rate, and cost per error via calc_skew_cost. |
| errors_per_day | Expected daily wrong predictions from query volume times the assumed skew error share. |
| query_rate | |
| skew_rate |
Methods
| Name | Description |
|---|---|
| cost_over | Accumulated USD over period at the constant modeled error-cost rate. |
cost_over
engine.scenario_types.SkewCostScenario.cost_over(period)Accumulated USD over period at the constant modeled error-cost rate.
Equation: (cost_rate * period).to('USD').
skew_rate
Type: float.
Declared default: required.
query_rate
Type: Quantity.
Declared default: required.
cost_per_error
Type: Quantity.
Declared default: required.
errors_per_day
engine.scenario_types.SkewCostScenario.errors_per_day
Expected daily wrong predictions from query volume times the assumed skew error share.
Equation: (skew_rate * query_rate * Q_(1, 'day')).to('').magnitude.
cost_rate
engine.scenario_types.SkewCostScenario.cost_rate
USD per day from error share, query rate, and cost per error via calc_skew_cost.
Equation: calc_skew_cost(skew_rate, query_rate, cost_per_error).to('USD/day').