RetrainingIntervalScenario
Statistics scenarios · Implementation: mlsysim/engine/scenario_definitions/statistics.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
Retraining a model whose accuracy decays as drift fits it, serving query_rate queries worth value_per_query per unit accuracy, at run_cost a run. footprint and accelerator_price price the compute share of a run; compared_intervals and scheduled_interval are cadences to compare with the optimum.
Notes
Equation: Average cost combines run_cost/interval with staleness_slope*interval/2; the optimum balances these terms.
Assumptions: Exponential accuracy decay is linearized locally, each retrain resets quality, and run cost is fixed.
Excludes: Nonlinear long-interval drift, retraining duration, concurrent serving interference, and adaptation effectiveness uncertainty.
References
Derivation: the scenario composes mlsysim.physics.economics.calc_optimal_retraining_interval, mlsysim.physics.economics.calc_retraining_cost_rate, mlsysim.physics.economics.calc_staleness_slope. Their contracts supply the underlying equations; property equations show the remaining unit conversions and accounting.
Examples
>>> from mlsysim import Scenarios
>>> scenario = Scenarios.RankingRetrainingInterval
>>> value = scenario.staleness_slope
>>> str(value.units)
'USD/d²'Attributes
| Name | Description |
|---|---|
| accelerator_price | |
| compared_intervals | |
| compute_cost | USD compute portion of one retraining run, device-hours times accelerator-hour price. |
| compute_share | Compute cost divided by full run cost, dimensionless. |
| drift | |
| footprint | |
| optimal_interval | Days minimizing fixed retraining cost per interval plus linearized staleness cost. |
| query_rate | |
| run_cost | |
| scheduled_interval | |
| staleness_slope | Linearized growth in loss rate, in USD/day², from queries, value, initial accuracy, and |
| value_per_query |
Methods
| Name | Description |
|---|---|
| cost_rate | USD/day for retraining at interval under the fixed-cost and linear-staleness |
cost_rate
engine.scenario_types.RetrainingIntervalScenario.cost_rate(interval)USD/day for retraining at interval under the fixed-cost and linear-staleness approximation.
Equation: calc_retraining_cost_rate(interval, run_cost, staleness_slope).to('USD/day').
drift
Type: AccuracyDecayScenario.
Declared default: required.
query_rate
Type: Quantity.
Declared default: required.
value_per_query
Type: Quantity.
Declared default: required.
run_cost
Type: Quantity.
Declared default: required.
footprint
Type: Optional[Any].
Declared default: None.
accelerator_price
Type: Optional[Quantity].
Declared default: None.
compared_intervals
Type: Optional[List[Quantity]].
Declared default: None.
scheduled_interval
Type: Optional[Quantity].
Declared default: None.
staleness_slope
engine.scenario_types.RetrainingIntervalScenario.staleness_slope
Linearized growth in loss rate, in USD/day², from queries, value, initial accuracy, and fitted decay.
Equation: calc_staleness_slope(query_rate, value_per_query, drift.initial_accuracy, drift.decay_rate).to('USD/day**2').
optimal_interval
engine.scenario_types.RetrainingIntervalScenario.optimal_interval
Days minimizing fixed retraining cost per interval plus linearized staleness cost.
Equation: calc_optimal_retraining_interval(run_cost, staleness_slope).to('day').
compute_cost
engine.scenario_types.RetrainingIntervalScenario.compute_cost
USD compute portion of one retraining run, device-hours times accelerator-hour price.
Equation: (footprint.retrain_accelerator_hours * accelerator_price).to('USD').