AccuracyDecayScenario
Reliability scenarios · Implementation: mlsysim/engine/scenario_definitions/reliability.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
A deployed model read at initial_accuracy and, elapsed later, at later_accuracy; noticed is when someone outside monitoring saw it. decay_rate fits A(t) = A0 exp(-gamma t) through the two readings.
Notes
Equation: Two observed accuracies determine gamma in A(t)=A0*exp(-gamma*t); threshold times invert the same curve.
Assumptions: Accuracies are positive, elapsed time is positive, and the selected exponential form remains applicable when extrapolated.
Excludes: Causal drift diagnosis, confidence intervals, and an empirical guarantee of future accuracy; inverse thresholds require nonzero gamma.
References
Derivation: the scenario composes mlsysim.physics.statistics.calc_exponential_decay_rate. Their contracts supply the underlying equations; property equations show the remaining unit conversions and accounting.
Examples
>>> from mlsysim import Scenarios
>>> scenario = Scenarios.RankingServiceAccuracyDrift
>>> value = scenario.decay_rate
>>> str(value.units)
'1/d'Attributes
| Name | Description |
|---|---|
| decay_rate | Inverse-time exponential decay rate fitted through the two supplied accuracy |
| drop | Initial minus later accuracy, in fractional accuracy units rather than relative percent. |
| elapsed | |
| initial_accuracy | |
| later_accuracy | |
| noticed |
Methods
| Name | Description |
|---|---|
| accuracy_at | Dimensionless accuracy at age from the fitted exponential curve. |
| time_to | Days until the fitted curve reaches the requested positive accuracy, requiring nonzero |
| trigger_for_floor | Accuracy threshold that decays to floor during the declared reset delay. |
accuracy_at
engine.scenario_types.AccuracyDecayScenario.accuracy_at(age)Dimensionless accuracy at age from the fitted exponential curve.
Equation: initial_accuracy * _math.exp(-(decay_rate * age).to('').magnitude).
time_to
engine.scenario_types.AccuracyDecayScenario.time_to(accuracy)Days until the fitted curve reaches the requested positive accuracy, requiring nonzero decay rate.
Equation: (_math.log(initial_accuracy / accuracy) / decay_rate).to('day').
trigger_for_floor
engine.scenario_types.AccuracyDecayScenario.trigger_for_floor(floor, delay)Accuracy threshold that decays to floor during the declared reset delay.
Equation: floor * _math.exp((decay_rate * delay).to('').magnitude).
initial_accuracy
Type: float.
Declared default: required.
later_accuracy
Type: float.
Declared default: required.
elapsed
Type: Quantity.
Declared default: required.
noticed
Type: Optional[Quantity].
Declared default: None.
decay_rate
engine.scenario_types.AccuracyDecayScenario.decay_rate
Inverse-time exponential decay rate fitted through the two supplied accuracy observations.
Equation: calc_exponential_decay_rate(initial_accuracy, later_accuracy, elapsed).
drop
engine.scenario_types.AccuracyDecayScenario.drop
Initial minus later accuracy, in fractional accuracy units rather than relative percent.
Equation: initial_accuracy - later_accuracy.