IterationBudgetScenario
Optimization scenarios · Implementation: mlsysim/engine/scenario_definitions/optimization.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
Two teams in one development window: a slow cycle against a fast one, with assumed gains per run.
Accuracies are in percent and gains in points per run. The slow team learns from every run; the fast team can run far more than it can learn from, so fast_effective_runs caps the runs that count. Both stop at ceiling.
Notes
Equation: final accuracy = min(start + useful runs * gain per run, ceiling).
Assumptions: Accuracy values are percentages and gains are percentage points. Run gains and effective fast runs are illustrative inputs, not learned response curves.
Excludes: A statistical learning-curve model and a causal claim that faster experiments improve accuracy by this amount.
References
Derivation: final accuracy = min(start + useful runs * gain per run, ceiling). The individual properties state units and intermediate accounting; this class introduces no fitted coefficient.
Examples
The named scenario supplies illustrative inputs; the output is calculated.
>>> from mlsysim import Scenarios
>>> case = Scenarios.SlowVersusFastIteration
>>> round(float(case.slow_runs), 3)
26.0Attributes
| Name | Description |
|---|---|
| ceiling | |
| fast_cycle | |
| fast_effective_runs | |
| fast_final | Fast team’s illustrative accuracy in percent after applying the shared ceiling. |
| fast_gain | |
| fast_possible_runs | Run-equivalents permitted by the fast cycle, before limiting useful learning |
| fast_start | |
| fast_uncapped | Illustrative accuracy in percent from the stipulated number of useful fast runs. |
| final_gap | Absolute difference in final illustrative accuracy, in percentage points. |
| ideas_ratio | Useful fast experiments per slow run-equivalent, a dimensionless opportunity |
| slow_cycle | |
| slow_final | Illustrative final accuracy in percent: starting accuracy plus per-run gain, |
| slow_gain | |
| slow_runs | Run-equivalents available in the slow team’s development window; fractional runs |
| slow_start | |
| window |
window
Type: Quantity.
Declared default: required.
slow_cycle
Type: Quantity.
Declared default: required.
slow_start
Type: float.
Declared default: required.
slow_gain
Type: float.
Declared default: required.
fast_cycle
Type: Quantity.
Declared default: required.
fast_start
Type: float.
Declared default: required.
fast_gain
Type: float.
Declared default: required.
fast_effective_runs
Type: int.
Declared default: required.
ceiling
Type: float.
Declared default: required.
slow_runs
engine.scenario_types.IterationBudgetScenario.slow_runs
Run-equivalents available in the slow team’s development window; fractional runs are retained.
Equation: return (window / slow_cycle).to('').magnitude.
slow_final
engine.scenario_types.IterationBudgetScenario.slow_final
Illustrative final accuracy in percent: starting accuracy plus per-run gain, capped at the ceiling.
Equation: return min(slow_start + slow_runs * slow_gain, ceiling).
fast_possible_runs
engine.scenario_types.IterationBudgetScenario.fast_possible_runs
Run-equivalents permitted by the fast cycle, before limiting useful learning opportunities.
Equation: return (window / fast_cycle).to('').magnitude.
fast_uncapped
engine.scenario_types.IterationBudgetScenario.fast_uncapped
Illustrative accuracy in percent from the stipulated number of useful fast runs.
Equation: return fast_start + fast_effective_runs * fast_gain.
fast_final
engine.scenario_types.IterationBudgetScenario.fast_final
Fast team’s illustrative accuracy in percent after applying the shared ceiling.
Equation: return min(fast_uncapped, ceiling).
ideas_ratio
engine.scenario_types.IterationBudgetScenario.ideas_ratio
Useful fast experiments per slow run-equivalent, a dimensionless opportunity ratio.
Equation: return fast_effective_runs / slow_runs.
final_gap
engine.scenario_types.IterationBudgetScenario.final_gap
Absolute difference in final illustrative accuracy, in percentage points.
Equation: return abs(fast_final - slow_final).