ClusterExpansionScenario
Optimization scenarios · Implementation: mlsysim/engine/scenario_definitions/optimization.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
A synchronous job grown from n_devices to factor times as many on the same checkpoint tier.
checkpoint supplies the write path; at each size its MTBF is replaced by the GPU-only interval component_mtbf / N and the overhead is the checkpoint-and-rework waste at the Young-Daly optimum. For fixed work, run time is proportional to step time over the retained fraction (one minus that overhead), so the larger job shortens the run only when its step speedup exceeds breakeven_speedup.
Notes
Equation: breakeven_speedup is the worked quantity; each property below states its equation and units.
Assumptions: GPU failures are independent and fixed work experiences the checkpoint solver’s steady-state waste.
Excludes: Non-GPU failures, changing failure distributions, and prediction of the supplied step speedup.
References
Derivation: The scenario binds declared inputs to calc_mtbf_cluster. Those implementations document their derivations and sources.
Examples
Execute the registered worked inputs; this is a modeled calculation.
>>> from mlsysim import Scenarios
>>> case = Scenarios.Gpt3ClusterDoubling1K
>>> value = case.breakeven_speedup
>>> round(value, 3)
1.007Attributes
| Name | Description |
|---|---|
| breakeven_speedup | Step speedup needed for equal fixed-work completion time after reliability overhead. |
| checkpoint | |
| component_mtbf | |
| expanded_devices | Expanded accelerator count rounded to the nearest integer. |
| expanded_interval | Expanded fleet optimal checkpoint interval from the checkpoint solver. |
| expanded_mtbf | Expanded fleet mean time between failures from the adjusted checkpoint scenario. |
| expanded_overhead | Expanded fleet checkpoint-and-rework waste fraction. |
| expanded_retained | Expanded fleet productive-time fraction, one minus waste. |
| factor | |
| interval | Original fleet optimal checkpoint interval from the checkpoint solver. |
| mtbf | Original fleet mean time between failures from the adjusted checkpoint scenario. |
| n_devices | |
| overhead | Original fleet checkpoint-and-rework waste fraction. |
| overhead_growth | Dimensionless expanded-to-original waste ratio. |
| retained | Original fleet productive-time fraction, one minus waste. |
| superlinear_threshold | Step speedup needed for equal accelerator-hours after device-count and reliability changes. |
Methods
| Name | Description |
|---|---|
| accelerator_hours_change | Fractional accelerator-hour change for the supplied positive step speedup. |
| at | Checkpoint scenario with GPU-only independent-failure MTBF at the supplied device count. |
| run_time_change | Fractional wall-time change for the supplied positive step speedup. |
accelerator_hours_change
engine.scenario_types.ClusterExpansionScenario.accelerator_hours_change(
step_speedup,
)Fractional accelerator-hour change for the supplied positive step speedup.
Fractional change in accelerator-hours (devices times run time) at a step speedup.
Equation: return self.superlinear_threshold / step_speedup - 1
at
engine.scenario_types.ClusterExpansionScenario.at(n_devices)Checkpoint scenario with GPU-only independent-failure MTBF at the supplied device count.
The checkpoint scenario against the GPU-only interval of n_devices.
Equation: return self.checkpoint.variant(mtbf=calc_mtbf_cluster(self.component_mtbf, n_devices).to(‘hour’))
run_time_change
engine.scenario_types.ClusterExpansionScenario.run_time_change(step_speedup)Fractional wall-time change for the supplied positive step speedup.
Fractional change in wall-clock run time at a measured step speedup.
Equation: return self.breakeven_speedup / step_speedup - 1
checkpoint
Type: CheckpointIntervalScenario.
Declared default: required.
component_mtbf
Type: Quantity.
Declared default: required.
n_devices
Type: int.
Declared default: required.
factor
Type: float.
Declared default: required.
expanded_devices
engine.scenario_types.ClusterExpansionScenario.expanded_devices
Expanded accelerator count rounded to the nearest integer.
Equation: return round(self.n_devices * self.factor)
mtbf
engine.scenario_types.ClusterExpansionScenario.mtbf
Original fleet mean time between failures from the adjusted checkpoint scenario.
Equation: return self.at(self.n_devices).mtbf
expanded_mtbf
engine.scenario_types.ClusterExpansionScenario.expanded_mtbf
Expanded fleet mean time between failures from the adjusted checkpoint scenario.
Equation: return self.at(self.expanded_devices).mtbf
interval
engine.scenario_types.ClusterExpansionScenario.interval
Original fleet optimal checkpoint interval from the checkpoint solver.
Equation: return self.at(self.n_devices).result.interval
expanded_interval
engine.scenario_types.ClusterExpansionScenario.expanded_interval
Expanded fleet optimal checkpoint interval from the checkpoint solver.
Equation: return self.at(self.expanded_devices).result.interval
overhead
engine.scenario_types.ClusterExpansionScenario.overhead
Original fleet checkpoint-and-rework waste fraction.
Equation: return self.at(self.n_devices).result.waste
expanded_overhead
engine.scenario_types.ClusterExpansionScenario.expanded_overhead
Expanded fleet checkpoint-and-rework waste fraction.
Equation: return self.at(self.expanded_devices).result.waste
overhead_growth
engine.scenario_types.ClusterExpansionScenario.overhead_growth
Dimensionless expanded-to-original waste ratio.
Equation: return self.expanded_overhead / self.overhead
retained
engine.scenario_types.ClusterExpansionScenario.retained
Original fleet productive-time fraction, one minus waste.
Equation: return 1 - self.overhead
expanded_retained
engine.scenario_types.ClusterExpansionScenario.expanded_retained
Expanded fleet productive-time fraction, one minus waste.
Equation: return 1 - self.expanded_overhead
breakeven_speedup
engine.scenario_types.ClusterExpansionScenario.breakeven_speedup
Step speedup needed for equal fixed-work completion time after reliability overhead.
Step speedup at which the larger job finishes the fixed work in the same time.
Equation: return self.retained / self.expanded_retained
superlinear_threshold
engine.scenario_types.ClusterExpansionScenario.superlinear_threshold
Step speedup needed for equal accelerator-hours after device-count and reliability changes.
Step speedup above which the larger job also uses fewer accelerator-hours.
Equation: return self.factor * self.breakeven_speedup