AllReduceScheduleScenario
Communication scenarios · Implementation: mlsysim/engine/scenario_definitions/communication.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
One buffer reduced by ring, simple tree, and parameter server at each of group_sizes.
run_steps and time_saved_fraction price a what-if: removing that share of the ring time on every step of a run.
Notes
Equation: Compose mlsysim.solvers.AllReduceAlgorithmModel.
Assumptions: Named algorithm times come from AllReduceAlgorithmModel; saved-time fractions are hypothetical and apply on every step.
Excludes: Application compute overlap and measurement-based validation of the assumed savings are absent.
References
Derivation: this scenario binds its input fields to the referenced implementation contracts: mlsysim.solvers.AllReduceAlgorithmModel. The methods below identify any additional arithmetic.
Examples
>>> from mlsysim import Scenarios
>>> case = Scenarios.DdpBucketRingNdr
>>> type(case.result).__name__
'AllReduceAlgorithmResult'Attributes
| Name | Description |
|---|---|
| alpha | |
| bandwidth | |
| group_sizes | |
| message | |
| result | Evaluate this scenario through mlsysim.solvers.AllReduceAlgorithmModel.solve. |
| run_steps | |
| time_saved_fraction |
Methods
| Name | Description |
|---|---|
| saved_device_hours | Aggregate device-time saved across the run. Equation: saved_time(n) * run_steps * n. |
| saved_time | Per-step ring time saved at a stated fractional improvement. Equation: |
saved_device_hours
engine.scenario_types.AllReduceScheduleScenario.saved_device_hours(n)Aggregate device-time saved across the run. Equation: saved_time(n) * run_steps * n. Units: scalar device-hours; every participating device is credited the same saved duration.
saved_time
engine.scenario_types.AllReduceScheduleScenario.saved_time(n)Per-step ring time saved at a stated fractional improvement. Equation: result.ring_time[n] * time_saved_fraction. Units: seconds; the improvement fraction is assumed.
message
Type: Quantity.
Declared default: required.
group_sizes
Type: List[int].
Declared default: required.
alpha
Type: Quantity.
Declared default: required.
bandwidth
Type: Quantity.
Declared default: required.
run_steps
Type: int.
Declared default: 1.
time_saved_fraction
Type: float.
Declared default: 0.0.
result
engine.scenario_types.AllReduceScheduleScenario.result
Evaluate this scenario through mlsysim.solvers.AllReduceAlgorithmModel.solve. Equation: pass the stated fields to that implementation; its contract defines the calculation and feasibility tests. Units: each returned field follows the referenced result schema; no independent equation is evaluated by this wrapper.