HierarchicalAllReduceScenario
Communication scenarios · Implementation: mlsysim/engine/scenario_definitions/communication.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
A buffer AllReduced over n_nodes nodes of gpus_per_node GPUs, in three phases or one flat ring.
Notes
Equation: Compose mlsysim.solvers.HierarchicalAllReduceModel.
Assumptions: Every node has the same GPU count and the hierarchy’s bandwidth/startup values are fixed.
Excludes: Topology placement and dynamic routing beyond the solver’s three-phase comparison are absent.
References
Derivation: this scenario binds its input fields to the referenced implementation contracts: mlsysim.solvers.HierarchicalAllReduceModel. The methods below identify any additional arithmetic.
Examples
>>> from mlsysim import Scenarios
>>> case = Scenarios.HierarchicalAllReduce8x8DgxH100
>>> round(case.n_devices, 4)
64Attributes
| Name | Description |
|---|---|
| gpus_per_node | |
| inter_alpha | |
| inter_bandwidth | |
| intra_bandwidth | |
| message | |
| n_devices | Accelerator count across all nodes. Equation: n_nodes * gpus_per_node. Units: |
| n_nodes | |
| result | Evaluate this scenario through mlsysim.solvers.HierarchicalAllReduceModel.solve. |
message
Type: Quantity.
Declared default: required.
n_nodes
Type: int.
Declared default: required.
gpus_per_node
Type: int.
Declared default: required.
intra_bandwidth
Type: Quantity.
Declared default: required.
inter_bandwidth
Type: Quantity.
Declared default: required.
inter_alpha
Type: Quantity.
Declared default: required.
n_devices
engine.scenario_types.HierarchicalAllReduceScenario.n_devices
Accelerator count across all nodes. Equation: n_nodes * gpus_per_node. Units: devices; node populations are identical.
result
engine.scenario_types.HierarchicalAllReduceScenario.result
Evaluate this scenario through mlsysim.solvers.HierarchicalAllReduceModel.solve. Equation: pass the stated fields to that implementation; its contract defines the calculation and feasibility tests. Units: each returned field follows the referenced result schema; no independent equation is evaluated by this wrapper.