DataParallelScaleOutScenario
Training scenarios · Implementation: mlsysim/engine/scenario_definitions/training.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
A data-parallel job of workload on nodes nodes, from a one-device profile.
Notes
Equation: Compose mlsysim.solvers.DataParallelScaleOutModel.
Assumptions: The stated fields are inputs to mlsysim.solvers.DataParallelScaleOutModel; its documented validity regime, units, and modeling choices apply unchanged.
Excludes: This scenario adds no measurement, empirical calibration, or mechanisms beyond mlsysim.solvers.DataParallelScaleOutModel.
References
Derivation: this scenario binds its input fields to the referenced implementation contracts: mlsysim.solvers.DataParallelScaleOutModel. The methods below identify any additional arithmetic.
Examples
>>> from mlsysim import Scenarios
>>> case = Scenarios.Gpt2DataParallelOneNode
>>> type(case.result).__name__
'DataParallelScaleOutResult'Attributes
| Name | Description |
|---|---|
| accumulation_steps | |
| devices_per_node | |
| inter_bandwidth | |
| intra_bandwidth | |
| node_hour_rate | |
| nodes | |
| per_device_batch | |
| precision | |
| result | Evaluate this scenario through mlsysim.solvers.DataParallelScaleOutModel.solve. |
| step_time | |
| training_time | |
| workload |
workload
Type: Workload.
Declared default: required.
precision
Type: str.
Declared default: 'fp16'.
step_time
Type: Quantity.
Declared default: required.
training_time
Type: Quantity.
Declared default: required.
per_device_batch
Type: int.
Declared default: required.
devices_per_node
Type: int.
Declared default: required.
nodes
Type: int.
Declared default: 1.
intra_bandwidth
Type: Quantity.
Declared default: required.
inter_bandwidth
Type: Optional[Quantity].
Declared default: None.
accumulation_steps
Type: int.
Declared default: 1.
node_hour_rate
Type: Optional[Quantity].
Declared default: None.
result
engine.scenario_types.DataParallelScaleOutScenario.result
Evaluate this scenario through mlsysim.solvers.DataParallelScaleOutModel.solve. Equation: pass the stated fields to that implementation; its contract defines the calculation and feasibility tests. Units: each returned field follows the referenced result schema; no independent equation is evaluated by this wrapper.