ParallelLayoutScenario
Training scenarios · Implementation: mlsysim/engine/scenario_definitions/training.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
workload trained on a TP x PP x DP layout: step time, efficiency, and state per device.
Notes
Equation: Compose mlsysim.solvers.ParallelLayoutModel.
Assumptions: Tensor, pipeline, and data parallelism form a rectangular product layout, and the solver’s precision and overlap inputs apply uniformly.
Excludes: The scenario does not independently validate physical placement or execution schedules.
References
Derivation: this scenario binds its input fields to the referenced implementation contracts: mlsysim.solvers.ParallelLayoutModel. The methods below identify any additional arithmetic.
Examples
>>> from mlsysim import Scenarios
>>> case = Scenarios.Gpt3DataParallel1K
>>> round(case.n_devices, 4)
1024Attributes
| Name | Description |
|---|---|
| accelerator | |
| batch_tokens | |
| device_memory | |
| dp_size | |
| flops_per_param_token | |
| gradient_precision | |
| link_bandwidth | |
| local_efficiency | |
| microbatches | |
| n_devices | Accelerator count consumed by the parallel layout. Equation: tp_size * pp_size * | | [overlap_fraction](#mlsysim.engine.scenario_types.ParallelLayoutScenario.overlap_fraction) | | | [pp_size](#mlsysim.engine.scenario_types.ParallelLayoutScenario.pp_size) | | | [precision](#mlsysim.engine.scenario_types.ParallelLayoutScenario.precision) | | | [result](#mlsysim.engine.scenario_types.ParallelLayoutScenario.result) | Evaluate this scenario throughmlsysim.solvers.ParallelLayoutModel.solve``. Equation: |
| tp_size | |
| window_resolution | |
| workload |
workload
Type: Workload.
Declared default: required.
tp_size
Type: int.
Declared default: 1.
pp_size
Type: int.
Declared default: 1.
dp_size
Type: int.
Declared default: required.
microbatches
Type: int.
Declared default: 1.
batch_tokens
Type: float.
Declared default: required.
flops_per_param_token
Type: float.
Declared default: required.
accelerator
Type: HardwareNode.
Declared default: required.
precision
Type: str.
Declared default: 'fp8'.
local_efficiency
Type: float.
Declared default: required.
link_bandwidth
Type: Quantity.
Declared default: required.
overlap_fraction
Type: float.
Declared default: required.
gradient_precision
Type: str.
Declared default: 'fp16'.
window_resolution
Type: Optional[Quantity].
Declared default: None.
device_memory
Type: Optional[Quantity].
Declared default: None.
n_devices
engine.scenario_types.ParallelLayoutScenario.n_devices
Accelerator count consumed by the parallel layout. Equation: tp_size * pp_size * dp_size. Units: devices; the three parallel dimensions form a rectangular layout.
result
engine.scenario_types.ParallelLayoutScenario.result
Evaluate this scenario through mlsysim.solvers.ParallelLayoutModel.solve. Equation: pass the stated fields to that implementation; its contract defines the calculation and feasibility tests. Units: each returned field follows the referenced result schema; no independent equation is evaluated by this wrapper.