Skip to main content

ZeroShardingScenario

Training scenarios · Implementation: mlsysim/engine/scenario_definitions/training.py.

Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.

workload’s training state per device at each ZeRO stage over n_devices.

Notes

Equation: Compose mlsysim.solvers.ZeroShardingModel.

Assumptions: The stated fields are inputs to mlsysim.solvers.ZeroShardingModel; its documented validity regime, units, and modeling choices apply unchanged.

Excludes: This scenario adds no measurement, empirical calibration, or mechanisms beyond mlsysim.solvers.ZeroShardingModel.

References

Derivation: this scenario binds its input fields to the referenced implementation contracts: mlsysim.solvers.ZeroShardingModel. The methods below identify any additional arithmetic.

Examples

>>> from mlsysim import Scenarios
>>> case = Scenarios.Llama2_7B_ZeroStages64
>>> type(case.result).__name__
'ZeroShardingResult'

Attributes

Name Description
activation_strategy
activations
link_bandwidth
n_devices
result Evaluate this scenario through mlsysim.solvers.ZeroShardingModel.solve. Equation:
workload

workload

Type: Workload.

Declared default: required.

n_devices

Type: int.

Declared default: required.

activations

Type: Optional[ActivationMemoryScenario].

Declared default: None.

activation_strategy

Type: str.

Declared default: 'full'.

result

engine.scenario_types.ZeroShardingScenario.result

Evaluate this scenario through mlsysim.solvers.ZeroShardingModel.solve. Equation: pass the stated fields to that implementation; its contract defines the calculation and feasibility tests. Units: each returned field follows the referenced result schema; no independent equation is evaluated by this wrapper.

Back to top