Skip to main content

GradientSyncScenario

Communication scenarios · Implementation: mlsysim/engine/scenario_definitions/communication.py.

Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.

A data-parallel step of workload: compute, then a ring AllReduce over each link.

Notes

Equation: Compose mlsysim.solvers.GradientSyncModel.

Assumptions: The stated fields are inputs to mlsysim.solvers.GradientSyncModel; its documented validity regime, units, and modeling choices apply unchanged.

Excludes: This scenario adds no measurement, empirical calibration, or mechanisms beyond mlsysim.solvers.GradientSyncModel.

References

Derivation: this scenario binds its input fields to the referenced implementation contracts: mlsysim.solvers.GradientSyncModel. The methods below identify any additional arithmetic.

Examples

>>> from mlsysim import Scenarios
>>> case = Scenarios.Gpt3GradientSync
>>> type(case.result).__name__
'GradientSyncResult'

Attributes

Name Description
compute_speedup
compute_time
energy_per_bit
links
precision
result Evaluate this scenario through mlsysim.solvers.GradientSyncModel.solve. Equation:
steps
workload

workload

Type: Workload.

Declared default: required.

precision

Type: str.

Declared default: 'fp16'.

compute_time

Type: Quantity.

Declared default: required.

energy_per_bit

Type: Optional[Quantity].

Declared default: None.

steps

Type: int.

Declared default: 1.

compute_speedup

Type: float.

Declared default: 1.0.

result

engine.scenario_types.GradientSyncScenario.result

Evaluate this scenario through mlsysim.solvers.GradientSyncModel.solve. Equation: pass the stated fields to that implementation; its contract defines the calculation and feasibility tests. Units: each returned field follows the referenced result schema; no independent equation is evaluated by this wrapper.

Back to top