DataParallelAttackSurfaceScenario
Reliability scenarios · Implementation: mlsysim/engine/scenario_definitions/reliability.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
workload trained on fleet with dp_degree data-parallel replicas: the gradient each step exchanges, the checkpoint it writes, and the traffic of a robust aggregate that gathers every replica’s gradient instead of a ring AllReduce of their sum.
Notes
Equation: Parameter count times element width gives each payload; gather and ring factors compare replicated traffic.
Assumptions: Equal data-parallel replicas and the supplied parameter widths; ratio comparisons require multiple replicas.
Excludes: Cryptographic protocol work, optimizer state beyond the declared checkpoint width, and communication overlap.
References
Derivation: Parameter count times element width gives each payload; gather and ring factors compare replicated traffic. Individual property contracts identify the exact accounting expression or reusable calculation. Registry entries supply the scenario inputs; this class does not turn them into measured outcomes.
Examples
Evaluate a named scenario using its declared inputs.
>>> from mlsysim import Scenarios
>>> scenario = Scenarios.ReferenceJobAttackSurface
>>> round(float(scenario.accelerators), 4)
1024.0Attributes
| Name | Description |
|---|---|
| accelerators | Accelerator count copied from the composed fleet definition. |
| accelerators_per_node | Accelerator count per node copied from the fleet’s node definition. |
| accelerators_per_replica | Mean accelerator count allocated to each data-parallel replica. |
| allreduce_factor | Dimensionless per-worker ring AllReduce bytes per gradient byte, from |
| checkpoint_bytes | Bytes in one parameter checkpoint, using the declared checkpoint width. |
| checkpoint_bytes_per_param | |
| dp_degree | |
| fleet | |
| gather_factor | Per-worker gathered gradient copies from the other data-parallel replicas. |
| gather_traffic_ratio | Dimensionless all-gather-to-ring-AllReduce traffic ratio; requires more than one |
| gradient_bytes | Bytes in one replica’s parameter gradients, using the declared gradient width. |
| gradient_bytes_per_param | |
| nodes | Node count copied from the composed fleet definition. |
| weight_bytes | Bytes in the optional serving checkpoint; requires weight_bytes_per_param. |
| weight_bytes_per_param | |
| workload |
workload
Type: Workload.
Declared default: required.
fleet
Type: Fleet.
Declared default: required.
dp_degree
Type: int.
Declared default: required.
gradient_bytes_per_param
Type: Quantity.
Declared default: required.
checkpoint_bytes_per_param
Type: Quantity.
Declared default: required.
weight_bytes_per_param
Type: Optional[Quantity].
Declared default: None.
accelerators
engine.scenario_types.DataParallelAttackSurfaceScenario.accelerators
Accelerator count copied from the composed fleet definition.
Equation: fleet.total_accelerators.
nodes
engine.scenario_types.DataParallelAttackSurfaceScenario.nodes
Node count copied from the composed fleet definition.
Equation: fleet.count.
accelerators_per_node
engine.scenario_types.DataParallelAttackSurfaceScenario.accelerators_per_node
Accelerator count per node copied from the fleet’s node definition.
Equation: fleet.node.accelerators_per_node.
accelerators_per_replica
engine.scenario_types.DataParallelAttackSurfaceScenario.accelerators_per_replica
Mean accelerator count allocated to each data-parallel replica.
Equation: accelerators / dp_degree.
gradient_bytes
engine.scenario_types.DataParallelAttackSurfaceScenario.gradient_bytes
Bytes in one replica’s parameter gradients, using the declared gradient width.
Equation: memory_from_params(workload.parameters, gradient_bytes_per_param).to('byte').
checkpoint_bytes
engine.scenario_types.DataParallelAttackSurfaceScenario.checkpoint_bytes
Bytes in one parameter checkpoint, using the declared checkpoint width.
Equation: memory_from_params(workload.parameters, checkpoint_bytes_per_param).to('byte').
weight_bytes
engine.scenario_types.DataParallelAttackSurfaceScenario.weight_bytes
Bytes in the optional serving checkpoint; requires weight_bytes_per_param.
Equation: memory_from_params(workload.parameters, weight_bytes_per_param).to('byte').
allreduce_factor
engine.scenario_types.DataParallelAttackSurfaceScenario.allreduce_factor
Dimensionless per-worker ring AllReduce bytes per gradient byte, from calc_ring_allreduce_data_factor.
Bytes each worker moves in a ring AllReduce, per byte of gradient.
Equation: calc_ring_allreduce_data_factor(dp_degree).
gather_factor
engine.scenario_types.DataParallelAttackSurfaceScenario.gather_factor
Per-worker gathered gradient copies from the other data-parallel replicas.
Bytes each worker receives when it gathers every other replica’s gradient.
Equation: dp_degree - 1.
gather_traffic_ratio
engine.scenario_types.DataParallelAttackSurfaceScenario.gather_traffic_ratio
Dimensionless all-gather-to-ring-AllReduce traffic ratio; requires more than one replica.
Gather over ring AllReduce traffic; equals dp_degree / 2.
Equation: gather_factor / allreduce_factor.