PipelineScheduleScenario

Training scenarios · Implementation: mlsysim/engine/scenario_definitions/training.py.

Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.

A synchronous pipeline of stages stages fed microbatches microbatches per step.

Notes

Equation: Compose mlsysim.physics.performance.calc_pipeline_bubble.

Assumptions: All pipeline stages take equal time and use the noninterleaved synchronous fill/drain abstraction.

Excludes: Communication, stage imbalance, and interleaved virtual stages are absent.

References

Derivation: this scenario binds its input fields to the referenced implementation contracts: mlsysim.physics.performance.calc_pipeline_bubble. The methods below identify any additional arithmetic.

Examples

>>> from mlsysim import Scenarios
>>> case = Scenarios.GPipeFourStageWalkthrough
>>> round(case.bubble, 4)
0.4286

Attributes

Name	Description
bubble	Idle fraction of the balanced synchronous pipeline schedule. Equation:
fill_slots	Initial pipeline fill depth. Equation: stages - 1. Units: slots; each stage consumes
idle_slots	Fill/drain slots without useful work at a pipeline stage. Equation: 2 * (stages - | | [microbatches](#mlsysim.engine.scenario_types.PipelineScheduleScenario.microbatches) | | | [phase_slots](#mlsysim.engine.scenario_types.PipelineScheduleScenario.phase_slots) | Slots in one forward or backward phase. Equation:microbatches + stages - 1. Units: | | [slots](#mlsysim.engine.scenario_types.PipelineScheduleScenario.slots) | Total forward-plus-backward scheduling slots. Equation:2 * (microbatches + stages -
stages

Methods

Name	Description
microbatches_for_bubble	Continuous microbatch count meeting the target bubble fraction. Equation: ``(stages - 1)

microbatches_for_bubble

engine.scenario_types.PipelineScheduleScenario.microbatches_for_bubble(target)

Continuous microbatch count meeting the target bubble fraction. Equation: (stages - 1) * (1 - target) / target. Units: microbatches; callers must round up for an integer schedule and use a positive target.

stages

Type: int.

Declared default: required.

microbatches

Type: int.

Declared default: required.

bubble

engine.scenario_types.PipelineScheduleScenario.bubble

Idle fraction of the balanced synchronous pipeline schedule. Equation: mlsysim.physics.performance.calc_pipeline_bubble(stages, microbatches). Units: dimensionless; imbalance and communication are excluded.

slots

engine.scenario_types.PipelineScheduleScenario.slots

Total forward-plus-backward scheduling slots. Equation: 2 * (microbatches + stages - 1). Units: equal-duration slots; stages are balanced.

idle_slots

engine.scenario_types.PipelineScheduleScenario.idle_slots

Fill/drain slots without useful work at a pipeline stage. Equation: 2 * (stages - 1). Units: slots; this is the noninterleaved schedule.

fill_slots

engine.scenario_types.PipelineScheduleScenario.fill_slots

Initial pipeline fill depth. Equation: stages - 1. Units: slots; each stage consumes one slot.

phase_slots

engine.scenario_types.PipelineScheduleScenario.phase_slots

Slots in one forward or backward phase. Equation: microbatches + stages - 1. Units: slots under the balanced-stage abstraction.