ClusterTrainingTimeScenario
Optimization scenarios · Implementation: mlsysim/engine/scenario_definitions/optimization.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
Training workload on n_devices x hardware at one end-to-end utilization (compute-bound).
Optional (added 2026-10-20): tokens_per_step turns the run into steps (a step costs 6P FLOPs per token); hourly_price is the rental price of one device for an hour. Energy is every device at its TDP for the whole run.
Notes
Equation: Compose mlsysim.physics.transformer.calc_transformer_training_flops.
Assumptions: The registered training work is compute bound at a fixed aggregate delivered rate. The supplied efficiency must already absorb any intended end-to-end losses.
Excludes: Memory capacity and explicit communication scheduling are not checked; TDP energy is modeled, not measured.
References
Derivation: this scenario binds its input fields to the referenced implementation contracts: mlsysim.physics.transformer.calc_transformer_training_flops. The methods below identify any additional arithmetic.
Examples
>>> from mlsysim import Scenarios
>>> case = Scenarios.Gpt3TrainingTime1KA100
>>> str(case.peak_rate)
'312.0 TFLOPs/s'Attributes
| Name | Description |
|---|---|
| delivered_rate | Aggregate arithmetic capacity after the supplied efficiency. Equation: n_devices * | | [device_hours](#mlsysim.engine.scenario_types.ClusterTrainingTimeScenario.device_hours) | Aggregate accelerator-time allocated to training. Equation:n_devices * |
| efficiency | |
| energy | Constant-TDP accelerator energy over the modeled training duration. Equation: |
| hardware | |
| hourly_price | |
| n_devices | |
| peak_precision | |
| peak_rate | Per-device peak arithmetic rate at the selected precision. Equation: catalog default |
| rental_cost | Rental charge for modeled aggregate accelerator-time. Equation: device_hours * | | [step_energy](#mlsysim.engine.scenario_types.ClusterTrainingTimeScenario.step_energy) | Constant-TDP accelerator energy for one modeled step. Equation:n_devices * |
| step_operations | Parameter-dominated work for the stated tokens per step. Equation: |
| step_time | Compute-bound duration of one training step. Equation: _time(step_operations). |
| steps | Number of equal-work steps in the training budget. Equation: ``workload.training_ops / |
| tokens_per_step | |
| training_time | Compute-bound duration for the registered total training work. Equation: |
| training_tokens | Parameter-dominated token count implied by total training work. Equation: total FLOPs |
| workload |
workload
Type: Workload.
Declared default: required.
hardware
Type: HardwareNode.
Declared default: required.
n_devices
Type: int.
Declared default: required.
efficiency
Type: float.
Declared default: required.
tokens_per_step
Type: Optional[int].
Declared default: None.
hourly_price
Type: Optional[Quantity].
Declared default: None.
peak_precision
Type: Optional[str].
Declared default: None.
peak_rate
engine.scenario_types.ClusterTrainingTimeScenario.peak_rate
Per-device peak arithmetic rate at the selected precision. Equation: catalog default when no override is supplied, otherwise precision_flops[peak_precision]. Units: FLOP/s; neither utilization nor device count is included yet.
training_time
engine.scenario_types.ClusterTrainingTimeScenario.training_time
Compute-bound duration for the registered total training work. Equation: _time(workload.training_ops) using work divided by aggregate delivered rate. Units: days; communication must already be reflected in efficiency.
delivered_rate
engine.scenario_types.ClusterTrainingTimeScenario.delivered_rate
Aggregate arithmetic capacity after the supplied efficiency. Equation: n_devices * peak_rate * efficiency. Units: TFLOP/s; efficiency is assumed rather than learned from measurements here.
training_tokens
engine.scenario_types.ClusterTrainingTimeScenario.training_tokens
Parameter-dominated token count implied by total training work. Equation: total FLOPs divided by calc_transformer_training_flops(parameters, 1). Units: count; this inverts the helper’s accounting approximation.
step_operations
engine.scenario_types.ClusterTrainingTimeScenario.step_operations
Parameter-dominated work for the stated tokens per step. Equation: mlsysim.physics.transformer.calc_transformer_training_flops(parameters, tokens_per_step). Units: TFLOPs; attention-over-context is excluded by the helper.
step_time
engine.scenario_types.ClusterTrainingTimeScenario.step_time
Compute-bound duration of one training step. Equation: _time(step_operations). Units: seconds; the same delivered rate applies throughout the run.
steps
engine.scenario_types.ClusterTrainingTimeScenario.steps
Number of equal-work steps in the training budget. Equation: workload.training_ops / step_operations. Units: steps returned as a scalar; a partial final step is not rounded.
energy
engine.scenario_types.ClusterTrainingTimeScenario.energy
Constant-TDP accelerator energy over the modeled training duration. Equation: n_devices * hardware.tdp * training_time. Units: kWh; host/facility overhead and varying power are excluded.
step_energy
engine.scenario_types.ClusterTrainingTimeScenario.step_energy
Constant-TDP accelerator energy for one modeled step. Equation: n_devices * hardware.tdp * step_time. Units: joules; this is an estimate, not meter data.
device_hours
engine.scenario_types.ClusterTrainingTimeScenario.device_hours
Aggregate accelerator-time allocated to training. Equation: n_devices * training_time. Units: device-hours; every device is counted throughout the run.
rental_cost
engine.scenario_types.ClusterTrainingTimeScenario.rental_cost
Rental charge for modeled aggregate accelerator-time. Equation: device_hours * hourly_price. Units: USD; storage and network charges are excluded.