AcceleratorInputFeedScenario
Data scenarios · Implementation: mlsysim/engine/scenario_definitions/data.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
accelerators copies of hardware training workload at sustained_rate samples a second each, fed from supply with encoded_sample bytes stored per sample and decoded_sample bytes after decode.
The arithmetic ceiling is peak FLOP/s over the training FLOPs of one sample (training_flops_factor times the forward pass). Demand is rate times bytes per sample; the feeding loss is the share of the training rate a slower supply removes (DataModel). worker_rate is what one host loader worker decodes and augments a second; host_decode_cap is where adding workers stops helping because the host’s cores run out.
Notes
Equation: sample ceiling = peak arithmetic / training FLOPs per sample; bandwidth demand = samples per second * bytes per sample.
Assumptions: Forward work and training multiplier define the arithmetic estimate; rates and encoded/decoded sample sizes refer to the same workload.
Excludes: Host-contention scaling, asynchronous pipeline transients, and automatic enforcement of the optional host decode cap.
References
Derivation: This configuration forwards its stored inputs to mlsysim.solvers.DataModel.solve. The solver contract records the equations and their provenance.
Examples
The named scenario supplies illustrative inputs; the output is calculated.
>>> from mlsysim import Scenarios
>>> case = Scenarios.ResNet50A100InputFeed
>>> round(float(case.training_flops_per_sample.to('flop').magnitude), 3)
24600000000.0Attributes
| Name | Description |
|---|---|
| accelerators | |
| achieved_flops | Useful training arithmetic rate in TFLOP/s at the supplied sample throughput. |
| busy_fraction | Accelerator busy fraction limited by host supply, capped at one. |
| ceiling_demand | Required encoded bandwidth in MB/s if every accelerator reached the arithmetic |
| ceiling_gap | Arithmetic ceiling divided by sustained throughput, a dimensionless gap factor. |
| ceiling_rate | Samples per second at peak arithmetic throughput for one accelerator. |
| decode_expansion | Decoded sample bytes divided by encoded sample bytes, a dimensionless expansion |
| decoded_sample | |
| encoded_sample | |
| hardware | |
| host_decode_cap | |
| host_supply | Host-decoded samples per second from worker count times per-worker throughput. |
| host_workers | |
| supplied_fraction | Fraction of encoded sustained demand delivered by storage, capped at one. |
| supply | |
| sustained_fraction | Measured or assumed sample throughput divided by the modeled arithmetic ceiling. |
| sustained_rate | |
| total_rate | Aggregate sustained samples per second for the declared accelerator count. |
| training_flops_factor | |
| training_flops_per_sample | Training work in FLOPs per sample, forward work times the declared training |
| worker_crossing | Fractional worker count whose aggregate decode rate matches one accelerator’s |
| worker_rate | |
| workers_per_accelerator | Whole loader-worker count per accelerator, rounding the crossing upward with |
| workers_pooled | Whole loader-worker count for aggregate demand in one shared pool, rounded upward. |
| workload |
Methods
| Name | Description |
|---|---|
| demand | Required MB/s for all accelerators, using encoded or decoded bytes and the |
| feeding_loss | Lost throughput fraction from DataModel.solve for the selected storage demand. |
| samples_supplied | Encoded samples per second supported by the supplied storage bandwidth. |
| streams_needed | Fractional count of equal-bandwidth streams needed to meet the selected demand. |
demand
engine.scenario_types.AcceleratorInputFeedScenario.demand(
decoded=False,
rate=None,
)Required MB/s for all accelerators, using encoded or decoded bytes and the selected per-device rate.
Equation: per = sustained_rate if rate is None else rate; size = decoded_sample if decoded else encoded_sample; return (accelerators * per * size / Q_(1, 'count')).to('MB/s').
feeding_loss
engine.scenario_types.AcceleratorInputFeedScenario.feeding_loss(decoded=False)Lost throughput fraction from DataModel.solve for the selected storage demand.
Equation: Delegates the declared fields to mlsysim.solvers.DataModel.solve; returned quantities and feasibility follow that contract.
samples_supplied
engine.scenario_types.AcceleratorInputFeedScenario.samples_supplied(bandwidth)Encoded samples per second supported by the supplied storage bandwidth.
Equation: return (bandwidth / encoded_sample * Q_(1, 'count')).to('count/second').
streams_needed
engine.scenario_types.AcceleratorInputFeedScenario.streams_needed(
bandwidth,
rate=None,
)Fractional count of equal-bandwidth streams needed to meet the selected demand.
Equation: target = ceiling_demand if rate is None else demand(rate=rate); return (target / bandwidth).to('').magnitude.
workload
Type: Workload.
Declared default: required.
hardware
Type: HardwareNode.
Declared default: required.
sustained_rate
Type: Quantity.
Declared default: required.
encoded_sample
Type: Optional[Quantity].
Declared default: None.
decoded_sample
Type: Optional[Quantity].
Declared default: None.
supply
Type: Any.
Declared default: None.
training_flops_factor
Type: float.
Declared default: 3.0.
accelerators
Type: int.
Declared default: 1.
worker_rate
Type: Optional[Quantity].
Declared default: None.
host_decode_cap
Type: Optional[Quantity].
Declared default: None.
host_workers
Type: Optional[int].
Declared default: None.
host_supply
engine.scenario_types.AcceleratorInputFeedScenario.host_supply
Host-decoded samples per second from worker count times per-worker throughput.
Equation: return (host_workers * worker_rate).to('count/second').
busy_fraction
engine.scenario_types.AcceleratorInputFeedScenario.busy_fraction
Accelerator busy fraction limited by host supply, capped at one.
Equation: return min(1.0, (host_supply / sustained_rate).to('').magnitude).
training_flops_per_sample
engine.scenario_types.AcceleratorInputFeedScenario.training_flops_per_sample
Training work in FLOPs per sample, forward work times the declared training multiplier.
Equation: return (workload.inference_flops * training_flops_factor).to('flop').
ceiling_rate
engine.scenario_types.AcceleratorInputFeedScenario.ceiling_rate
Samples per second at peak arithmetic throughput for one accelerator.
Equation: return (hardware.compute.peak_flops / training_flops_per_sample).to('count/second').
sustained_fraction
engine.scenario_types.AcceleratorInputFeedScenario.sustained_fraction
Measured or assumed sample throughput divided by the modeled arithmetic ceiling.
Equation: return (sustained_rate / ceiling_rate).to('').magnitude.
achieved_flops
engine.scenario_types.AcceleratorInputFeedScenario.achieved_flops
Useful training arithmetic rate in TFLOP/s at the supplied sample throughput.
Equation: return (sustained_rate * training_flops_per_sample / Q_(1, 'count')).to('TFLOP/s').
ceiling_gap
engine.scenario_types.AcceleratorInputFeedScenario.ceiling_gap
Arithmetic ceiling divided by sustained throughput, a dimensionless gap factor.
Equation: return 1 / sustained_fraction.
total_rate
engine.scenario_types.AcceleratorInputFeedScenario.total_rate
Aggregate sustained samples per second for the declared accelerator count.
Equation: return (accelerators * sustained_rate).to('count/second').
ceiling_demand
engine.scenario_types.AcceleratorInputFeedScenario.ceiling_demand
Required encoded bandwidth in MB/s if every accelerator reached the arithmetic ceiling.
Equation: return demand(rate=ceiling_rate).
decode_expansion
engine.scenario_types.AcceleratorInputFeedScenario.decode_expansion
Decoded sample bytes divided by encoded sample bytes, a dimensionless expansion factor.
Equation: return (decoded_sample / encoded_sample).to('').magnitude.
supplied_fraction
engine.scenario_types.AcceleratorInputFeedScenario.supplied_fraction
Fraction of encoded sustained demand delivered by storage, capped at one.
Equation: return min((supply.bandwidth / demand()).to('').magnitude, 1.0).
worker_crossing
engine.scenario_types.AcceleratorInputFeedScenario.worker_crossing
Fractional worker count whose aggregate decode rate matches one accelerator’s sustained demand.
Equation: return (sustained_rate / worker_rate).to('').magnitude.
workers_per_accelerator
engine.scenario_types.AcceleratorInputFeedScenario.workers_per_accelerator
Whole loader-worker count per accelerator, rounding the crossing upward with floating-point tolerance.
Equation: return math.ceil(worker_crossing - 1e-09).
workers_pooled
engine.scenario_types.AcceleratorInputFeedScenario.workers_pooled
Whole loader-worker count for aggregate demand in one shared pool, rounded upward.
Equation: return math.ceil((total_rate / worker_rate).to('').magnitude - 1e-09).