BatchCapChoiceScenario
Serving scenarios · Implementation: mlsysim/engine/scenario_definitions/serving.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
Two batch caps at one arrival rate, batches waiting until full: throughput against formation wait.
Notes
Equation: The calculation members below define formation_wait, throughput_gain, capacity_over_load.
Assumptions: Batches wait until full under the supplied arrival rate; service throughput comes from the same profile.
Excludes: Timeout-triggered partial batches and worker backlog.
References
Derivation: Two batch caps at one arrival rate, batches waiting until full: throughput against formation wait. Each member records the corresponding algebra or named physics function.
Examples
Evaluate the named configuration and display a modeled result.
>>> from mlsysim import Scenarios
>>> case = Scenarios.BatchCap16Or32At500Qps
>>> value = case.throughput_gain
>>> print(f"{value:.3g}")
0.769Attributes
| Name | Description |
|---|---|
| arrival_rate | |
| capacity_over_load | Small-cap saturated throughput divided by arrival rate, dimensionless. |
| large_cap | |
| service | |
| small_cap | |
| throughput_gain | Fractional throughput increase from small to large batch cap. |
Methods
| Name | Description |
|---|---|
| formation_wait | Mean full-batch formation wait in ms via calc_batch_formation_wait for the supplied cap. |
formation_wait
engine.scenario_types.BatchCapChoiceScenario.formation_wait(cap)Mean full-batch formation wait in ms via calc_batch_formation_wait for the supplied cap.
Calculation: mlsysim.physics.serving.calc_batch_formation_wait.
service
Type: BatchingKneeScenario.
Declared default: required.
arrival_rate
Type: Quantity.
Declared default: required.
small_cap
Type: int.
Declared default: required.
large_cap
Type: int.
Declared default: required.
throughput_gain
engine.scenario_types.BatchCapChoiceScenario.throughput_gain
Fractional throughput increase from small to large batch cap.
Equation: (service.throughput(large_cap) / service.throughput(small_cap)).to('').magnitude - 1.
capacity_over_load
engine.scenario_types.BatchCapChoiceScenario.capacity_over_load
Small-cap saturated throughput divided by arrival rate, dimensionless.
Equation: (service.throughput(small_cap) / arrival_rate).to('').magnitude.