BatchServingCostScenario
Economics scenarios · Implementation: mlsysim/engine/scenario_definitions/economics.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
One accelerator rented at price, serving batches back to back: cost per request by batch size.
Notes
Equation: The calculation members below define cost_per_million, cost_ratio, added_time.
Assumptions: The accelerator is rented continuously and runs batches back to back at the supplied service curve.
Excludes: Formation delay, idle utilization, queueing, and autoscaling costs.
References
Derivation: One accelerator rented at price, serving batches back to back: cost per request by batch size. Each member records the corresponding algebra or named physics function.
Examples
Evaluate the named configuration and display a modeled result.
>>> from mlsysim import Scenarios
>>> case = Scenarios.ResNet50BatchOneVersusEightCostH100
>>> value = case.cost_ratio
>>> print(f"{value:.3g}")
7.51Attributes
| Name | Description |
|---|---|
| added_time | Last-batch service time minus first-batch service time in ms. |
| batch_sizes | |
| cost_ratio | Dimensionless first-to-last batch-size cost-per-request ratio. |
| price | |
| service |
Methods
| Name | Description |
|---|---|
| cost_per_million | USD per million requests at a given batch size, rent rate divided by saturated throughput. |
cost_per_million
engine.scenario_types.BatchServingCostScenario.cost_per_million(batch_size)USD per million requests at a given batch size, rent rate divided by saturated throughput.
Equation: (price / service.throughput(batch_size) * 1000000.0).to('dollar').
service
Type: BatchingKneeScenario.
Declared default: required.
price
Type: Quantity.
Declared default: required.
batch_sizes
Type: List[int].
Declared default: required.
cost_ratio
engine.scenario_types.BatchServingCostScenario.cost_ratio
Dimensionless first-to-last batch-size cost-per-request ratio.
Cost per request at the first batch size over the last.
Equation: (cost_per_million(first) / cost_per_million(last)).to('').magnitude.
added_time
engine.scenario_types.BatchServingCostScenario.added_time
Last-batch service time minus first-batch service time in ms.
Service time of the last batch size beyond the first.
Equation: (service.service_time(last) - service.service_time(first)).to('ms').