ServingInstanceCostScenario
Economics scenarios · Implementation: mlsysim/engine/scenario_definitions/economics.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
Instances with an hourly price and a saturated request rate: cost per request, and when the small instance sized to the load undercuts one large instance.
Notes
Equation: The calculation members below define cost_per_million, requests_per_hour, price_ratio, breakeven_rate, small_within_large_price, small_fleet_rate.
Assumptions: Instance rates are saturated throughput; the small fleet adds capacities linearly.
Excludes: Queueing targets, idle periods, and economies of scale beyond supplied prices.
References
Derivation: Instances with an hourly price and a saturated request rate: cost per request, and when the small instance sized to the load undercuts one large instance. Each member records the corresponding algebra or named physics function.
Examples
Evaluate the named configuration and display a modeled result.
>>> from mlsysim import Scenarios
>>> case = Scenarios.ResNet50CloudInstanceCost
>>> value = case.breakeven_rate
>>> print(f"{value:.3g~}")
3.02e+03 / sAttributes
| Name | Description |
|---|---|
| breakeven_rate | Requests per second obtainable from small instances at the large instance’s price, allowing fractional instances. |
| large | |
| price_ratio | Large-to-small hourly price ratio, dimensionless. |
| prices | |
| rates | |
| small | |
| small_fleet_rate | Saturated requests per second from the integer small fleet below the large-instance price. |
| small_within_large_price | Largest integer small fleet costing strictly less than one large instance. |
Methods
| Name | Description |
|---|---|
| cost_per_million | USD per million requests for the named instance at its saturated request rate. |
| requests_per_hour | Requests served in one hour at the named instance’s saturated rate. |
cost_per_million
engine.scenario_types.ServingInstanceCostScenario.cost_per_million(name)USD per million requests for the named instance at its saturated request rate.
Equation: (prices[name] / rates[name] * 1000000.0).to('dollar').
requests_per_hour
engine.scenario_types.ServingInstanceCostScenario.requests_per_hour(name)Requests served in one hour at the named instance’s saturated rate.
Equation: (rates[name] * Q_(1, 'hour')).to('').magnitude.
prices
Type: Dict[str, Quantity].
Declared default: required.
rates
Type: Dict[str, Quantity].
Declared default: required.
small
Type: str.
Declared default: required.
large
Type: str.
Declared default: required.
price_ratio
engine.scenario_types.ServingInstanceCostScenario.price_ratio
Large-to-small hourly price ratio, dimensionless.
Equation: (prices[large] / prices[small]).to('').magnitude.
breakeven_rate
engine.scenario_types.ServingInstanceCostScenario.breakeven_rate
Requests per second obtainable from small instances at the large instance’s price, allowing fractional instances.
Load at which the large instance’s price buys the same capacity of small instances.
Equation: (price_ratio * rates[small]).to('1/second').
small_within_large_price
engine.scenario_types.ServingInstanceCostScenario.small_within_large_price
Largest integer small fleet costing strictly less than one large instance.
Most small instances that cost less than one large instance.
Equation: _math.ceil(price_ratio) - 1.
small_fleet_rate
engine.scenario_types.ServingInstanceCostScenario.small_fleet_rate
Saturated requests per second from the integer small fleet below the large-instance price.
Equation: (small_within_large_price * rates[small]).to('1/second').