PrecomputeVersusOnDemandScenario
Serving scenarios · Implementation: mlsysim/engine/scenario_definitions/serving.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
items inputs answered one at a time on demand, or precomputed at a large-batch offline_rate.
Notes
Equation: The calculation members below define on_demand_total, offline_total, ratio.
Assumptions: Every item is processed once; on-demand work is serial while offline throughput already includes batching.
Excludes: Storage, refresh, and unused precomputed answers.
References
Derivation: items inputs answered one at a time on demand, or precomputed at a large-batch offline_rate. Each member records the corresponding algebra or named physics function.
Examples
Evaluate the named configuration and display a modeled result.
>>> from mlsysim import Scenarios
>>> case = Scenarios.PhotoLibraryPrecomputeH100
>>> value = case.ratio
>>> print(f"{value:.3g}")
316Attributes
| Name | Description |
|---|---|
| items | |
| latency_budget | |
| offline_rate | |
| offline_total | Seconds to precompute every item at the stated offline throughput. |
| on_demand_time | |
| on_demand_total | Seconds to answer every item serially at the on-demand per-item time. |
| ratio | Dimensionless serial on-demand time divided by offline batch time. |
items
Type: int.
Declared default: required.
on_demand_time
Type: Quantity.
Declared default: required.
offline_rate
Type: Quantity.
Declared default: required.
latency_budget
Type: Quantity.
Declared default: required.
on_demand_total
engine.scenario_types.PrecomputeVersusOnDemandScenario.on_demand_total
Seconds to answer every item serially at the on-demand per-item time.
Equation: (items * on_demand_time).to('second').
offline_total
engine.scenario_types.PrecomputeVersusOnDemandScenario.offline_total
Seconds to precompute every item at the stated offline throughput.
Equation: (items / offline_rate).to('second').
ratio
engine.scenario_types.PrecomputeVersusOnDemandScenario.ratio
Dimensionless serial on-demand time divided by offline batch time.
Equation: (on_demand_total / offline_total).to('').magnitude.