ServingTaxScenario
Serving scenarios · Implementation: mlsysim/engine/scenario_definitions/serving.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
A request’s latency target split between model execution and everything outside it.
Notes
Equation: outside-model budget = latency target - model time; network time = sequential exchange count * exchange latency.
Assumptions: Network exchanges are sequential and low/high exchange latencies are supplied bounds.
Excludes: Queueing tails, overlap, serialization work, and enforcement of an end-to- end SLA.
References
Derivation: outside-model budget = latency target - model time; network time = sequential exchange count * exchange latency. The individual properties state units and intermediate accounting; this class introduces no fitted coefficient.
Examples
The named scenario supplies illustrative inputs; the output is calculated.
>>> from mlsysim import Scenarios
>>> case = Scenarios.InteractiveRequestServingTax
>>> round(float(case.outside_model.to('ms').magnitude), 3)
30.0Attributes
| Name | Description |
|---|---|
| exchange_high | |
| exchange_low | |
| exchanges | |
| exchanges_high | Milliseconds consumed by all sequential exchanges at the high per-exchange |
| exchanges_low | Milliseconds consumed by all sequential exchanges at the low per-exchange latency. |
| latency_target | |
| model_time | |
| outside_model | Latency budget remaining outside model execution in milliseconds; negative means |
latency_target
Type: Quantity.
Declared default: required.
model_time
Type: Quantity.
Declared default: required.
exchanges
Type: int.
Declared default: required.
exchange_low
Type: Quantity.
Declared default: required.
exchange_high
Type: Quantity.
Declared default: required.
outside_model
engine.scenario_types.ServingTaxScenario.outside_model
Latency budget remaining outside model execution in milliseconds; negative means no feasible remainder.
Equation: return (latency_target - model_time).to('ms').
exchanges_low
engine.scenario_types.ServingTaxScenario.exchanges_low
Milliseconds consumed by all sequential exchanges at the low per-exchange latency.
Equation: return (exchanges * exchange_low).to('ms').
exchanges_high
engine.scenario_types.ServingTaxScenario.exchanges_high
Milliseconds consumed by all sequential exchanges at the high per-exchange latency.
Equation: return (exchanges * exchange_high).to('ms').