NoisyNeighborScenario
Serving scenarios · Implementation: mlsysim/engine/scenario_definitions/serving.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
Tenants sharing a KV pool and a replica, one of them bursting, with and without quotas.
Notes
Equation: Other tenant share=(pool-swollen_share)/(tenants-1); sequences=truncate(other_share/baseline_bytes_per_sequence).
Assumptions: Equal allocation among nonbursting tenants and a fixed per-sequence KV footprint.
Excludes: Computing p99 from arrival distributions; the scenario carries p99 values as inputs.
References
Derivation: After one tenant takes its supplied allocation, the remaining pool is divided equally among the other tenants. Dividing each remaining allocation by baseline bytes per sequence gives its whole-sequence capacity.
Examples
The named registry scenario supplies the stated inputs; this is a modeled calculation, not a new measurement.
>>> from mlsysim import Scenarios
>>> scenario = Scenarios.SharedPoolNoisyNeighbor
>>> round(scenario.batch_reduction, 6)
0.625Attributes
| Name | Description |
|---|---|
| base_rate | |
| base_sequences | |
| base_share | Equal-share KV allocation per tenant, preserving the pool’s byte units. |
| batch_reduction | Fractional loss of sequence capacity relative to base_sequences. |
| burst_multiple | Dimensionless offered-load multiplier burst_rate / base_rate; base rate must be positive. |
| burst_rate | |
| kv_pool | |
| other_sequences | Whole sequences each remaining tenant can retain, truncating its allocation divided by the baseline bytes per sequence. |
| other_share | KV bytes per remaining tenant after the bursting tenant takes swollen_share; requires more than one tenant. |
| pool_tenants | |
| protected_p99 | |
| quota_rate | |
| slo | |
| swollen_share | |
| unprotected_p99_high | |
| unprotected_p99_low |
kv_pool
Type: Quantity.
Declared default: required.
pool_tenants
Type: int.
Declared default: required.
base_sequences
Type: int.
Declared default: required.
base_rate
Type: float.
Declared default: required.
burst_rate
Type: float.
Declared default: required.
slo
Type: Quantity.
Declared default: required.
unprotected_p99_low
Type: Quantity.
Declared default: required.
unprotected_p99_high
Type: Quantity.
Declared default: required.
quota_rate
Type: float.
Declared default: required.
protected_p99
Type: Quantity.
Declared default: required.
other_sequences
engine.scenario_types.NoisyNeighborScenario.other_sequences
Whole sequences each remaining tenant can retain, truncating its allocation divided by the baseline bytes per sequence.
Equation: The calculation follows the stated accounting rule; the method body shows each intermediate term.
batch_reduction
engine.scenario_types.NoisyNeighborScenario.batch_reduction
Fractional loss of sequence capacity relative to base_sequences.
Equation: 1 - self.other_sequences / self.base_sequences
burst_multiple
engine.scenario_types.NoisyNeighborScenario.burst_multiple
Dimensionless offered-load multiplier burst_rate / base_rate; base rate must be positive.
Equation: self.burst_rate / self.base_rate