RequestZonesScenario
Serving scenarios · Implementation: mlsysim/engine/scenario_definitions/serving.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
One serialized request laid out in zones, the leading ones an invariant shared prefix.
Notes
Equation: Total tokens=sum(zones); prefix tokens=sum(first shared_zones entries).
Assumptions: Dictionary insertion order is serialization order and the leading prefix is identical across requests.
Excludes: Tokenizer execution and checking that prefix contents actually match.
References
Derivation: A serialized sequence is a concatenation of zones. Summing all zone lengths gives the request length; summing the leading subset gives the shareable prefix. The prefix fraction divides those two counts.
Examples
The named registry scenario supplies the stated inputs; this is a modeled calculation, not a new measurement.
>>> from mlsysim import Scenarios
>>> scenario = Scenarios.RepairRequestZones
>>> scenario.shared_tokens
14336Attributes
| Name | Description |
|---|---|
| shared_share | Dimensionless fraction of request tokens in the invariant prefix: shared_tokens / total_tokens; needs a nonempty request. |
| shared_tokens | Token count in the first shared_zones entries; only that contiguous leading prefix is shareable. |
| shared_zones | |
| total_tokens | Serialized request token count, summing every zone in insertion order. |
| zones |
zones
Type: Dict[str, int].
Declared default: required.
total_tokens
engine.scenario_types.RequestZonesScenario.total_tokens
Serialized request token count, summing every zone in insertion order.
Equation: sum(self.zones.values())