WeightedRoutingScenario
Serving scenarios · Implementation: mlsysim/engine/scenario_definitions/serving.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
Traffic split over server pools in proportion to each pool’s per-server capacity.
Notes
Equation: Fleet capacity=sum(servers*per_server_capacity); per-server route weight=capacity/fleet_capacity.
Assumptions: Offered requests split proportionally to constant per-server capacities.
Excludes: Queueing, routing overhead, and heterogeneous request costs.
References
Derivation: A pool contributes server count times per-server capacity. Dividing one server’s capacity by the fleet sum assigns a per-server routing probability; offered rate times that probability gives its mean load.
Examples
The named registry scenario supplies the stated inputs; this is a modeled calculation, not a new measurement.
>>> from mlsysim import Scenarios
>>> scenario = Scenarios.HeterogeneousH100A100Routing
>>> round(scenario.total_capacity, 6)
22000.0Attributes
| Name | Description |
|---|---|
| capacity | |
| equal_split_load | Requests/s sent to each server under a uniform split, independent of its capacity. |
| offered | |
| servers | |
| total_capacity | Fleet request capacity in requests/s, summing each pool’s server count times per-server capacity. |
Methods
| Name | Description |
|---|---|
| equal_split_utilization | Dimensionless utilization of a server in pool under equal traffic per server. |
| utilization | Dimensionless per-server load under capacity-weighted routing: offered rate times per-server weight divided by that server’s capacity. |
| weight | Dimensionless traffic fraction for one server in pool: its capacity divided by fleet capacity; multiply by pool size for a pool-wide share. |
equal_split_utilization
engine.scenario_types.WeightedRoutingScenario.equal_split_utilization(pool)Dimensionless utilization of a server in pool under equal traffic per server.
Equation: self.equal_split_load / self.capacity[pool]
utilization
engine.scenario_types.WeightedRoutingScenario.utilization(pool)Dimensionless per-server load under capacity-weighted routing: offered rate times per-server weight divided by that server’s capacity.
Equation: self.offered * self.weight(pool) / self.capacity[pool]
weight
engine.scenario_types.WeightedRoutingScenario.weight(pool)Dimensionless traffic fraction for one server in pool: its capacity divided by fleet capacity; multiply by pool size for a pool-wide share.
Equation: self.capacity[pool] / self.total_capacity
servers
Type: Dict[str, int].
Declared default: required.
capacity
Type: Dict[str, float].
Declared default: required.
offered
Type: float.
Declared default: required.
total_capacity
engine.scenario_types.WeightedRoutingScenario.total_capacity
Fleet request capacity in requests/s, summing each pool’s server count times per-server capacity.
Equation: sum((self.servers[k] * self.capacity[k] for k in self.servers))
equal_split_load
engine.scenario_types.WeightedRoutingScenario.equal_split_load
Requests/s sent to each server under a uniform split, independent of its capacity.
Equation: self.offered / sum(self.servers.values())