LittlesLawScenario
Serving scenarios · Implementation: mlsysim/engine/scenario_definitions/serving.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
Requests in flight at arrival_rate, and replicas sized from a batch service time.
Notes
Equation: N=lambda*T; replica capacity=B/S; provisioned replicas=ceil(lambda/(u*capacity)).
Assumptions: Stationary mean rates and identical replica service times; replica properties require all optional sizing fields.
Excludes: Burst distributions and tail-latency guarantees.
References
Implementation references: mlsysim.physics.serving.calc_littles_law_occupancy. These helper contracts state the source assumptions; the scenario binds their inputs and derives the displayed ratios and budgets.
Examples
The named registry scenario supplies the stated inputs; this is a modeled calculation, not a new measurement.
>>> from mlsysim import Scenarios
>>> scenario = Scenarios.VisionServingLittlesLaw
>>> round(scenario.in_flight, 6)
100.0Attributes
| Name | Description |
|---|---|
| arrival_rate | |
| batch_service_time | |
| batch_size | |
| executing_positions | Total batch slots in the provisioned replicas: replicas_needed * batch_size; a count, not mean occupancy. |
| in_flight | Mean outstanding request count from Little’s law, arrival_rate * time_in_system; delegates to calc_littles_law_occupancy. |
| mean_executing | Mean busy batch positions under the fixed service-time approximation: arrival_rate * batch_service_time, dimensionless. |
| replica_throughput | Saturated capacity of one replica in 1/s: batch_size / batch_service_time; requires both optional fields. |
| replicas_needed | Integer replica count allowing the declared utilization headroom; ceiling of demand divided by target-utilization capacity. |
| service_utilization | Offered load divided by provisioned saturated capacity; dimensionless and distinct from the input utilization target. |
| throughput_floor | Integer replica count at full utilization, rounding arrival_rate / replica_throughput upward. |
| time_in_system | |
| utilization_target |
arrival_rate
Type: Quantity.
Declared default: required.
time_in_system
Type: Quantity.
Declared default: required.
batch_size
Type: Optional[int].
Declared default: None.
batch_service_time
Type: Optional[Quantity].
Declared default: None.
utilization_target
Type: Optional[float].
Declared default: None.
in_flight
engine.scenario_types.LittlesLawScenario.in_flight
Mean outstanding request count from Little’s law, arrival_rate * time_in_system; delegates to calc_littles_law_occupancy.
Equation: The calculation follows the stated accounting rule; the method body shows each intermediate term.
replica_throughput
engine.scenario_types.LittlesLawScenario.replica_throughput
Saturated capacity of one replica in 1/s: batch_size / batch_service_time; requires both optional fields.
Equation: (self.batch_size / self.batch_service_time).to(‘1/second’)
throughput_floor
engine.scenario_types.LittlesLawScenario.throughput_floor
Integer replica count at full utilization, rounding arrival_rate / replica_throughput upward.
Equation: _math.ceil((self.arrival_rate / self.replica_throughput).to(’’).magnitude)
replicas_needed
engine.scenario_types.LittlesLawScenario.replicas_needed
Integer replica count allowing the declared utilization headroom; ceiling of demand divided by target-utilization capacity.
Equation: _math.ceil((self.arrival_rate / (self.utilization_target * self.replica_throughput)).to(’’).magnitude)
executing_positions
engine.scenario_types.LittlesLawScenario.executing_positions
Total batch slots in the provisioned replicas: replicas_needed * batch_size; a count, not mean occupancy.
Equation: self.replicas_needed * self.batch_size
mean_executing
engine.scenario_types.LittlesLawScenario.mean_executing
Mean busy batch positions under the fixed service-time approximation: arrival_rate * batch_service_time, dimensionless.
Equation: (self.arrival_rate * self.batch_service_time).to(’’).magnitude
service_utilization
engine.scenario_types.LittlesLawScenario.service_utilization
Offered load divided by provisioned saturated capacity; dimensionless and distinct from the input utilization target.
Equation: (self.arrival_rate / (self.replicas_needed * self.replica_throughput)).to(’’).magnitude