BatchingKneeScenario
Serving scenarios · Implementation: mlsysim/engine/scenario_definitions/serving.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
Batch service time S(B) = t_0 + B t_req: where throughput turns over.
Notes
Equation: S(B)=fixed_time+B*per_request_time; throughput=B/S(B).
Assumptions: Positive affine service costs and back-to-back full batches.
Excludes: Queueing, batch formation, and batch-dependent efficiency changes.
References
Implementation references: mlsysim.physics.serving.calc_batch_knee, mlsysim.physics.serving.calc_batch_throughput_share. These helper contracts state the source assumptions; the scenario binds their inputs and derives the displayed ratios and budgets.
Examples
The named registry scenario supplies the stated inputs; this is a modeled calculation, not a new measurement.
>>> from mlsysim import Scenarios
>>> scenario = Scenarios.VisionBatchingKnee
>>> round(scenario.knee, 6)
2.0Attributes
| Name | Description |
|---|---|
| batch_sizes | |
| fixed_time | |
| knee | Batch size where fixed and request-dependent service costs are equal; dimensionless fixed_time / per_request_time via calc_batch_knee. |
| per_request_time |
Methods
| Name | Description |
|---|---|
| fixed_share | Dimensionless fixed-overhead share: fixed_time / service_time(batch_size). |
| service_time | Time for one batch in ms: fixed overhead plus batch_size * per_request_time. |
| throughput | Saturated request rate in 1/s: batch_size / service_time(batch_size); excludes queueing and batch formation. |
| throughput_share | Fraction of asymptotic throughput at positive batch_size; B / (B + knee) via calc_batch_throughput_share. |
| time_per_request | Amortized service cost in ms per request: batch service time divided by positive batch size. |
service_time
engine.scenario_types.BatchingKneeScenario.service_time(batch_size)Time for one batch in ms: fixed overhead plus batch_size * per_request_time.
Equation: (self.fixed_time + batch_size * self.per_request_time).to(‘ms’)
throughput
engine.scenario_types.BatchingKneeScenario.throughput(batch_size)Saturated request rate in 1/s: batch_size / service_time(batch_size); excludes queueing and batch formation.
Equation: (batch_size / self.service_time(batch_size)).to(‘1/second’)
time_per_request
engine.scenario_types.BatchingKneeScenario.time_per_request(batch_size)Amortized service cost in ms per request: batch service time divided by positive batch size.
Equation: (self.service_time(batch_size) / batch_size).to(‘ms’)
fixed_time
Type: Quantity.
Declared default: required.
per_request_time
Type: Quantity.
Declared default: required.
batch_sizes
Type: List[int].
Declared default: [].
knee
engine.scenario_types.BatchingKneeScenario.knee
Batch size where fixed and request-dependent service costs are equal; dimensionless fixed_time / per_request_time via calc_batch_knee.
Equation: The calculation follows the stated accounting rule; the method body shows each intermediate term.