BatchDispatchScenario
Serving scenarios · Implementation: mlsysim/engine/scenario_definitions/serving.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
A dual-trigger batch: it may leave at timeout after its oldest arrival, then waits for a worker.
Notes
Equation: The calculation members below define formation_end, dispatch, backlog_wait, batch_time, oldest_latency, slack, timer_only_latency.
Assumptions: Arrivals are chronologically ordered; the timer expires after the oldest arrival and dispatch waits for the worker.
Excludes: An independently simulated queue, size-trigger timing, and inter-request scheduling.
References
Derivation: A dual-trigger batch: it may leave at timeout after its oldest arrival, then waits for a worker. Each member records the corresponding algebra or named physics function.
Examples
Evaluate the named configuration and display a modeled result.
>>> from mlsysim import Scenarios
>>> case = Scenarios.DualTriggerBatchFourRequests
>>> value = case.oldest_latency
>>> print(f"{value:.3g~}")
25.6 msAttributes
| Name | Description |
|---|---|
| arrivals | |
| backlog_wait | Worker-availability delay after the timer trigger, in ms. |
| batch_time | Service duration for the observed arrival-count batch from the service profile. |
| deadline | |
| dispatch | Actual dispatch time in ms, later of timer trigger and worker availability. |
| formation_end | Timer-trigger time in ms, oldest arrival plus timeout. |
| oldest_latency | Oldest request’s response latency in ms including formation, backlog, service, and response path. |
| response_path | |
| service | |
| slack | Deadline minus oldest-request latency in ms; negative means missed deadline. |
| timeout | |
| timer_only_latency | Timeout plus batch service in ms, deliberately excluding worker backlog and response path. |
| worker_free |
arrivals
Type: List[Quantity].
Declared default: required.
timeout
Type: Quantity.
Declared default: required.
worker_free
Type: Quantity.
Declared default: required.
response_path
Type: Quantity.
Declared default: required.
deadline
Type: Quantity.
Declared default: required.
service
Type: BatchingKneeScenario.
Declared default: required.
formation_end
engine.scenario_types.BatchDispatchScenario.formation_end
Timer-trigger time in ms, oldest arrival plus timeout.
Equation: (arrivals[0] + timeout).to('ms').
dispatch
engine.scenario_types.BatchDispatchScenario.dispatch
Actual dispatch time in ms, later of timer trigger and worker availability.
Equation: max(formation_end, worker_free.to('ms')).
backlog_wait
engine.scenario_types.BatchDispatchScenario.backlog_wait
Worker-availability delay after the timer trigger, in ms.
Equation: (dispatch - formation_end).to('ms').
batch_time
engine.scenario_types.BatchDispatchScenario.batch_time
Service duration for the observed arrival-count batch from the service profile.
Equation: service.service_time(len(arrivals)).
oldest_latency
engine.scenario_types.BatchDispatchScenario.oldest_latency
Oldest request’s response latency in ms including formation, backlog, service, and response path.
Equation: (dispatch + batch_time + response_path - arrivals[0]).to('ms').
slack
engine.scenario_types.BatchDispatchScenario.slack
Deadline minus oldest-request latency in ms; negative means missed deadline.
Equation: (deadline - oldest_latency).to('ms').
timer_only_latency
engine.scenario_types.BatchDispatchScenario.timer_only_latency
Timeout plus batch service in ms, deliberately excluding worker backlog and response path.
The answer that ignores the busy worker and the response path.
Equation: (timeout + batch_time).to('ms').