CreditWindowScenario
Agents scenarios · Implementation: mlsysim/engine/scenario_definitions/agents.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
A producer feeding a slower model consumer, unbounded or behind a credit window.
The consumer serves one item per tokens_per_item at its decode rate. With no bound the queue grows at the rate difference; a saturated window of W credits holds each item about W / mu by Little’s law.
Notes
Equation: Service rate is decode_rate/tokens_per_item; fluid backlog is (producer_rate-service_rate)*horizon; saturated window residence is credits/service_rate.
Assumptions: Rates and per-item work remain constant, the initial backlog is zero, and the growth calculation describes overload.
Excludes: Bursts, integer event scheduling, and a nonnegative queue clamp; underload produces a negative fluid backlog.
References
Derivation: Service rate is decode_rate/tokens_per_item; fluid backlog is (producer_rate-service_rate)*horizon; saturated window residence is credits/service_rate. This is declared scenario accounting rather than a fitted empirical law.
Examples
>>> from mlsysim import Scenarios
>>> scenario = Scenarios.DiffReviewCreditWindow
>>> value = scenario.service_rate
>>> str(value.units)
'1/s'Attributes
| Name | Description |
|---|---|
| backlog | Fluid backlog count accumulated from an initially empty queue over horizon; not |
| backlog_wait | Seconds needed to drain the fluid backlog at the fixed service rate. |
| credits | |
| decode_rate | |
| growth_rate | Net queue growth in items per second; negative values indicate producer demand below |
| horizon | |
| producer_rate | |
| residence_target | |
| service_rate | Consumer completions per second, dividing decode tokens per second by tokens needed per |
| tokens_per_item | |
| window_for_target | Largest integer credit count within the requested mean residence time. |
| window_residence | Saturated mean item residence in seconds from credit count divided by service rate. |
producer_rate
Type: Quantity.
Declared default: required.
decode_rate
Type: Quantity.
Declared default: required.
tokens_per_item
Type: Quantity.
Declared default: required.
horizon
Type: Quantity.
Declared default: required.
credits
Type: int.
Declared default: required.
residence_target
Type: Quantity.
Declared default: required.
service_rate
engine.scenario_types.CreditWindowScenario.service_rate
Consumer completions per second, dividing decode tokens per second by tokens needed per item.
Equation: (decode_rate / tokens_per_item).to('1/s').
growth_rate
engine.scenario_types.CreditWindowScenario.growth_rate
Net queue growth in items per second; negative values indicate producer demand below service capacity.
Equation: (producer_rate - service_rate).to('1/s').
backlog
engine.scenario_types.CreditWindowScenario.backlog
Fluid backlog count accumulated from an initially empty queue over horizon; not clamped at zero.
Equation: (growth_rate * horizon).to('').magnitude.
backlog_wait
engine.scenario_types.CreditWindowScenario.backlog_wait
Seconds needed to drain the fluid backlog at the fixed service rate.
Equation: (backlog / service_rate).to('s').
window_residence
engine.scenario_types.CreditWindowScenario.window_residence
Saturated mean item residence in seconds from credit count divided by service rate.
Equation: (credits / service_rate).to('s').
window_for_target
engine.scenario_types.CreditWindowScenario.window_for_target
Largest integer credit count within the requested mean residence time.
Equation: _math.floor((residence_target * service_rate).to('').magnitude + 1e-09).