CompilationPaybackScenario

Serving scenarios · Implementation: mlsysim/engine/scenario_definitions/serving.py.

Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.

A compiled decode step and the executions that repay its build.

The eager step splits into shares (matrix multiplies, element-wise kernels, launch overhead); compilation removes removed[k] of each share. A build is repaid after build / saved_per_token tokens; shared by replicas it is repaid in a fraction of that time, and rebuilt by each it costs replicas x gpus_per_replica x build.

Notes

Equation: remaining step = eager step * sum(component share * retained fraction); repayment tokens = build time / time saved per token.

Assumptions: Component fractions describe the same eager workload, and build repayment requires a positive saving per token.

Excludes: Compiler-specific prediction, overlap between removed components, and build- cache invalidation.

References

Derivation: remaining step = eager step * sum(component share * retained fraction); repayment tokens = build time / time saved per token. The individual properties state units and intermediate accounting; this class introduces no fitted coefficient.

Examples

The named scenario supplies illustrative inputs; the output is calculated.

>>> from mlsysim import Scenarios
>>> case = Scenarios.Decode13BCompilationPayback
>>> round(float(case.step_fraction), 3)
0.635

Attributes

Name	Description
builds
compiled_rate	Tokens per second implied by the reciprocal compiled step time.
compiled_step	Compiled token-step duration in milliseconds under the component-removal
gpus_per_replica
removed
replica_life
replicas
saved_per_token	Milliseconds saved per token relative to the eager step.
shares
speedup	Compiled token rate divided by the supplied eager rate; these inputs must describe
step_fraction	Fraction of eager-step time remaining after removing each declared component’s
step_time
token_rate

Methods

Name	Description
build_share	Selected build duration divided by replica lifetime, a dimensionless startup
rollout_cost	Total build device-hours when each replica builds independently on its allocated
shared_repay	Per-replica repayment seconds when the pool shares a single build cost.
time_to_repay	Minutes of compiled generation needed to produce the repayment token count.
tokens_to_repay	Token count whose cumulative saved time equals the selected build duration.

build_share

engine.scenario_types.CompilationPaybackScenario.build_share(build)

Selected build duration divided by replica lifetime, a dimensionless startup fraction.

Equation: return (builds[build] / replica_life).to('').magnitude.

rollout_cost

engine.scenario_types.CompilationPaybackScenario.rollout_cost(build)

Total build device-hours when each replica builds independently on its allocated devices.

Equation: return (builds[build] * gpus_per_replica * replicas).to('hour').

shared_repay

engine.scenario_types.CompilationPaybackScenario.shared_repay(build)

Per-replica repayment seconds when the pool shares a single build cost.

Equation: return (time_to_repay(build) / replicas).to('second').

time_to_repay

engine.scenario_types.CompilationPaybackScenario.time_to_repay(build)

Minutes of compiled generation needed to produce the repayment token count.

Equation: return (tokens_to_repay(build) / compiled_rate).to('minute').

tokens_to_repay

engine.scenario_types.CompilationPaybackScenario.tokens_to_repay(build)

Token count whose cumulative saved time equals the selected build duration.

Equation: return (builds[build] / saved_per_token).to('').magnitude.