TokenEnergyScenario

Sustainability scenarios · Implementation: mlsysim/engine/scenario_definitions/sustainability.py.

Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.

Joules per generated token: node power over aggregate token rate, well used and poorly used.

The node draws the accelerator’s power plus a host share whether a step carries one sequence or many. concurrent_requests at per_request_rate set the well-used token rate; at low_use_fraction of that rate the accelerator still draws low_use_accelerator_power.

Notes

Equation: Per-token energy is attributed node watts divided by aggregate tokens per second, then multiplied by response token count for response joules.

Assumptions: Power and throughput are steady within each declared utilization regime; host power is attributed equally to both regimes.

Excludes: Idle transitions, a power trace, other tenants, and prediction of low-use power from throughput.

References

Derivation: Per-token energy is attributed node watts divided by aggregate tokens per second, then multiplied by response token count for response joules. This is declared scenario accounting rather than a fitted empirical law.

Examples

>>> from mlsysim import Scenarios
>>> scenario = Scenarios.H100ChatEnergyPerToken
>>> value = scenario.node_power
>>> str(value.units)
'W'

Attributes

Name	Description
accelerator_power
concurrent_requests
host_power
joules_per_token	Joules per token at the declared well-used aggregate rate.
low_use_accelerator_power
low_use_fraction
low_use_joules_per_token	Joules per token under the separate low-use power and throughput assumptions.
low_use_power	Low-use accelerator plus the same attributed host power, in watts.
low_use_ratio	Low-use per-token energy divided by well-used per-token energy, dimensionless.
low_use_token_rate	Aggregate tokens per second after applying the low-use throughput fraction.
node_power	Accelerator plus attributed host power in watts.
per_request_rate
response_energy	Joules allocated to one response at the well-used per-token energy.
response_tokens
token_rate	Aggregate token rate from concurrency times per-request token rate, in tokens per

Methods

Name	Description
energy_per_token	Joules per token from selected node power divided by positive aggregate `token_rate`.

energy_per_token

engine.scenario_types.TokenEnergyScenario.energy_per_token(
    token_rate,
    low_use=False,
)

Joules per token from selected node power divided by positive aggregate token_rate.

Equation: (power / token_rate).to('joule').