Skip to main content

RingAttentionScenario

Communication scenarios · Implementation: mlsysim/engine/scenario_definitions/communication.py.

Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.

Ring attention on accelerator: the smallest chunk that hides each K/V tile, per link.

Notes

Equation: Compose mlsysim.physics.communication.calc_ring_attention_min_chunk.

Assumptions: The selected arithmetic peak is multiplied by a supplied local efficiency, and overlap follows the ring-attention helper’s tile model.

Excludes: Kernel scheduling, nonideal overlap, and link contention are absent.

References

Derivation: this scenario binds its input fields to the referenced implementation contracts: mlsysim.physics.communication.calc_ring_attention_min_chunk. The methods below identify any additional arithmetic.

Examples

>>> from mlsysim import Scenarios
>>> case = Scenarios.H100RingAttentionChunk
>>> round(next(iter(case.min_chunk_tokens.values())), 4)
1098.8889

Attributes

Name Description
accelerator
element_bytes
links
local_efficiency
min_chunk_tokens Smallest chunk whose compute can hide K/V tile transfer on each link. Equation:
precision

accelerator

Type: HardwareNode.

Declared default: required.

precision

Type: str.

Declared default: 'fp16_dense'.

local_efficiency

Type: float.

Declared default: required.

element_bytes

Type: Quantity.

Declared default: required.

min_chunk_tokens

engine.scenario_types.RingAttentionScenario.min_chunk_tokens

Smallest chunk whose compute can hide K/V tile transfer on each link. Equation: mlsysim.physics.communication.calc_ring_attention_min_chunk with effective peak rate, element bytes, and link bandwidth. Units: tokens per link; the helper’s overlap assumptions apply.

Back to top