EmbeddingTableScenario
Serving scenarios · Implementation: mlsysim/engine/scenario_definitions/serving.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
tables embedding tables of rows rows, each width values wide, against hardware.
Notes
Equation: Row bytes are width*value_width; total storage multiplies by rows and table count.
Assumptions: Fully allocated equal-shaped tables and labeled memory capacity.
Excludes: Index metadata, replicas, allocator reserve, and cached or compressed tables.
References
Derivation: A dense embedding table contains rows*width values. Multiplying by bytes_per_value and table count gives allocated storage. Comparing that storage with labeled capacity tests only table residency, not the rest of the serving process.
Examples
>>> from mlsysim import Scenarios
>>> s = Scenarios.HundredMillionItemTableOnA100
>>> s.total_bytes == s.table_bytes*s.tables
TrueAttributes
| Name | Description |
|---|---|
| bytes_per_value | |
| fits | Whether table storage alone fits labeled device capacity; runtime reserve is excluded. |
| hardware | |
| labeled_share | All tables over the device’s labeled capacity. |
| row_bytes | Bytes in one embedding vector. |
| rows | |
| table_bytes | Decimal GB in one fully allocated embedding table. |
| tables | |
| total_bytes | Decimal GB across all identically shaped tables. |
| width |
rows
Type: int.
Declared default: required.
width
Type: int.
Declared default: required.
bytes_per_value
Type: Quantity.
Declared default: required.
hardware
Type: HardwareNode.
Declared default: required.
tables
Type: int.
Declared default: 1.
row_bytes
engine.scenario_types.EmbeddingTableScenario.row_bytes
Bytes in one embedding vector.
Equation: (width * bytes_per_value).to('byte').
table_bytes
engine.scenario_types.EmbeddingTableScenario.table_bytes
Decimal GB in one fully allocated embedding table.
Equation: (rows * row_bytes).to('GB').
total_bytes
engine.scenario_types.EmbeddingTableScenario.total_bytes
Decimal GB across all identically shaped tables.
Equation: (table_bytes * tables).to('GB').
fits
engine.scenario_types.EmbeddingTableScenario.fits
Whether table storage alone fits labeled device capacity; runtime reserve is excluded.
Equation: labeled_share <= 1.