EmbeddingGatherScenario
Serving scenarios · Implementation: mlsysim/engine/scenario_definitions/serving.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
One recommender request on hardware: one row of width values per categorical feature of dataset is gathered, then the rows and the bottom MLP’s output interact by pairwise dot products.
Notes
Equation: One embedding row is gathered per categorical feature; vectors*(vectors-1)/2 dot products model interactions.
Assumptions: One lookup per feature, equal embedding widths, and one bottom-MLP vector.
Excludes: Caching, random-access efficiency, MLP execution, and overlap between host and device transfers.
References
Derivation: The declared categorical-feature count determines gathered rows. Adding the bottom-MLP output gives the interacting vector count; choosing every unordered pair gives n*(n-1)/2 dot products. Dividing a bandwidth by gathered bytes/request yields an ideal request-rate ceiling.
Examples
>>> from mlsysim import Scenarios
>>> s = Scenarios.CriteoShapedRequestOnH100
>>> s.pairs == s.vectors*(s.vectors-1)//2
TrueAttributes
| Name | Description |
|---|---|
| bandwidth_ratio | Dimensionless ideal device-memory request rate divided by ideal host-link rate. |
| bytes_per_value | |
| dataset | |
| device_rate | Requests a second if every row streamed at device-memory bandwidth (an upper bound). |
| features | Categorical-feature count that determines rows gathered per request. |
| hardware | |
| host_rate | Requests a second if every row crossed the host link (an upper bound). |
| intensity | Interaction FLOPs (two per multiply-add) per gathered byte. |
| interaction_macs | Multiply-accumulate count for all pairwise dot products. |
| pairs | Unordered pair count among gathered embeddings and the bottom-MLP vector. |
| request_bytes | Bytes gathered across all categorical features in one request. |
| row_bytes | Bytes fetched for one embedding row. |
| vectors | The gathered rows and the bottom MLP’s output. |
| width |
dataset
Type: DatasetProfile.
Declared default: required.
width
Type: int.
Declared default: required.
bytes_per_value
Type: Quantity.
Declared default: required.
hardware
Type: HardwareNode.
Declared default: required.
features
engine.scenario_types.EmbeddingGatherScenario.features
Categorical-feature count that determines rows gathered per request.
Equation: dataset.categorical_features.
row_bytes
engine.scenario_types.EmbeddingGatherScenario.row_bytes
Bytes fetched for one embedding row.
Equation: (width * bytes_per_value).to('byte').
request_bytes
engine.scenario_types.EmbeddingGatherScenario.request_bytes
Bytes gathered across all categorical features in one request.
Equation: (features * row_bytes).to('byte').
vectors
engine.scenario_types.EmbeddingGatherScenario.vectors
The gathered rows and the bottom MLP’s output.
Equation: features + 1.
pairs
engine.scenario_types.EmbeddingGatherScenario.pairs
Unordered pair count among gathered embeddings and the bottom-MLP vector.
Equation: vectors * (vectors - 1) // 2.
interaction_macs
engine.scenario_types.EmbeddingGatherScenario.interaction_macs
Multiply-accumulate count for all pairwise dot products.
Equation: pairs * width.
intensity
engine.scenario_types.EmbeddingGatherScenario.intensity
Interaction FLOPs (two per multiply-add) per gathered byte.
Equation: (Q_(2 * interaction_macs, 'flop') / request_bytes).to('flop/byte').
device_rate
engine.scenario_types.EmbeddingGatherScenario.device_rate
Requests a second if every row streamed at device-memory bandwidth (an upper bound).
Equation: (hardware.memory.bandwidth / request_bytes).to('1/s').
host_rate
engine.scenario_types.EmbeddingGatherScenario.host_rate
Requests a second if every row crossed the host link (an upper bound).
Equation: (hardware.interconnect.bandwidth / request_bytes).to('1/s').
bandwidth_ratio
engine.scenario_types.EmbeddingGatherScenario.bandwidth_ratio
Dimensionless ideal device-memory request rate divided by ideal host-link rate.
Equation: (device_rate / host_rate).to('').magnitude.