LayerStackScenario
Training scenarios · Implementation: mlsysim/engine/scenario_definitions/training.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
Weight-only count of a network built from convolution and dense layers, input first (biases omitted).
A multilayer perceptron has no convolution layers; a small CNN ends in a dense classifier head. (The full forward and training costs of a dense network belong to the neural-computation solvers; this counts weights.)
Notes
Equation: Total weights are the sum of convolution and dense matrix counts; the dense share divides its subtotal by the total.
Assumptions: A nonempty dense-layer sequence with compatible widths and positive total weight count.
Excludes: Biases, activation state, and training work; use neural solvers for those costs.
References
Derivation: Convolution filters and dense matrices are disjoint parameter sets, so their counts add. Dividing the dense subtotal by the combined count expresses its share. Biases are omitted consistently from both numerator and denominator.
Examples
>>> from mlsysim import Scenarios
>>> s = Scenarios.MnistMlpLayerStack
>>> s.weights == s.conv_weights + s.dense_weights
TrueAttributes
| Name | Description |
|---|---|
| conv_layers | |
| conv_weights | Sum of convolution filter counts; dense weights and biases are excluded. |
| dense_layers | |
| dense_share | Share of the weights in the dense layers (a CNN’s classifier head). |
| dense_weights | Sum of dense matrix counts; convolution filters and biases are excluded. |
| macs_per_example_dense | Multiply-adds of the dense layers per example (one per weight). |
| weights | Total convolution and dense weight count, with biases omitted. |
| widths | Dense widths, input first. |
conv_layers
Type: List[ConvLayerScenario].
Declared default: [].
dense_layers
Type: List[DenseLayerScenario].
Declared default: required.
widths
engine.scenario_types.LayerStackScenario.widths
Dense widths, input first.
Equation: [dense_layers[0].in_features] + [d.out_features for d in dense_layers].
conv_weights
engine.scenario_types.LayerStackScenario.conv_weights
Sum of convolution filter counts; dense weights and biases are excluded.
Equation: sum((c.weights for c in conv_layers)).
dense_weights
engine.scenario_types.LayerStackScenario.dense_weights
Sum of dense matrix counts; convolution filters and biases are excluded.
Equation: sum((d.weights for d in dense_layers)).
weights
engine.scenario_types.LayerStackScenario.weights
Total convolution and dense weight count, with biases omitted.
Equation: conv_weights + dense_weights.
macs_per_example_dense
engine.scenario_types.LayerStackScenario.macs_per_example_dense
Multiply-adds of the dense layers per example (one per weight).
Equation: dense_weights.