RecurrentStepScenario
Training scenarios · Implementation: mlsysim/engine/scenario_definitions/training.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
One step of a recurrent layer: input_dim inputs, a hidden_dim state, and gates affine maps per step (one for a plain RNN, four for an LSTM); steps steps in a sequence.
Notes
Equation: Each gate uses H*I input MACs and H*H recurrent MACs; all gates and steps share their matrices.
Assumptions: Dense gates and a retained hidden vector of H values at the declared width.
Excludes: Biases, nonlinear gate work, LSTM cell-state extras, and recomputation policies.
References
Derivation: Each recurrent gate applies a matrix to the input and another to the preceding hidden state, giving H*I + H*H MACs. Gates multiply that work; time steps reuse the same weights but retain separate states when full history is stored.
Examples
>>> from mlsysim import Scenarios
>>> s = Scenarios.ElmanRnnStep
>>> s.macs_per_step == s.input_macs + s.recurrent_macs
TrueAttributes
| Name | Description |
|---|---|
| bytes_per_value | |
| gate_rows | Rows of the fused input and recurrent weight matrices. |
| gates | |
| hidden_dim | |
| input_dim | |
| input_macs | Input-projection multiply-accumulate count across all gates for one step. |
| macs_per_step | Combined input and recurrent multiply-accumulate count for one time step. |
| recurrent_macs | Recurrent-state multiply-accumulate count across all gates for one step. |
| state_bytes | One state vector: what inference keeps at any sequence length. |
| steps | |
| training_states | States backpropagation through time keeps: one per step. |
| weights | Input and recurrent weights (biases omitted); one multiply-add each per step. |
input_dim
Type: int.
Declared default: required.
bytes_per_value
Type: Quantity.
Declared default: required.
gates
Type: int.
Declared default: 1.
steps
Type: int.
Declared default: 1.
gate_rows
engine.scenario_types.RecurrentStepScenario.gate_rows
Rows of the fused input and recurrent weight matrices.
Equation: gates * hidden_dim.
recurrent_macs
engine.scenario_types.RecurrentStepScenario.recurrent_macs
Recurrent-state multiply-accumulate count across all gates for one step.
Equation: gate_rows * hidden_dim.
input_macs
engine.scenario_types.RecurrentStepScenario.input_macs
Input-projection multiply-accumulate count across all gates for one step.
Equation: gate_rows * input_dim.
macs_per_step
engine.scenario_types.RecurrentStepScenario.macs_per_step
Combined input and recurrent multiply-accumulate count for one time step.
Equation: recurrent_macs + input_macs.
weights
engine.scenario_types.RecurrentStepScenario.weights
Input and recurrent weights (biases omitted); one multiply-add each per step.
Equation: macs_per_step.
state_bytes
engine.scenario_types.RecurrentStepScenario.state_bytes
One state vector: what inference keeps at any sequence length.
Equation: (hidden_dim * bytes_per_value).to('byte').
training_states
engine.scenario_types.RecurrentStepScenario.training_states
States backpropagation through time keeps: one per step.
Equation: steps.