LowRankUpdateScenario
Training scenarios · Implementation: mlsysim/engine/scenario_definitions/training.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
A rank-rank update to one out_dim by in_dim weight matrix.
Notes
Equation: Dense parameters=in_dim*out_dim; rank-r factors contain rank*(in_dim+out_dim) parameters.
Assumptions: Bias-free matrix and positive rank/dimensions; byte properties require bytes_per_param.
Excludes: Accuracy effects, optimizer state, and the frozen backbone when interpreting update_bytes.
References
Implementation references: mlsysim.physics.calc_low_rank_update_params. These helper contracts state the source assumptions; the scenario binds their inputs and derives the displayed ratios and budgets.
Examples
The named registry scenario supplies the stated inputs; this is a modeled calculation, not a new measurement.
>>> from mlsysim import Scenarios
>>> scenario = Scenarios.TransformerLayerRank4Update
>>> scenario.update_params
6144Attributes
| Name | Description |
|---|---|
| bytes_per_param | |
| compression | Dense-to-factor parameter-count ratio; applies to replacing the matrix, not to storing LoRA alongside its backbone. |
| full_bytes | Dense matrix storage in MB at the optional declared bytes-per-parameter width. |
| full_params | Dense matrix parameter count: output width times input width, excluding bias. |
| in_dim | |
| out_dim | |
| rank | |
| reduction | Fraction of trainable dense parameters removed by the low-rank update; can be negative for excessive rank. |
| update_bytes | Two-factor storage in MB at that same width; excludes the frozen backbone and optimizer state. |
| update_params | Parameter count in two rank-r factors via calc_low_rank_update_params: rank * (in_dim + out_dim). |
in_dim
Type: int.
Declared default: required.
out_dim
Type: int.
Declared default: required.
rank
Type: int.
Declared default: required.
full_params
engine.scenario_types.LowRankUpdateScenario.full_params
Dense matrix parameter count: output width times input width, excluding bias.
Equation: self.in_dim * self.out_dim
update_params
engine.scenario_types.LowRankUpdateScenario.update_params
Parameter count in two rank-r factors via calc_low_rank_update_params: rank * (in_dim + out_dim).
Equation: The calculation follows the stated accounting rule; the method body shows each intermediate term.
reduction
engine.scenario_types.LowRankUpdateScenario.reduction
Fraction of trainable dense parameters removed by the low-rank update; can be negative for excessive rank.
Equation: 1 - self.update_params / self.full_params
bytes_per_param
Type: Optional[Quantity].
Declared default: None.
compression
engine.scenario_types.LowRankUpdateScenario.compression
Dense-to-factor parameter-count ratio; applies to replacing the matrix, not to storing LoRA alongside its backbone.
Equation: self.full_params / self.update_params
full_bytes
engine.scenario_types.LowRankUpdateScenario.full_bytes
Dense matrix storage in MB at the optional declared bytes-per-parameter width.
Equation: (self.full_params * self.bytes_per_param).to(‘MB’)
update_bytes
engine.scenario_types.LowRankUpdateScenario.update_bytes
Two-factor storage in MB at that same width; excludes the frozen backbone and optimizer state.
Equation: (self.update_params * self.bytes_per_param).to(‘MB’)