GradientChainScenario
Training scenarios · Implementation: mlsysim/engine/scenario_definitions/training.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
The activation-slope factor a gradient keeps after depth layers, at each slope’s peak.
Notes
Equation: The activation-only gradient factor is max_sigmoid_slope**depth.
Assumptions: Every layer is evaluated at the sigmoid derivative maximum; weight Jacobians are treated separately.
Excludes: Weight-matrix amplification, residual paths, and actual training-gradient distributions.
References
Derivation: The chain rule multiplies activation derivatives across layers. The maximum sigmoid derivative is supplied by mlsysim.physics.neural.calc_sigmoid_slope at zero; raising it to depth isolates the activation-only factor without assigning values to the weight Jacobians.
Examples
>>> from mlsysim import Scenarios
>>> s = Scenarios.TwentySigmoidLayers
>>> s.variant(depth=s.depth+1).surviving_fraction <= s.surviving_fraction
TrueAttributes
| Name | Description |
|---|---|
| activation | |
| depth | |
| peak_slope | Maximum sigmoid derivative, evaluated at zero; other activation names are rejected. |
| surviving_fraction | Dimensionless product of the peak sigmoid slopes over the configured depth. |
depth
Type: int.
Declared default: required.
activation
Type: str.
Declared default: 'sigmoid'.
peak_slope
engine.scenario_types.GradientChainScenario.peak_slope
Maximum sigmoid derivative, evaluated at zero; other activation names are rejected.
Equation: calc_sigmoid_slope(0.0).
surviving_fraction
engine.scenario_types.GradientChainScenario.surviving_fraction
Dimensionless product of the peak sigmoid slopes over the configured depth.
Equation: peak_slope ** depth.