MeanShiftTestScenario
Statistics scenarios · Implementation: mlsysim/engine/scenario_definitions/statistics.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
A feature mean that moves from reference_mean to observed_mean over n_samples requests.
The test assumes a known reference mean, independent requests, and the stated spread feature_sd: z is the move in standard errors of the mean.
Notes
Equation: The standardized mean shift is the mean difference divided by the standard error.
Assumptions: Independent samples, known reference mean, and supplied positive feature standard deviation.
Excludes: Uncertainty in the reference mean and multiple-testing correction.
References
Derivation: The standardized mean shift is the mean difference divided by the standard error. Individual property contracts identify the exact accounting expression or reusable calculation. Registry entries supply the scenario inputs; this class does not turn them into measured outcomes.
Examples
Evaluate a named scenario using its declared inputs.
>>> from mlsysim import Scenarios
>>> scenario = Scenarios.FeatureMeanShiftTest
>>> round(float(scenario.mean_shift), 4)
0.05Attributes
| Name | Description |
|---|---|
| feature_sd | |
| mean_shift | Observed-minus-reference feature mean, in the feature’s numerical units. |
| n_samples | |
| observed_mean | |
| p_value | Two-sided standard-normal tail probability at the absolute observed z-score. |
| reference_mean | |
| significance_bound | |
| standard_error | Mean’s standard error in feature units, using independent samples and the supplied |
| z_score | Dimensionless mean shift divided by its standard error, treating the reference mean as |
reference_mean
Type: float.
Declared default: required.
observed_mean
Type: float.
Declared default: required.
feature_sd
Type: float.
Declared default: required.
n_samples
Type: int.
Declared default: required.
significance_bound
Type: Optional[float].
Declared default: None.
mean_shift
engine.scenario_types.MeanShiftTestScenario.mean_shift
Observed-minus-reference feature mean, in the feature’s numerical units.
Equation: observed_mean - reference_mean.
standard_error
engine.scenario_types.MeanShiftTestScenario.standard_error
Mean’s standard error in feature units, using independent samples and the supplied feature spread.
Equation: calc_standard_error(feature_sd, n_samples).
z_score
engine.scenario_types.MeanShiftTestScenario.z_score
Dimensionless mean shift divided by its standard error, treating the reference mean as known.
Equation: mean_shift / standard_error.
p_value
engine.scenario_types.MeanShiftTestScenario.p_value
Two-sided standard-normal tail probability at the absolute observed z-score.
Two-sided normal-tail probability of a move at least this large from noise alone.
Equation: 2 * (1 - NormalDist().cdf(abs(z_score))).