SequentialTestScenario
Statistics scenarios · Implementation: mlsysim/engine/scenario_definitions/statistics.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
Wald’s SPRT between an unacceptable failure rate p_unacceptable and an acceptable one p_acceptable at error rates alpha and beta; the score after trace_clean_runs clean runs and one failure; and the fixed zero-failure test for success 1 - p_unacceptable at the same alpha.
Notes
Equation: Success and failure log-likelihood increments accumulate until a Wald boundary is crossed.
Assumptions: Independent Bernoulli outcomes under the two fixed hypotheses and the specified SPRT error budgets.
Excludes: Changing hypotheses, dependent outcomes, and a guaranteed stopping-time distribution.
References
Derivation: Success and failure log-likelihood increments accumulate until a Wald boundary is crossed. Individual property contracts identify the exact accounting expression or reusable calculation. Registry entries supply the scenario inputs; this class does not turn them into measured outcomes.
Examples
Evaluate a named scenario using its declared inputs.
>>> from mlsysim import Scenarios
>>> scenario = Scenarios.LatchSequentialTest
>>> round(float(scenario.success_increment), 4)
0.0953Attributes
| Name | Description |
|---|---|
| accept | Upper log-likelihood boundary for accepting the lower-failure hypothesis. |
| alpha | |
| beta | |
| failure_increment | Log-likelihood increment after a failed trial under the same two Bernoulli hypotheses. |
| false_accept_bound | Nominal exponential false-accept bound implied by the upper boundary. |
| false_reject_bound | Nominal exponential false-reject bound implied by the lower boundary. |
| fixed_rejects_acceptable | Probability the fixed zero-failure rule sees at least one failure under the acceptable |
| fixed_runs | Zero-failure fixed-sample trial count for the unacceptable-rate ceiling at confidence |
| p_acceptable | |
| p_unacceptable | |
| reject | Lower log-likelihood boundary for rejecting the lower-failure hypothesis. |
| runs_to_accept | Ceiling number of consecutive successful trials needed to reach the acceptance boundary |
| success_increment | Log-likelihood increment after a successful trial, from calc_sprt_increments. |
| trace_clean_runs | |
| trace_score | Log-likelihood score of the illustrative trace: declared successful trials plus one |
p_unacceptable
Type: float.
Declared default: required.
p_acceptable
Type: float.
Declared default: required.
alpha
Type: float.
Declared default: required.
beta
Type: float.
Declared default: required.
trace_clean_runs
Type: int.
Declared default: required.
success_increment
engine.scenario_types.SequentialTestScenario.success_increment
Log-likelihood increment after a successful trial, from calc_sprt_increments.
Equation: calc_sprt_increments(p_unacceptable, p_acceptable)[0].
failure_increment
engine.scenario_types.SequentialTestScenario.failure_increment
Log-likelihood increment after a failed trial under the same two Bernoulli hypotheses.
Equation: calc_sprt_increments(p_unacceptable, p_acceptable)[1].
accept
engine.scenario_types.SequentialTestScenario.accept
Upper log-likelihood boundary for accepting the lower-failure hypothesis.
Equation: calc_sprt_boundaries(alpha, beta)[0].
reject
engine.scenario_types.SequentialTestScenario.reject
Lower log-likelihood boundary for rejecting the lower-failure hypothesis.
Equation: calc_sprt_boundaries(alpha, beta)[1].
runs_to_accept
engine.scenario_types.SequentialTestScenario.runs_to_accept
Ceiling number of consecutive successful trials needed to reach the acceptance boundary from zero.
Equation: _math.ceil(accept / success_increment).
false_accept_bound
engine.scenario_types.SequentialTestScenario.false_accept_bound
Nominal exponential false-accept bound implied by the upper boundary.
Ville’s inequality: e^-A = alpha / (1 - beta).
Equation: _math.exp(-accept).
false_reject_bound
engine.scenario_types.SequentialTestScenario.false_reject_bound
Nominal exponential false-reject bound implied by the lower boundary.
Equation: _math.exp(reject).
trace_score
engine.scenario_types.SequentialTestScenario.trace_score
Log-likelihood score of the illustrative trace: declared successful trials plus one failure.
Equation: trace_clean_runs * success_increment + failure_increment.
fixed_runs
engine.scenario_types.SequentialTestScenario.fixed_runs
Zero-failure fixed-sample trial count for the unacceptable-rate ceiling at confidence one minus alpha.
Equation: calc_zero_failure_sample_size(1.0 - p_unacceptable, 1.0 - alpha).
fixed_rejects_acceptable
engine.scenario_types.SequentialTestScenario.fixed_rejects_acceptable
Probability the fixed zero-failure rule sees at least one failure under the acceptable rate.
Chance that a policy at the acceptable rate fails the fixed zero-failure test.
Equation: 1.0 - (1.0 - p_acceptable) ** fixed_runs.