ConfidenceRegimesScenario
Statistics scenarios · Implementation: mlsysim/engine/scenario_definitions/statistics.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
Stipulated proposer confidence against task success in several operating regimes.
Notes
Equation: confidence gap = supplied confidence - supplied success for the named regime.
Assumptions: Each tuple contains a regime name and compatible probability-scale values.
Excludes: Estimating calibration from samples or interpreting confidence as a safety guarantee.
References
Derivation: confidence gap = supplied confidence - supplied success for the named regime. The individual properties state units and intermediate accounting; this class introduces no fitted coefficient.
Examples
The named scenario supplies illustrative inputs; the output is calculated.
>>> from mlsysim import Scenarios
>>> case = Scenarios.ProposerConfidenceRegimes
>>> round(float(case.gap('In distribution')), 3)
0.02Attributes
| Name | Description |
|---|---|
| regimes |
Methods
| Name | Description |
|---|---|
| gap | Supplied confidence minus task-success fraction for the named regime; raises |
| row | The stored (regime, confidence, success) tuple; this accessor performs no |
gap
engine.scenario_types.ConfidenceRegimesScenario.gap(regime)Supplied confidence minus task-success fraction for the named regime; raises KeyError if absent.
Equation: Follow the gap calculation below; branch conditions and rounding are part of its definition.
row
engine.scenario_types.ConfidenceRegimesScenario.row(regime)The stored (regime, confidence, success) tuple; this accessor performs no calibration.
Equation: Follow the row calculation below; branch conditions and rounding are part of its definition.
regimes
Type: List[Any].
Declared default: required.