AutomationBiasScenario

Statistics scenarios · Implementation: mlsysim/engine/scenario_definitions/statistics.py.

Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.

A human reviews an AI’s output; the human accepts miss_acceptance of the AI’s misses.

The best case assumes the reviewer catches every other miss and never overturns a correct detection, so team sensitivity is at most one minus the accepted misses.

Notes

Equation: Remaining misses equal AI miss rate times human acceptance of those misses; sensitivity is the complement.

Assumptions: Every rejected AI miss is corrected and the reviewer introduces no additional misses.

Excludes: Correlated human errors, false positives, and an empirically validated team model.

References

Derivation: Remaining misses equal AI miss rate times human acceptance of those misses; sensitivity is the complement. Individual property contracts identify the exact accounting expression or reusable calculation. Registry entries supply the scenario inputs; this class does not turn them into measured outcomes.

Examples

Evaluate a named scenario using its declared inputs.

>>> from mlsysim import Scenarios
>>> scenario = Scenarios.RadiologyAutomationBias
>>> round(float(scenario.ai_miss_rate), 4)
0.05

Attributes

Name	Description
acceptance_high
acceptance_low
ai_miss_rate	Probability an actual positive is missed by the AI, complementary to its sensitivity.
ai_sensitivity
gain_over_ai	Absolute sensitivity gain of that optimistic team bound over the AI alone.
human_sensitivity
miss_acceptance
team_sensitivity_bound	Optimistic team sensitivity when every unaccepted AI miss is corrected and no new misses

Methods

Name	Description
accepted_misses	Fraction of actual positives remaining missed when the human accepts the specified

accepted_misses

engine.scenario_types.AutomationBiasScenario.accepted_misses(acceptance=None)

Fraction of actual positives remaining missed when the human accepts the specified fraction of AI misses.

Equation: a = miss_acceptance if acceptance is None else acceptance; result = ai_miss_rate * a.