SubgroupEvaluationBudgetScenario
Statistics scenarios · Implementation: mlsysim/engine/scenario_definitions/statistics.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
The labeled data a per-group evaluation needs: one group’s rate within half_width at confidence (worst case rate 0.5), the collection random sampling needs to reach a target_per_group from a group of group_share, and the per-group count that pins a gap between two equal-size groups to gap_half_width.
Notes
Equation: Normal-error half-widths determine sample counts; random collection divides required subgroup count by its population share.
Assumptions: Samples are independent Bernoulli observations; equal-size groups and the declared rate underlie gap planning.
Excludes: A guaranteed subgroup yield from random sampling, exact small-sample coverage, and cluster-sampling design effects.
References
Derivation: the scenario composes mlsysim.physics.fairness.calc_gap_samples_per_group, mlsysim.physics.statistics.calc_proportion_gap_se, mlsysim.physics.statistics.calc_samples_for_half_width. Their contracts supply the underlying equations; property equations show the remaining unit conversions and accounting.
Examples
>>> from mlsysim import Scenarios
>>> scenario = Scenarios.OnePercentGroupEvaluationBudget
>>> value = scenario.z
>>> round(float(value), 3)
1.96Attributes
| Name | Description |
|---|---|
| confidence | |
| gap_half_width | |
| group_share | |
| half_width | |
| random_collection | Expected total random samples needed to collect the target subgroup count; not a |
| random_multiple | Expected random collection count per targeted subgroup example, reciprocal of subgroup |
| rate | |
| samples_needed | Normal-approximation count needed for the declared one-group proportion half-width. |
| target_per_group | |
| variance | Bernoulli outcome variance rate * (1 - rate), dimensionless. |
| z | Two-sided standard-normal quantile for the requested confidence, dimensionless. |
Methods
| Name | Description |
|---|---|
| gap_half_width_at | Normal-approximation gap half-width at n_per_group independent observations in each |
| gap_samples_per_group | Per-group sample count for a two-group gap half-width using the shared planning helper. |
gap_half_width_at
engine.scenario_types.SubgroupEvaluationBudgetScenario.gap_half_width_at(
n_per_group,
)Normal-approximation gap half-width at n_per_group independent observations in each group.
Equation: z * calc_proportion_gap_se(rate, rate, n_per_group, n_per_group).
gap_samples_per_group
engine.scenario_types.SubgroupEvaluationBudgetScenario.gap_samples_per_group(
half_width=None,
)Per-group sample count for a two-group gap half-width using the shared planning helper.
Equation: calc_gap_samples_per_group(rate, half_width or gap_half_width, z).
rate
Type: float.
Declared default: 0.5.
half_width
Type: float.
Declared default: required.
confidence
Type: float.
Declared default: required.
target_per_group
Type: int.
Declared default: required.
gap_half_width
Type: Optional[float].
Declared default: None.
z
engine.scenario_types.SubgroupEvaluationBudgetScenario.z
Two-sided standard-normal quantile for the requested confidence, dimensionless.
Equation: NormalDist().inv_cdf(0.5 + confidence / 2).
variance
engine.scenario_types.SubgroupEvaluationBudgetScenario.variance
Bernoulli outcome variance rate * (1 - rate), dimensionless.
Equation: rate * (1 - rate).
samples_needed
engine.scenario_types.SubgroupEvaluationBudgetScenario.samples_needed
Normal-approximation count needed for the declared one-group proportion half-width.
Equation: calc_samples_for_half_width(rate, half_width, z).
random_collection
engine.scenario_types.SubgroupEvaluationBudgetScenario.random_collection
Expected total random samples needed to collect the target subgroup count; not a high-probability bound.
Equation: target_per_group / group_share.
random_multiple
engine.scenario_types.SubgroupEvaluationBudgetScenario.random_multiple
Expected random collection count per targeted subgroup example, reciprocal of subgroup share.
Equation: random_collection / target_per_group.