ActiveLearningBudgetScenario
Data scenarios · Implementation: mlsysim/engine/scenario_definitions/data.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
Active learning against a fixed label budget when scoring the pool is charged to the same budget.
Counts are examples; prices are USD. compute_share of the budget pays for scoring a narrowed candidate pool, and the rest buys labels.
Notes
Equation: budget = scoring spend + label spend; affordable counts divide each allocation by its unit price.
Assumptions: Required label counts are stipulated for the target quality and unit prices remain constant.
Excludes: Predicting an acquisition strategy’s accuracy or discovering the required label count empirically.
References
Derivation: budget = scoring spend + label spend; affordable counts divide each allocation by its unit price. The individual properties state units and intermediate accounting; this class introduces no fitted coefficient.
Examples
The named scenario supplies illustrative inputs; the output is calculated.
>>> from mlsysim import Scenarios
>>> case = Scenarios.ActiveLearningTenMillionPool
>>> round(float(case.random_label_cost.to('USD').magnitude), 3)
500000.0Attributes
| Name | Description |
|---|---|
| active_label_cost_high | Currency needed for the upper stipulated active-learning label count. |
| active_label_cost_low | Currency needed for the lower stipulated active-learning label count. |
| active_labels_high | |
| active_labels_low | |
| active_total_high | Currency for full-pool scoring plus the upper active-label requirement. |
| active_total_low | Currency for full-pool scoring plus the lower active-label requirement. |
| affordable_labels | Labels purchasable after reserving the scoring budget; fractional counts are |
| baseline_accuracy | |
| candidate_pool | Candidates that the scoring budget can process at the fixed per-candidate price. |
| compute_budget | Currency allocated to candidate scoring by the declared compute share. |
| compute_share | |
| full_pool_scoring_cost | Currency charged to score every candidate in the full pool once. |
| label_budget | |
| label_price | |
| pool | |
| random_label_cost | Currency needed for the stipulated random-sampling label count. |
| random_labels_needed | |
| scoring_price | |
| target_accuracy |
pool
Type: float.
Declared default: required.
label_budget
Type: Quantity.
Declared default: required.
label_price
Type: Quantity.
Declared default: required.
scoring_price
Type: Quantity.
Declared default: required.
random_labels_needed
Type: float.
Declared default: required.
active_labels_low
Type: float.
Declared default: required.
active_labels_high
Type: float.
Declared default: required.
baseline_accuracy
Type: float.
Declared default: required.
target_accuracy
Type: float.
Declared default: required.
random_label_cost
engine.scenario_types.ActiveLearningBudgetScenario.random_label_cost
Currency needed for the stipulated random-sampling label count.
Equation: return random_labels_needed * label_price.
active_label_cost_low
engine.scenario_types.ActiveLearningBudgetScenario.active_label_cost_low
Currency needed for the lower stipulated active-learning label count.
Equation: return active_labels_low * label_price.
active_label_cost_high
engine.scenario_types.ActiveLearningBudgetScenario.active_label_cost_high
Currency needed for the upper stipulated active-learning label count.
Equation: return active_labels_high * label_price.
full_pool_scoring_cost
engine.scenario_types.ActiveLearningBudgetScenario.full_pool_scoring_cost
Currency charged to score every candidate in the full pool once.
Equation: return pool * scoring_price.
active_total_low
engine.scenario_types.ActiveLearningBudgetScenario.active_total_low
Currency for full-pool scoring plus the lower active-label requirement.
Equation: return full_pool_scoring_cost + active_label_cost_low.
active_total_high
engine.scenario_types.ActiveLearningBudgetScenario.active_total_high
Currency for full-pool scoring plus the upper active-label requirement.
Equation: return full_pool_scoring_cost + active_label_cost_high.
compute_budget
engine.scenario_types.ActiveLearningBudgetScenario.compute_budget
Currency allocated to candidate scoring by the declared compute share.
Equation: return label_budget * compute_share.
candidate_pool
engine.scenario_types.ActiveLearningBudgetScenario.candidate_pool
Candidates that the scoring budget can process at the fixed per-candidate price.
Equation: return (compute_budget / scoring_price).to('').magnitude.
affordable_labels
engine.scenario_types.ActiveLearningBudgetScenario.affordable_labels
Labels purchasable after reserving the scoring budget; fractional counts are retained.
Equation: return ((label_budget - compute_budget) / label_price).to('').magnitude.