QueryExtractionBudgetScenario
Reliability scenarios · Implementation: mlsysim/engine/scenario_definitions/reliability.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
An extractor that needs extraction_queries answers from a priced API, under three billing regimes: pay per query, a paid tier capped per day, and a lifetime per-principal allowance after which every query is billed at the per-query price.
Notes
Equation: Per-query cost multiplies answer count; tier cost multiplies quota-limited collection days by daily fee.
Assumptions: Answers fill the daily quota continuously and prorated tier charges are permitted.
Excludes: Calendar rounding, failed queries, and a prediction that the supplied answer budget reaches target fidelity.
References
Derivation: Per-query cost multiplies answer count; tier cost multiplies quota-limited collection days by daily fee. Individual property contracts identify the exact accounting expression or reusable calculation. Registry entries supply the scenario inputs; this class does not turn them into measured outcomes.
Examples
Evaluate a named scenario using its declared inputs.
>>> from mlsysim import Scenarios
>>> scenario = Scenarios.ResNet50ApiExtractionBudget
>>> round(float(scenario.collection_days), 4)
500.0Attributes
| Name | Description |
|---|---|
| allowance_fees | USD in tier fees needed to exhaust the lifetime allowance. |
| billing_period | |
| collection_days | Days required to collect the full target answer set at the daily tier quota. |
| extraction_queries | |
| full_set_lifetime | USD for the full target answer set under the lifetime-allowance policy. |
| full_set_pay_per_query | USD for all target answers under pure per-query billing. |
| full_set_tier | USD for all target answers under the daily-quota tier model. |
| lifetime_allowance | |
| overage_cost | USD billed per query for the nonnegative excess above the lifetime allowance. |
| price_per_query | |
| target_fidelity | |
| tier_daily_quota | |
| tier_price | |
| tier_share_of_training | Dimensionless tier-extraction bill divided by the victim model’s stated training cost. |
| training_cost |
Methods
| Name | Description |
|---|---|
| lifetime_cost | USD for tier collection up to the lifetime allowance plus per-query charges above it. |
| pay_per_query_cost | USD billed for queries answers at the supplied per-query price. |
| tier_cost | USD in prorated tier fees for the days needed to collect queries answers. |
| tier_days | Collection days for queries answers when the daily tier quota is fully used; |
lifetime_cost
engine.scenario_types.QueryExtractionBudgetScenario.lifetime_cost(queries)USD for tier collection up to the lifetime allowance plus per-query charges above it.
Tier fees up to the allowance, then every query billed per query.
Equation: within = min(queries, lifetime_allowance); overage = max(queries - lifetime_allowance, 0); result = (tier_cost(within) + pay_per_query_cost(overage)).to('USD').
pay_per_query_cost
engine.scenario_types.QueryExtractionBudgetScenario.pay_per_query_cost(queries)USD billed for queries answers at the supplied per-query price.
Equation: (queries * price_per_query).to('USD').
tier_cost
engine.scenario_types.QueryExtractionBudgetScenario.tier_cost(queries)USD in prorated tier fees for the days needed to collect queries answers.
Daily quota only: the tier fee for every day of collection.
Equation: (tier_days(queries) * _tier_fee_per_day()).to('USD').
tier_days
engine.scenario_types.QueryExtractionBudgetScenario.tier_days(queries)Collection days for queries answers when the daily tier quota is fully used; fractional days are allowed.
Equation: queries / tier_daily_quota.
training_cost
Type: Quantity.
Declared default: required.
extraction_queries
Type: float.
Declared default: required.
target_fidelity
Type: float.
Declared default: required.
price_per_query
Type: Quantity.
Declared default: required.
tier_price
Type: Quantity.
Declared default: required.
tier_daily_quota
Type: float.
Declared default: required.
billing_period
Type: Quantity.
Declared default: required.
lifetime_allowance
Type: float.
Declared default: required.
collection_days
engine.scenario_types.QueryExtractionBudgetScenario.collection_days
Days required to collect the full target answer set at the daily tier quota.
Equation: tier_days(extraction_queries).
full_set_pay_per_query
engine.scenario_types.QueryExtractionBudgetScenario.full_set_pay_per_query
USD for all target answers under pure per-query billing.
Equation: pay_per_query_cost(extraction_queries).
full_set_tier
engine.scenario_types.QueryExtractionBudgetScenario.full_set_tier
USD for all target answers under the daily-quota tier model.
Equation: tier_cost(extraction_queries).
allowance_fees
engine.scenario_types.QueryExtractionBudgetScenario.allowance_fees
USD in tier fees needed to exhaust the lifetime allowance.
Equation: tier_cost(lifetime_allowance).
overage_cost
engine.scenario_types.QueryExtractionBudgetScenario.overage_cost
USD billed per query for the nonnegative excess above the lifetime allowance.
Equation: pay_per_query_cost(max(extraction_queries - lifetime_allowance, 0)).
full_set_lifetime
engine.scenario_types.QueryExtractionBudgetScenario.full_set_lifetime
USD for the full target answer set under the lifetime-allowance policy.
Equation: lifetime_cost(extraction_queries).