DelegationTokenBreakEvenScenario
Agents scenarios · Implementation: mlsysim/engine/scenario_definitions/agents.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
Delegation at a token multiple m per attempt: cost per accepted task scales by m p_single / p_multi.
With cost proportional to tokens, delegation breaks even only if it multiplies acceptance by m, which is possible only while single-agent acceptance is at most 1 / m.
Notes
Equation: The calculation members below define token_multiple, max_single_acceptance, acceptance_multiplier, cost_ratio_if_gain_is_acceptance, required_multi_acceptance.
Assumptions: Cost is proportional to tokens; the score-gain interpretation as acceptance improvement is explicitly hypothetical.
Excludes: Equating benchmark score and acceptance probability without independent evidence.
References
Derivation: Delegation at a token multiple m per attempt: cost per accepted task scales by m p_single / p_multi. Each member records the corresponding algebra or named physics function.
Examples
Evaluate the named configuration and display a modeled result.
>>> from mlsysim import Scenarios
>>> case = Scenarios.ResearchSystemTokenBreakEven
>>> value = case.cost_ratio_if_gain_is_acceptance
>>> print(f"{value:.3g}")
1.97Attributes
| Name | Description |
|---|---|
| acceptance_multiplier | Hypothetical acceptance multiplier formed as one plus the reported relative gain. |
| agent_tokens_vs_chat | |
| cost_ratio_if_gain_is_acceptance | Delegated-to-single cost per accepted task if the reported score gain is an acceptance gain. |
| max_single_acceptance | Largest single-agent success probability permitting delegation break-even, reciprocal token multiple. |
| multi_agent_tokens_vs_chat | |
| reported_gain | |
| required_multi_acceptance | Delegated acceptance needed for break-even, token multiple times single acceptance; None if unspecified. |
| single_acceptance | |
| token_multiple | Delegated-to-single-agent token volume ratio, dimensionless. |
agent_tokens_vs_chat
Type: float.
Declared default: required.
multi_agent_tokens_vs_chat
Type: float.
Declared default: required.
reported_gain
Type: float.
Declared default: required.
single_acceptance
Type: Optional[float].
Declared default: None.
token_multiple
engine.scenario_types.DelegationTokenBreakEvenScenario.token_multiple
Delegated-to-single-agent token volume ratio, dimensionless.
Equation: multi_agent_tokens_vs_chat / agent_tokens_vs_chat.
max_single_acceptance
engine.scenario_types.DelegationTokenBreakEvenScenario.max_single_acceptance
Largest single-agent success probability permitting delegation break-even, reciprocal token multiple.
Largest single-agent acceptance at which break-even is possible: 1 / m.
Equation: 1 / token_multiple.
acceptance_multiplier
engine.scenario_types.DelegationTokenBreakEvenScenario.acceptance_multiplier
Hypothetical acceptance multiplier formed as one plus the reported relative gain.
Equation: 1 + reported_gain.
cost_ratio_if_gain_is_acceptance
engine.scenario_types.DelegationTokenBreakEvenScenario.cost_ratio_if_gain_is_acceptance
Delegated-to-single cost per accepted task if the reported score gain is an acceptance gain.
Delegated over single cost per accepted task if the gain multiplied acceptance.
Equation: token_multiple / acceptance_multiplier.
required_multi_acceptance
engine.scenario_types.DelegationTokenBreakEvenScenario.required_multi_acceptance
Delegated acceptance needed for break-even, token multiple times single acceptance; None if unspecified.
Delegated acceptance that breaks even against single_acceptance: m p_single.
Equation: None if single_acceptance is None else token_multiple * single_acceptance.