ToolDriftScenario
Agents scenarios · Implementation: mlsysim/engine/scenario_definitions/agents.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
Baseline and candidate distributions of tool calls over the same tools, compared by the Jensen-Shannon divergence (natural logarithms).
Notes
Equation: divergence is the worked quantity; each property below states its equation and units.
Assumptions: Both probability mappings use the same tool keys; natural logarithms measure divergence in nats.
Excludes: A significance test or handling candidate-only keys absent from the baseline mapping.
References
Derivation: The scenario binds declared inputs to calc_jensen_shannon_divergence. Those implementations document their derivations and sources.
Examples
Execute the registered worked inputs; this is a modeled calculation.
>>> from mlsysim import Scenarios
>>> case = Scenarios.TestLintToolDrift
>>> value = case.divergence
>>> round(value, 3)
0.021Attributes
| Name | Description |
|---|---|
| baseline | |
| candidate | |
| divergence | Jensen-Shannon divergence in nats over the baseline tool keys. |
| maximum | Maximum binary-mixture Jensen-Shannon divergence, ln(2) nats. |
| share_of_maximum | Dimensionless observed divergence divided by its theoretical maximum. |
baseline
Type: Dict[str, float].
Declared default: required.
candidate
Type: Dict[str, float].
Declared default: required.
divergence
engine.scenario_types.ToolDriftScenario.divergence
Jensen-Shannon divergence in nats over the baseline tool keys.
Equation: keys = list(self.baseline) return calc_jensen_shannon_divergence([self.baseline[k] for k in keys], [self.candidate[k] for k in keys])
maximum
engine.scenario_types.ToolDriftScenario.maximum
Maximum binary-mixture Jensen-Shannon divergence, ln(2) nats.
ln 2, reached by two distributions with no tool in common.
Equation: return _math.log(2)