MinHashBandingScenario
Data scenarios · Implementation: mlsysim/engine/scenario_definitions/data.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
A MinHash signature cut into bands bands of rows hashes over shingle_words-word shingles.
Notes
Equation: threshold is the worked quantity; each property below states its equation and units.
Assumptions: Hash matches and bands follow the independence approximation of the shared LSH helpers.
Excludes: Correlated hashes, exact index overhead, and empirical recall calibration.
References
Derivation: The scenario binds declared inputs to calc_lsh_candidate_probability, calc_lsh_threshold, calc_minhash_standard_error. Those implementations document their derivations and sources.
Examples
Execute the registered worked inputs; this is a modeled calculation.
>>> from mlsysim import Scenarios
>>> case = Scenarios.TrajectoryMinHashScreen
>>> value = case.threshold
>>> round(value, 3)
0.42Attributes
| Name | Description |
|---|---|
| bands | |
| hash_bytes | |
| hashes | Hash count in the full signature, bands times rows. |
| index_bytes | Raw signature-index storage in bytes; requires records and hash_bytes. |
| max_standard_error | Largest Jaccard standard error, evaluated at probability one half with the full hash count. |
| probe_similarities | |
| records | |
| rows | |
| shingle_words | |
| signature_bytes | One record’s signature storage in bytes; requires hash_bytes. |
| threshold | Dimensionless approximate LSH transition threshold from the shared retrieval helper. |
Methods
| Name | Description |
|---|---|
| candidate_probability | Candidate probability for the supplied Jaccard similarity under independent bands. |
candidate_probability
engine.scenario_types.MinHashBandingScenario.candidate_probability(similarity)Candidate probability for the supplied Jaccard similarity under independent bands.
Equation: return calc_lsh_candidate_probability(similarity, self.bands, self.rows)
bands
Type: int.
Declared default: required.
rows
Type: int.
Declared default: required.
shingle_words
Type: Optional[int].
Declared default: None.
probe_similarities
Type: List[float].
Declared default: [].
hash_bytes
Type: Optional[Quantity].
Declared default: None.
records
Type: Optional[int].
Declared default: None.
max_standard_error
engine.scenario_types.MinHashBandingScenario.max_standard_error
Largest Jaccard standard error, evaluated at probability one half with the full hash count.
Largest standard error of the Jaccard estimate (at J = 0.5) from the full signature.
Equation: return calc_minhash_standard_error(0.5, self.hashes)
signature_bytes
engine.scenario_types.MinHashBandingScenario.signature_bytes
One record’s signature storage in bytes; requires hash_bytes.
Equation: return (self.hashes * self.hash_bytes).to(‘byte’)
index_bytes
engine.scenario_types.MinHashBandingScenario.index_bytes
Raw signature-index storage in bytes; requires records and hash_bytes.
Equation: return (self.records * self.signature_bytes).to(‘byte’)
hashes
engine.scenario_types.MinHashBandingScenario.hashes
Hash count in the full signature, bands times rows.
Equation: return self.bands * self.rows
threshold
engine.scenario_types.MinHashBandingScenario.threshold
Dimensionless approximate LSH transition threshold from the shared retrieval helper.
Equation: return calc_lsh_threshold(self.bands, self.rows)