Data Scenarios

Data preparation, selection, storage, and delivery.

All domains · Classes A–Z

Source locations

• Inputs and calculations: mlsysim/engine/scenario_definitions/data.py.
• Named configurations: mlsysim/engine/scenario_catalog/data/.
• Result schemas: mlsysim/engine/result_types/data.py.

Public imports and existing class reference URLs remain stable.

Scenario contracts

• AcceleratorInputFeedScenario: accelerators copies of hardware training workload at sustained_rate samples a second each, fed from supply with encoded_sample bytes stored per sample and decoded_sample bytes after decode.
• ActiveLearningBudgetScenario: Active learning against a fixed label budget when scoring the pool is charged to the same budget.
• ActiveLearningRoundsScenario: Active learning from initial_labels adding labels_per_round for rounds rounds, each round paying round_scoring_cost to score the pool, against random labeling that needs random_labels at the same label_price.
• AdmissionOrderScenario: Independent admission checks (pass probability, slot time) run in each of orders.
• AllocationGrowthScenario: A compute-optimal allocation X_opt ~ C^exponent when compute grows by compute_multiple.
• AlwaysOnDetectorTargetsScenario: An always-on detector’s product contract: what it must catch, how rarely it may wake falsely, what it may occupy on the device, and the data budget and schedule that must deliver it.
• AttemptGroupContrastScenario: attempts independent attempts per fixture at each of pass_rates.
• BatchVersusStreamingIngestScenario: One event stream ingested by always-on streaming cores or by a batch job each hour.
• BudgetMatchedLabelingScenario: A pool to label at label_price under budget: random labeling spends the budget, active learning reaches the same quality with active_labels.
• CheckpointStormScenario: Every one of replicas writes workload’s full resumable state through one ingest path.
• CodecEpochScenario: epoch_bytes decompressed on the host each epoch for epochs epochs, at each codec’s decode_rates.
• CohortDispositionScenario: A finite cohort of attempts by disposition at a cutoff, its total cost, and the disposition that emits a learning artifact.
• ColumnProjectionScenario: A model that uses columns_used of columns_total equal-size columns, read from a row or a columnar file.
• CompressedReadScenario: Compressed shards read at read_bandwidth, decoded on CPU cores at each codec’s output rate.
• CoresetRunScenario: workload trained for epochs over dataset on one hardware at utilization, against a keep_fraction coreset scored by one forward pass of workload or proxy.
• CorpusDenominatorsScenario: A retained pool of record classes (records, stored tokens per record, selected target tokens per record); focus is the class whose shares are read, admitted the classes a route admits.
• CorpusRepetitionScenario: A training run (ClusterTrainingTimeScenario) set against the unique tokens of its workload’s own corpus.
• CorpusTierScenario: An audio corpus (dataset) kept on storage tiers at storage_rates and read at read_bandwidths.
• CoverageFloorScenario: A keep_fraction coreset drawn without regard to class from pool examples, of which rare_share belong to a rare class that needs at least floor examples.
• CurriculumScheduleScenario: A curriculum that reaches target in curriculum_epochs against baseline_epochs shuffled, with an optional linear warmup of warmup_epochs (epoch j serves j/W of the data).
• DataBudgetScenario: A data budget split into shares by category; the labeling share buys labels at label_price plus review_overhead for quality review.
• DataCostBreakdownScenario: A supervised dataset’s bill: acquisition, labels at label_price, storage_bytes kept for storage_duration at storage_rate, and runs training runs of run_time each on accelerators at accelerator_rate.
• DataEchoingScenario: A host that prepares host_rate samples a second feeding an accelerator that consumes accelerator_rate, over epochs of examples, with each prepared sample used echo_factor times.
• DataScalingReturnScenario: Growing a dataset from base_examples to scaled_examples for accuracy_gain points.
• DataStallScenario: A batch delivered in io_time against compute_time of compute, in sequence or overlapped.
• DesignTradeoffScenario: One design-space move of a knob from baseline to alternative, with an illustrative quality gain from gain_low to gain_high (fractions); costs scale by ratio.
• DuplicateRemovalScenario: Near-duplicate shares removed from run’s corpus: the training time and energy each share returns, against one read of the corpus and the MinHash signatures of its documents.
• FailureTriageScenario: runs trajectories a month failing at failure_rate, each probe’s attributed share, and a fine-tuning run on every failure of workload on devices x hardware.
• FeatureLookupScenario: An online service at request_rate that reads reads_per_request stored features per request.
• HostIoCpuScenario: devices each consuming requests_per_device reads per second, at each path’s CPU service time.
• InformationComputeRatioScenario: Accuracy gained per training FLOP for one epoch over all of dataset and over a keep_fraction subset, with gain_full and gain_subset points of gain.
• InputDemandScenario: consumers each read samples_per_step samples per step_time of compute.
• LabelGranularityScenario: Label entries for one image: a per-pixel mask against boxes boxes of coords_per_box.
• LabelingVersusTrainingScenario: A labeling budget against one training run: epochs over dataset_examples at throughput_per_accelerator on accelerators, priced at a low and a high hourly rate.
• LaborEffortScenario: items labeled by hand at time_per_item each: total hours and full-time person-years.
• MinHashBandingScenario: A MinHash signature cut into bands bands of rows hashes over shingle_words-word shingles.
• ObjectStorageBillScenario: A dataset kept in object storage for a billing period and read reads times across a boundary.
• PrefetchDepthScenario: Ready batches that cover each tier’s tail read, and host memory at a safety depth.
• PretrainingAmortizationScenario: tasks tasks trained from scratch at scratch_compute each against one pretraining run of pretrain_compute plus finetune_compute per task, with the labels each path needs.
• RunAmortizationScenario: A one-time investment of build_cost plus operating_cost that saves saving_per_run on every training run that reuses its output.
• RunSplitLeakageScenario: attempts runs per task, each assigned to training with probability train_fraction.
• SelectionBudgetScenario: Selection that takes selection_time and leaves a subset that trains in subset_time, against full training in full_time (the selection inequality in hours).
• SemiSupervisedCostScenario: A supervised baseline with supervised_labels and supervised_compute against a semi-supervised run with ssl_labels and ssl_compute, labels at label_price.
• ShardPackagingScenario: files samples packaged into shards large files on a file system rated at rated_bandwidth.
• StagingBurstScenario: A collector of rollouts concurrent rollouts feeding slots verifiers, whose checks stall at timeout, with records charged at record_ceiling against store.
• StorageFootprintScenario: What a training run of workload reads once, saves every interval, and archives.
• StreamVsStageScenario: Stream every epoch from remote object storage, or stage the dataset locally once per run.
• SyntheticFootprintScenario: Generated samples with provenance metadata, read once per verification pass.
• TieredCacheScenario: A fast tier in front of a slow one: a stated hit rate, or capacity over a uniformly sampled corpus.
• TieredCheckpointScenario: One save of workload’s state on fleet: local write per node, then a durable drain.
• UtilizationLossScenario: fleet’s accelerators billed at device_hour_rate while utilization falls.