models.types.Workload
models.types.Workload()Layer A (Workload Demand): Base representation of an ML model or task.
A Workload defines the computational requirements of a task without any knowledge of the hardware it will run on. It must implement lower() to project its architectural definition down into a ComputationGraph.
inference_flops semantics — the unit of work DIFFERS BY MODEL FAMILY (documented 2026-06-06; the field is dimensioned only as flop, so the family convention is the contract):
- Autoregressive LMs (
TransformerWorkloaddecoder-only,SparseTransformerWorkload): FLOPs per generated token, following the 2 x params convention — 2 x TOTAL params for dense models, 2 x ACTIVE params for MoE. - Encoder models (BERT-family): FLOPs per sequence at the reference sequence length recorded in the entry’s provenance (BERT-Base: 22 GFLOP at seq=128).
- Vision / CNN / recommendation models: FLOPs per input (one image / one query forward pass).
NEVER compare inference_flops across families without normalizing — the magnitudes differ by orders of magnitude purely from the work unit.
Methods
| Name | Description |
|---|---|
| lower | Project stored workload facts into a computation graph. |
| size_in_bytes | Calculate stored model weights at the requested precision. |
lower
models.types.Workload.lower(precision=BYTES_FP16)Project stored workload facts into a computation graph.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| precision | Quantity or str | Bytes per stored element, or a registered format such as ‘fp16’. A byte quantity specifies the width directly; it must be positive. | BYTES_FP16 |
Returns
| Name | Type | Description |
|---|---|---|
| ComputationGraph | Work in FLOPs, resident weight bytes, parameter count, arithmetic intensity in FLOP/byte, and the available layer count. |
Notes
Equation: W = P * b and I = F / W. F uses inference_flops when supplied; otherwise F = 2 * P. If P is absent but model_size exists, W = model_size, P = 0, and missing FLOPs become zero.
Assumptions: one graph represents the work unit described by the workload family. With parameter count present, this base implementation computes W from P and b rather than preferring model_size. A zero-byte size-only workload has zero arithmetic intensity by convention.
Excludes: activation or cache storage, backward work, hardware efficiency, and precision-dependent operation counts. The 2P fallback is an approximation, not a graph trace.
References
Derivation: multiply parameter count by storage width to obtain weight bytes, then divide the declared or approximated work by those bytes. This is a demand-accounting projection; it does not predict device time.
Examples
>>> from mlsysim.models.types import Workload
>>> from mlsysim.core.units import Q_
>>> model = Workload(name='Example', architecture='dense', parameters=Q_(100, 'count'))
>>> graph = model.lower('fp16')
>>> graph.weight_bytes == model.parameters.magnitude * Q_(2, 'byte')
Truesize_in_bytes
models.types.Workload.size_in_bytes(precision=BYTES_FP16)Calculate stored model weights at the requested precision.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| precision | Quantity or str | Bytes per stored element, or a registered format such as ‘fp16’. A byte quantity specifies the width directly; it must be positive. | BYTES_FP16 |
Returns
| Name | Type | Description |
|---|---|---|
| Quantity | Resident weight size with data units, normally bytes. A supplied model_size retains its original compatible units when returned. |
Notes
Equation: W = P * b when parameter count P is available. A supplied model_size takes precedence at FP16/BF16; if P is missing it is used at every precision because there is no count from which to rescale.
Assumptions: all parameters use one element width. An explicit model_size is an authoritative footprint at the default storage format. Without parameters or model_size, the method raises NotImplementedError.
Excludes: gradients, optimizer states, activations, KV caches, quantization scales, packing alignment, and allocator reserve.
References
Derivation: multiply the number of stored elements by bytes per element. The model_size precedence is an implementation convention for workloads whose stored footprint is known but whose parameter inventory is absent.
Examples
>>> from mlsysim.models.types import Workload
>>> from mlsysim.core.units import Q_
>>> model = Workload(name='Example', architecture='dense', parameters=Q_(100, 'count'))
>>> model.size_in_bytes('fp16') == 2 * model.size_in_bytes('int8')
True