Notation

Machine learning systems span machine learning (computer science and statistics) and systems (computer architecture and hardware). Each field developed its notation independently, and many symbols mean different things across communities and publications. This collision creates real confusion when the disciplines merge. The notation below distinguishes quantities used in both fields.

Consider a simple statement: “Increasing $\BatchSize$ improves throughput.” To an ML researcher, $\BatchSize$ means batch size. To a hardware engineer, $\Glyph{B}$ means bandwidth. Both interpretations are correct in their respective fields, but in machine learning systems we need both concepts in the same equation—hence the need for a single, consistent convention.

The Iron Law of ML Systems

Across the Data · Algorithm · Machine (D·A·M) axes, execution time factors into the product of workload scale, algorithmic operations per sample, and realized machine time per operation:

\[\Time = \underbrace{\left(\frac{\text{Samples}}{\text{Task}}\right)}_{\text{Data}} \times \underbrace{\left(\frac{\text{FLOPs}}{\text{Sample}}\right)}_{\text{Algorithm}} \times \underbrace{\left(\frac{\text{Seconds}}{\text{FLOP}}\right)}_{\text{Machine}}\]

where the first two factors multiply to set total floating-point operations $\Operations$. For serialized execution phases on physical hardware, unpacking how the machine executes those operations gives the three-term iron-law performance decomposition, where each variable is chosen deliberately to avoid collision with standard ML terminology:

\[\Time = \underbrace{\frac{\DataVolume}{\Bandwidth}}_{\text{Memory Time}} + \underbrace{\frac{\Operations}{\PeakRate \cdot \EfficiencyHw}}_{\text{Compute Time}} + \underbrace{\Latency}_{\text{Latency Overhead}}\]

Symbol	Definition	Unit	Notes
$\Time$	Execution Time	seconds	Total execution time in the iron-law performance decomposition.
$\DataVolume$	Data Volume	bytes	Avoids collision with $\DatasetSize$ (Dataset Size). In scaling laws and the D·A·M product, $\DatasetSize$ counts logical training samples or tokens. Here we need physical bytes moved through memory. The subscript disambiguates.
$\Bandwidth$	Bandwidth	bytes/s	Avoids collision with $\BatchSize$ (Batch Size). Systems literature often uses $\Glyph{B}$ for bandwidth, while ML literature often uses $\BatchSize$ for batch size. The notation preserves the ML convention.
$\Operations$	Operations	FLOPs	Total floating-point operations, the product of $\text{Samples/Task}$ and $\text{FLOPs/Sample}$. Clean in equations (vs. “$\text{Ops}$”).
$\PeakRate$	Peak Rate	FLOP/s	Avoids collision with $\Parameters$ (Parameters). Roofline presentations often use $\Glyph{P}$ for performance, while ML literature commonly uses $\Parameters$ for parameter count. The notation preserves the ML convention.
$\EfficiencyHw$	Efficiency	—	Hardware utilization $(0 \le \EfficiencyHw \le 1)$. Measures compute-unit utilization in the three-term decomposition, or end-to-end model FLOPs utilization when data movement and overhead are folded into the single-term training law (Model Training). Avoids collision with learning rate $(\LearningRate)$.
$\Latency$	Latency	seconds	Avoids collision with $\LossFunction$ (Loss). Fixed overhead time (kernel launch, network RTT). The subscript distinguishes from the loss function.

Why these choices matter

Without careful notation, sentences become ambiguous:

“Reducing $\DatasetSize$ improves performance.”

The sentence has two possible readings:

Reducing dataset size (fewer training samples) can speed training but may reduce accuracy.
Reducing data volume moved (for example, through compression or quantization) can speed inference, with accuracy effects that depend on the technique and calibration.

With our notation, we can write precisely and eliminate this ambiguity so that a statement like “$\Bandwidth$ limits throughput” has only one reading:

“Reducing $\DataVolume$ through FP32-to-INT8 quantization cuts parameter memory traffic to one quarter while $\DatasetSize$ (training data) remains unchanged.”

Subscripted variants

The root names the quantity, and an upright subscript names which instance of it is meant. The convention distinguishes closely related quantities:

$\DiskBandwidth$, $\NetworkBandwidth$, $\AcceleratorMemoryBandwidth$: bandwidth at three points on the data path
$\DataVolume$, $\PeakRate$, $\Latency$, $\EfficiencyHw$: iron-law terms, each held clear of a common ML symbol
$\EnergyPerByteMoved$, $\EnergyPerOperation$, $\TotalEnergy$: one energy budget split into components

The subscript identifies where a measurement applies. Disk and accelerator bandwidth differ by orders of magnitude, so a reported bandwidth is meaningful only if readers know where it was measured.

The Degradation Equation

The degradation equation is an approximation fitted to observed outcomes over a limited range. Divergence alone does not determine whether accuracy rises or falls, so estimate $\FittedSensitivity$ from labeled outcomes. Some symbols below, such as $\ResponseThreshold$, appear in chapter prose rather than in the equation. \[\AccuracyAfterDeployment(\DeploymentAge) \approx \InitialAccuracy - \FittedSensitivity \cdot \StatisticalDivergence\]

Symbol	Definition	Type or unit	Notes
$\AccuracyAfterDeployment(\DeploymentAge)$	Accuracy at Time $\DeploymentAge$	Scalar	Model accuracy after the model has been deployed for time $\DeploymentAge$.
$\InitialAccuracy$	Initial Accuracy	Scalar	Model accuracy at deployment time.
$\FittedSensitivity$	Fitted Sensitivity	Scalar	Local coefficient estimated from outcomes over a stated range; not a universal constant. (Not wavelength.)
$\CurrentDistribution$	Current Distribution	Distribution	The data distribution at time $\DeploymentAge$. (Not parameters—use $\Parameters$ for parameter count.)
$\TrainingDistribution$	Training Distribution	Distribution	The data distribution at training time.
$\StatisticalDivergence$	Statistical Divergence	Scalar $\ge 0$	Measures how far $\CurrentDistribution$ has drifted from $\TrainingDistribution$. Common choices: KL divergence, total variation, Wasserstein. (Calligraphic to avoid collision with $\DatasetSize$ = dataset size.)
$\ResponseThreshold$	Response Threshold	Scalar $> 0$	Operational threshold for investigation; it does not automatically trigger retraining. The same symbol marks a classifier’s decision threshold.
$\DivergenceMeasure$	Divergence Measure	Function	Divergence function that measures how far one probability distribution lies from another, with the specific measure left open.
$\KolmogorovSmirnovStatistic$	Kolmogorov Smirnov Statistic	Dimensionless	Two-sample Kolmogorov-Smirnov statistic, the largest absolute difference between two empirical cumulative distribution functions.
$\TrainingEmpiricalCdf$	Training Empirical CDF	Function	Empirical cumulative distribution function of a feature in the training-baseline sample.
$\MonitoredFeatureLevel$	Monitored Feature Level	Varies	One value of a monitored feature, the argument at which its distribution or empirical cumulative distribution function is evaluated. The maximization variable of the Kolmogorov-Smirnov statistic.
$\ServingEmpiricalCdf$	Serving Empirical CDF	Function	Empirical cumulative distribution function of a feature in the current serving sample.
$\KolmogorovSmirnovCriticalThreshold$	Kolmogorov Smirnov Critical Threshold	Dimensionless	Critical threshold that the two-sample Kolmogorov-Smirnov statistic must exceed to reject equal distributions at a given significance level.
$\DriftTestSampleSize$	Drift Test Sample Size	Integer	Number of observations in one of the two samples a two-sample drift test compares.
$\KlDivergence$	KL Divergence	Function	Kullback-Leibler divergence, the expected extra code length from encoding one distribution with a code built for another.
$\ShannonEntropy$	Shannon Entropy	bits or nats	Average uncertainty of a probability distribution, the expected negative log probability of its outcomes. The log base sets the unit: base 2 gives bits, natural log nats.

The Energy Corollary

Effective costs per byte moved and per operation give the following approximation for workload energy. The symbols in the table below separate data-movement energy from arithmetic energy and list the circuit-level quantities that govern dynamic power: \[\TotalEnergy \approx \DataVolume \times \EnergyPerByteMoved + \Operations \times \EnergyPerOperation\]

Symbol	Definition	Unit	Notes
$\TotalEnergy$	Total Energy	joules	Total energy consumed by an ML workload, decomposed into data-movement and compute terms.
$\EnergyPerByteMoved$	Energy per Byte Moved	joules/byte	Effective movement cost; total energy also depends on bytes moved and operations executed.
$\EnergyPerOperation$	Energy per Operation	joules/FLOP	Energy cost of a single arithmetic operation.
$\SwitchedCapacitance$	Switched Capacitance	farads	Capacitance of the wires and gates a circuit charges when it switches.
$\SupplyVoltage$	Supply Voltage	volts	Supply voltage that powers a digital circuit, the quantity dynamic voltage and frequency scaling lowers.
$\ClockFrequency$	Clock Frequency	hertz	Rate at which a processor’s clock ticks, setting how many switching cycles occur per second.
$\ClusterPower$	Cluster Power	watts	Power drawn by a training cluster, including accelerators and hosts. Average power multiplied by training time gives training energy.
$\DevicePower$	Device Power	watts	Electrical power one device draws. Integrated over time it gives the device’s energy.
$\DynamicPower$	Dynamic Power	watts	Power a chip draws switching its transistors, proportional to switched capacitance, supply voltage squared, and clock frequency.
$\AveragePower$	Average Power	watts	Power drawn averaged over a stated period.

Deep Learning Notation

This book follows standard deep learning conventions (Goodfellow et al. 2016) with explicit disambiguation for systems variables. Where standard deep learning symbols would collide with hardware or serving quantities, the table below assigns distinct symbols and states the disambiguation rule explicitly.

Goodfellow, Ian, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. MIT Press.

Symbol	Definition	Type or unit	Notes
$\BatchSize$	Batch Size	Integer	Number of examples grouped in a training or inference batch. (Never bandwidth.)
$\Parameters$	Parameter Count	Integer	The total count of scalar model parameters, including weights and biases. (Never peak FLOP/s.)
$\DatasetSize$	Dataset Size	Integer	Number of training samples or tokens. (Never data volume in bytes—use $\DataVolume$.)
$\SequenceLength$	Sequence Length	Integer	Number of tokens or time steps in an input sequence.
$\HiddenDimension$	Hidden Dimension	Integer	Size of the hidden state vector of a layer or representation.
$\LayerInputWidth$	Layer Input Width	Integer	Number of input features in a layer.
$\LayerOutputWidth$	Layer Output Width	Integer	Number of output features in a layer.
$\AttentionHeadDimension$	Attention Head Dimension	Integer	Per-head hidden dimension in attention layers.
$\LayerCount$	Number of Layers	Integer	Total number of layers in a network.
$\AttentionHeadCount$	Number of Attention Heads	Integer	Number of attention heads in a multi-head attention layer.
$\KvHeadCount$	Number of Key-Value Heads	Integer	Number of key-value heads in grouped-query or multi-query attention.
$\LayerIndex$	Layer Index	Integer	Index for a layer. Use instead of bare $\Glyph{L}$ when indexing layers, since $\Glyph{L}$ collides with loss and latency. A numeric subscript marks a norm instead ($\ell_1$, $\ell_2$).
$\LossFunction$	Loss Function	Scalar	The objective function minimized during training.
$\LearningRate$	Learning Rate	Scalar	Step size for the optimizer. (Never bare for hardware efficiency—use $\EfficiencyHw$.)
$\ModelParameters$	Model Parameters	Vector	All learnable model parameters, treated as a vector.
$\LossGradient$	Gradient	Vector	Gradient of the loss with respect to the parameters, $\nabla_\ModelParameters \LossFunction$; a subscript $\OptimizerStepIndex$ marks the optimizer step.
$\InputDistribution$	Distribution	Probability mass/density	Distribution for a random variable. Use lowercase $\ProbabilityFunction(\cdot)$ for generic distributions to avoid collision with $\Parameters$ (parameter count).
$\ConditionalDistribution$	Conditional Distribution	Conditional probability/density	Use this form for generic label relationships; reserve $\TrainingDistribution$ and $\CurrentDistribution$ for the training/current distributions of drift analysis.
$\PredictedOutput$	Predicted Output	Vector	A model’s output for one example, such as class probabilities; $\LabelVector$ is the label vector.
$\DistillationTemperature$	Distillation Temperature	Dimensionless	Softmax temperature in knowledge distillation. It is dimensionless, unlike execution time.
$\Pr(\Glyph{E})$	Event Probability	Probability	Probability of an event $\Glyph{E}$. Use for event statements such as $\Pr(\text{batch}=0)$; use $\InputDistribution$ and $\ConditionalDistribution$ for distributions.
$\TrainingCompute$	Training Compute	FLOPs	Total floating-point operations required for model training.
$\ExampleLabel$	Example Label	Random variable	The label paired with one input, treated as a random variable.
$\ModelInput$	Model Input	Random variable	Input of one example, the data a model receives to produce a prediction.
$\FiniteChange$	Finite Change	Operator	Finite change in the quantity it prefixes, between two states, configurations or points in time. Prefix operator, as in the return-on-compute ratio of accuracy change to compute-cost change.
$\BatchIndex$	Batch Index	Index	Position of one batch in the sequence delivered by a data loader or runtime.
$\ProbabilityFunction$	Probability Function	Function	Probability mass or density function of a random variable, applied to its argument.
$\ImageHeight$	Image Height	Integer	Number of pixel rows in an image or its segmentation mask.
$\ImageWidth$	Image Width	Integer	Number of pixel columns in an image or its segmentation mask.
$\ModelWeight$	Model Weight	Dimensionless	Value of one learnable weight of a model, such as the weight on one input of a neuron.
$\LayerInputComponent$	Layer Input Component	Dimensionless	One component, or feature, of a layer’s input vector.
$\InputFeatureVector$	Input Feature Vector	Vector	Input feature vector of one example, written as a row vector.
$\NeuronBias$	Neuron Bias	Dimensionless	Bias term added to one neuron’s weighted sum. Indexed by neuron: $\NeuronBias_j$.
$\NeuronPreactivation$	Neuron Preactivation	Dimensionless	Preactivation of one neuron, its weighted input sum plus bias before the activation function. At the output layer, $\NeuronPreactivation_j$ is the logit for one class, the argument of softmax.
$\ActivationFunction$	Activation Function	Function	Activation function of a neuron or layer, applied element-wise to the preactivation. A layer superscript names the layer (f^{(l)}); f’ is its derivative.
$\WeightVector$	Weight Vector	Vector	Weight vector of one neuron, one weight per input.
$\PredictedOutputComponent$	Predicted Output Component	Dimensionless	One entry of a model’s prediction, such as the predicted probability for one class.
$\WeightMatrix$	Weight Matrix	Matrix	Matrix of learned weights one layer multiplies its input by, or the kernel tensor of a convolution layer.
$\ActivationInput$	Activation Input	Dimensionless	Scalar argument an activation function is applied to in its defining formula.
$\SigmoidFunction$	Sigmoid Function	Function	Logistic sigmoid function, one over one plus e to the minus x. Maps any real input to the open interval (0, 1).
$\ClassCount$	Class Count	Integer	Number of classes a classifier distinguishes, or vocabulary entries it scores, the length of its softmax output.
$\PreactivationVector$	Preactivation Vector	Vector	Preactivation vector of one layer for one example, the logit vector at the output layer.
$\BiasVector$	Bias Vector	Vector	Vector of learned biases one layer adds to its weighted sums, one per neuron or output channel. Broadcast across the batch.
$\LayerWidth$	Layer Width	Integer	Number of units in the layer at a given position of the network, with $\LayerWidth_0$ the input width. Indexed by layer position.
$\ActivationVector$	Activation Vector	Vector	Activation vector of one layer for one example.
$\ActivationMatrix$	Activation Matrix	Matrix	Activation matrix of one layer for a batch, one example per row.
$\LabelVector$	Label Vector	Vector	True label of one example written as a vector, such as a one-hot class vector.
$\ModelPredictor$	Model Predictor	Function	Trained model viewed as a function that maps an input to its prediction.
$\PreactivationMatrix$	Preactivation Matrix	Matrix	Preactivation matrix of one layer for a batch, one example per row.
$\LayerInputMatrix$	Layer Input Matrix	Matrix	Input to a layer in matrix form, one row per example or token, or a single-channel image.
$\ClassIndex$	Class Index	Index	Index of an output class.
$\InputChannelCount$	Input Channel Count	Integer	Number of channels in the feature map a convolution layer reads.
$\KernelSize$	Kernel Size	Integer	Side length, in input positions, of a square convolution kernel.
$\HiddenActivation$	Hidden Activation	Vector	Activation vector a hidden layer outputs and passes to the next layer.
$\OutputChannelCount$	Output Channel Count	Integer	Number of channels, one per filter, in the feature map a convolution layer produces.
$\LayerFunction$	Layer Function	Function	Function computed by a layer, operator, or recurrent cell, mapping inputs to outputs.
$\LayerInputVector$	Layer Input Vector	Vector	Vector of activations one layer receives as input.
$\FeatureMapHeight$	Feature Map Height	Integer	Height, in pixels, of the image or feature map a convolution layer reads.
$\FeatureMapWidth$	Feature Map Width	Integer	Width, in pixels, of the image or feature map a convolution layer reads.
$\OutputFeatureMapHeight$	Output Feature Map Height	Integer	Number of output rows a convolution produces.
$\OutputFeatureMapWidth$	Output Feature Map Width	Integer	Number of output columns a convolution produces.
$\AttentionQueryMatrix$	Attention Query Matrix	Matrix	Matrix of query vectors, one row per token, in an attention layer.
$\AttentionKeyMatrix$	Attention Key Matrix	Matrix	Matrix of key vectors, one row per token, in an attention layer.
$\AttentionValuesMatrix$	Attention Values Matrix	Matrix	Matrix of value vectors, one row per token, that attention weights combine into the output.
$\AttentionWeightMatrix$	Attention Weight Matrix	Matrix	Matrix of attention weights, the row-wise softmax of scaled query-key scores.
$\IdentityMatrix$	Identity Matrix	Matrix	Square matrix with ones on the diagonal and zeros elsewhere. Size follows from context.
$\MiniBatchSet$	Mini Batch Set	Set	Set of examples processed together as one mini-batch.
$\NormalizationScale$	Normalization Scale	Dimensionless	Learned factor a normalization layer multiplies each normalized activation by.
$\NormalizationScaleVector$	Normalization Scale Vector	Vector	Vector of learned per-channel or per-feature factors a normalization layer multiplies its normalized output by.
$\NormalizationShiftVector$	Normalization Shift Vector	Vector	Vector of learned per-channel or per-feature offsets a normalization layer adds to its scaled output.
$\TraceIntermediateVariable$	Trace Intermediate Variable	Dimensionless	Intermediate variable in the evaluation trace of a differentiated program, its primal value.
$\TraceIntermediateAdjoint$	Trace Intermediate Adjoint	Dimensionless	Reverse-mode adjoint of an intermediate variable, the derivative of the output with respect to it.
$\FunctionInputVector$	Function Input Vector	Vector	Vector a layer or differentiated function takes as input.
$\FunctionInputDimension$	Function Input Dimension	Integer	Number of inputs of a differentiated function, the column count of its Jacobian.
$\FunctionInputElement$	Function Input Element	Dimensionless	One element of the input vector of a layer or elementwise function.
$\MatrixRowIndex$	Matrix Row Index	Index	Position of a row within a matrix.
$\MatrixColumnIndex$	Matrix Column Index	Index	Column index of a matrix entry.
$\AttentionScoreMatrix$	Attention Score Matrix	Matrix	Matrix of attention scores, the scaled dot products of every query with every key before the softmax.
$\SoftmaxInputVector$	Softmax Input Vector	Vector	Vector of scores an online softmax normalizes block by block.
$\SoftmaxRunningMaximum$	Softmax Running Maximum	Dimensionless	Largest score seen so far in a block of the online softmax, subtracted before exponentiation. Superscripts (1), (2), new mark the block it covers.
$\SoftmaxNormalizer$	Softmax Normalizer	Dimensionless	Sum of the max-shifted exponentials of a softmax row or block, the softmax denominator, kept as a running sum by online softmax. Not the class logit z of the data appendix.
$\AttentionHeadOutput$	Attention Head Output	Matrix	Output of one attention head before the output projection, the attention weights applied to the value matrix. Distinct from the output-projection result of a whole attention layer.
$\DataScalingExponent$	Data Scaling Exponent	Dimensionless	Fitted power-law exponent at which loss falls as the training dataset or token count grows.
$\DerivativeOperator$	Derivative Operator	Operator	Differential operator d used in an ordinary derivative.
$\OptimizerStepIndex$	Optimizer Step Index	Index	Position of one optimizer update in the sequence of training steps.
$\EpochCount$	Epoch Count	Integer	Number of full passes a training run makes over its dataset. Training work scales with it times dataset size.
$\OutputLogit$	Output Logit	Dimensionless	Raw score a model outputs for one class before the softmax.
$\InputResolution$	Input Resolution	Integer	Input image resolution of a convolutional model. Subscript 0 marks the base model in compound scaling.
$\QuantizationScale$	Quantization Scale	Varies	Real-valued step between adjacent integer levels of a uniform quantizer.
$\QuantizedIntegerCode$	Quantized Integer Code	Integer	Integer code a quantizer assigns to one real value.
$\LayerWeightEntry$	Layer Weight Entry	Dimensionless	One scalar entry of a layer’s weight matrix, picked out by a layer superscript and row and column subscripts.
$\WeightMatrixEntry$	Weight Matrix Entry	Dimensionless	One entry of a weight matrix at a given row and column; italic because it is a scalar.
$\FusedConvolutionWeight$	Fused Convolution Weight	Dimensionless	Convolution weights after the following batch normalization is folded into them, so inference runs one layer instead of two.
$\QuantizerInput$	Quantizer Input	Varies	Full-precision real value, a weight or activation, that a quantizer maps to an integer code. Unit: units of the quantized tensor.
$\ClippingRangeLowerBound$	Clipping Range Lower Bound	Varies	Lower end of the real-valued range a quantizer represents, below which values are clipped. Calibration estimates it.
$\ClippingRangeUpperBound$	Clipping Range Upper Bound	Varies	Upper end of the real-valued range a quantizer represents, above which values are clipped.
$\QuantizationBitWidth$	Quantization Bit Width	Integer	Number of bits in each integer code a quantizer produces.
$\BatchNormRunningStandardDeviation$	Batch Norm Running Standard Deviation	Vector	Per-channel square root of the running variance plus the numerical stabilizer that batch normalization uses at inference.
$\StructuredSparsityNonzeroCount$	Structured Sparsity Nonzero Count	Integer	Number of values kept nonzero in each group of an N:M structured-sparsity pattern.
$\StructuredSparsityGroupSize$	Structured Sparsity Group Size	Integer	Number of consecutive values in one group of an N:M structured-sparsity pattern.
$\KernelHeight$	Kernel Height	Integer	Height of a convolution kernel, in input positions.
$\KernelWidth$	Kernel Width	Integer	Width of a convolution kernel, in input positions.
$\TopClassCount$	Top Class Count	Integer	Number of highest-scoring classes a top-k prediction returns or checks for the correct label.
$\ServiceRate$	Service Rate	items/s	Rate at which one server, such as a worker, consumer, or model endpoint, completes items or calls.
$\ModelPrediction$	Model Prediction	Dimensionless	Prediction a model outputs for one example, compared with its label by the loss.
$\LayerOutputMatrix$	Layer Output Matrix	Matrix	Output matrix of a layer’s matrix multiplication of the input matrix by the weight matrix, one row per example. Shape batch size by layer output width; before any bias or activation function.
$\SoftmaxRowMaximum$	Softmax Row Maximum	Dimensionless	Largest score in a softmax row or block, subtracted before exponentiation to prevent overflow.
$\SoftmaxInputScore$	Softmax Input Score	Dimensionless	One entry of the score vector the online softmax normalizes. Written with an index and a block superscript, $\SoftmaxInputScore_j^{(1)}$.
$\SymmetricClippingThreshold$	Symmetric Clipping Threshold	Varies	Largest magnitude a symmetric quantizer represents, so values are clipped to plus or minus this bound.

Latin matrices and vectors are set in bold ($\WeightMatrix$, $\InputFeatureVector$); scalars, dimensions, indices, and individual matrix entries stay italic ($\Glyph{N}$, $\HiddenDimension$, $\MatrixRowIndex$, $\WeightMatrixEntry$). The generic parameter symbol $\ModelParameters$ is the one conventional exception and stays italic.

Local matrix algebra follows the same bold rule. Matrix operands are bold ($\GemmLeftOperand$, $\GemmRightOperand$, $\GemmOutputMatrix$), as in general matrix multiply (GEMM) $\GemmOutputMatrix=\GemmProductScale \GemmLeftOperand\GemmRightOperand+\GemmAccumulatorScale \GemmOutputMatrix$. The bold form marks a matrix operand and is distinct from the italic scalar batch size $\BatchSize$, which remains batch size in scalar contexts.

A GEMM’s scalar dimensions are $\GemmRowDimension$, $\GemmColumnDimension$, and $\GemmInnerDimension$ ($\GemmLeftOperand$ is $\GemmRowDimension \times \GemmInnerDimension$, $\GemmRightOperand$ is $\GemmInnerDimension \times \GemmColumnDimension$). Asymptotic cost is written $\mathcal{O}(\cdot)$, which keeps it distinct from the operation count $\Operations$.

Performance, Serving, and Memory Notation

Reusable performance and serving quantities follow the same collision-avoidance rule as the iron law. Rates use $\Glyph{R}$ or descriptive Greek symbols; request counts use $\RequestConcurrency$ rather than overloading $\Glyph{N}$; queue utilization uses a subscripted $\Glyph{\rho}$ so bare $\Glyph{\rho}$ remains available for other ratio models.

Symbol	Definition	Type or unit	Notes
$\ArithmeticIntensity$	Arithmetic Intensity	FLOP/byte	Workload FLOPs per byte moved. The Roofline Model uses $\ArithmeticIntensity$ as the independent variable.
$\RooflineRidgePoint$	Roofline Ridge Point	FLOP/byte	$\RooflineRidgePoint = \PeakRate/\Bandwidth$. Prefer this explicit form over starred shorthand so the meaning remains clear in prose.
$\AttainableComputeRate$	Attainable Compute Rate	FLOP/s	Roofline bound: $\AttainableComputeRate \leq \min(\PeakRate, \ArithmeticIntensity \times \Bandwidth)$. Uses $\Glyph{R}$, not $\Time$, because this quantity is a rate.
$\ModelFlopsUtilization$	Model FLOPs Utilization	Dimensionless	Useful model FLOP/s divided by available peak FLOP/s. Text acronym avoids overloading $\LearningRate$.
$\RequestConcurrency$	Request Concurrency	requests	Average in-flight requests in a stable serving system ($\RequestConcurrency = \ArrivalRate \cdot \RequestTimeInSystem$ via Little’s Law). Avoids overloading $\Glyph{N}$, which counts devices, elements, or samples elsewhere.
$\ArrivalRate$	Arrival Rate	requests/s	Request arrival rate. The subscript keeps bare $\FittedSensitivity$ free for fitted sensitivities.
$\IronLawMemoryTime$	Iron-Law Memory Time	seconds	Data-movement term $\DataVolume/\Bandwidth$ in the iron law.
$\IronLawComputeTime$	Iron-Law Compute Time	seconds	Compute term $\Operations/(\PeakRate \cdot \EfficiencyHw)$ in the iron law.
$\AcceleratorCount$	Accelerator Count	Integer	Number of accelerators in one job.
$\RequestTimeInSystem$	Request Time in System	seconds	Average time a request spends queueing and being served, as used in Little’s law. Distinct from $\Latency$, the fixed-latency term in the iron law.
$\EndToEndRequestLatency$	End-to-End Request Latency	seconds	Request-path latency budget, $\NetworkLatency + \PreprocessingLatency + \InferenceLatency + \PostprocessingLatency + \QueueingLatency$. $\RequestTimeInSystem$ covers only the time inside the serving system.
$\BatchServiceTime(\BatchSize)$	Batch Service Time	seconds	Time to serve a batch of size $\BatchSize$.
$\EffectiveServiceRate(\BatchSize)$	Effective Service Rate	requests/s	Batched service rate, typically $\EffectiveServiceRate(\BatchSize) = \BatchSize/\BatchServiceTime(\BatchSize)$. The unbatched service rate is $\ServiceRate = \EffectiveServiceRate(1)$.
$\ServingUtilization$	Serving Utilization	Dimensionless	Queue/server utilization. The subscript keeps bare $\Glyph{\rho}$ free for other ratio models.
$\TotalMemoryFootprint$	Total Memory Footprint	bytes	Sum of explicit memory components; avoids bare $\Glyph{M}$ ambiguity.
$\WeightMemory$	Weight Memory	bytes	Memory occupied by model weights ($\Parameters \times \text{bytes-per-parameter}$).
$\GradientMemory$	Gradient Memory	bytes	Memory occupied by stored gradients.
$\OptimizerStateMemory$	Optimizer-State Memory	bytes	Momentum, variance, master weights, and related optimizer buffers.
$\ActivationMemory$	Activation Memory	bytes	Retained activations for the backward pass or serving intermediates.
$\KvCacheMemory$	KV-Cache Memory	bytes	Memory occupied by the key-value cache during autoregressive inference.
$\ElementStorageSize$	Element Storage Size	bytes/element	Bytes per stored tensor element.
$\SampleStorageSize$	Sample Storage Size	bytes/sample	Bytes per stored training or serving sample.
$\LossScaleFactor$	Loss-Scale Factor	Dimensionless	Multiplier applied to the loss in reduced-precision training so small gradients stay representable.
$\PipelinedExecutionTime$	Pipelined Execution Time	seconds	Ideal steady-state execution time when data movement and computation overlap, approximated by the slower stage plus fixed latency.
$\AcceleratedFractionSpeedup$	Accelerated Fraction Speedup	Dimensionless	Speedup factor applied to the accelerated fraction of a workload in Amdahl’s law, whether it comes from faster hardware or from more workers.
$\TrainingTime$	Training Time	seconds	Wall-clock time to complete one training run.
$\IoTransferTime$	I/O Transfer Time	seconds	Time to move a workload’s data over a storage or network link, the input/output stage of a pipelined step. Overlaps with compute in the pipelined iron law; the step is I/O bound when it is the largest stage.
$\NetworkBandwidth$	Network Bandwidth	bytes/s	Sustained rate at which a network path delivers bytes between machines.
$\NetworkLatency$	Network Latency	seconds	Time a request and its response spend crossing the network, including propagation, routing, and protocol handling.
$\DeviceMemoryCapacity$	Device Memory Capacity	bytes	Total memory a device provides for a model’s weights and activations.
$\AcceleratorGain$	Accelerator Gain	Dimensionless	Speedup an optimization or accelerator gives the fraction of execution it applies to, in Amdahl’s law.
$\FirstFloatingPointAddend$	First Floating Point Addend	Dimensionless	First of three floating-point numbers added in either grouping to show that floating-point addition is not associative.
$\SecondFloatingPointAddend$	Second Floating Point Addend	Dimensionless	Second of three floating-point numbers added in either grouping to show that floating-point addition is not associative.
$\ThirdFloatingPointAddend$	Third Floating Point Addend	Dimensionless	Third of three floating-point numbers added in either grouping to show that floating-point addition is not associative.
$\StorageBandwidth$	Storage Bandwidth	bytes/s	Sustained rate at which the storage system delivers bytes to the input pipeline.
$\TrainingStepTime$	Training Step Time	seconds	Wall-clock time of one training step, bounded below by the slower of compute and input I/O.
$\SquareGemmDimension$	Square GEMM Dimension	Integer	Side length of the square matrices in a matrix multiply, the shared row, column, and inner dimension.
$\GemmRowDimension$	GEMM Row Dimension	Integer	Number of rows of the left operand and of the output in a matrix multiply.
$\GemmInnerDimension$	GEMM Inner Dimension	Integer	Shared inner dimension of a general matrix multiply, the number of products summed per output element.
$\GemmColumnDimension$	GEMM Column Dimension	Integer	Number of columns of the right operand and of the output in a matrix multiply.
$\MicroUnitPrefix$	Micro Unit Prefix	Operator	SI unit prefix micro, one millionth, written before a unit such as seconds.
$\TensorElementCount$	Tensor Element Count	Integer	Number of elements in one tensor.
$\ChainedOperationCount$	Chained Operation Count	Integer	Number of consecutive operations that read and write the same intermediate tensor and can be combined through kernel fusion.
$\BatchTransferTime$	Batch Transfer Time	seconds	Time to copy one batch of input data from host memory to the device that consumes it.
$\GemmOutputMatrix$	GEMM Output Matrix	Matrix	Output matrix of a general matrix multiply, of size M by N, which the operation updates in place.
$\GemmProductScale$	GEMM Product Scale	Dimensionless	Scalar multiplier applied to the matrix product in a general matrix multiply.
$\GemmAccumulatorScale$	GEMM Accumulator Scale	Dimensionless	Scalar multiplier applied to the existing output matrix before the product is added in a general matrix multiply.
$\GemmLeftOperand$	GEMM Left Operand	Matrix	Left input matrix of a general matrix multiply, of size M by K.
$\GemmRightOperand$	GEMM Right Operand	Matrix	Right input matrix of a general matrix multiply, of size K by N.
$\DiskBandwidth$	Disk Bandwidth	bytes/s	Sustained rate at which a storage disk delivers bytes.
$\SustainedComputeRate$	Sustained Compute Rate	FLOP/s	Operation rate an accelerator or system actually sustains on a workload.
$\HostPreparationTime$	Host Preparation Time	seconds	Time the host takes to prepare one batch for the accelerator, from read through batching.
$\AcceleratorMemoryBandwidth$	Accelerator Memory Bandwidth	bytes/s	Sustained rate at which bytes move between an accelerator’s own memory, such as HBM, and its compute units.
$\TimeInstant$	Time Instant	seconds	A point in continuous wall-clock time, the variable over which time-varying quantities are integrated.
$\InferenceLatency$	Inference Latency	seconds	Time the model takes to compute a request’s output.
$\IntermediateTensorSize$	Intermediate Tensor Size	bytes	Bytes in the intermediate tensor that a sequence of operations passes between its steps.
$\NonzeroCount$	Nonzero Count	Integer	Number of nonzero values a sparse matrix format stores.
$\SparseMatrixRowCount$	Sparse Matrix Row Count	Integer	Number of rows in a sparse matrix. Compressed sparse row (CSR) storage uses one more row pointer than the number of rows.
$\DecodeTokenTimeFloor$	Decode Token Time Floor	seconds	Lower bound on the time to generate one token at batch one, bytes streamed per step divided by memory bandwidth.
$\QueueingLatency$	Queueing Latency	seconds	Time a request waits in queues before it is served.
$\PreprocessingLatency$	Preprocessing Latency	seconds	Time spent preparing a request’s input before inference.
$\PostprocessingLatency$	Postprocessing Latency	seconds	Time spent turning a model’s output into the response.
$\NumericBitWidth$	Numeric Bit Width	Integer	Number of bits used to represent one numeric value, such as an arithmetic operand or a quantized weight or activation.
$\DatasetTransferTime$	Dataset Transfer Time	seconds	Ideal line-rate time to move a dataset over a network or storage path: data volume divided by bandwidth.
$\AcceleratorBatchTime$	Accelerator Batch Time	seconds	Time the accelerator takes to run one training step on one batch, excluding input preparation.

Statistical and Operational Notation

Statistical thresholds, workload fractions, and system constraints follow standard statistical and operational conventions. Each symbol in the table below carries an explicit domain and unit so that hypothesis tests, queueing bounds, and fairness metrics can be combined without ambiguity:

Symbol	Definition	Type or unit	Notes
$\InterAnnotatorAgreement$	Cohen’s Kappa	Dimensionless	Inter-annotator agreement coefficient $\InterAnnotatorAgreement \in [-1, 1]$.
$\CompletionDeadline$	Deadline	seconds	Latency deadline for query completion or batch execution.
$\OperationalOverheadFraction$	Operational Overhead Fraction	Dimensionless	Fraction of wall time spent outside productive step execution ($\OperationalOverheadFraction$).
$\ParallelFraction$	Parallel Fraction	Dimensionless	Fraction of workload execution that can be parallelized in Amdahl’s Law ($0 \le \ParallelFraction \le 1$).
$\SignificanceLevel$	Significance Level	Dimensionless	Statistical significance threshold $\SignificanceLevel$ for hypothesis tests.
$\NumericalStabilityConstant$	Numerical Stability Constant	Scalar	Small numerical constant $\NumericalStabilityConstant > 0$ for numerical stability.
$\WorkerCount$	Worker Count	Integer	Number of parallel worker threads, processes, or devices.
$\ObservationTimestamp$	Observation Timestamp	seconds	Time at which a labeled observation, such as a churn event, occurred. Upper bound on the feature timestamps an as-of join may use.
$\ErroneousPredictionCost$	Erroneous Prediction Cost	dollars	Average business cost incurred by one erroneous prediction. Multiplies the skew rate and query volume in the training-serving skew cost.
$\StorageAccessFactor$	Storage Access Factor	Dimensionless	Fraction of peak storage throughput a workload’s access pattern achieves.
$\SubgroupIndex$	Subgroup Index	Index	Label of one population subgroup whose outcomes are compared against a reference group.
$\FeedbackIntervalCount$	Feedback Interval Count	Integer	Number of fixed feedback intervals, such as retraining cycles, elapsed since the disparity was first measured.
$\BiasFeedbackFactor$	Bias Feedback Factor	Dimensionless	Factor by which a group’s disparity is multiplied over one feedback interval, fitted from observed disparities. Above 1 the feedback loop is self-reinforcing; at or below 1 it is stable or damped.
$\GroupDisparity$	Group Disparity	Dimensionless	Size of the performance gap between one group and a reference group; the feedback analysis tracks how it grows from one feedback interval to the next.
$\AnonymitySetSize$	Anonymity Set Size	Integer	Smallest number of records that share every quasi-identifier value in a k-anonymous release.
$\DeploymentDistribution$	Deployment Distribution	Distribution	Joint distribution of inputs and labels a deployed model must serve, the population a verification guarantee is stated over.
$\TaskDistance$	Task Distance	Function	Task-specific distance between a model’s prediction and the correct label, the error measure a verification guarantee bounds.
$\TaskErrorTolerance$	Task Error Tolerance	Dimensionless	Largest task distance between a prediction and its label that still counts as an acceptable output.
$\ToleratedFailureProbability$	Tolerated Failure Probability	Dimensionless	Maximum allowed probability that a deployed model exceeds its task error tolerance. The confidence level used to estimate this probability is specified separately.
$\TrainingServingSkew$	Training Serving Skew	Dimensionless	Expected absolute difference between the serving-path and training-path model scores on the same inputs, over a stated comparison-input distribution. Measurable without labels; a risk signal, not an accuracy-loss equation.
$\ServingPathPredictor$	Serving Path Predictor	Function	Serving code path viewed as a function from a raw input to the model’s output, including its preprocessing and runtime.
$\TrainingPathPredictor$	Training Path Predictor	Function	Training code path viewed as a function from a raw input to the model’s output, as offline validation evaluates it.
$\TargetCompletionQuantile$	Target Completion Quantile	Dimensionless	Required probability that a request completes within its latency deadline, the completion quantile a service level states.

Units and Precision

Physical units: This book uses SI (metric) units throughout, including meters, kilograms, seconds, watts, and °C, consistent with standard engineering and scientific practice. Where source data was originally reported in imperial units, the book converts to SI and notes the original values parenthetically. A space always separates the number from the unit (for example, 100 ms, 2 TB/s).
Data and memory: This book uses decimal SI prefixes only: KB = $10^3$ bytes, MB = $10^6$, GB = $10^9$, TB = $10^{12}$. Binary-prefixed units do not appear in prose; model sizes, activation footprints, throughputs, and transfer volumes are reported in decimal units (for example, 2 TB/s, 102 MB). Hardware memory capacities retain the vendor’s nameplate figure and label. Calculations requiring an exact byte count use the device’s documented capacity.
Compute: The notation distinguishes operation counts from rates. Total work uses FLOPs (for example, GFLOPs, TFLOPs), while throughput uses FLOP/s with decimal prefixes (for example, GFLOP/s, TFLOP/s).
- 1 TFLOP = $10^{12}$ FLOPs
- 1 TFLOP/s = $10^{12}$ FLOPs per second
- Arithmetic intensity conventionally uses FLOP/byte as a unit ratio (floating-point operations per byte moved). This is a ratio unit, not a total-work symbol or throughput symbol.
Power: Power carries a subscript ($\AveragePower$, $\DevicePower$, $\ClusterPower$); bare $\Parameters$ remains parameter count.
Latency percentiles: Written p50, p95, and p99 in upright type, inside math as well as in prose.
Currency: Dollar amounts use the dollar sign ($); unless otherwise noted, dollar-denominated costs are U.S. dollars (USD).
Precision:
- FP64: Double precision (8 bytes)
- FP32: Single precision (4 bytes)
- TF32: TensorFloat-32 (19-bit Tensor Core compute mode; inputs and storage remain FP32)
- FP16: Half precision (2 bytes, standard range)
- BF16: Brain float (2 bytes, wide dynamic range)
- FP8: Quarter precision (1 byte, E4M3 or E5M2 format)
- FP4: 4-bit floating-point format
- INT8: 8-bit integer (1 byte)
- INT4: 4-bit integer; lower integer precisions follow the same uppercase INTn pattern (for example, INT3, INT2)

Quick Reference: Resolving Collisions

ML conventions generally take precedence for single letters, while systems quantities use subscripts or descriptive symbols. This preserves familiar ML notation while distinguishing systems quantities across the most common notation collisions in machine learning systems literature:

Symbol	ML Meaning	Systems Meaning	Book Convention
$\BatchSize$	Batch Size	Bandwidth	Batch Size. Use $\Bandwidth$ for bandwidth.
$\Parameters$	Parameter Count	Peak FLOP/s	Parameter Count. Use $\PeakRate$ for peak rate.
$\DatasetSize$	Dataset Size	Data Volume	Dataset Size. Use $\DataVolume$ for bytes moved.
$\Glyph{L}$	Loss	Latency	Loss $(\LossFunction)$. Use $\Latency$ for latency.
$\LearningRate$	Learning Rate	Efficiency	Learning Rate. Use $\EfficiencyHw$ for efficiency.