Interactive Labs

Lab 01: The AI Triad
Step 1 · Read the Scenario
Incoming Message · CTO
“Our model accuracy dropped 8%. The infra team wants 4× more GPUs. The data team says training data is 18 months old. Who is right?”
— Dr. Priya Mehta, MedVision Health
→
Step 2 · Commit Your Prediction
The team buys 4× more GPUs.
What happens to accuracy?
Improves ~4× (proportional)
Improves modestly (~1.3×)
No change — bottleneck elsewhere
Simulator locked until you predict
→
Step 3 · See What Actually Happens
✔ Correct
The bottleneck is stale data, not compute. 4× GPUs at $120K each = $0 accuracy gain.
D-A-M Diagnostic — ResNet-50
Data
Algorithm
Machine
Binding
Adequate
Surplus
LAB 01 · The AI Triad | PARTS 4 + Synthesis | STATUS ready Click anywhere to launch →
Launch Lab 01 → Start Orientation Share Feedback
Back to top

Lab 01 is one instance of a pattern that every lab repeats.

How Labs Work

Each lab asks you to test a quantitative result that is often surprising. The design rests on one observation. A student who predicts wrong and then discovers why has learned more than a student who reads a correct answer. The prediction lock makes that possible. You cannot passively watch the simulator; you have to commit first.

The Predict-Discover-Explain Cycle

Every part within every lab follows the same rhythm:

  1. Stakeholder Scenario: A fictional but realistic message from a CTO, VP of Engineering, or ML lead frames a real-world problem, of the kind engineers decide in practice.

  2. Prediction Lock: Before seeing any data, you must commit a structured prediction (multiple choice or numeric estimate). The simulator is locked until you predict. This forces you to surface your assumptions.

  3. Interactive Instruments: Sliders, toggles, and charts powered by the mlsysim physics engine let you explore the design space. Every number traces to a specific textbook claim, with no unexplained constants.

  4. Prediction Reveal: The lab shows you what you predicted versus what actually happened, with specific numbers: “You predicted 2\(\times\). Actual: 50\(\times\). You were off by 25\(\times\).” This gap is the learning moment.

  5. Math Peek: A collapsible accordion reveals the governing equation. You can always see the physics behind the simulator.

Structure of Each Lab

Briefing          ~2 min    Learning objectives, prerequisites, core question
Part A            ~12 min   Calibration --- correct a wrong prior with data
Part B            ~12 min   Deepening --- quantify the mechanism behind Part A
Part C            ~12 min   Cross-context --- same system, different hardware
Part D            ~12 min   Design challenge --- make a decision with trade-offs
Synthesis         ~5 min    Key takeaways, connections, self-assessment

At least one part includes a failure state: push a slider too far and the system crashes (OOM, SLA violation, thermal throttle). These failures are reversible and instructive: the point is to find the boundary, not to punish.

The Design Ledger

Your predictions and design decisions persist across labs in the Design Ledger, a browser-based save system. Lab 08’s training memory budget builds on Lab 05’s activation analysis, which builds on Lab 01’s magnitude calibration. The capstone labs (Lab 16 in Volume I and Lab 17 in Volume II) synthesize your full Design Ledger into a portfolio.

Lab Inventory

Volume I: Foundations

I. Foundations

Lab 00 · Orientation
The Architect's Portal

How do these labs work? A 5-minute walkthrough of the predict-discover-explain ritual every lab follows.
Lab 01 · Introduction
The AI Triad: Data, Algorithm, Machine

If a model fails for three different physical reasons on three hardware targets, how do you diagnose which axis to fix?
Lab 02 · System Paradigms
Physics of Deployment

If you double compute power, why does latency not halve?
Lab 03 · ML Lifecycle
The Constraint Tax

Why does discovering a deployment constraint late cost 16× more than finding it early?
Lab 04 · Data Engineering
Data Gravity & Ingestion

When is moving compute to data cheaper than moving data to compute?

II. Build

Lab 05 · Neural Computation
The Activation Tax

ReLU and Sigmoid produce similar accuracy, so why does activation memory dictate cache fit?
Lab 06 · Network Architectures
The Architecture Tax

Why does self-attention scale quadratically O(N²) while convolutions scale linearly?
Lab 07 · ML Frameworks
The Framework Tax & Kernel Fusion

Why does compiled execution run 17× faster than eager mode without changing a single weight?
Lab 08 · Model Training
The Training Gauntlet

Why does a 7B parameter model need 112 GB of memory before storing a single activation?

III. Optimize

Lab 09 · Data Selection
The Selection Paradox

When does curating data produce more accuracy per dollar than adding more raw data?
Lab 10 · Model Compression
The Compression Frontier

Can you compress a model 4× without losing accuracy? Where is the cliff?
Lab 11 · Hardware Acceleration
The Hardware Roofline

Is your workload compute-bound or memory-bound, and why does the answer change the design?

IV. Deploy

Lab 12 · Benchmarking
The Benchmarking Trap

Amdahl's Law says 5% sequential code limits speedup to 20× regardless of parallelism. Is that right?
Lab 13 · Model Serving
The Tail Latency Trap

Your server looks healthy at 50% utilization, so why does latency climb steeply at 80%?
Lab 14 · ML Operations
Silent Degradation & Monitoring

Your model shipped Monday. By Friday it lost 3 accuracy points while your dashboard is green. Why?

Capstone

Lab 15 · Responsible Engineering
No Free Fairness

Fairness costs accuracy, explanations cost latency, and all of it costs carbon. How do you budget?
Lab 16 · Capstone Synthesis
The Architect's Audit

Synthesize every invariant and physical constraint from the earlier labs into one deployment decision.

Volume II: Scaling

I. Foundations

Lab 01 · Introduction to Scale
The Scale Illusion

If you add 10× more GPUs, do you get 10× more throughput?
Lab 02 · Compute Infrastructure
The Compute Wall

Why does Model FLOPs Utilization rarely exceed 50% on real datacenter hardware?
Lab 03 · Network Fabrics
Network Fabric Design

When does topology, latency, bandwidth, or bisection become the real system limit?
Lab 04 · Data Storage
The Storage Wall

Can your distributed storage hierarchy feed your GPUs fast enough, or are they starving?

II. Build

Lab 05 · Collective Communication
Collective Communication

Which collective algorithm fits your topology, message size, and residual risk?
Lab 06 · Distributed Training
The Parallelism Puzzle

Data, tensor, or pipeline parallelism, and which fits your model and your cluster?
Lab 07 · Fault Tolerance
When Failure Is Routine

At 10,000 GPUs, what is the probability of zero failures in 24 hours?
Lab 08 · Fleet Orchestration
The Scheduling Trap

FIFO scheduling wastes 40% of your cluster. Can gang scheduling and DRF do better?

III. Optimize

Lab 09 · Performance Engineering
The Optimization Trap

How does FlashAttention SRAM tiling eliminate memory bandwidth bottlenecks?
Lab 10 · Inference at Scale
The Inference Economy

How does PagedAttention virtual memory eliminate KV-cache fragmentation at scale?
Lab 11 · Edge Intelligence
Edge Thermodynamics

When does moving inference to the edge save energy vs. trigger thermal throttling?

IV. Deploy

Lab 12 · Ops at Scale
The Silent Fleet

How do grey failures and stragglers silently amplify tail latency across a fleet?
Lab 13 · Security & Privacy
The Price of Privacy

Differential privacy adds noise. How much accuracy do you lose for how much privacy?
Lab 14 · Robust AI
The Robustness Budget

Adversarial training costs 8× compute. When is it worth it?
Lab 15 · Sustainable AI
The Carbon Budget

Moving your training from Iowa to Quebec cuts carbon 10×. Why?

Capstone

Lab 16 · Responsible AI
The Fairness Budget

How do you allocate a finite fairness budget across competing metrics?
Lab 17 · Capstone Synthesis
The Fleet Synthesis

Design and defend a production fleet balancing cost, latency, fairness, and carbon.

Volume IV: Physical AI

Lab 1
The Digital Twin
Lab 2
Simulation to Real Hardware
Lab 3
Learned Control
Lab 4
Real Data & Fine-Tuning
Lab 5
Stress-Testing
Lab 6
Safety Governor
Capstone
Physical Release Defense

Run Offline

Optional: Run Offline

The labs already run in your browser, so nothing needs to be installed. Users who want offline access or want to modify the simulations can install the package:

python3 -m pip install -r labs/requirements.txt
python3 -m pip install -e mlsysim
cd labs
marimo run vol1/lab_01_ml_intro.py

Part of the MLSysBook Ecosystem

Textbook

A four-volume open textbook on machine learning systems.

Lecture Slides

Beamer lecture decks with speaker notes for each volume and TinyML.

Labs

Interactive Marimo notebooks that measure the book's claims.

Hardware Kits

Hands-on embedded ML deployment on real devices.

Instructor Hub

Course maps, syllabi, and adoption resources for teaching.

TinyTorch

A machine learning framework built from scratch, one module at a time.

MLSys·im

The analytical modeling engine behind the book's quantitative figures.