Skip to main content

For Instructors

Reproducible, hardware-independent problem sets, paired with the lecture decks for Volumes I and II.

MLSys·im provides a framework for assigning analytically grounded problem sets where every answer is deterministic and reproducible, regardless of the hardware students have access to. Combined with the companion lecture slides, it covers the tools needed for an ML systems course.

Why MLSys·im for Teaching

Challenge How MLSys·im Helps
Students lack GPU access All analysis runs on a laptop, with no cloud credits needed
Homework answers vary by hardware Vetted registry specs produce identical results everywhere
Hard to grade open-ended systems questions Analytical solvers give deterministic, verifiable outputs
Specifications become stale Registry updated from official datasheets; one update propagates everywhere
Students memorize without understanding Analytical walkthroughs and parameter sweeps build genuine intuition
No time to build slides from scratch Beamer lecture decks for Volumes I and II, with speaker notes, in-class exercises, and SVG diagrams

The Teaching Ecosystem

MLSys·im is one component of a larger open teaching toolkit:

Resource What It Provides Link
Textbook Four-volume open textbook: Volumes I and II online (Volume II as a preview), Volumes III and IV in preparation mlsysbook.ai
Lecture Slides Beamer lecture decks for Volumes I and II, with speaker notes on every slide Slides Portal
MLSys·im Six primary workflows backed by typed resolver classes, a hardware registry, and deterministic assignments Getting Started
TinyML Courseware A four-course sequence of slide decks for embedded ML TinyML Slides
Teaching Guide Semester plan, in-class activity types, and how to customize the decks Teaching Guide

Course Integration Patterns

Pattern 1: Textbook Companion (Full Semester)

Map MLSys·im tutorials and assignments directly to textbook chapters and lecture decks. The table below shows one possible 16-week arrangement using Volume I slides.

Week Lecture Slides Textbook Topic MLSys·im Assignment
2 Introduction The Iron Law of ML Systems Read Hello, Roofline warmup: identify bottleneck equation
5 NN Computation FLOPs, memory footprint Hello, Roofline: roofline analysis, batch size sweep
8 Model Training Training memory budget Solver Guide: TrainingMemoryModel, ZeRO stages
11 HW Acceleration Roofline model, accelerator comparison Hardware comparison assignment (see below)
13 Model Serving TTFT, ITL, KV-cache Two Phases of Inference: serving latency analysis

For a Volume II course on distributed systems:

Week Lecture Slides Textbook Topic MLSys·im Assignment
3 Compute Infrastructure GPU clusters, interconnects TCO analysis with EconomicsModel
6 Distributed Training 3D parallelism, scaling Scaling to 1000 GPUs: parallelism strategies
7 Fault Tolerance Checkpointing, MTBF ReliabilityModel: Young-Daly checkpoint interval
10 Performance Engineering Profiling, optimization Multi-solver composition (see capstone ideas below)
15 Sustainable AI Energy, carbon, water Geography is a Systems Variable: carbon footprint
TipSemester Plans

The Teaching Guide describes how the Volume I decks fit a semester.

Pattern 2: Standalone Labs

Use individual tutorials as self-contained lab assignments in any systems course. Each tutorial is an executable walkthrough with live code and analytical verification:

Tutorial Duration Key Concepts Example mlsysim Output Pairs With Slides
Hello, Roofline 15 min Roofline model, memory vs. compute bound A100 to H100 ResNet-50 speedup: 1.08× (batch 1) vs. 2.94× (batch 256) HW Acceleration
Geography is a Systems Variable 20 min Energy, carbon footprint, regional grids Iowa (680 gCO₂/kWh) vs. Quebec (20 gCO₂/kWh): 34× carbon gap Sustainable AI
Two Phases of Inference 25 min TTFT vs. ITL, KV-cache pressure Llama-3.1-70B KV-cache at 4K context: 1.34 GB per request Model Serving
Scaling to 1000 GPUs 30 min Data/tensor/pipeline parallelism 3D parallelism scaling efficiency across cluster tiers Distributed Training

Pattern 3: Capstone Projects

Advanced students compose multiple solvers to answer research-style questions. See Writing a Custom Solver for the custom solver API.

Assignment Ideas

Homework: Hardware Comparison (30 min)

Using Engine.solve(), compare ResNet-50 inference latency on the A100, H100, and Jetson AGX at batch sizes 1, 32, and 256. For each configuration, state whether the workload is memory-bound or compute-bound and explain why the bottleneck shifts with batch size.

Pairs with: HW Acceleration slides (roofline model, ridge point) and Benchmarking slides (measurement methodology).

Homework: Training Memory Budget (30 min)

Using TrainingMemoryModel, calculate the memory required to train GPT-2 (1.5B parameters) in FP16 with Adam optimizer under ZeRO Stage 0, Stage 1, and Stage 3. Explain why each stage reduces memory and what trade-off it introduces.

Pairs with: Model Training slides and Distributed Training slides.

Lab: Carbon-Aware Training (45 min)

Using the SustainabilityModel, calculate the carbon footprint of training GPT-3 on a 256-GPU H100 cluster in Quebec (20 gCO₂/kWh) vs. US Average (429 gCO₂/kWh) vs. Poland (820 gCO₂/kWh, 41× Quebec). Produce a table and a 2-paragraph analysis of why datacenter location matters more than hardware choice for carbon.

Pairs with: Sustainable AI slides (grid carbon intensity, PUE).

Lab: LLM Serving Capacity Planning (45 min)

Using the ServingModel, determine the maximum sequence length at which Llama-3.1-70B can serve a single request on an 8-GPU H100 node without exceeding memory. Then calculate TTFT and ITL at sequence lengths of 1K, 4K, and 16K tokens, and identify the sequence length at which KV-cache pressure dominates.

Pairs with: Model Serving slides and Inference at Scale slides.

Exam Question: Back-of-Envelope

A GPU has 989 TFLOP/s peak dense compute (FP16) and 3.35 TB/s memory bandwidth. Compute (a) the ridge point in FLOP/Byte (295 FLOP/Byte), (b) whether a model layer with arithmetic intensity of 50 FLOP/Byte is compute-bound or memory-bound, and (c) which regime a layer with arithmetic intensity of 400 FLOP/Byte occupies and what that implies about the benefit of moving to a GPU with 2× the bandwidth. Show all intermediate steps.

Pairs with: HW Acceleration slides (roofline model, ridge point derivation).

Capstone: Multi-Solver Design Study (1 week)

Design a training cluster for a 70.6B-parameter model. Use the DistributedModel to select a parallelism strategy, the EconomicsModel for TCO over 6 months, the SustainabilityModel to compare three datacenter locations, and the ReliabilityModel to determine checkpoint frequency. Present the analysis as a 3-page technical memo with quantitative justification for each decision.

Pairs with: the full Volume II slide set, covering infrastructure, training, fault tolerance, and sustainability.

Grading Notes

Because MLSys·im produces deterministic output from vetted specifications:

  • Answer keys are stable: the same mlsysim version produces identical numbers for every student, every semester
  • Partial credit is straightforward: grade the reasoning (which solver, which inputs, which bottleneck explanation), not only the number
  • Analytical predictions are easy to assess: students can compute analytical bounds from the closed-form equations before running mlsysim to verify their work
NoteVersion Pinning

Pin the exact version in the assignment instructions (pip install mlsysim==<course-version>) so answer keys remain valid even after new releases update specifications.

Reproducibility Guarantee

All specifications in the MLSys Zoo are:

  • Sourced from official manufacturer datasheets and published benchmarks
  • Typed with pint.Quantity for dimensional correctness, so unit errors are caught at runtime
  • Frozen per release, so an exact pin such as mlsysim==0.3.0 always produces the same answers

An answer key therefore works for every student, every semester.

Jupyter & Quarto Compatibility

All tutorials run in:

  • Jupyter Notebooks: standard .ipynb workflow
  • Quarto documents: render to HTML, PDF, or slides with quarto render
  • Google Colab: pip install mlsysim in the first cell, then run the tutorial

No GPU runtime required. CPU-only environments are sufficient because MLSys·im computes from equations, not from profiling.

Getting Started

  1. Point students to the Getting Started guide for installation
  2. Assign the Hello, Roofline tutorial as a warmup
  3. Browse the Solver Guide to select solvers for the course topics
  4. Pair each assignment with the relevant lecture slides for classroom context
  5. Use the MLSys Zoo for available hardware, model, and infrastructure specifications
Back to top