Introduction to Machine Learning Systems

Syllabus · Semester plan

Textbook Volume I: Introduction to Machine Learning Systems
Prerequisites Python, linear algebra, basic calculus, and introductory probability
Format A 16-week semester or a 10-week quarter, with two 75-minute lectures per week
Materials Slides · Labs · TinyTorch (instructor guide for autograding) · no GPUs needed

Description

Students study how data pipelines, training, optimization, and deployment work on a single machine with up to eight accelerators. Along the way they build a small deep learning framework, TinyTorch, one module at a time.

Learning outcomes

By the end of the course, students can:

  • Build a small deep learning framework that trains a model.
  • Measure where time and memory go in a training or inference workload.
  • Choose optimizations, such as compression, against deployment constraints.
  • Deploy and evaluate a model against latency, memory, and quality targets.

Reading schedule

In the semester plan, students read one Volume I chapter each week; the quarter plan combines some weeks. Each chapter has a matching lab, and the first eleven weeks also have a TinyTorch module. The instructor decides which labs and modules to assign.

Week Topic Reading Slides
Part I: The Physics of AI
1 Why ML Systems? Introduction Ch 1
2 The ML Systems Landscape System Paradigms Ch 2
3 The ML Lifecycle ML Lifecycle Ch 3
4 Data Engineering Data Engineering Ch 4
Part II: Building the Stack
5 Neural Network Computation Neural Computation Ch 5
6 Neural Network Architectures NN Architectures Ch 6
7 ML Frameworks ML Frameworks Ch 7
8 Training Training Ch 8
Part III: Optimization
9 Data Selection and Curation Data Selection Ch 9
10 Model Compression Model Compression Ch 10
11 Hardware Acceleration Hardware Acceleration Ch 11
12 Benchmarking Benchmarking Ch 12
Part IV: Deployment & Production
13 Model Serving Model Serving Ch 13
14 ML Operations ML Operations Ch 14
15 Responsible Engineering Responsible Engineering Ch 15
16 Capstone: The AI Olympics Conclusion Ch 16

Four pairs of weeks are combined. Benchmarking (week 12) and Responsible Engineering (week 15) become optional reading.

Week Topic Reading Slides
1 Why ML Systems? + The ML Systems Landscape Introduction · System Paradigms Ch 1 · Ch 2
2 The ML Lifecycle ML Lifecycle Ch 3
3 Data Engineering + Neural Network Computation Data Engineering · Neural Computation Ch 4 · Ch 5
4 Neural Network Architectures NN Architectures Ch 6
5 ML Frameworks + Training ML Frameworks · Training Ch 7 · Ch 8
6 Data Selection and Curation Data Selection Ch 9
7 Model Compression Model Compression Ch 10
8 Hardware Acceleration Hardware Acceleration Ch 11
9 Model Serving + ML Operations Model Serving · ML Operations Ch 13 · Ch 14
10 Capstone: The AI Olympics Conclusion Ch 16

Assessment

A suggested split; each offering sets its own.

Component Weight
TinyTorch modules (8, auto-graded) 35%
Lab Decision Logs (short written analyses) 25%
Design challenges (the open-ended part of the labs) 20%
Capstone: deploy one application on several hardware targets, with a design report 20%

Suggested papers

Optional readings that pair with the chapters.

Week Topic Paper
4 Data Engineering Sambasivan et al., “Everyone Wants to Do the Model Work, Not the Data Work” (CHI 2021)
7 ML Frameworks Chen et al., “TVM: An Automated End-to-End Optimizing Compiler” (OSDI 2018)
10 Model Compression Dettmers et al., “LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale” (NeurIPS 2022)
11 HW Acceleration Jouppi et al., “In-Datacenter Performance Analysis of a Tensor Processing Unit” (ISCA 2017)
13 Model Serving Yu et al., “Orca: A Distributed Serving System for Transformer-Based Models” (OSDI 2022)
14 ML Operations Sculley et al., “Hidden Technical Debt in Machine Learning Systems” (NeurIPS 2015)

Adapting this syllabus

  • Each offering sets its own dates, deadlines, and policies on late work, collaboration, and AI tools.
  • Two semesters: this course leads into Scaling Machine Learning Systems.
Back to top