Machine Learning Systems

An open textbook series on machine learning systems by Vijay Janapa Reddi (Harvard University), free to read online: Introduction to Machine Learning Systems and Scaling Machine Learning Systems.
Keywords

Machine Learning Systems, MLSys Book, AI Engineering, Physical AI, Agentic AI Systems, Distributed Machine Learning, Edge AI, TinyML, Harvard CS249r, Machine Learning Systems Textbook

FOUR-VOLUME SERIESOPEN TEXTBOOK SERIES

Machine Learning
Systems.

The physics of AI engineering.

A rigorous, principles-first treatment of how ML systems are built, optimized, and deployed, from single machines and distributed scale to autonomous agents and physical embodimentto distributed scale.

Harvard University

Actively maintained Last updated October 2026 Release notes

Volume I cover

Volume I

Introduction to Machine Learning SystemsThe MIT Press · 2026
Formats
HTML
Preview Volume II cover

Volume II

Scaling Machine Learning SystemsThe MIT Press · 2027
Formats
HTML
In the works Two more volumes are taking shape. Volumes III and IV carry the series further. Be the first to read the preview. Get notified →
In preparation Volume III cover

Volume III

Agentic Machine Learning Systems
Formats
HTML
In preparation Volume IV cover

Volume IV

Physical AI
Formats
HTML
Explore the Curriculum
↓

A complete curriculum for AI engineering.

Choose a path: read the books, explore trade-offs in labs, build the internals with TinyTorch, model constraints with MLSys·im, deploy on real hardware, practice with StaffML, or plan a course with the Instructor Hub.

Play a short systems game →

For Students & Learners

EXPLORE

Labs

Interactive Marimo notebooks in which students change a parameter and watch which constraint binds.

Lab 17 · Fleet Synthesis Explore

BUILD

TinyTorch

An ML framework built from scratch in 20 modules, each adding one part of the framework.

tinytorch — tensor.py class Tensor: def __init__(self, data): self.data = data self.grad = 0.0 self._backward = lambda: None

MODEL

MLSys·im

Analytical performance models of training and inference, from microcontrollers to datacenter clusters.

$ mlsysim eval Llama3_70B H100 --batch-size 1 mem-bound compute-bound b=1 b=32 b=128 Arithmetic Intensity FLOP/s

DEPLOY

Hardware Kits

Labs that deploy ML models to Arduino, Seeed, Grove, and Raspberry Pi boards, within their memory and power limits.

Arduino · Seeed · Grove · Raspberry Pi

For Career & Instructors

PRACTICE

StaffML

Interview questions for ML systems roles, with a question vault, drills, and mock interviews.

Systems Design L5 · Staff A 70B model needs 1,000 req/s. Walk through your hardware selection and parallelism strategy. Hardware Parallelism Trade-offs Cloud Edge Mobile TinyML

ADOPT

Instructor Hub

A syllabus for each course, with weekly readings, slides, and labs.

Instructor Hub Courses · Syllabi · Teaching Guidance Syllabi Introduction · Scaling · TinyML More Courses Agentic · Physical AI · Edge AI Reading Schedules Week · Topic · Reading Lecture Slides A deck for every chapter READY

TEACH

Lecture Slides

44 Beamer lecture decks with speaker notes and 1,502 original SVG diagrams, ready to teach or adapt.

Intro Systems DNN Training Accel Deploy Ethics The Iron Law of ML Systems T = D/BW + O/(R·η) + L Data Term — memory bandwidth Compute Term — utilization η ≤ 0.7 Latency Term — orchestration overhead Harvard University · ML Systems 12 / 38

FOLLOW

Newsletter

Updates on the curriculum, new chapters, and what the community is building.

MLSysBook Weekly 4 New: Vol II Ch. 7 — Fault Tolerance Updated: TinyTorch Module 20 Community: 1,100+ PRs merged Milestone: 28,000+ GitHub stars Join 1,100+ subscribers
Support the Mission
↓

OUR MISSION

AI education should be
free and open to everyone.

Everyone calls AI the new electricity, but electricity is useless without engineers who can build the grid. For AI to be efficient, reliable, and safe, the world needs engineers who understand how to build it.

That knowledge should be accessible to anyone willing to learn. This curriculum is our commitment to making it so.

Live readership: 18,389 readers a month on average across 204 countries and territories

Top countries: United States · India · Singapore · China · Vietnam

313,063
readers
204
countries &
territories
18,389
avg. a month

Our goal: one million learners by 2030

Star on GitHub 28,106

Every star helps another learner find this book.
Our sponsors and partners keep it free for all of them.