Volume II

Scaling Machine Learning Systems

Lecture decks for Volume II, which takes the systems of Volume I to fleets of machines.

All PDFs (ZIP) All PowerPoint (ZIP) Source

Introduction

Course OverviewEngineering the fleet behind one large modelPDF PowerPoint Source

IntroductionWhy adding machines does not divide the timeRead the chapterPDF PowerPoint Source

Part I: The Fleet

Compute InfrastructureFrom one accelerator to a podRead the chapterPDF PowerPoint Source

Network FabricsHow collectives complete on a real fabricRead the chapterPDF PowerPoint Source

Data StorageFeeding and saving the fleetRead the chapterPDF PowerPoint Source

Part II: Distributed ML

Collective CommunicationPricing the exchanges of a training stepRead the chapterPDF PowerPoint Source

Distributed TrainingSplitting one training job across many acceleratorsRead the chapterPDF PowerPoint Source

Fault ToleranceCheckpointing and recovery when failure is routineRead the chapterPDF PowerPoint Source

Fleet OrchestrationWho gets which accelerators, when, and whereRead the chapterPDF PowerPoint Source

Part III: The Fleet in Production

Fleet EfficiencyWhere a fleet's capacity goesRead the chapterPDF PowerPoint Source

Serving at ScaleServing language models within latency targetsRead the chapterPDF PowerPoint Source

Edge IntelligenceLearning on the devices the fleet servesRead the chapterPDF PowerPoint Source

ML Operations at ScaleOperating hundreds of models as one graphRead the chapterPDF PowerPoint Source

Part IV: The Responsible Fleet

Security and PrivacyAttacks through the interfaces a model exposesRead the chapterPDF PowerPoint Source

Robust AIDiagnosing drift, attacks, poisoning, and silent faultsRead the chapterPDF PowerPoint Source

Sustainable AIPower, energy, and carbon for a fleetRead the chapterPDF PowerPoint Source

Responsible AIMeasured obligations for models that decide about peopleRead the chapterPDF PowerPoint Source

Conclusion

ConclusionFollowing the binding constraint across the fleetRead the chapterPDF PowerPoint Source

Read the matching chapter at mlsysbook.ai/vol2.

Tip

PPTX files are image-based (300 DPI) and visually identical to the PDF. Use them for PowerPoint presenter mode and slide annotations. For editable slides, download the LaTeX source.