Volume IV

Physical AI: Machine Learning Systems That Sense and Act

Lecture decks for Volume IV, which covers machine learning systems that sense the physical world and act on it, where a learned output becomes a physical action that cannot be taken back. Every lecture comes as a PDF for projection, a PowerPoint file whose notes pane holds the speaker notes, and the LaTeX source.

All PDFs (ZIP) All PowerPoint (ZIP) Source

Introduction

Course OverviewEngineering learned models that move machinesPDF PowerPoint Source

The Causal BoundaryWhere a learned output becomes physical workRead the chapterPDF PowerPoint Source

Part I: The Machine Anatomy

The Physical BodyThe physical budgets every action draws onRead the chapterPDF PowerPoint Source

The Cognitive BrainLearned proposers, the memory wall, and action chunksRead the chapterPDF PowerPoint Source

The Nervous SystemBounded delays from the proposer to the drivesRead the chapterPDF PowerPoint Source

Part II: Teaching the Machine

Physical DataDemonstration data that teaches a robot to recoverRead the chapterPDF PowerPoint Source

Policy TrainingWhere a trained policy can be trustedRead the chapterPDF PowerPoint Source

Closed-Loop EvaluationHow much evidence a success rate providesRead the chapterPDF PowerPoint Source

Part III: Running the Machine

Sensor PerceptionTime, frame, and error of each observationRead the chapterPDF PowerPoint Source

Spatial MemoryHow long a remembered scene may move a robotRead the chapterPDF PowerPoint Source

Grounded IntentFrom a spoken request to an expiring, bounded targetRead the chapterPDF PowerPoint Source

Trajectory PlanningMotions that carry their own timing and stopRead the chapterPDF PowerPoint Source

Safety EnforcementGranting permission on every control tickRead the chapterPDF PowerPoint Source

Workload PlacementPlacing computations so safety checks meet deadlinesRead the chapterPDF PowerPoint Source

Part IV: Governing the Machine

Supervisory InterventionHanding a moving machine to a human operatorRead the chapterPDF PowerPoint Source

Adversarial VerificationTesting behavior by injecting faults on purposeRead the chapterPDF PowerPoint Source

Deployment ReleaseDeciding when a machine may operate among peopleRead the chapterPDF PowerPoint Source

Responsible Physical AIWho answers for a machine that movesRead the chapterPDF PowerPoint Source

Conclusion

Engineering Physical AIThe open premises that set what to build nextRead the chapterPDF PowerPoint Source

Read the matching chapter at mlsysbook.ai/vol4.

Tip

PPTX files are image-based (300 DPI) and visually identical to the PDF. Use them for PowerPoint presenter mode and slide annotations. For editable slides, download the LaTeX source.