Learning Resources

TinyTorch teaches you to build ML systems. These resources help you understand the why behind what you’re building.

Companion Textbook

Machine Learning Systems

mlsysbook.ai by Prof. Vijay Janapa Reddi (Harvard University)

TinyTorch began as hands-on labs for this textbook. While TinyTorch can be used standalone, the ML Systems book provides the theoretical depth and production context behind every module you build.

What it teaches: Systems engineering for production ML—memory hierarchies, performance optimization, deployment strategies, and the engineering decisions behind modern ML frameworks.

How it connects to TinyTorch:

• TinyTorch modules directly implement concepts from the book’s chapters

• The book explains why PyTorch, TensorFlow, and JAX make certain design decisions

• Together, they provide both hands-on implementation and theoretical understanding

When to use it: Read in parallel with TinyTorch. When you implement Module 05 (Autograd), read the book’s chapter on automatic differentiation to understand the systems engineering behind your code.

Related Academic Courses

• CS 329S: Machine Learning Systems Design (Stanford) Production ML systems and deployment

• TinyML and Efficient Deep Learning (MIT 6.5940) Edge computing, model compression, and efficient ML

• CS 249r: Tiny Machine Learning (Harvard) TinyML systems and resource-constrained ML

• CS 231n: Convolutional Neural Networks (Stanford) Computer vision - complements TinyTorch Modules 08-09

• CS 224n: Natural Language Processing (Stanford) Transformers and NLP - complements TinyTorch Modules 10-13

Other Textbooks

• Deep Learning by Goodfellow, Bengio, Courville Mathematical foundations behind what you implement in TinyTorch

• Hands-On Machine Learning by Aurélien Géron Practical implementations using established frameworks

Minimal Frameworks

Alternative approaches to building ML from scratch:

• micrograd by Andrej Karpathy Autograd in 100 lines. Perfect 2-hour intro before TinyTorch.

• nanoGPT by Andrej Karpathy Minimalist GPT implementation. Complements TinyTorch Modules 12-13.

• tinygrad by George Hotz Performance-focused educational framework with GPU acceleration.

Production Framework Internals

• PyTorch Internals by Edward Yang How PyTorch actually works under the hood

• PyTorch: Extending PyTorch Custom operators and autograd functions

Ready to start? See the Quick Start for a 15-minute hands-on introduction.