Pipeline Pacer

Keep the GPUs fed without bubbling.

Press Space or tap to admit microbatches. Keep the GPUs busy without filling activation memory.
throughput 0 · score 0 tap to schedule FWD · watch memory limits · R retry

How to play

  1. Tap the canvas or press Space to launch a microbatch into the pipeline.
  2. Launch quickly enough to avoid idle bubbles, but leave memory headroom for activations waiting on backward.
  3. Survive the timer with as many completed passes as possible. Press R to retry.

The Systems Concept

Pipeline parallelism is a scheduling tradeoff. More microbatches hide bubbles and improve device utilization, but each in-flight forward pass keeps activations alive until backward consumes them. Good schedules keep GPUs busy without turning memory into the bottleneck.

Part of MLSysBook Playground. Found a bug? Report an issue.