Topology Tycoon
Build the fabric, avoid the bottlenecks.
How to play
You have 8 GPUs and 20 seconds to build their network topology. Click between two nodes to toggle their connection: None, InfiniBand (blue, slow, unlimited), or NVLink (green, fast, limited to 6). Once the run phase begins, packets will flood the network. NVLinks can handle heavy traffic, but InfiniBand links will quickly bottleneck, turning red and severely dropping throughput. Maximize your delivered GB/s!
The Systems Concept
In distributed training, Bisection Bandwidth—the worst-case bandwidth between two halves of your cluster—often becomes the primary bottleneck. A cluster’s effective throughput isn’t just about raw GPU compute; it’s heavily dictated by the fabric connecting them. Fast local connections like NVLink can handle massive all-to-all traffic, while slower cross-node links (like standard InfiniBand setups) quickly turn into traffic jams under load.
Part of MLSysBook Playground. Found a bug? Report an issue.