Milestone 01: The Perceptron (1958)

NoteMilestone Info

Foundation Milestone | Difficulty: ●○○○ | Time: 15–30 min | Prerequisites: Modules 01–03

TipWhat You’ll Learn
  • Why random weights produce random results
  • How tensors, activations, and layers form a perceptron
  • Why training infrastructure is needed before models can learn

Overview

You just finished the forward-pass foundation. Your Tensor (Module 01), activations (Module 02), and Linear layer (Module 03) are working. This milestone runs the simplest possible model those pieces can drive — and the one that started the field.

It’s 1958. Computers fill entire rooms and can barely add numbers. Then Frank Rosenblatt makes an outrageous claim: he’s built a machine that can learn from examples. Not through hand-written rules, but through adjustable weights.

The press goes wild. The Navy funds research expecting machines that will “walk, talk, see, write, reproduce itself and be conscious of its existence.” The New York Times runs the headline: “New Navy Device Learns by Doing.”

The optimism was premature. The insight wasn’t. You’re about to recreate the moment machine learning was born — with components YOU built yourself.

What You’ll Recreate

A single-layer perceptron for binary classification that demonstrates the forward path before training enters the curriculum:

  1. The Architecture — tensors flow through a Linear layer and sigmoid activation
  2. The Limitation — random weights produce arbitrary predictions
Figure 1: The 1958 Perceptron forward pass and the learning dilemma. The forward path passes input features through your Linear layer and Sigmoid activation to classify points. Without a learning loop (loss, autograd, and optimizer), weights remain random, demonstrating why learning infrastructure is essential for intelligence.

Prerequisites

Table 1 lists the modules you need to have completed before starting.

Table 1: Prerequisite modules for the Perceptron milestone.
Module Component What It Provides
01 Tensor YOUR data structure
02 Activations YOUR sigmoid activation
03 Layers YOUR Linear layer

Running the Milestone

Finish Modules 01–03 first. Check your progress:

tito module status
tito milestone run 01

Expected Results

Table 2 records the accuracy you should expect to see and what the milestone checks.

Table 2: Expected accuracy and pass condition for the Perceptron milestone scripts.
Script Accuracy Pass Condition What It Shows
01 (Forward Only) Whatever the random weights give (0% in the run we measured) YOUR output matches \(\sigma(XW + b)\) computed in NumPy from the model’s own weights The model has not learned anything

The Aha Moment: Learning IS the Intelligence

You’ll run a single forward-only script using YOUR Linear layer and YOUR sigmoid. Its accuracy is whatever the random weights happen to give, sometimes near zero and occasionally high by luck, because nothing has been learned yet.

What’s missing? Not the model. Not the data. The learning loop.

output = model(input)           # YOUR code computes
loss = loss_fn(output, target)  # YOUR code measures
# No backward(), no optimization, no learning
# Result: Random weights stay random

Run the script and watch YOUR Linear layer make random guesses. The accuracy is whatever the random line happens to give, often all right or all wrong. That failure is the point: the architecture is ready, but learning requires the modules that come next. Because the accuracy is luck, the milestone grades the arithmetic instead: it recomputes the same forward pass in NumPy from your model’s weights and exits with status 1 if YOUR Linear and Sigmoid disagree.

You just recreated the forward pass of Rosenblatt’s perceptron. The next milestones add the training machinery that turns this architecture into a learner.

Your Code Powers This

Table 3 names the TinyTorch components that power this milestone.

Table 3: TinyTorch components that power the Perceptron milestone.
Component Your Module What It Does
Tensor Module 01 Stores inputs and weights
Sigmoid Module 02 YOUR activation function
Linear Module 03 YOUR fully-connected layer

Historical Context

Rosenblatt didn’t just publish — he built. The Mark I Perceptron was custom hardware: a 20×20 grid of photocells wired to motor-driven potentiometers that physically adjusted the weights. The 1958 paper established the two ideas under every modern model: trainable weights and error-driven learning. Eleven years later, Minsky and Papert’s Perceptrons (1969) proved single-layer networks couldn’t learn XOR. Funding collapsed. The first AI winter began.

Systems Insights

  • Memory: O(n) parameters for n input features
  • Compute: O(n) operations per sample
  • Limitation: Can only solve linearly separable problems

Read why

The companion book’s chapter Synthesis I: From Perceptrons to Rumelhart’s MLP covers Milestones 01 through 03 together. Read it in TinyTorch: From Tensors to Transformers (PDF) after you have run the milestone.

What’s Next

Linear separability — the Perceptron’s hard ceiling — sparked the first AI winter. Milestone 02 runs your network on XOR and watches that limit appear.

Further Reading

Back to top