Milestone 01: The Perceptron (1958)

NoteMilestone Info

Foundation Milestone | Difficulty: ●○○○ | Time: 15–30 min | Prerequisites: Modules 01–03

TipWhat You’ll Learn
  • Why random weights produce random results
  • How tensors, activations, and layers form a perceptron
  • Why training infrastructure is needed before models can learn

Overview

You just finished the forward-pass foundation. Your Tensor (Module 01), activations (Module 02), and Linear layer (Module 03) are working. This milestone runs the simplest possible model those pieces can drive — and the one that started the field.

It’s 1958. Computers fill entire rooms and can barely add numbers. Then Frank Rosenblatt makes an outrageous claim: he’s built a machine that can learn from examples. Not through hand-written rules, but through adjustable weights.

The press goes wild. The Navy funds research expecting machines that will “walk, talk, see, write, reproduce itself and be conscious of its existence.” The New York Times runs the headline: “New Navy Device Learns by Doing.”

The optimism was premature. The insight wasn’t. You’re about to recreate the moment machine learning was born — with components YOU built yourself.

What You’ll Build

A single-layer perceptron for binary classification that demonstrates the forward path before training enters the curriculum:

  1. The Architecture — tensors flow through a Linear layer and sigmoid activation
  2. The Limitation — random weights produce random predictions (~50% accuracy)
Input (features) --> Linear --> Sigmoid --> Output (0 or 1)

Prerequisites

Table 1 lists the modules you need to have completed before starting.

Table 1: Prerequisite modules for the Perceptron milestone.
Module Component What It Provides
01 Tensor YOUR data structure
02 Activations YOUR sigmoid activation
03 Layers YOUR Linear layer

Running the Milestone

Finish Modules 01–03 first. Check your progress:

tito module status
cd milestones/01_1958_perceptron

# See the forward-pass problem
python 01_rosenblatt_forward.py
# Expected: ~50% accuracy (random guessing)

# Or run it from the TinyTorch project root
tito milestone run 01

Expected Results

Table 2 records the accuracy and runtime you should expect to see.

Table 2: Expected accuracy for the Perceptron milestone scripts.
Script Accuracy What It Shows
01 (Forward Only) ~50% Random weights = random guessing

The Aha Moment: Learning IS the Intelligence

You’ll run a single forward-only script using YOUR Linear layer and YOUR sigmoid. It should land near 50% accuracy because the weights are still random.

What’s missing? Not the model. Not the data. The learning loop.

output = model(input)           # YOUR code computes
loss = loss_fn(output, target)  # YOUR code measures
# No backward(), no optimization, no learning
# Result: Random weights stay random

Run the script and watch YOUR Linear layer make random guesses — near coin-flip accuracy. That failure is the point: the architecture is ready, but learning requires the modules that come next.

You just recreated the forward pass of Rosenblatt’s perceptron. The next milestones add the training machinery that turns this architecture into a learner.

Your Code Powers This

Table 3 names the TinyTorch components that power this milestone.

Table 3: TinyTorch components that power the Perceptron milestone.
Component Your Module What It Does
Tensor Module 01 Stores inputs and weights
Sigmoid Module 02 YOUR activation function
Linear Module 03 YOUR fully-connected layer

Historical Context

Rosenblatt didn’t just publish — he built. The Mark I Perceptron was custom hardware: a 20×20 grid of photocells wired to motor-driven potentiometers that physically adjusted the weights. The 1958 paper established the two ideas under every modern model: trainable weights and error-driven learning. Eleven years later, Minsky and Papert’s Perceptrons (1969) proved single-layer networks couldn’t learn XOR. Funding collapsed. The first AI winter began.

Systems Insights

  • Memory: O(n) parameters for n input features
  • Compute: O(n) operations per sample
  • Limitation: Can only solve linearly separable problems

What’s Next

Linear separability — the Perceptron’s hard ceiling — sparked the first AI winter. Milestone 02 runs your network on XOR and watches that limit appear.

Further Reading

Back to top