A guided experiment · about 5 minutes
How does a computer find edges?
You see a square. A computer gets a grid of numbers. Discover how comparing neighboring pixels can reveal a shape’s outline—no background in AI required.
Learn three ideas: pixels, filters, and feature maps
Your experiment: find the edges
no math or coding neededA digital picture is made of tiny colored squares called pixels. Here we use just light and dark pixels. An edge is a place where brightness changes, like the boundary between this light square and its dark background.
Brighter means a stronger edge response. The result shows the inner area we scanned.
The left and right sides of the square light up. The middle stays dark. The detected edges are in the center, following the square.
Move it left or right. The same detector finds the edges in each new position.
What is the computer looking for?
It compares nearby pixels. To find a vertical edge, it looks for a difference between the left and right. For a horizontal edge, it compares above and below.
This little pattern detector is called a filter. Repeating its calculation across the picture is the idea behind convolution.
The result is a new picture showing where the filter responded, called a feature map. It highlights a useful pattern rather than naming the object.
Try to catch it out
Choose Solid bright image. It has more bright pixels than the square. Before you try, predict: will that produce more edges?
Then switch back to Square and move it. You changed the picture’s position, but the detector’s rule stayed the same.
You now have the starting idea
Pixels make a picture. Filters look for patterns. Feature maps show where those patterns appear. Finding edges is one small step toward recognizing something in an image.
Choose another question to explore →How does this become AI?
The filters in this experiment are hand-designed rules called Sobel filters. They find brightness changes; they have not learned what a square is.
A convolutional neural network, or CNN, learns useful filters from examples during training. It combines many filters across layers, building on simple patterns to help recognize more complex ones.
Edges can be useful clues, but recognizing a face or a cat takes much more. The full lab below lets you explore how these pieces fit together.
For a deeper explanation: OpenCV’s Sobel tutorial and Stanford’s guide to convolutional networks.
Open the full lab and technical explanationsThe Convolution Bench · paint pixels, explore filters, and see the math
fig. 01 — convolve · rectify · pool: one full CNN stage, by hand
Convolution: one small window, slid everywhere
A fully-connected network treats every pixel as its own independent input — move the subject two pixels left and, as far as the math is concerned, it’s a brand-new image. ConvolutionSliding a small grid of weights across an image and recording how strongly each spot responds. makes the opposite bet: a pattern worth detecting is worth detecting anywhere. So instead of learning one weight per pixel, you learn a tiny 3×3 window of weights — a kernelA tiny grid of weights (here 3×3) slid across an image, asking the same question at every position. — and reuse it at every position.
At each stop, the kernel does one multiply–accumulate: nine weights times nine pixels, summed to a single number. That number is the answer to one question — how much does this patch look like my pattern? — and the grid of answers is the feature mapThe grid of responses a kernel writes as it slides — bright where the image matches its pattern. you watched build in the bench above. Weight sharing is also why CNNs are cheap: the bench’s 16×16 input would need 256 weights per neuron in an MLP; the kernel gets by with nine, total.
deeper cut · the arithmetic
One stop of the kernel is a dot product: S(i,j) = Σₘ Σₙ I(i+m, j+n) · K(m,n) — nine multiplies and eight adds writing one cell of the feature map. (Pedants’ corner: that’s technically cross-correlation; a true convolution flips the kernel 180° first. Deep-learning frameworks skip the flip, because a learned kernel simply learns itself pre-flipped.)
Output size is one formula: out = ⌊(n + 2p − k)/s⌋ + 1 for input width n, padding p, kernel k, stride s. The bench: n=16, k=3, p=0, s=1 → 14 — which is exactly why the map you painted came out 14×14. Add a one-pixel border of zeros (p=1, “same” padding) and the map returns to 16×16.
And the cheapness claim in real numbers: a layer of 32 such kernels costs 32 · (9 + 1) = 320 weights (the +1 is each kernel’s bias); one fully-connected layer from these 256 pixels to 256 hidden units costs 65,792. That ratio, not any cleverness, is why vision went convolutional.
Kernels are pattern detectors
Before deep learning, vision engineers designed kernels by hand — Sobel for edges, Gaussian for smoothing, Laplacian for outlines. Each one is just a different 3×3 arrangement of weights, and each asks the image a different question. Flip through them below: the input never changes, only the question does.
The deep-learning move was to stop designing them. A CNN learns its kernels by gradient descent — and when you inspect a trained network’s first layer, it has usually rediscovered these same shapes: oriented edges, color contrasts, little gradients. The difference is it learned the questions worth asking for its task, and it asks hundreds of them at once.
deeper cut · reading a kernel like a derivative
sobel-x, weight by weight
Read Sobel-x by columns: left negative, right positive — it computes right minus left, a finite difference approximating ∂I/∂x — while the 1-2-1 down each column averages vertically so a single noisy pixel can’t fake an edge. The weights sum to zero, and that’s the deeper pattern: a zero-sum kernel stays silent on flat patches — it only speaks when its pattern actually shows up.
The Gaussian is the opposite species: all-positive, summing to one, so flat stays flat. It hides one more trick — it’s separable: the 3×3 is the outer product of [1 2 1]/4 with itself, so you can blur rows then columns and pay 2k multiplies per pixel instead of k². The Laplacian is the second derivative: blind to steady ramps, loud at changes of slope — which is why it traces outlines.
Max pooling: keep the what, loosen the where
Feature maps are wasteful. If a vertical edge fired strongly at one spot, it also fired weakly at the neighbors — you don’t need all of that. Max poolingShrinking a feature map by keeping a summary (usually the strongest response) of each small neighborhood. slides a 2×2 window and keeps only the strongest response, shrinking the map to a quarter of the pixels while preserving that the feature was found, just not exactly where.
That blur on position is a feature, not a bug: it buys a little translation invariance, so the network stops caring whether the edge was at pixel 6 or pixel 7. Average pooling keeps the mean instead — gentler, but it waters down strong detections. Toggle between the two and notice which one lets the bright responses survive.
deeper cut · gradients through a switch
Pooling owns no weights, but it still must pass gradients backwards, and the two variants do it in character. Max pooling routes like a switch: the winning cell receives the entire gradient; the other three get exactly zero — so only the detector that actually fired gets trained harder. Average pooling splits the gradient into four equal quarters, diluting credit exactly the way it diluted the signal.
The bookkeeping: a 2×2, stride-2 pool halves each side, so after L pools a map’s side is n/2ᴸ. The classic ImageNet pipeline pools 224 → 112 → 56 → 28 → 14 → 7 — five halvings before the classifier reads the 7×7 summary.
Receptive fields: depth buys context
One 3×3 kernel sees three pixels across — enough for an edge, hopeless for an eye. The fix isn’t bigger kernels, it’s stacking: a second layer’s kernel slides over the first layer’s feature maps, so each of its cells indirectly watches a 5×5 patch of the original image. Add pooling and a few more layers and the deepest cells see most of the picture.
This is the feature hierarchy your image classifier learned without being told to: edges → textures → parts → objects. Layer one finds oriented lines, layer three composes them into corners and fur, the last layers respond to whole faces. Each layer asks its questions about the answers of the layer before.
deeper cut · the growth formula
Receptive-field growth is a one-line recurrence: r ← r + (k − 1) · j, where j is the product of all strides so far (start at r=1, j=1). Two 3×3 convs: 1 → 3 → 5, exactly as the tracer shows. Insert a 2×2 stride-2 pool (r=6, j doubles to 2) and the next 3×3 jumps straight to r=10 — pooling is a receptive-field accelerator: every later kernel step covers twice the original pixels.
The stacking economics, per channel: two 3×3 layers see 5×5 using 18 weights and two nonlinearities; one 5×5 layer needs 25 weights and bends once. Three 3×3s match a 7×7: 27 weights against 49. Same context, fewer weights, more bends — that arithmetic is VGG’s whole thesis, and it’s why 3×3 became the default everywhere.
Bridge
Convolution’s superpower is also its limit: every cell only ever sees its neighborhood, and context has to climb the stack one layer at a time. What if, instead, every position could look directly at every other position and decide for itself what’s relevant? That single idea — attention — is why transformers displaced recurrence, and it’s waiting at the next station.
Next station · 04
Transformers — The Attention Lens
Every token looks at every other token.