The Convolution Bench

A fully-connected network treats every pixel as its own independent input — move the subject two pixels left and, as far as the math is concerned, it’s a brand-new image. ConvolutionSliding a small grid of weights across an image and recording how strongly each spot responds. makes the opposite bet: a pattern worth detecting is worth detecting anywhere. So instead of learning one weight per pixel, you learn a tiny 3×3 window of weights — a kernelA tiny grid of weights (here 3×3) slid across an image, asking the same question at every position. — and reuse it at every position.

At each stop, the kernel does one multiply–accumulate: nine weights times nine pixels, summed to a single number. That number is the answer to one question — how much does this patch look like my pattern? — and the grid of answers is the feature mapThe grid of responses a kernel writes as it slides — bright where the image matches its pattern. you watched build in the bench above. Weight sharing is also why CNNs are cheap: the bench’s 16×16 input would need 256 weights per neuron in an MLP; the kernel gets by with nine, total.

Before deep learning, vision engineers designed kernels by hand — Sobel for edges, Gaussian for smoothing, Laplacian for outlines. Each one is just a different 3×3 arrangement of weights, and each asks the image a different question. Flip through them below: the input never changes, only the question does.

The deep-learning move was to stop designing them. A CNN learns its kernels by gradient descent — and when you inspect a trained network’s first layer, it has usually rediscovered these same shapes: oriented edges, color contrasts, little gradients. The difference is it learned the questions worth asking for its task, and it asks hundreds of them at once.

Feature maps are wasteful. If a vertical edge fired strongly at one spot, it also fired weakly at the neighbors — you don’t need all of that. Max poolingShrinking a feature map by keeping a summary (usually the strongest response) of each small neighborhood. slides a 2×2 window and keeps only the strongest response, shrinking the map to a quarter of the pixels while preserving that the feature was found, just not exactly where.

That blur on position is a feature, not a bug: it buys a little translation invariance, so the network stops caring whether the edge was at pixel 6 or pixel 7. Average pooling keeps the mean instead — gentler, but it waters down strong detections. Toggle between the two and notice which one lets the bright responses survive.

One 3×3 kernel sees three pixels across — enough for an edge, hopeless for an eye. The fix isn’t bigger kernels, it’s stacking: a second layer’s kernel slides over the first layer’s feature maps, so each of its cells indirectly watches a 5×5 patch of the original image. Add pooling and a few more layers and the deepest cells see most of the picture.

This is the feature hierarchy your image classifier learned without being told to: edges → textures → parts → objects. Layer one finds oriented lines, layer three composes them into corners and fur, the last layers respond to whole faces. Each layer asks its questions about the answers of the layer before.