TRY IT FIRST / COMPUTER VISION
Play with pixels.
Start with four pixels. Slide a small filter across an image. Build a pattern finder and try to fool it, then test the same idea on a real photograph. The lecture connects these experiments to CNNs, learning and the camera games.
EXERCISE 1 / MAKE A PREDICTION
Which diagonal is brighter?
Click a pixel to switch between dark (−1) and bright (+1). The four fixed weights compare the two diagonals. Predict the answer, then reveal the calculation.
Your 2 × 2 image
Fixed weights
A positive score favors bright top-left and bottom-right: a backslash (\). A negative score favors a slash (/). A zero score means this rule has no preference. These weights are chosen by us; no training has happened.
Python: one small step at a time
A function is a reusable recipe. Here we give it four named inputs, so you do not need to understand image indexing first. Bright is +1; dark is −1.
- Multiply: each input has its own line and a fixed weight.
- Add: the four products become one score.
- Choose:
if,elifandelsemean “if this”, “otherwise if”, and “otherwise”. - Return: send the score, word label and four products back together.
Loading Python source…Try four numbers
answer = compare_four_pixels(
top_left=1,
top_right=-1,
bottom_left=-1,
bottom_right=1
)
print(answer["score"]) # 4
print(answer["label"]) # backslashanswer["score"] reads the number saved under the name “score”. Change one input, then predict how the number changes.
EXERCISE 2 / SLIDE THE SAME FILTER
Find the boundary of a bright square.
Change the image pixels or any kernel weight. Select an output cell to see its input patch and nine products. Move to the next patch and compare a positive edge, a negative edge and a flat region.
Input · 7 × 7
Highlighted pixels form the selected patch.
Kernel · 3 × 3
Stride 1 · no padding · kernel is not flipped.
Response · 5 × 5
Blue: negative · orange: positive.
| Kernel position | Pixel × weight | Product |
|---|
The same weights answer the same local question at every position. ReLU keeps positive responses and replaces negative responses with zero. This is the cross-correlation operation commonly called convolution in CNN libraries.
Python: every multiplication and every patch
row moves down and column moves right. A for loop repeats the indented steps. append adds one item to a list. The helper _shape only checks the input dimensions.
Loading Python source…EXERCISE 3 / BUILD IT, THEN FOOL IT
Can a filter recognize an eye?
Choose a local pattern filter, edit its weights and adjust the threshold. Compare the face-like drawing with a rotated drawing and a striped decoy. Click an output cell to inspect a match.
Editable pattern · 9 × 9
Pink outlines mark centers of matching patches.
Your filter · 3 × 3
Positive responses · 7 × 7
A match must be strictly greater than the threshold.
A local match is not a recognized eye. Hand-chosen filters respond to contrast, including similar contrast in non-face images. A CNN learns many filters from data and combines their responses. Try to create a strong match in the striped decoy, then explain what your filter actually measures.
Python: find local patterns without inventing labels
Visit one response at a time. Keep it only when its score is above the threshold. The list is a list of matching patches, not a list of recognized body parts.
Loading Python source…EXERCISE 4 / BUILD A FILTER FROM SCRATCH
Sixteen weights. A pattern you can see.
Choose a preview to open a small pixel workshop. Build one 4 × 4 kernel using only −1, 0 and +1. Click a weight to highlight the exact pixel it multiplies in the inspected eye patch and in the photo.
A practical problem: find a repeated visual feature in a controlled image. The same sliding-filter idea can be used to inspect a fixed label for a printed mark. Here, the feature is an eye’s dark band. Averaging the face into a small grayscale view makes the task visible and manageable.
These are cropped, averaged versions of real photographs at a fixed scale. The whole displayed pixel image is searched; the eye checking regions do not guide the search.
EXERCISE 5 / MAKE THE INVISIBLE VISIBLE
Paint the photograph with a filter.
A tiny filter searches a big picture. Find vertical boundaries, horizontal boundaries, or fine outlines. The purple overlay marks pixels with a strong response. Change the nine weights and see what changes.
Your 3 × 3 filter
Positive weights add brightness; negative weights subtract it. Zero ignores a pixel. Click a weight to highlight its matching input pixel.
The patch under this filter
Try a visual mission
Find a vertical cup handle or face boundary. Switch to horizontal edges: which marks disappear? Then try outlines on the cat’s fur or the cup rim.
Click the same location in any view to connect the photo, the response and the purple overlay.
How the mask maps back to the picture
At every complete 3 × 3 window, multiply the nine grayscale brightness values by the nine weights and add. The signed sum is the filter response. We use its absolute value so both dark-to-light and light-to-dark boundaries can appear.
The response map is two pixels smaller in each dimension because there is no padding. Output (row, column) maps to the center pixel (row + 1, column + 1) in the original working image. Border pixels have no output. The mask keeps responses whose strength reaches your threshold; those positions are painted purple over the photo.
The response-map colors are contrast enhanced to make weak edges visible; the mask threshold uses the unenhanced strength. Strength = |sum of pixel × weight| / sum of |weight|. This keeps the threshold scale stable when weights change. It is a contrast measure, not a probability or an object label. An outline filter can respond to parts of a circle, but does not recognize a circle.
Choose the simplest method that fits the task.
A hand-designed rule can be enough when the pattern is small and well defined. A learned CNN can help when examples vary in ways that one fixed rule cannot describe. Compare their errors and complexity for the task you actually need to solve.
You have seen a fixed classifier, shared filters and local pattern responses. Next, connect them to learned features, model outputs and the camera games. Explore how to choose a tool.
Continue to the lecture →