UConn AI Club / VISION LAB Updated

PIXELS → FEATURES → PLAY

Make the computer see.

Build an edge detector, draw in the air, then investigate what makes a hand tracker lose its way. No example collection is needed here.

↑ Back to the guided route

EXERCISE 1 / FILTER DETECTIVE

What does a filter notice?

Predict which filter will respond to a vertical boundary. Click pixels to change the image; click a response cell to inspect its exact calculation. Keyboard: Tab enters each grid; arrow keys move between cells, Home/End move across a row, and Space or Enter activates the cell.

8 × 8 pixels

3 × 3 filter weights

6 × 6 response

3 × 3 max pooling

The same nine weights slide across every 3 × 3 patch: multiply matching values, then add. We use stride 1 and no padding or bias. Pooling keeps the largest value in each 2 × 2 block. These filters are hand-designed; this page does not train a CNN.

Team mission

Draw a letter or symbol. Find a filter that highlights it. Turn ReLU off: what happened to the opposite edge? Blur the image: where did the fractional values come from? Explain why an edge detector alone cannot tell a cat from a hand.

↑ Back to the guided route

EXERCISE 2 / AIR ART

Your index finger is a paintbrush.

Freehand strokes need mouse/touch or optional camera input. Keyboard-only route: complete the filter grid, then try the party timer and scoring controls; these checks do not test drawing or hand tracking.

Move your index fingertip to aim. Pinch your thumb and index together to paint; separate them to lift the brush. Challenge: draw a star, initials, or a mystery object for your partner to guess.

Camera: pinch to draw; hold an open palm for 0.35 seconds to erase. Pinch again to use the pen.

Mouse / touch mode: drag on the drawing board. Camera mode uses the bundled pretrained model.

AIR ART / PARTY MODE

Draw something ridiculous.

One artist, everyone guessing, one ridiculous drawing. Play together to reach a shared goal—or switch to teams for friendly competition.

Game rules

Artist: no talking or writing letters. Someone clicks Correct when the group guesses it. Swap artists each round. Mirror flips horizontal control; vanishing ink fades while you draw. Surprise mix gives every team the same twist in each round. Hiding the camera hides the preview only; tracking continues until Stop camera. Before revealing the prompt, ask guessers to look away. This shared screen cannot hide a word from people watching it. Hide the prompt before starting. If camera tracking fails, pause the timer or switch to mouse/touch. Camera and mouse matches should use the same input mode for every team. Scores last until reset or refresh.

ARTIST ONLY / OTHERS LOOK AWAY

Remember the word. Draw without speaking or writing letters.

Camera is off

The video shows the computer's landmarks; the board mirrors your hand. Painting is a rule on landmark distances, not a learned drawing classifier. Video stays in this browser. Only Save drawing exports the painted board.

↑ Back to the guided route

EXERCISE 3 / BREAK IT, EXPLAIN IT

Become a model detective.

Keep one hand in view for five seconds per condition. Change only one thing. Watch the landmarks and “hand found” message. Record your own observation below; these notes are not automatic accuracy measurements.

Design a better controller: should it pause when a hand disappears? Require a short hold before a pinch? Give visible feedback? Sketch one change and explain which failure it addresses.

Notes stay on this page until refresh. Copy them into your team's worksheet to keep them.

Final exercise: can movement reveal a future gesture?

Now combine tracking, features, training and a fair test. Rock–paper–scissors is the final experiment: compare recognizing a hand now with predicting its later move.

Open the final challenge →
Connect the game to the mathematicsGo deeper: gradients and learning →Try the final challenge