This homework must be submitted as a zip file containing all code files and images. The HTML file may need to be modified to include links to your code and image files.

Supporting material can be found on the class webpage.

All videos were generated using this game.

All problems after question 1 may be solved in any language (Python, Java, C++, MATLAB, etc.) as long as you are utilizing the OpenCV library or MATLAB.

Question 1

Write a program that detects all movement in the input video and outputs the motion history as a new video. That is, the output video must contain only the pixels that moved in the last T frames and their color must reflect how recent was their movement (bright white most recent; dark gray most distant). This part must be written in both MATLAB and OpenCV.

Part A

This portion should be written in MATLAB.

Part B

This portion should be written in OpenCV.

Question 2

In this part you will use the FFmpeg command line utility to encode every fifth frame from a series of images into a video. An example usage of FFmpeg with some sections replaced with ...’s. You should modify this command to output a video that is around 24 seconds (the same length as the input) at 30 frames per second.

        ffmpeg -framerate ... -i ....jpg -vf "select=...,setpts=N/FRAME_RATE/TB" -vsync vfr output.mp4

FFmpeg and its documentation may be downloaded from https://www.ffmpeg.org/.

The original video was recorded at 30 frames per second. And its frames were extracted and stored in the "in" folder for this question.

images/p2/in/1/*

images/p2/in/2/*

Question 3

In this part you will write a program to detect when a line is cleared and how many lines were cleared. Once your program detects a line clear, it should draw text showing the number of lines cleared somewhere visible on the video (for several frames).

Input Images:

Question 4

In this part your program should extract the number of points the user has, convert them to text, and display them somewhere else on the screen. You will have to create your own template for each digit to use as structuring elements. You are allowed to specify/hardcode a region of interest in the video frame in which to look for the score.

Question 5

Write a program that detects the center of mass of the moving object and draws its trail as it is falling down, rotating, or moving sideways. The output video should consist of the original video with the overlayed trail (in some bright color). The trail should not decay.

Question 6

In this part your task is to write a program to recolor all orange blocks in green (the same color as the green objects). And save the result as a new video file.

Question 7

Write a program that changes all blocks of the same color to a different random color. This must be done for all colors and the mapping must be consistent for all frames. For example, if red is mapped to brown only the square bricks should be colored brown. All yellow objects will be mapped to a new random color, etc.

Question 8

Mark all red square tetrominos with a purple bounding box. When a part of a red square is removed the bounding box should also be removed.

Question 9

In this part you should overlay the name and the color of the future tetromino on the input video. The future tetromino can be seen in the top right of the input video. You can find names for all of the tetrominos here.

Question 10

In this part you should draw a number on the left of each line of the game area that represents the number of green blocks on that line.

Question EC 1

This part is similar to part 5, except you must track multiple blocks falling throughout the video. In this case, the trails should decay and eventually disappear to reduce clutter.

Question EC 2

In this part you need to record a video while playing your favorite game. Using this footage, write a program that tracks or detects an object in the video based on motion and color.

This document was last modified on 2026-02-26.