Analyzing Videos for Fun and Profit

People's phones and all of the various sensors that may be built in to them is a source of scientific data logging that almost everyone carries around. Although the selection of sensors varies from phone to phone, they almost all have a camera. In this article, I take a look at a piece of software called Tracker that can be used to analyze videos you take of experiments.

You can download Tracker directly from the project page. A lot of good documentation is available at the main website, including examples of how you might be able to use it from the community of other users. Tracker is written in Java, however, so you also need to have a JVM installed on your system before you can use it. For example, on a Debian-based system, you can install a very good JVM with the command:

sudo apt-get install openjdk-8-jre

You then can download the relevant installer for your system and run it from a terminal window. You probably will need to make the installer executable with a command like this:

chmod +x

Don't forget to use the sudo command so that you have the correct permissions to do the installation.

Once it's installed, you should have a new entry in your applications menu system. Starting Tracker will give you a new empty project for beginning your video analysis.

Figure 1. When you start Tracker, you get a new empty project.

Although you could start analyzing your own videos immediately, you may want to use simpler videos while learning how Tracker works. When you ran the initial installer for Tracker, it should have asked you whether you also wanted to install example files. Assuming you said yes, you now can load one of those within Tracker. When you click on the menu item File→Open File, you'll see a new window where you can select either a movie file or a Tracker file.

Figure 2. The open file dialog box allows you to open movie files or Tracker files.

To begin with, let's look at one of the Tracker files from the experiments folder. In Figure 3, I have loaded the experiment file named BallToss.trk. The main window displays the movie that is being analyzed as part of the experiment. The pane in the bottom right-hand side shows a table of x and y coordinates for a series of time units. A similar table is generated for each tracked object within the movie. In this example experiment, the only tracked object is the ball. The top right-hand pane contains the associated plot for the data stored in the table. This way, you can have a visual representation of the tracked motion.

Figure 3. Loading a Tracker file opens all of the parts of a previous analysis.

So, what can Tracker actually track? At the bottom of the movie display is a set of controls. If you click on the green arrow, the movie will start to play. As it does, Tracker actually registers the movement of the tracked object and also updates the data panes on the right-hand side.

Figure 4. As the movie progresses, Tracker updates the data panes on the right-hand side for the tracked object.

Although these panes do give you lots of raw data, there really isn't any analysis being done yet. Clicking the View→Data Tool menu item pops open a new window where you can use this raw data to figure out what is happening in the video.

Figure 5. The Data Tool window provides a set of analysis tools to look at the raw data pulled from the video.

The Measure and Analyze buttons at the top of the plot window provide a set of different sections that you can display. This gives you several different analysis options for your data. Clicking the Measure button gives you the options Coordinates, Slope and Area. When you select one or more of those options, values are displayed at the bottom of the plot representing each of the selected options.

When you click the Analyze button, you get a drop-down with the Statistics, Curve Fits and Fourier Spectrum options. Each of those options actually opens a new pane in the Data Tool window. The exception is the Fourier Spectrum option. This option actually opens a new window where a Fourier analysis is done on the raw data from the tracked object.

Figure 6. Tracker also can do a Fourier analysis of the observed motion.

The Statistics option gives the data displayed in the top right pane. This is the set of descriptive statistics—that is, the maximum, minimum, mean and standard deviation, among other values.

Selecting the Curve Fits option gives you the pane at the bottom of the plot window. Here you can select what type of curve is being fitted to your raw data. You can use a number of default functions to try to fit your data. In this case, the linear function is the obvious choice for a fitting function. You can see the function being used, x=A*t+B in this case, along with the calculated values for the parameters A and B. If you are tracking an object that has a really odd motion, you can click the Fit Builder button to pop up a new window where you define a new fitting function.

Figure 7. You define your own fitting functions as part of the data analysis.

What do you do as part of a new project? Let's say you want to analyze the pendulum video from the sample video folder. You can set some calibration items within the video by either selecting the relevant buttons at the top of the window or selecting the items under the menu entry Track→New→Calibration Tools. You can set up items like a calibration stick or a reference set of axes.

Figure 8. You can set up calibration tools within your video to be analyzed.

One way to do the analysis is to step through the video, frame by frame, selecting the object to be tracked within each frame. This may be the only way to collect the raw data, but humans are lazy. If the object being tracked is relatively consistent across the duration of the video clip in question, you can try using the autotracker. Clicking the autotracker button will pop up a new window where you can control what is being tracked.

Figure 9. The autotracker tries to follow an object from frame to frame automatically.

Tracker helpfully displays reminder information on how to use the software. In this case, it reminds you that you need to press Ctrl, Shift and left-click the mouse button on the object of interest. If you then click the search button within the autotracker window, it will move through each frame, looking for the object in question. Once it is done, you should have a full set of data from the tracked object. From the plot in Figure 10, you can see right away that the pendulum follows the expected motion. You also can track multiple objects by creating a new track for each object.

Figure 10. The autotracker can generate all of your raw data automatically.

Now that you've taken a quick look at Tracker and the kind of analysis you can do with it, you should be comfortable enough to start experimenting with it. Don't forget to share any interesting ideas you come up with on the community page and add to a great resource for citizen science.

Load Disqus comments