Linux-Based 8mm Telecine
A Short History of 8mm Film
The 8mm film format was developed by Eastman Kodak and released on the market in 1932 to create a home movie format that was less expensive than 16mm. The film spools actually contained 16mm film, which was exposed only along half its width. When the film reached its end, the camera was opened, and the spools in the camera were flipped. The same film was exposed along the side of the film left unexposed on the first loading. During processing, the film was split down the middle. This fit four times as many frames in the same amount of 16mm film. In 1965, Super8 film was released. It featured a bigger image area, resulting in a better quality image. It also moved the location of the sprocket hole and changed the hole size. Naturally, having two standards (see Resources) complicates both the software and hardware for an 8mm Telecine.
The next step is to find the first sprocket hole. Because we are searching in a B&W image, we use a simplified correlation method. The search is done on a vertical line that spans the centerline we just found. If we find a white line, we add its value in to the correlation for that point. Black lines add zero. We have to look only at points inside the correlation window. Outside the window, the correlation value is zero. The process is sometimes called xor correlation, because addition replaces multiplication. The peak of the correlation function marks the edge of the sprocket hole.
With the edges of the first sprocket hole located, we know approximately where the centerline of the next sprocket hole should be. Simple line searches left and right from that centerline are used to find the next set of sprocket hole edges. The search ends at the last sprocket hole in the segment. Once we have found the left and right edges, we search up and down to locate the top and bottom edges. The film in Figure 1 shows the sprocket hole and frame markup after scanning.
If everything were that simple, we would be done. Naturally, it's not. The film segment in Figure 2 illustrates two problems. First, Kodak edge-marks its film. It says “safety film”. Second, the image is not restricted to the frame area and has overlapped into the sprocket hole. Parts of the top and bottom edges of the sprocket hole have vanished in the B&W image. This will cause an edge-detection failure. There is a variety of heuristic methods to treat edge-detection failures. For left or right edge failures, I substitute the expected location based on the approximate sprocket center and the standard for the sprocket hole width. For top or bottom failures, my choice is to post-process the table of edges. When I find a missing edge or a run of missing edges, I average the edge values on either side of the gap and use the average as the location of the missing edge. It's important not to have abrupt changes in the sprocket hole locations, as this leads to visible jitter in the movie image.
Once all of the sprocket holes are found, the image frames are written to separate files in the frames subdirectory. The sprocket hole edge locations are written out to the log file. Although I have not yet needed to do so, at some point, I expect to encounter a film segment where I cannot locate all the sprocket hole edges. Heuristic methods will take you only so far. It will be easier to use GIMP to find the elusive sprocket hole edges and edit the log file table with the correct coordinates. A modified version of the frame finding program could read in the corrected log file table and use that data to generate the image frames.
The images in the first movie I converted would get brighter and then get dimmer with a cycle of about 2–3 seconds. It was very visible and made the movie unusable. I'm scanning 45–46 image frames in each segment of film. At 18f/s, that's about 2.5 seconds of film. I'm using the film backlight removed from the cover of the scanner. It's a cold cathode fluorescent lamp with a white plastic diffuser in front of it. It was intended to backlight 35mm slides. It turns out that its light output is not uniform from end to end. Like most fluorescent lamps, it's slightly dimmer at the ends. Projector manufacturers go to significant lengths to make sure that the film is uniformly illuminated. See the link in Resources on Köhler Illuminators for more details. Replacing the lamp with a longer one didn't fix the problem.
An e-mail conversation with Richard J. Kinch led me to put illumination compensation into the software. I scanned a piece of neutral density film. Don't have any available? I didn't either. I cheated. I cut up a gray anti-static storage bag into strips. Two layers of the plastic film brought the resulting image into the middle of the gray scale. Then, I divided the scan into segments and sampled the image at the center of each segment. Not surprisingly, there was about a 30% variation from each end to the center. As the individual frame files are written out, a location-dependent compensation value is applied. This eliminated the illumination variation from the movie.
The final step is to remove the duplicate images where the scan segments overlap. The amount of overlap depends on how far you advance the film between scans. For this Telecine design, we have traded frame-accurate mechanical registration for software registration. We are not trying to be precise with the film advance. Typical scan segments overlap by two or more frames. The method for detecting a match between frames is called correlation. If two image files are identical, their correlation will be 1.0. If they differ, it will be less than 1.0. In practice, image frames of the same image scanned at either end of the scanner do not match precisely. The program for removing duplicates copies and renumbers frames to the end of the current segment. It matches the next-to-last frame of the current segment with the first five frames of the next segment. The frame with the highest correlation is the matching frame. The next segment becomes the current segment, and frame copying and renumbering begins with the frame after the best match. The process ends when there is no next segment.
At this point, we have converted the movie. It's just not in a format that is very usable. Some video editing software is capable of importing a sequence of image files and then writing out a movie file. Many do not. However, we are not really interested in editing the movie. We want to convert it and give it back to the customer. Using an editing program would be cumbersome. Instead, we use FFmpeg to read in the image frames and create a movie file in a format that's ready to burn on a DVD. A sample command line looks like this:
ffmpeg -r 18 -i movie/sam.%4d.tiff \ -target ntsc-dvd -aspect 4:3 sam.mpg
-r 18 tells FFmpeg that the input file is at 18 frames/second.
-i movie/sam.%4d.tiff implies the input files are named sam.0001.tiff, sam.002.tiff and so on.
-target ntsc-dvd -aspect 4:3 uses FFmpeg presets to create an .mpg movie file suitable for burning to DVD.
sam.mpg is the generated movie file.
Consult the on-line documentation and the reference cited in the Resources section for more information. At this point, our job is done. A variety of Linux tools is available for authoring DVDs and burning DVD disks. Both are beyond the scope of this article.
This project demonstrates that customized, relatively sophisticated, image processing can be handled easily with Linux-based tools. It also describes embedded hardware development in a Linux environment. This project is continuing to evolve. Sprocket hole edges can be checked for abrupt changes. Once the frame files are extracted, there are opportunities for additional improvements. I have experimented with the ImageMagick toolset to sharpen the images and remove dust specks. The Python programs for image processing as well as the C code and other engineering documents for the film transport are both available from the LJ FTP site.
Practical books for the most technical people on the planet. Newly available books include:
- Agile Product Development by Ted Schmidt
- Improve Business Processes with an Enterprise Job Scheduler by Mike Diehl
- Finding Your Way: Mapping Your Network to Improve Manageability by Bill Childers
- DIY Commerce Site by Reven Lerner
Plus many more.
- diff -u: What's New in Kernel Development
- Giving Silos Their Due
- What's New in 3D Printing, Part III: the Software
- Server Hardening
- 22 Years of Linux Journal on One DVD - Now Available
- Controversy at the Linux Foundation
- Don't Burn Your Android Yet
- February 2016 Issue of Linux Journal
- Firefox OS