Linux and Scooby-Doo
Scooby-Doo, the computer-generated dog in the Warner Brothers film of the same name, was created using Linux. Scooby-Doo was released on June 14, 2002 and stars Sarah Michelle Gellar, from the popular TV show, Buffy the Vampire Slayer. Live footage for the film was shot in Australia, and the Scooby-Doo character was added electronically later.
Animators at the Los Angeles post-production studio Rhythm & Hues used Maya, Houdini, Film GIMP and proprietary Linux-based tools. "We utilized about a hundred Linux desktops to create Scooby-Doo", says Technology VP Mark Brown. "My biggest problem was all the animators yelling at me for more Linux boxes."
Film GIMP is the motion picture version of the popular open-source GIMP image editing program. Scooby-Doo was in production at the time I visited the studio for my article, "Film GIMP at Rhythm & Hues", which appeared in the March issue of Linux Journal. Both a developer and a user of Film GIMP, Rhythm & Hues keeps a few Windows and Mac OS X machines around, mainly for compatibility with Adobe Photoshop.
After the article appeared, some readers asked why Photoshop is being used rather than the GIMP. Film GIMP developer Caroline Dahllöf, a programmer at Rhythm & Hues, "Photoshop handles more layers with big images better". Matte painting artists at Rhythm & Hues create large backgrounds with perhaps forty layers and use a lot of specialized plugins. Working on single large images is quite different from the typical Film GIMP tasks of retouching film frames to remove dust or wire rigs. To get rid of Photoshop completely would require investing a lot of developer resources.
"I really wish that there would be an official effort and that I had more time to contribute", says Dahllöf. "Right now we're really busy, but I hope to have more time for Film GIMP this summer".
I myself am joining the project. The first things I want to accomplish are updating the Film GIMP web site and providing a source tarball so it isn't necessary to check out Film GIMP from CVS.
Film GIMP development perhaps has a renewed urgency because Apple recently acquired not only Nothing Real's Shake (see my May article in Linux Journal, "Tippett Studio and Nothing Real's Shake") but also Silicon Grail's RAYZ. Film GIMP, Shake and RAYZ are the three available Linux compositors; all the other Linux-based compositors are proprietary, internal to the studios that developed them.
Steve Jobs reportedly visited motion picture studios months back and took copious notes about to how best position Apple in the motion picture business. Buying Shake brings Apple the leading commercial film compositor, and in buying RAYZ, it has acquired the most significant Linux challenger. Apple stated that they intend to continue Linux support for at least one more version of Shake, but users worry that Apple seems lukewarm in their support for Linux. More Linux compositors, however, are on the horizon.
At the National Association of Broadcasters (NAB) convention in April, Discreet showed their Combustion product ported to Linux, though not yet released. Digital Domain says it may release NUKE, its proprietary compositor that has won two Scientific and Technical Achievement Academy Awards. ILM also has a highly regarded Linux compositor called CompTime (described in my July 2002 Linux Journal article, "Industrial Light & Magic"), but there are no plans to release it. A source at Adobe says there also are no plans to port After Effects to Linux, but they did release Adobe Acrobat for Linux in May without fanfare.
Rhythm & Hues has 125 Linux desktops and 300 SGI machines. Brown expects to complete the phasing out of SGI desktops by the middle of 2003. "Those doing the heaviest work are using Linux for performance", says Brown. "Productivity using Linux is through the ceiling. Interactively, Linux is five to six times faster than the SGI workstations being replaced".
"Our desktops are all dual-processor rackmounts, split 50-50 between P3s and Athlon MP 1800+", said Brown. Animator desktop machines are remote rackmounts kept in the machine room, connected with Cybex KVM extenders. Brown says that 3U racks were chosen to avoid any weird AGP risers. The graphics cards are ATI FireGL 2. "We're looking at FireGL 8800 Radeon cards", notes Brown, "but the drivers are not ready yet." They like the FireGL 2 cards because of the overlay planes (which work well with their software) and because they are good at manipulating heavy, complex 3-D geometries. Their machines use single monitors, not dual head.
The renderfarm, where the individual motion picture frames are computed, has 150 dual Pentium 1Ghz and 60 dual Athlon MP 1800+ machines. "AMD chips scream for our applications", says Brown. "I can't tell you how impressed I am. An Athlon MP 1800+ gives about the same performance as a 2.2G Pentium Xeon but at a third of the price, if that." The render PCs all have separate IP addresses. Rhythm & Hues uses its own custom queue for batch control, which also uses the desktop machines as render nodes during their idle cycles.
"We've ported our software and have all that running on Linux", reports Brown. "We're using Red Hat 7.2 and XFree86 4.1. We use the ATI OpenGL libraries, the SGI GLU libraries and the Mesa 3.4.2 GLUT." Mesa recommends the SGI GLU library version 1.3 over its own 1.2 implementation because SGI's is more up-to-date and reliable. Brown created scripts to switch between various library permutations for testing and benchmarking.
"Linux is stabilizing for us", says Brown. "For instance, normal operations are fine, but the Thunder K7 Tyan AGP 4x motherboard will wedge in our fire-hose tests". Brown says they probably will switch to ASUS or Gigabyte motherboards. Blue Arc, Network Appliance and a custom Sun box are the backend NFS servers. "You just can't serve terabytes of data off a Linux box now", states Brown. "Throughput is about a third of what it should be".
Rhythm & Hues chose Angstrom for their rackmount PCs. "Angstrom does a good engineering job and has a good team", says Brown. "They did well with the burn-ins, and their prices are good. We get monster machines for $2,500. If I had the money, I'd throw out every SGI now and get Athlons".
Located in Boston, Angstrom Microsystems creates ultra-dense, high performance rackmount servers. "Our cooling and our quality are what make us unique", says president Lalit Jain. "We're moving heavily into the render market. Customers include Pixar and Rhythm & Hues". Angstom sold several thousand servers to Akamai last year for streaming web pages and internet video.
"The Quad2000 just released is the highest compute-density, four-processor rackmount available", says chief architect Ravi Soundararajan. "That uses the Athlon 2000+ rated for 2100. Angstrom does a lot of customized cooling, which is how we get to this density". Despite the density, the 4-CPU 1U system is air cooled.
Asked if Angstrom was considering low power chips for its servers, like the architecture of the Transmeta Green Destiny supercomputer, Jain says no. "Not the Transmeta Crusoe, but we scrapped a dual Mobile P3 design we were working on. It would have been four to ten times the price of a comparable P4 system and didn't save any power due to the added components."
Green Destiny is a 240-blade Beowolf cluster that operates air cooled in an 80-degree warehouse at Los Alamos National Laboratory. (Linus Torvalds was among the luminaries at the unveiling in May 2002.) Soundararajan points out, "Angstrom servers can run reliably in 100-degree rooms." He also says that Angstrom servers are working perfectly in foreign countries where there is no A/C. Less air-conditioning also conserves machine room power, a difference felt not so much in power bills (electricity) but in avoiding the costs of building wiring and A/C upgrades (electricians).
Soundararajan says Angstrom has been working closely with AMD and ATI on a graphics bug related to AGP. "There's corruption of video and lockups. We've tested in-house on a number of different AMD boards with the 760 MPX chipset and all have the same problem." The Linux community has been discussing, on the Linux-smp list, a related bug that causes AGP data to be lost during SMP operations. (See the sidebar for more details.) Angstrom sent a machine with a reproducible bug configuration to AMD for testing. Angstrom testing is continuing with NVIDIA cards and Pentium motherboards; Soundararajan found that some of those machines are crashing similarly. Finding solutions to these sorts of problems is part of Angstrom's appeal over generic hardware.
"I think everything 32-bit will be replaced by 64-bit in a year and a half" says Jain, "and AMD will be phasing out 32-bit servers and desktops". AMD's 64-bit CPU is backwards compatible to run 32-bit applications, but temperatures tend to rise with performance. "If AMD continues increasing cooling requirements, we'll have to go to liquid cooling. We're looking at whole rack cooling with water." Angstrom is focusing on 64-bit systems and on software to manage clusters and network operations centers.
Film GIMP is the only open-source compositor used in feature motion picture work. Scooby-Doo is the latest motion picture to utilize Film GIMP, a list for Rhythm & Hues that includes Harry Potter, Cats & Dogs, Dr. Dolittle 2, Little Nicky, The Grinch, Sixth Day, Stuart Little and Planet of the Apes. While there may be some growing pains, the motion picture industry is finding Linux very much to its liking. Compatibility with proprietary UNIX-based studio applications, performance, cost and the Linux community's highly regarded responsiveness in correcting issues are all factors in this movement.
email: [email protected]