Industry of Change: Linux Storms Hollywood

Before the summer of 2001, Linux
supporters often pointed to any of a number of single-company
deployments as a measure of success for the fledgling operating
system. There was Burlington Northern, which committed in February
1999 to deploy Linux in 250 US stores. That was followed by Japan's
Lawson, which struck a deal with IBM to supply that convenience
store retailer with 15,000 IBM Linux-based eServers running on Red
Hat software. Ford announced a plan where they would deploy 33,000
Linux desktops. These were big wins for the open-source faithful.
But they were corporate waves in a sea of change. What Linux needed
was a tidal wave--an industry-wide migration--to signal that the
penguin had come of age.Enter the visual effects industry, the collection of studios
that produce special effects, or VFX in industry parlance, for
movies and animated tales like Toy Story and
Shrek. This is an industry ripe for change, an
industry struggling to shake the bondage of single-vendor solutions
and high-priced specialized hardware. It's also an industry that
tested the waters of Windows and found it flowing in the wrong
direction.This isn't a story about one or two studios adopting Linux as
servers in their renderfarms, those back rooms full of servers used
to produce the individual sets of frames used in a movie. We're
talking about the entire industry--from Rhythm & Hues to Pixar,
from Digital Domain to DreamWorks. DreamWorks-PDI had over 2,000
Linux-based CPUs on-line by the summer of 2001. Their summer
blockbuster Shrek was rendered on 1,000+
mostly Linux machines (see GFX:
"DreamWorks Feature Linux and Animation", August 2001 issue
of LJ). Pixar has only deployed 15 stations in
production and 25 in software development, but VP of Technology
Darwin Peachey says the studio is on the verge of a major purchase
and deployment of desktops to replace their current SGI desktops.
Even Industrial Light & Magic is considering a major switch to
the penguin OS.And this isn't the infrastructure saying they will support
Linux, like IBM or Compaq or HP announcing they will support the
OS--it's the end users demanding it from suppliers of applications
and hardware. Back in June 2001, Ray Feeney, technology committee
chair of the Visual Effects Society said, "For the high-end part of
movie making, 80-90% will be Linux-based inside of 18 months.
Everything is going Linux." This sort of mass migration has never
happened before in the Linux world. The tidal wave is here.Signs before the TsunamiUnderstanding how this wave was formed requires some
understanding of the industry itself. Effects studios talk about
movie production as pipelines, the set of processes required to
create effects and integrate them into a movie. A pipeline has two
distinct sides to it: the graphic workstation and the renderfarm.
The latter is like any other room full of servers, crunching away
on any given problem. In this case, the problem is producing the
3-D imagery from models fed to the farm by the many artists working
for the studio. The artists work on the other end of the pipeline,
on the graphic workstations.The first ripple in this tidal surge came with the use of
Linux by Digital Domain to render frames for the movie
Titanic. Involved in this film was well-known
Linux graphics guru Daryll Strauss, who
covered this story for
Linux Journal back in February 1998. At the
time, Daryll used a room full of Alpha-based Linux systems
networked together to render some of the water scenes used in the
movie. In this early stage, Linux still was used in its traditional
role as a back-end server. The front-end graphics workstations were
still primarily the domain of SGI IRIX systems.In 1999, SideFX software ported their very popular (and very
expensive) high-end 3-D modeling and animation package, Houdini, to
Linux. Linux Journal again covered the story,
this time in an interview I did with
SideFX's Director of Research and Development, Paul
Salvini. Houdini is an artist's tool used to create the models that
renderfarms crunch on. At the time that Houdini was ported, Linux
still had graphic-related limitations, such as a lack of support
for hardware-accelerated OpenGL (a de facto
industry standard for doing 3-D applications and games). This
created a chicken-and-egg problem, according to Salvini. Doing a
product like this for Linux required hardware acceleration to make
it really viable, but hardware acceleration often requires
applications in order to warrant drivers to be written." Drivers
from video card makers weren't being written because there were no
applications that needed them, and applications weren't being
written because no drivers were available. SideFX sidestepped the
issue by using software-accelerated OpenGL, a slower and
problematic alternative that didn't require special video card
drivers. Still, it was enough to entice the VFX industry toward
Linux. It also provided motivation to graphics card vendors to
provide both assistance to XFree86 and to begin work on their own
proprietary drivers.The Wave CrestsWith the operating system proving viable on the renderfarm
side of production and improvements being made on the graphics
workstation side, the stage was set for the earthquake that would
shape the VFX industry's tidal force: the VES 2000 Linux Summit.
The Visual Effects Society (VES) is the professional member
organization for the VFX industry. That summer, 45 representatives
from 24 effects companies met in Santa Barbara, California to talk
about the need to move away from SGI IRIX and into Linux. The
event, organized by the VFX Society's Technology Chair Ray Feeney
and Kate Swanborge of DreamWorks, began the process of thawing the
usually frozen communications between studios. Stress was placed on
the need for third-party applications to be ported. At the time,
only five of the more than 20 standard graphics packages used by
the industry had been ported to Linux. And worse, application
vendors felt that Linux was not the operating system being used by
the industry.Most studios began porting their own in house software over
to Linux on their own. Meanwhile, the VES invited software vendors
into workshops to discuss the need for ports of their applications.
By moving in unison, the VES membership felt they could apply
enough pressure on the vendors to get applications ported
quickly.Some applications were already available. Dana Batali,
Pixar's director of RenderMan product development, said that they
actually had the PRMan renderer available on Linux for a couple of
years. SideFX's Houdini port was the first publicly announced
product. The pressure applied by the society worked. By the time
SIGGRAPH 2001 rolled around in August 2001, ported applications
included Alias|Wavefront's complete Maya modeling and rendering
toolset (see "GFX: Alias|Wavefront Maya
4", October 2001 issue of LJ), Avid's
Softimage XSI image compositor, Kaydara's Filmbox content authoring
package, Silicon Grail's RAYZ compositor and Nothing Real's
compositor, Shake.Anatomy of a Wave: Cost vs. PerformanceThe VFX industry's migration to Linux exposes some
interesting interactions that open-source advocates might not have
noticed previously. For example, the issue of cost isn't
necessarily important when compared to Microsoft products, and it
also isn't a factor when considered without the underlying
commodity hardware.Toronto-based Axyz Animation did much of the early adopter
testing for SideFX's Houdini on Linux. John Coldrick, senior
animator of Axyz, which has already replaced all of their
workstations with Linux-based PCs, says the migration was a cost
issue when coming from IRIX, but a technology issue when coming
from NT:[Linux] doesn't offer more than IRIX except it's
substantially less expensive. But it is the scalability on Linux
that is phenomenal. If you start with eight workstations with NT
you're fine, but if you have to balloon up to 70 or 100 you run
into some major problems. You do that with Linux with no
problems.Cost wasn't the most important issue for Pixar either. "Most
people tend to focus on cost" when it comes to migration issues,
says Pixar's Vice President of Technology, Darwin Peachey:But cost isn't the most important thing at these
price levels. The most important thing is to look at what is the
best performing hardware. Right now that's Intel-based
workstations, with NVIDIA or ATI equipped graphics solutions. These
have now eclipsed traditional RISC workstations (such as SGIs) in
terms of graphics and CPU performance. Quite apart from
price/performance, if you look at absolute performance, it appears
that this is the way the industry has to go.DreamWorks Animation Head of Technology Ed Leonard says that,
these days, computing costs have to be recouped with each
film:Historically we purchased a large amount of SGIs.
Those were amortized over several films. You'd want to get five
years out of some of that hardware due to the expense of that
investment. With the Intel/Linux strategy today, we're moving
toward what we refer to as disposable computing. Film productions
are generally two years long, and during that time the technology
takes several steps ahead. We normally anticipate to recoup a large
portion of our hardware costs with every production. So with each
new movie, we go out and purchase a new renderfarm.But it was SideFX's Salvini who put it most bluntly: "It's
one thing to have your product run on Linux, but so what if you
save $200 on the OS and you need a $5,000 graphics card?"The use of commodity components is allowing the industry to
remove their dependency on SGI, a company with an expensive,
specialized hardware and a questionable future. And, Peachey adds,
there are only two ways to go with the Intel solution: Microsoft NT
or Linux. Feeney says NT turned out to be both a technical problem
as well as a political one.Anatomy of a Wave: Why not Microsoft?"Once upon a time there was a great focus on the part of
Microsoft that the VFX industry would be the next realm they would
conquer", says Feeney, who is also the founder of VFX studio
Silicon Grail. "They would fix big data transfer issues in their OS
and so forth, taking Windows from a consumer tool to the
enterprise." But that never happened, he says,This industry is like a team sport--it's a
collaborative effort. The ability to share data and other material,
aside from office-style documents, is extremely complicated and
made more so by the limitations of the Microsoft environment. So it
was for technical reasons that the industry is looking back to the
[Linux] market.Axyz's John Coldrick agrees and adds that the idea of going
NT sent shivers through him. "We were used to working in a UNIX
environment where we had control and networking and stability. It's
more like NT doesn't offer enough for us: the networking is awful,
there are no links and the stability is not as good." Moving to
Linux provided less technical problems from a porting standpoint.
Because Linux is UNIX for all practical purposes, porting from IRIX
was far easier than going to NT.A recent Giga survey found that a large group of Microsoft
customers not related to the VFX industry were not willing to
upgrade because of the new licensing rules for XP, which locked
them into two-year upgrade cycles. Instead, many are planning to
migrate to other options, mostly because they plan to keep their
PCs for longer periods. Surprisingly, the group that wants to
upgrade fairly often--the VFX industry--didn't get much support
from Microsoft. Feeney says,They're off to work on issues that don't solve
the set of issues relative to the high-end effects industry. They
decided that they would be better off spending their time
elsewhere, like on the Web with Hailstorm and .NET. So they never
bit on the enterprise market.But it isn't just that Linux is more like UNIX. Before the
first Linux Summit, the studios were contemplating their upgrade
policies. "This industry is a relatively new industry that grew up
in the past five or seven years", said DreamWorks' Leonard, "And
that is about the lifespan of the SGI hardware."Industry insiders began wondering what to do about
that--either buy new SGIs and deal with the issues of corporate
stability that might imply, or go with commodity platforms. Leonard
says that Intel's IA-64 is going to make a quantum leap for the VFX
market. "It will force everyone to migrate to commodity hardware
over the next 18 months", matching the IA-64 release cycle.As part of their migration, the industry has had to consider
upgrade policies in a new way. Studios find that upgrading
commodity hardware doesn't make much sense. Desktops and
renderfarms have a two- to three-year lifetime, matching the time
frame for a given film production, after which faster hardware
makes them replaceable. As new systems are brought in, new software
installations are too, and that means verifying applications
continue to work on both. In many cases, studios will continue to
depend on a single-source vendor--though they now have a choice of
vendors--to supply Linux and third-party applications certified for
their hardware. From the start of the migration the vendor with the
most interest in the industry movement has been
Hewlett-Packard.Anatomy of a Wave: Praise for HPMuch of the early work in migrating companies like Pixar,
DreamWorks and Axyz Animation was helped in no small part by the
energetic team at HP's Ft. Collins graphics group.Both SideFX's Salvini and Axyz's Coldrick praised HP for
their help in getting the initial port of Houdini working.
Recognizing the lack of accelerated hardware support in XFree86, HP
ported their own server from HP/UX over to Linux along with a
supporting OpenGL environment. This solution, while not completely
open-source oriented, moved the process along to the point prior to
the first VES Linux Summit.Nothing Real had ported their Shake compositing software to
Linux by April 2000. The company would have had it sooner, except
for the lack of hardware-accelerated drivers. They credit HP for
helping them make the move to Linux, as well as video card maker
NVIDIA, which they say moved more slowly but eventually came around
to providing proprietary drivers of their own.But while these early offerings from companies like HP and
NVIDIA included some closed-source software components, such
situations are considered interim solutions only. Ed Leonard says
that the industry is willing to accept these short-term,
closed-source solutions because of the early stage of the overall
migration. But they still want to see long term strategies that
include open source. "We've said to vendors like HP, 'In order for
us to partner, we really want to see you embrace Linux and open
source.' That gives the industry more flexibility in choosing
hardware. The industry is driving open-source solutions from
vendors." And HP, for their part, agrees. They have already noted
their desire to exit the X server business, leaving that work to
both the XFree86 group and video card makers.Solution providers such as Hewlett-Packard and IBM offer VFX
studios complete system solutions running various distributions of
Linux, with the industry default standard being Red Hat Linux. HP's
digital content creation (a.k.a. DCC) systems are the x2000 and
dual processor x4000, and both are certified for use with Red Hat.
IBM's Linux Digital Studio Solution is that company's DCC offering
and includes their IntelliStation M Pro workstation, the eServer
xSeries for rendering duties and various other hardware for storage
options. Threshold Entertainment produced the Berkeley Breathed
short Edward Fudwupper Fibbed Big, shown this
year at SIGGRAPH 2001, entirely on IBM's offering.Pixar's PRMan, Alias|Wavefront's Maya, Nothing Real's Shake,
Silicon Grail's RAYZ and SideFX's Houdini are all certified to run
on Red Hat. Softimage, which makes the XSI compositing package,
plans to support multiple distributions. But there has to be a
limit to the number of distributions they can support simply based
on the requirements of having those distributions ready and
available for testing. Each distribution has a good chance of
working, but the management of all those distributions would be too
much for any application vendor to handle.For this reason studios will continue to rely on single
source vendors. Leonard says that in DreamWork's case, partnering
with HP for their initial work just made sense:They owned the entire stack, just like SGI did.
Not by directive or choice. It just happened that they manufactured
the hardware, such as their FX line of graphics cards. They had
their own drivers for their graphics cards. If we had a problem,
they could own that problem through to resolution.But eventually, when open-source drivers are readily
available for the NVIDIA or ATI cards of choice, DreamWorks and
other studios will expect to get the open-source solutions from any
vendor. The studios can keep a single source to get the complete
solution if they choose, they'll just have more than one vendor to
choose from to get it.Cooperation: the Next Wave for VFXThe industry has learned that the secretive nature of their
business is in its twilight, soon to be replaced by the more
sharing nature of the open-source world. The cooperative spirit
required to move the industry to Linux is allowing studios to solve
common problems more quickly. Pixar's Peachey says his studio
welcomes the change:We're all competitors in one sense or another,
but [migrating to Linux] has helped us share a little more than we
historically have about our thoughts and plans. Not surprisingly,
those problems are very common across the many studios. We're
starting to see that if someone solves a problem that isn't central
to the art we do [as individual studios], and there is a
technological component to that art, but many of the things we have
to solve are not related to that art, there is suddenly a feeling
we can all benefit from it. That's encouraging. It's fun to
see.DreamWorks' Leonard says he'd still like to see the Open
Source community look toward entertainment as a partner in
innovation as well as recognition:One of the hard parts of dealing with open source
is that it's still viewed as a bit of a hacker's world, as long as
you're willing to hack at the code you'll get what you want. A
reality for the VFX industry is that as real businesses we need to
find a way to channel the talent in the open-source world so that
we can get value from it.Leonard adds that the industry is probably easier to work
with for open-source developers than other industries might
be:The VFX industry is willing to take risks. Our
solutions don't have to be wrapped with a pretty bow--we're willing
to work together to make these things work. I think the application
for what we solve together with open source can be applied to a
much larger community.Ray Feeney agrees. "We're the canary in the mine shaft", he
says of the willingness the industry has to push Linux in new
ways.Still, many in the industry have concerns about working with
open-source developers. One early example of conflict came about
while making the GIMP, the open-source world's answer to Photoshop,
better suited to the needs of film industry. GIMP is highly thought
of in the industry, but the 1.2 version lacks 16 bit color
channels, something the industry needs to maintain the high quality
color found in films. A separate Hollywood branch of GIMP was
created to solve this problem, but the solutions it offered, which
were made available back to the Open Source community, weren't
accepted directly by The GIMP developers. Instead, they opted for a
redesign that would be more practical over the long term, but also
not available for a much longer period (they still aren't available
more than a year later).Unfortunately, says Pixar's Peachey, this left a bit of a bad
taste in the industry's collective mouth:Extensions were done by the VFX industry but
weren't picked up by the project. This left some members of the
industry feeling that because we're a market segment different than
what open-source developers are interested in, the industry can't
get any open-source developer seriously interested in what the
industry needs."The industry", Peachey says, "is interested in doing
open-source work, but if you do the work and no one accepts it, how
do you get that work to survive?"Feeney agrees. "The Linux community works well with the
development savvy, but how do you get film makers and open-source
developers communicating? For the industry, it's a complicated
issue." For example, The GIMP always has lacked a professional
color management facility. There are plenty of external proprietary
systems that are available and could (with the right technical
work) be added to the program. But there are logistical and
political issues to resolve as well, such as how those proprietary
systems can be made to work with GPL code like The GIMP.Does the VFX industry know what the GPL is or what it means
and how it relates to proprietary software? Feeney says probably
not. "In our community, the open-source push is great when you need
to reach consensus and standards. But what our customers do
requires differentiation through their own specialized approach.
There is a big difference between open source and shared source."
Feeney thinks that the application level that the VFX world works
in is more of a shared-source world.Other WavesDespite concerns on how to work with open-source developers,
the VFX industry continues its wholesale conversion to Linux. The
operating system gives them more control of their own futures, both
individually as studios and collectively as an industry.At this point most of the major tools used by the industry
are available. The major conference and tradeshow for the industry,
SIGGRAPH, was full of Linux offerings in August 2001. This includes
modeling and rendering tools, tools for distributing rendering work
and 2-D compositing tools such as Avid's Softimage XSI.While VFX is the first major industry to adopt, others may
follow very soon. If Ford does deploy 33,000 Linux desktops,
industry insiders say that it will likely push GM, Daimler and
others to make similar moves due to the cost competitive nature of
the auto business. And that will make Hollywood's Visual Effects
industry move look like a silent film.ResourcesWeb SitesAlias|WavefrontAvid/SoftimageDigital DomainDreamWorksPixarSide Effects
SoftwareSilicon
GrailVisual
Effects SocietyNews StoriesFord
StoryIBM
Deal to Supply LawsonIBM's
Linux Digital Studio AnnouncementThe
Little Engine That Could, Millimeter
MagazineMicrosoft
Licensing SurveySIGGRAPH
2000: News and Views From the FloorPixar/PDI debate in 1999
which left the industry OS-agnostic turned to a push for Linux,
thanks in part to the backing of Alias/Wavefront.Michael J. Hammel is a noted
author with books on The GIMP and GTK+ to his credit, a graphics
artist wannabe and a software developer currently living in Houston
with his wife, Brinda, and daughter Ryann. Most recently, Michael
worked as a senior editor for Linux Weekly
News
(LWN.net).

email: mjhammel@graphics-muse.org