DreamWorks Animation "Shrek the Third": Linux Feeds an Ogre

DreamWorks Animation pushes the limits of CG filmmaking with Linux.

All the big film studios primarily use Linux for animation and visual effects. Perhaps no commercial Linux installation is larger than DreamWorks Animation, with more than 1,000 Linux desktops and more than 3,000 server CPUs.

“For Shrek 3, we will consume close to 20 million CPU render hours for the making of the film”, says DreamWorks Animation CTO Ed Leonard. “Each of our films continues to push the edge of what's possible, requiring more and more compute power.” Everyone knows Moore's Law predicts that compute power will double every one and a half years. A little known corollary is that feature cartoon animation CPU render hours will double every three years. In 2001, the original Shrek movie used about 5 million CPU render hours. In 2004, Shrek 2 used more than 10 million CPU render hours. And in 2007, Shrek 3 is using 20 million CPU render hours.

“At any given time, we are working on more than a dozen films”, says Leonard. “Each of those films has its own creative ambition to push the limits of CG filmmaking.” DreamWorks Animation employs about 1,200 people, with about two-thirds in their Glendale studio and the rest in their PDI studio in Redwood City linked by a 2Gb network. (Note that DreamWorks Animation, a publicly traded company led by Jeffrey Katzenberg, isn't Steven Spielberg's DreamWorks live-action that merged with Paramount recently.)

“There were many specific technical advancements on the movie, including advancements in hair, clothing, costuming and crowds as well as bringing the secondary character animation [crowds] to a whole new level of performance”, says Leonard. About 350 people are working on Shrek 3, with about 300 at PDI and 50 in Glendale.

Long Flowing Hair and Running in Long Dresses

In Shrek 3, Fiona transforms a bevy of classic “rescue me” fairy-tale princesses into action figures to defend the kingdom of Far Far Away from usurper Prince Charming. How to convert Sleeping Beauty's narcolepsy into a weapon or get neat-freak Snow White to dirty her nails fighting bad guys seem like minor challenges compared to the technical obstacles involved.

“DreamWorks Animation R&D provides the tools, libraries and software infrastructure for the creation of world-class CG films”, says Leonard. “We develop and support a suite of application tools for our films, including a proprietary animation system, lighting, rendering and compositing tools, and effects tools for things like fire, water, clothing and crowds, to name just a few.” Leonard estimates they have several millions lines of custom code, mostly written in C (legacy code) and C++ (newer code).

Andrew Pearce leads the DreamWorks Animation R&D group based at PDI. “Long hair may be the biggest technology advance in Shrek 3”, says Pearce. “In all of animation in the past you've seen long hair very little.” “It took months to do the hero-hair flick on Shrek 2”, notes Visual Effects Supervisor Philippe Gluckman. “Hair is everywhere in Shrek 3.” How hair glides across a shoulder looks easy but is very complicated to model. “The way the hair moves had to become much more automated”, says Gluckman. There isn't time for animators to position each hair by hand.

“With clothing we have more interactions, including ripping of the cloth”, says Pearce. “Fast motion is always difficult. In the real world, there's only so fast you can move, but nobody has told our animators that. If you went from 0 to 500 mph in one second, you'd probably leave some clothing behind in the real world.” Animation reality is as much art as physics.

Figure 1. Our print copy of this photo surely doesn't do justice to the astounding detail in the hair of the characters. (Photo credit: all Shrek photos in this article courtesy of DreamWorks Animation LLC.)

It's Getting Crowded in Here

It isn't just the challenge of animating some clothing, it's how much clothing. “We have a lot more characters in the same shot”, notes Shrek 3 Co-Director Raman Hui. “Shrek 3 has a huge cast with 48 characters”, says Shrek 3 Director Chris Miller. “We have huge crowd scenes with 40 to 50 characters on a stage and 2,500 in the audience.” “The challenge in crowds is each character needs to look different”, says Pearce.

“If we had to do one setup for each character, that would take too long”, says Character TD Supervisor Lucia Modesto. “We take a generic character and warp that character. We have Man A, Man B and Woman. The big variation you get in a crowd scene is the silhouette of hair and hat. For characters in Shrek, we had one generic man with three variations, now that's 16. Women went from five to 25 variations and 13 hairstyles.” A lot of work is done to make known characters like Shrek look better, but without looking different.

______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

The improvements being made

Miami DUI Lawyer's picture

The improvements being made in CGI for movies and video games is apparent with each new film or game that is released. The characters are looking more and more life-like. I think the most incredible part is the attention that graphic artists give even to the background characters. It's not just stagnant background people anymore like in earlier versions of CGI productions. The background characters move around and do things in the same great detail as the main character of movies and games. It's amazing what has already come out and I'm excited to see what CGI artists will continue to come out with.

paslanmaz sac

paslanmaz sac's picture

Ty

Thank You

RHM's picture

Thank You

i watched the movie, but

Tarihte Bugun Bir Sey Oldu's picture

i watched the movie, but other Shrek movies were better i think..

i watched the movie, but

Tarihte Bugun Bir Sey Oldu's picture

i watched the movie, but other Shrek movies were better i think..

Thanks

Anonymous's picture

Thank You

Good info

Pharmf866's picture

Hello! dfekkbk interesting dfekkbk site!

I'm surprised, Imm good. My

Pisadam's picture

I'm surprised, Imm good.

My Page : www.forumline.net

Can you come pls ?

I'm surprised, Imm good. My

Pisadam's picture

I'm surprised, Imm good.

My Page : www.forumline.net

Can you come pls ?

türkçe rap müzik

rap's picture

Rendering time...

tomaxxi's picture

So, what is real time to render movie like "Shrek the Third"?
Two days? Two weeks? Two months?

tomaxxi

It is norly

Anonymous's picture

In 2008, Intel had at their disposal 9,000 servers used for verification of their cpus in development. Now in 2009 I expect that number to be even larger. http://7daysport.blogspot.com

Daz Software or E-fronteir Poser?

Michael REMY's picture

hi !

it's strange...in a documentary, i heard that was Poser7 software (not an open-source software one) which was majoritaly used to produce Shrek3 face animation and characters movings.....

so, what 's the true ?

A movie from Linux, but not for Linux

Juanjo's picture

Yesterday my girlfriend gave me a Shrek the Third DVD as a Chrismas gift, I wan't it because I already have the first and the sendcond movie in DVD, but, at this moment I'm trying to see it on my desktop computer, and on my laptop, both running linux, with updated mplayer an xine, and it doesn't work, maybe it's a problem with anticopy protection, it have a notice saying it's only playable from a windows computer. I've also tried it on my legacy XBOX with XMMP, where I get the sound, but not the image.

Dreamworks movie staff does an incredible job making the movies with Linux, but Dreamworks packagers are promoting piracy, because xvid encoded movies work anywhere and original DVDs doesn't.

I wan't to play my expensive DVDs on my linux powered computers. Now I have to download it, because I can't play original DVDs.

Thanks Dreamworks for promoting Linux, but you are also promoting piracy.

How to play encrypted DVD's on xine using linux ?

William's picture

Refer this URL

http://spreadlinux.org.in/HowToCompileXine.html

Section:
Steps to install the multi-media codes and libdvdcss rpm (for playing encryped dvd's)

In short you need to install libdvdcss rpm to play encrypted DVD's
I have watched lots of encrypted DVD's particularly from Dreamworks and Pixar using this rpm installed with xine video player.

Thanks

Dreamworks rock!

kidnames's picture

Dreamworks have revolutionized the movie industry, and mostly with open source, this guys rock.

The moviemaking

Ann's picture

I'm so glad I read this article. I had no idea how it was made until reading this. The process of creating this animation is simply amazing from sketch to miracle ;)

Free for me, None for you!

filasopher's picture

I find it curious when I read articles like this regarding studios whose foundation is touted as "Linux" and the "outstanding" role they play as pioneers in this frontier.
All too often I read that these monster money makers are happy to gleen off of the Open source community, but flinch at the idea of offering something back! These companies to me are nothing more than Judas in corporate atire.
I get frustrated by the copouts given by these organizations. The fact is anything that doesn't directly or indirectly put money in their pocket isn't worth pursuing. I applaud any of the so called advancements that they may have made, but I often wonder how many of these advancements are due to some young upstart college individual who'll never be named.
These articles always read "free for me, cost for you!" Coming from the perspective of the Corporations. It is like that little kid who never learned to share. "My toys! Mine!"
I agree that the article was good. I agree that Linux only made sense coming from IRIX. I also agree that IRIX may have been a pinnacle we didn't realize! Finally, I also agree that it will be a great day when Linux has formidible repetoire of video applications! I mean AVID? Come on!

cool

remix's picture

The reason linux is widely used by film studios is because all their software used to run on irix-workstations. Linux is very close to irix featurewise

Shrek III

Seduzione's picture

I tried the videogame on XBOX360 on Shrek The Third and it's really good! :)

Humble servants ought to be promoted.

M. Miyojim's picture

I see the higher level artistic work for movies being done on proprietary computer platforms and the humble, weight-lifting jobs done by GNU/Linux render farms, as if the tools originally created by the open-source community were only good for secondary roles.

The movie studios should set apart a budget for a small team to extract and wrap byproducts useful for general PCs out there, as payment for the benefits obtained with the success of the movies.

I guess 20% of the savings made by not using proprietary licenses for the thousands of CPUs in the render farm would be enough to pay for this little team. And they should spend some effort to flesh out multiplatform applications replacing the proprietary ones in use, or just kickoff the open-source development for them. It is a fair payback to the community.

3000 CPUs is not really impressive nowadays

Ju's picture

For "who's got the biggest ?" contest, you may check the top500 list
... HPC nowadays uses systems with 10^5 CPUs.

The reason linux is widely used

Anonymous's picture

The reason linux is widely used by film studios is because all their software used to run on irix-workstations. Linux is very close to irix featurewise.

Linux its a probed tech

Luis Medina's picture

This is a great news for me, I do use Linux on Desktop but read this and see where Linux its used say to me what i already know Linux rocks!

Largest commercial linux installation

Daniel's picture

I'm pretty sure Google has them beat

http://en.wikipedia.org/wiki/Google_platform

At least Grid Engine can be

Grid's picture

At least Grid Engine can be used to replace LSF:

Grid Engine Project Home

It's opensource and free.

If you use SGE, you still

Anonymous's picture

If you use SGE, you still have to pay to get "official" support though. That can surpass the cost of LSF.

are they using 5Dwm (SGI clone desktop on linux)

Eric Masson's picture

Just wondering if they were using the Indigo Magic Desktop for Linux a.k.a 5dwm ?

cheers,

EM

http://5dwm.org

Credit to Linux

Rick Stanley's picture

I would like to encourage the Film Industry to recognize Linux and Open Source in the Film Credits. If Linux is contributing so much to the advancements in animation, then they should give credit, and thanks! Let them show their support. They might even give a cameo role to Tux sometime too! ;^)

To Rick Stanley

Fernando Barajas's picture

AFAIK, "Wheezy", the penguin on Toy Story II is a homage to Tux :-)

Sadly no lite versions

Trizt's picture

I know it's would take a lot time from the big companies and cost them money, but it had been nice if they could have released some lite version of their software for home users, not necessarily as opensource but at least as precompiled version for x86, x86_64, PPC, PPC64. Another option could be to release those codes they think are obsolete and release those as they are for the public to use as they wish.

I have seen quite a lot of people asking for this kind of software for Linux, there are those heavy and expensive programs that no normal end-user will afford and sadly they can't get rid of the last dependency they have to microsoft.

I know I'm wishing for too much...

Film making

chris's picture

I do agree with you. I wish I could get rid of Microsoft for film editing. I am curently using Première and Canopus professional boards and codec. Linux software on the subject is very poor.

linux is better than microsoft

grace's picture

with your comment, I respect it, but you just said that so because you were just using microsoft, have you tried linux? It's better to do it in linux.... try to have one..

What Makes Linux Better

ADAC's picture

Getting past the usual religious "Microsoft is better" "No Linux is Better".

I use strictly MS and Photoshop products for my graphics. Is there a good reason why I would want to switch to, or at least have one box operating on Linux? would there really be a tangible effort that would be worth the expense or retraining?

Large commercial linux installations

Anonymous's picture

I'm curious to know why Dreamworks was investigated over Pixar. From all I've been able to glean, Pixar's pipeline runs non-commercialized linux, and has more processors in its farm. "Perhaps no commercial Linux installation is larger than DreamWorks Animation" indeed.

For that matter, I expect Oracle's Austin Datacenter which two years ago housed over 10,000 servers would have eclipsed Dreamworks' as well.

DreamWorks Animation has a

Anonymous's picture

DreamWorks Animation has a lot more CPU's than they are reporting in this article (certainly over double and possibly quadruple).

Thanks for a very well written article!

For shear computing power.....

Anonymous's picture

Anyone interested in this may want to see: "At the heart of the matter" Eweek magazine (trade journal - www.eweek.com), May 21, 2007, pages 25 - 30. They examine the computing grid put together at CERN for the LHC (Large Hadron Collider). A quote from the story: "About 200 institutions in 80 countries - some with their own large data centers - will participate in the grid to process an expected 15 petabytes of data generated per year by the LHC."
No numbers are tallied for the whol grid, but "CERN itself will contribute about 10 percent of the 100,000 or so processors needed for the job. In all, CERN will provide about 8,000 systems - using both single and dual-core chips - to the task".
Presumably this would be the largest NON-commercial concentration of raw computing power using commodity (off-the-shelf) hardware?

In 2004, Intel had at their

Adam Krolnik's picture

In 2004, Intel had at their disposal 6,000 servers used for verification of their cpus in development. Now in 2007 I expect that number to be even larger.

4K rendering, compositing, editing, transfer. whats the pipeline

siddharth's picture

not a discussion on linux as microsoft 64 and vista are going multi processor so its yet to be found who the winner would be. ofcourse savings from not having to use properitery software will be a major consideration.

besides the discussion i had a question...

i wish to find out the production pipeline, the software and equipment used for 3D 4K movie making > Rendering > Compositing > Editing > Transfer to Films > Backup on DLT etc...

my knowledge is 3ds max / maya / ZBrush for 3D >>> Combustion for compositing but not for 4K >>> Premiere for editing but not for 4K >>> what comes next

i need it for 4K. even if the matter is not completely relevant but if someone can take the trouble to answer then i'll be really thankful

thanks and regards

siddharth

Webinar
One Click, Universal Protection: Implementing Centralized Security Policies on Linux Systems

As Linux continues to play an ever increasing role in corporate data centers and institutions, ensuring the integrity and protection of these systems must be a priority. With 60% of the world's websites and an increasing share of organization's mission-critical workloads running on Linux, failing to stop malware and other advanced threats on Linux can increasingly impact an organization's reputation and bottom line.

Learn More

Sponsored by Bit9

Webinar
Linux Backup and Recovery Webinar

Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs. However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.” After all, before data can be restored to a system, there must be a system to restore it to.

In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.

Learn More

Sponsored by Storix