Industrial Light and Magic

Discussing the move to Linux on ILM's renderfarm, with speed and stability comes responsibility.
______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

The visuals in Episode II

Atlanta Personal Injury Lawyer's picture

The visuals in Episode II were incredible though Hayden Christianson was not great. George Lucas has never been known for his dialogue writing either but seriously, the visuals got better with every film. Each world was so believable and the lightsabers and other weapons in the series looked so amazing every new movie.

what is useful for spacial

Monesh's picture

what is useful for spacial effect?
lighting or modeling
why?

so that's what they use

tuggle's picture

It's interesting to see what graphic software is used for elite needs, such as motion pictures. You know they can't do what they do with consumer grade software, and it's too expensive to build their own, so you're always left wondering.

hmmm

PK's picture

Actually, it is a magic pixie dust. The article mentions nothing about hardware upgrades whatsoever, you know why? Because there were none. Linux is faster, it essentially *is* a magic pixie dust because it utilizes hardware much better than any other operating system available today. Linux is extremely advanced and extremely powerful. Move to Windows XP? Ha! You would see a tremendous drop in performance simply because Windows doesn't send instructions fast enough and hogs resources to itself - and there isn't anything you can do about it. Linux can even be specialized, there are versions dedicated to being routers, extremely secure, extremely graphical, minimal, customizable, user friendly, or in this case, optimized for graphics. Linux is the ultimate operating system, and I would actually find it interesting to get a custom version of OS depending on your usage intents - gaming, web server, etc etc.

So when can I watch these movies on Linux?

Howard's picture

All of this is great, and I'm glad they're getting such wonderful performance, etc. etc. However, I have a MAJOR complaint. I cannot legally watch these movies on Linux!!!!! What kind of crap is this? The kind that says thanks for making the platform now go away, you bother me.

Sorry, but I'm fed up with hearing how ILM and Dreamworks are "doing wonders" in their movies with Linux workstations, only to be threatened with DMCA violations if I use Linux to watch the freaking things.

As far as I'm concerned ILM and Dreamworks and every other movie company should go back to using the Windows they're trying to force me to use to watch their movies!!

why actually not?

Jorg's picture

sorry for dumb question, but why you cannot watch these movies on linux?
How you will vilolate DMCA if you will use linux player?

Comparisons

sundaresh's picture

I am not so sure but secrecy and exclusivity are overrated, with regards to knowledge or programs for that matter. From what I've discovered,it requires little or no brain matter to come up with startling, even absolutely mind boggling solutions. Nobody has a
power and exclusive hold over creativity or knowledge, which is totally an inward and internal thing. But that apart, there are probably ways to harness and develop this power, and probably use
it to do wonderous things.

But this form of commercialsim, selling or making money, only undervalues, diminishes, cheapens their own efforts.

But there is one thing to be said about and learnt from them
though. They know the value of crap, and even sell it, at an
exorbitant rate. Afterall, excrement by another name is manure.

There is light if you have the eyes, and there is freedom, if you
have the wings.

Re: GFX: Industrial Light & Magic

Anonymous's picture

First off they were using Unix (IRIX from Silicon Graphics) on machines based on the MIPS pocessor and not windows on an Intel based machine. The MIPS processor has not been upgraded for some time now. The point of the artical is that they could migrate from one Unix based system to another Unix based system in the middle of doing productive work and as side-effect the newer hardware (and using Linux as the OS) was much faster. The graphics cards on modern Intel based boxes seems much faster as the older SGI boxes. I has on O2 box when it was new and it was very impressive as multi-meduim box back then. Yes Linux could be run on this older SGI boxes and can extend their duty as the OS is not such a hog on the processor.

What processor ???

billy ranson's picture

What intel-processor would you recommend for this kind of jobs. The Dual-Core-Ones ... or the old Xeon-Line ???

AMD rules

Michael Mueller's picture

I've just watched this video on a PC with a new double core processor by AMD. It looks brilliant.

Re:What processor ???

Jackke's picture

You mean - for doing a movie ???

Re:What processor ???

billy ranson's picture

Yes. I think, that in the meantime the processors were soooo good, that you could do this movie on your home-computer. Or am I wrong ???

Wrong.

linuxpenguin's picture

Well, technically, you could, but it'd take a long time (maybe a few weeks? Possibly even months depending on the length of the movie and complexity of the models and scenes) to render such complex scenes and stuff. You'd want a rendering farm. Unless you're just doing like short clips and stuff.

I'm sure there's plenty of tutorials on how to create a render farm. Plus, even if your render farm isn't the fastest, you've still got your regular PC free to use for other things.

According to this Wikipedia

linuxpenguin's picture

According to this Wikipedia entry, each frame takes about an hour to render. So it'd take a LONG time unless you're doing just a short clip or aren't using as complex scenes, models, and textures as the professionals are.

Re: GFX: Industrial Light & Magic

Giochi's picture

Industrial Light & Magic (ILM) is still porting its code and will begin
using Linux on its next movie after Star Wars, Episode II (release date May
2002). Director of research and development Andy Hendrickson says, "We're
on schedule to replace about 20 percent of our 600 desktops and 20 percent
of our renderfarm with Linux PCs in October.

Re: GFX: Industrial Light & Magic

Foto Ragazze's picture

Another one is given the Gadget was useless as a weapon how on earth was the
US able to get Japan to surrender? Even after Nagasaki in our timeline
there were still those would fight no matter what. How do you get
hardliners who are running to believe you? Also given the spies he had why
did Stalin agree to declair war on Japan in this timeline - seems to be in
his best intrest for the US and Japan to pound each other silly

Also with Illusion and Creation spells available for movies and TV why would
Ray Harryhausen do stop motion and model work or Lucus create Industrial
Light and Magic? Also what path would anime take in a world where things
like the Demon City or the Evil Dynasty really could exist?

Re: GFX: Industrial Light & Magic

Foto Incontri's picture

This opens up the issue of what falls under 'magic' For D&D one tends to
focus on spells almost to the exclusion of all else but GURPS Fantasy
reminds you that is more MUCH MORE.

First you have kinds of magic items: Natural, Alchemy (which in D&D is
really wimpy compared to GURPS), Enchantment, Fetiches, and Holy Relics.

Re: GFX: Industrial Light & Magic

Anonymous's picture

I'm using maya 4.5 on RHat8 and in the past have used maya 4 in RHat 7.2, ive also driven it on NT4/w2k/XP

My experiences show me.... linux is much faster, less buggy.

Simple example : hypershade is more responsive, click the colour swatch and the colourpicker is instant, NT i had to wait 2-3 seconds for the box to open.

Importing hi res obj meshes, 2-3xfaster

I can leave my machine on for days with maya loaded on linux, in windows i would have to reboot 2-3x a day to clear memory, or the machine just slooowed down.

Viewport speed is smoother and faster under poly load.

The machine i use is a dual processor intel system with 1.5gb of memory built for 3D.

In fairness i would say rendering times are about the same on all platforms. Interactive/working speeds are not. Linux is a clear winner.

Re: GFX: Industrial Light & Magic

Anonymous's picture

I will belive the facts

stated by LMI more so than some clowns debating the articles substance for truth. Are we now going to question the experiances of a company with there technical background and credintials.

Come on boy's, it time to grow up and be men.

Re: GFX: Industrial Light & Magic

Anonymous's picture

If you try to run Windoze XXXX for many days with a multitasking/multithreaded process, using heavily the CPU, memory and disk...

...be sure you can get all the work done after two or three reboots and two or three times the elapsed time Linux can do. The STABILITY is TIME, and TIME is MONEY.

ILM knows it, then use Linux.

Eduardo Jaunez

Re: GFX: Industrial Light & Magic

Anonymous's picture

Yep Gents, this is all true. But if you haven't noticed, these guys don't compare Linux to Windoze. They compare it to Irix, the Unix of

SGI's O2. And Irix does not crash.

Using the same logic that you Linux promoters use, one can say that

Linux has only half the speed. You compare a new PC with a 2 GHz

processor with a nearly decade old O2 with a 200 MHz CPU. By your logic, it should run

10 times faster. If it only runs 5 times faster with Linux, it can only mean that Linux drags it down...

Gents, this is a complex issue of MHz, cache, graphics hardware acceleration, you name it. Yep, a stable opsys helps.

And I have nothing against Linux. But if you claim credit for something

that is not due to Linux, it shows that you Linux users are not much more mature than Windoze users whom you despise so much.

And you are well on your way to turn an otherwise good thing into

another silly religion.

I'm sorry, but somehow I expected a great deal more professionalism

from Linux devotees. It seems that I was wrong.

Re: GFX: Industrial Light & Magic

Anonymous's picture

BTW, do you have any figures that show that Windows is more stable/efficient/faster than Linux?

I think that all poeples

myspace's picture

I think that all poeples already tired from disputes that more quickly Linux applications or Windows. Linux VS Microsoft - brr...:(

Figures for linux being stabler than Win

netuno's picture

Hello,

I searched for some figures of this, too, but I did not find some solid numbers. There is an interesting discussion in extremtech:

http://discuss.extremetech.com/forums/11/594752120/ShowPost.aspx

So if anybody has some real numbers, please let us know.

Re: GFX: Industrial Light & Magic

Anonymous's picture

My GranDad lives in NY. He used to drive his all black Model T to see

us in Miami every summer.

Now he sold the T and bought a black Ferrari with yellow leather seats.

He says that now he can drive the distance in one fifth of the time.

It is amazing what these yellow leather seats can do for you!

M Somogyi

Re: GFX: Industrial Light & Magic

Anonymous's picture

As stated, technical questions demand technical answers, not metaphors.

Re: GFX: Industrial Light & Magic

Anonymous's picture

I agree that moving to linux can speed up older hardware. The thing is, windows adds so many extra programs and processes that it slows down the machine. Acctually, there is code in windows that slows down the diplay of widgets in the start menu and other menus. They do this to make it more visually appealing I guess. The nice thing about linux is that you deside what goes on the machine and what doesent. Another thing is the support for older machines. Windows XP will not run on an old 386 or 486, but Linux will do so. By moving to an OS that can utalize hardware resources better is always a speed advantage. And those OS's that add extra bloat and don't let you remove the bloat, will allways be slower and require much more expensive machines to run. I would like to see someone remove Internet Explorer completely off of a Windows XP box - Can't do it -

Re: GFX: Industrial Light & Magic

Anonymous's picture

My GranDad lives in NY. He used to drive his all black Model T to see

us in Miami every summer.

Now he sold the T and bought a black Ferrari with yellow leather seats.

He says that now he can drive the distance in one fifth of the time.

It is amazing what these yellow leather seats can do for you!

M Somogyi

Re: GFX: Industrial Light & Magic

Anonymous's picture

Technical questions demand technical answers, not metaphors.

Re: GFX: Industrial Light & Magic

Anonymous's picture

sounds a little like that Unisys/Microsoft ad.

Re: GFX: Industrial Light & Magic

Anonymous's picture

>I would like to see someone remove Internet Explorer completely off of a Windows XP box - Can't do it

I don't care, I luv my PS2!

Re: GFX: Industrial Light & Magic

Anonymous's picture

Why spend wasteful time on the mundane, when you have the power to change things?

http://www.lindows.com/lindows_home_go_walmart.php

Re: GFX: Industrial Light & Magic

Anonymous's picture

5x gain - linux + intel

Anonymous's picture

they gained 5x speed-up by changing BOTH hardware and OS.

Re: 5x gain - linux + intel

Anonymous's picture

of course they did... duh... how fast do you think an IRIX O2 runs compared to today's typical (new) Linux machines? I'm only guessing, but I wouldn't be surprised if the O2 was only running at 400MHz, and the Linux machines were 2+GHz...

That wasn't the point of the article, tho. They also were comparing cost and uptime and how easy they were managing the transition to a new platform. Have you ever done a platform migration? It's usually not very easy and keeps most people from migrating in the first place.

In the Linux world, it doesn't hurt to toot your own horn every now and then... :)

Re: 5x gain - linux + intel

Anonymous's picture

Actually, the standard O2 runs on a 4XXX series 195MHz 64-bit CPU.

Re: 5x gain - linux + intel

Anonymous's picture

I agree, OTH moving from one UNIX(R) based system (IRIX), to another (Linux), should be *relatively* straight forward. That's the power of standards!

References:

1) www.unix-systems.org/unix98.html

2) www.kernel.org

Whither Windows?

Anonymous's picture

No closed, proprietary, inflexible OS that I know of, has ever been at the cutting edge of computing, or will ever be.

For a hint of this, check the TOP500 Supercomputer List at: www.top500.org

Re: GFX: Industrial Light & Magic

movielad's picture

What I want to know is exactly what graphics cards ILM are using?

I do as well, it certainly

Anonymous's picture

I do as well, it certainly is important for processing larger amounts of data.

the graphics hardware they use...

Anonymous's picture

The other article says that they were using NVIDIA Quadra 2 Pro cards.

Given that the Quadro 4's are out with roughly twice the frame buffer and twice the horsepower (see below), for only twice the price... ($1000 street as compared to $500 street for the Quadra 2 Pro (source: pricewatch))...

And he said their new P4/linux boxes were outworking them then... just imagine what it must be like now...

-jer

Quadro4 900 XGL

  • Memory: 128MB DDR
  • Triangles/Sec.: 60 Million
  • MedMCAD-01: 93 fps
  • ProCDRS-03: 104 fps

Compared to what ILM used for SWII:

Quadro2 Pro

  • Memory: 64MB DDR
  • Triangles/Sec.: 31 Million
  • MedMCAD-01: (not listed)
  • ProCDRS-03: (not listed)

Re: GFX: Industrial Light & Magic

Anonymous's picture

If this is a hoax, I've seen better ones on 1st April.

If this is not a hoax, "merging" CPU power, compilers, graphics cards

and Linux/Unix together does not help to maintain the good name of Linux.

5X faster. . . yea right.

Anonymous's picture

We tried it. Sat Maya for Linux right next to the winblowz version. Both were configured with the exact same hardware. The scene to be rendered was from highend3d.com. Both machines rendered the image within seconds of each other. 7 minutes 42 seconds if I remember right.

There is more to the story. They must have some wicked clustering or multithreading going that you just can't do if you don't have access to the source code of the OS you are using. But standard apps ported to linux using standard hardware don't just magically go faster.. . .

Re: 5X faster. . . yea right.

Anonymous's picture

ILM is not comparing Linux to Windows.

Windows is simply not an option for ILM.

OSX might be an option.

They have an aweful lot of inhouse unix code.. and a lot of unix programming experience under their belt....

Linux allows ILM to essentually reproduce what they had before just on cheap intel hardware. Linux runs XWindows, Irix runs XWindows.... simple as that.

Porting their systems to windows would practically mean rewriting there software.

While OSX is an option (being unix based) It doesn't run on cheap intel hardware either. Still apple might be able to blackmail some studios into using it for compositing etc by buying out products like shake.

Linux also runs on a number of other architectures so you have the widest range of flexibility when choosing hardware. Render nodes only need to be equiped wih processor, ram, HDD and network card.

Essentually On the same budget you would get a faster linux system (depending on which software you use of course) simply because it runs on mass market hardware and there are no licensing fees.

Re: 5X faster. . . yea right.

Anonymous's picture

They're replacing IRIX machines. You don't replace brand new high-end machines like that. These are old dogs. A big chunk of the performance increase has to be coming from the newer processor/architecture. The rest is probably due to the OS optimizations available in Linux.

Re: 5X faster. . . yea right.

Anonymous's picture

There is more to the story. They must have some wicked clustering

I'd be surprised if they DIDN'T since it is a render FARM. The ONLY thing that counts in rendering is raw number-crunching power, which is spelled B-e-o-w-u-l-f in my Linux lexicon.

or multithreading going that you just can't do if you don't have access to the source code of the OS you are using.

It has very little to do with having access to the source of the OS and more to do with access to the source of the apps they are using. ILM has developed almost all the apps it uses in-house and is, therefore, in a position to parallelize their code to take advantage of the clustering.

But standard apps ported to linux using standard hardware don't just magically go faster ... .

As a general rule, an app ported to Linux from Windows may or may not run marginally faster, depending on the quality of the porting job. In general, the Linux kernel and system services tend to consume less of the system's resources than Windows does. By extension, this means more resources are available to make the app run faster. OTOH, a shabby job of porting can negate this advantage very quickly.

Re: 5X faster. . . yea right.

Anonymous's picture

Uhhh...

they kept saying they were using their own custom apps....

Re: 5X faster. . . yea right.

Anonymous's picture

Instead of Windows, it is possible to compile your entire OS from scratch with all optimizations for your specific platform you wish. Especialy when you're using gcc 3+. This can provide a major speed increase.

Re: 5X faster. . . yea right.

Anonymous's picture

ummm, last time I checked, gcc wasn't much of an optimizing compiler. In fact, it's main selling point is portability. I'd be willing to bet that the Microsoft compiler is much better at optimizing code. At any rate, compiling with pentium optimizations doesn't usually give you that much of a performance increase either.

White Paper
Linux Management with Red Hat Satellite: Measuring Business Impact and ROI

Linux has become a key foundation for supporting today's rapidly growing IT environments. Linux is being used to deploy business applications and databases, trading on its reputation as a low-cost operating environment. For many IT organizations, Linux is a mainstay for deploying Web servers and has evolved from handling basic file, print, and utility workloads to running mission-critical applications and databases, physically, virtually, and in the cloud. As Linux grows in importance in terms of value to the business, managing Linux environments to high standards of service quality — availability, security, and performance — becomes an essential requirement for business success.

Learn More

Sponsored by Red Hat

White Paper
Private PaaS for the Agile Enterprise

If you already use virtualized infrastructure, you are well on your way to leveraging the power of the cloud. Virtualization offers the promise of limitless resources, but how do you manage that scalability when your DevOps team doesn’t scale? In today’s hypercompetitive markets, fast results can make a difference between leading the pack vs. obsolescence. Organizations need more benefits from cloud computing than just raw resources. They need agility, flexibility, convenience, ROI, and control.

Stackato private Platform-as-a-Service technology from ActiveState extends your private cloud infrastructure by creating a private PaaS to provide on-demand availability, flexibility, control, and ultimately, faster time-to-market for your enterprise.

Learn More

Sponsored by ActiveState