LinuxDNA Supercharges Linux with the Intel C/C++ Compiler
Exciting news from the LinuxDNA project, which earlier this month successfully compiled a recent Linux kernel with the Intel C/C++ compiler (ICC). This is not just a compile without errors, this is — for the most part — a fully bootable, compatible Linux kernel that can boot into a full Linux system. The full system is based on Gentoo Linux, and utilizes kernel version 2.6.22.
The project's goal is to be able to maintain an Intel-compiler-compatible kernel source along side the current kernel. A few remaining issues that need to be resolved before moving to 2.6.23 and beyond are under investigation.
One might ask: Why compile the kernel with something besides gcc?
The answer: Performance.
Earlier work at compiling the Linux kernel with ICC found that ICC provided up to a 40% boost in performance. Ingo A. Kubblin, a German developer that worked on the original ICC porting project in 2004, gave the following quote:
This early work was based on version 8 of ICC; current efforts are using versions 10.1 and 11.
The Intel compiler often produces faster code as the result of two major optimizations: IPO (Inter Procedural Optimization) and PGO (Profile Guided Optimization). IPO is a heuristics based optimization mechanism, while PGO uses several code stages, first to build code with special modifications that analyse code use, and then a recompile of the code for better execution based on that usage. As PGO is based on usage patterns, it can custom design an optimized kernel for a specific purpose: HPC, DCC, gaming, servers, etc. Although, PGO is also available in gcc, ICC is generally regarded as producing a superior result.
Among the contributors to LinuxDNA are:
- LuYi Cheng: A Chinese kernel hacker responsible for the code becoming fully operational.
- Feilong H: An Intel employee who laid the groundwork for needed patching.
- An unnamed kernel hacker at Broadcom who contributed advice and technical assistance.
- Claude Tyler McAdams: Hacker & spokesperson.
Currently, closed source drivers will not install out of the box, though the developers believed this issue should be easily fixable.
LinuxDNA is an Open Source project that aims to bring a compatible, current kernel source to the high-performance Intel Linux compiler.
Is there a story you'd like to see in Breaking News? Submit your tip by email
or to News Editor Justin Ryan via IRC in #linuxjournal on freenode.
Justin Ryan is a Contributing Editor for Linux Journal.
Realizing the promise of Apache® Hadoop® requires the effective deployment of compute, memory, storage and networking to achieve optimal results. With its flexibility and multitude of options, it is easy to over or under provision the server infrastructure, resulting in poor performance and high TCO. Join us for an in depth, technical discussion with industry experts from leading Hadoop and server companies who will provide insights into the key considerations for designing and deploying an optimal Hadoop cluster.
Sponsored by AMD
Built-in forensics, incident response, and security with Red Hat Enterprise Linux 6
Every security policy provides guidance and requirements for ensuring adequate protection of information and data, as well as high-level technical and administrative security requirements for a system in a given environment. Traditionally, providing security for a system focuses on the confidentiality of the information on it. However, protecting the data integrity and system and data availability is just as important. For example, when processing United States intelligence information, there are three attributes that require protection: confidentiality, integrity, and availability.
Learn more about catching the bad guy in this free white paper.
Sponsored by DLT Solutions
| Designing Electronics with Linux | May 22, 2013 |
| Dynamic DNS—an Object Lesson in Problem Solving | May 21, 2013 |
| Using Salt Stack and Vagrant for Drupal Development | May 20, 2013 |
| Making Linux and Android Get Along (It's Not as Hard as It Sounds) | May 16, 2013 |
| Drupal Is a Framework: Why Everyone Needs to Understand This | May 15, 2013 |
| Home, My Backup Data Center | May 13, 2013 |
- New Products
- Linux Systems Administrator
- Senior Perl Developer
- Technical Support Rep
- Web & UI Developer (JavaScript & j Query)
- UX Designer
- Designing Electronics with Linux
- Dynamic DNS—an Object Lesson in Problem Solving
- Using Salt Stack and Vagrant for Drupal Development
- Making Linux and Android Get Along (It's Not as Hard as It Sounds)
- Nice article, thanks for the
2 hours 30 min ago - I once had a better way I
8 hours 16 min ago - Not only you I too assumed
8 hours 34 min ago - another very interesting
10 hours 27 min ago - Reply to comment | Linux Journal
12 hours 20 min ago - Reply to comment | Linux Journal
19 hours 14 min ago - Reply to comment | Linux Journal
19 hours 30 min ago - Favorite (and easily brute-forced) pw's
21 hours 22 min ago - Have you tried Boxen? It's a
1 day 3 hours ago - seo services in india
1 day 7 hours ago
Enter to Win an Adafruit Pi Cobbler Breakout Kit for Raspberry Pi

It's Raspberry Pi month at Linux Journal. Each week in May, Adafruit will be giving away a Pi-related prize to a lucky, randomly drawn LJ reader. Winners will be announced weekly.
Fill out the fields below to enter to win this week's prize-- a Pi Cobbler Breakout Kit for Raspberry Pi.
Congratulations to our winners so far:
- 5-8-13, Pi Starter Pack: Jack Davis
- 5-15-13, Pi Model B 512MB RAM: Patrick Dunn
- 5-21-13, Prototyping Pi Plate Kit: Philip Kirby
- Next winner announced on 5-27-13!
Featured Jobs
| Linux Systems Administrator | Houston and Austin, Texas | Host Gator |
| Senior Perl Developer | Austin, Texas | Host Gator |
| Technical Support Rep | Houston and Austin, Texas | Host Gator |
| UX Designer | Austin, Texas | Host Gator |
| Web & UI Developer (JavaScript & j Query) | Austin, Texas | Host Gator |
Free Webinar: Hadoop
How to Build an Optimal Hadoop Cluster to Store and Maintain Unlimited Amounts of Data Using Microservers
Realizing the promise of Apache® Hadoop® requires the effective deployment of compute, memory, storage and networking to achieve optimal results. With its flexibility and multitude of options, it is easy to over or under provision the server infrastructure, resulting in poor performance and high TCO. Join us for an in depth, technical discussion with industry experts from leading Hadoop and server companies who will provide insights into the key considerations for designing and deploying an optimal Hadoop cluster.
Some of key questions to be discussed are:
- What is the “typical” Hadoop cluster and what should be installed on the different machine types?
- Why should you consider the typical workload patterns when making your hardware decisions?
- Are all microservers created equal for Hadoop deployments?
- How do I plan for expansion if I require more compute, memory, storage or networking?



Comments
help Full
The only code that's being modified is the Linux kernel. It would be crazy to think they could get away with putting a back door in to that!!!
Umm . . . Sun
Sun's compilers are free and run natively on Linux. I have seen performance improvements with Sun and I suspect Intel will be as good and likely much better. Problem with GCC is that it supports too many machine code variants. Ya know, jack of all trades, master of none. I think a compiler specific to the machine code will always be far better than one that compromises to support many.
compiling as a "project"
It's interesting that apparently just trying to compile something with Intel's compiler is a "project". I also tried compiling a piece of performance-critical code with Intel's compiler once. I gave up after a day or two--for me, there were just too many incompatibilities with the input flags and the output code. Instead of writing their own incompatible compiler, I'd really like to see Intel donate patches to gcc...
like with like
It's not clear that this is comparing like-with-like. Is the gcc code also PGO optimised? Also it's always a temptation to tweak benchmarks to favour your code and even if you resist that to code to the benchmarks.
The monopoly argument for open-source is invalid
The monopoly = lack of progress = laziness argument is really weak for open-source projects like gcc.
These projects
So, the fact that you absolutely need competition to ensure progress is just bogus. Linux kernel (and even gcc) have been progressing fairly fast despite lack of serious competitors. If Linus criticized gcc it was probably due to *too many* performance features rather than too few. The kernel is being aggressively optimized already at source level, so its main requirement from the compiler is correctness, not optimizations.
ICC has become these days imho largely irrelevant, unless you are developing proprietary apps perhaps. The openness of gcc / llvm allows for optimizations in the "logical level" that icc
cannot even dream of. (I.e., you can hack the compiler to introduce extra optimization passes specific to your application)
The only field that is still largely open for competition is that of JIT/lifetime optimizations/
static & dynamic analysis etc where lots of problems remain unsolved and there is no "boxed"
open-source soft yet that solves them. LLVM is mainly interesting for going after this goal
but it still has a long way to go, especially in the front end area.
In the meantime gcc is getting support for plugins real soon now and it won't be too surprising
to see it become competent in the static analysis area pretty fast.
In closing, if ICC manages to optimize a few functions in a common codepath so that they run 40% (!!)
faster, then this imho should be considered a kernel bug and the *source* should be reworked
so that (large part of) this speedup is gained without need for any compiler help.
llvm
http://llvm.org/ can be successor of the current gcc mid-level. It supports many advanced optimization features. It has a very good chance to became the best compiler.
In my tests the gcc's PGO was better than icc's.
It's both good and bad
Good:
1) GCC=monopoly=slow progress=laziness. Even Linus Torvalds criticized gcc by saying he'd really love there to be a viable alternative to gcc.
2) When there's only one viable player people never get to see the whole picture - which is a breeding ground for fake and justified reasons against/in favor of GCC/ICC.
3) GCC is lucky because ICC is shooting itself in the foot by being closed source and non-free.
Bad:
1) Intel's reputation is questionable - it has been found guilty in the past by providing a compiler that spoils the code that targets AMD processors. Period.
2) A closed source and non-free compiler is much worse (intolerable!) than a 10% slower open source and free one.
ICC is for intel's architecture dude!
1st. ICC isn't "exactly" an alternative to GCC, but can be used for certain applications where it kicks GCC's butt in optimization/performance - a fact!!
use an intel build of firefox in linux, and it becomes really obvious!
2nd. ICC isn't "shooting itself in the foot". lots of people are using ICC and it isn't an "ICC vs. GCC war" (like you are clearly participating in!). it is used where it can be useful and it is FREE for non-commercial use. is that really so horrible???
sure it is closed source, but intel is always working on making it better and fixing bugs, they offer decent support, so i don't have any problem with that, as my CPU is intel-based and this is a great compiler with a lot of potential for the future...
3rd. as far as intel's reputation? it is an "intel" compiler, so i wouldn't expect it to work perfectly on AMD processers, so your statement is redundant and you are probably complaining because you use an AMD processer or something....
intel makes quality CPU's that i prefer to that of AMD. my last laptop(AMD) and i couldn't use OSX on it, like i do on mine now. intel is the standard!
4th. worse and intolerable??? grow up. don't use it if you don't like it, and stop your complaining. 10% slower is a lot, if you think about it...i can always use a boost in performance under linux, so far linux doesn't out benchmark OSX, so why not?
in conclusion, GCC is great and is "the standard" in linux(obviously), but there are instances where ICC is very useful for various applications...so i am glad that intel has made their Compiler suite and offered it "free for non-commercial use". which when we are talking about linux, a lot of the time we are talking about non-commercial anyway!
Shrewd on Intel's part
This sounds like a shrewd move on Intel's part. If a particular Linux kernel will run faster on their processors, then they've created a demand for their processors. AMD needs to start their own compiler development (if they haven't already).
uhm...the portland group?
uhm...the portland group? http://www.pgroup.com/ icc still beats the pants off pgicc even on AMD hardware though
More likely AMD should help
More likely AMD should help fund improvement GCC. That way they don't need to maintain a development staff and everyone benefits. (In fact, I wouldn't be surprised if they have, but a little preliminary googling didn't this up.)
There is AMD's
There is AMD's Complier...
http://developer.amd.com/open64
this is what google showed me.. any feedback have any one used it?
I'm not an advanced computer
I'm not an advanced computer scientist but i think Intel' compiler will make their own distro for Intel based arch. (e.g., Moblin) performs really good.
And i think it's better for Chinese (and other country known bad in computing) to join FOSS. It's good for them and us all if they put their skill on the place that it should rather than they're busy creating warez, cracks, and viruses.
HPC tend to run the Application - not the kernel.
When doing performance profiling for a typical HPC (parallel) application...
One sees that one's application is typically sucking up 99.9% of the CPU, (or you have a badly tuned app!)
Improving the kernel performance significantly would only improve poorly written applications.
Help to HPC applications would come from better realtime debug & performance metrics.
(Ever tried to determine how much I/O is being done by a linux app, from outside the application process?)
But Linus has said...
...that for these apps, page-table-managment is frequently a 10-15% hidden overhead.
If a more-optimizing compiler can speed that up, it's a big-$ win.
ICC is seldom used to just optimze only the kernel though
When HPC companies like SGI use Intel chips to create huge clusters they use ICC (along with all of the other tools Intel provides) to not only compile the kernel with ICC but optimize the system code as well. This combination scales quite nicely. Add in that the program algorithms are optimized with the Intel libs as well and you have a purpose built cluster that excels way past what GCC can provide alone.
I heard that BSD folks dug
I heard that BSD folks dug up ancient "pcc" (portable c compiler, first c compiler written in c itself) and try now to make it working again.
As kernels go, Linux is in the same boat with BSD. GCC is badly optimized for such low level tasks, often introducing unexpected breakages.
Needless to mention that GCC is overall badly optimized. Unfortunately, good code optimization requires enormous effort on part of developers. Commercial companies can afford it - GCC mostly can't. (My personal metric: I can read asm code produced by e.g. aCC or SunStudio and understand what it does - but GCC optimizer's decision making often results in puzzling code.)
In the end, I'd say that GCC is great value for the money. I know number of project which would never had materialized without ease of portability GCC offers (but no commercial counterpart does). As CPU performance becomes cheaper and cheaper (I/O being mostly first and biggest bottleneck) pressure on GCC to produce near-excellent code is also diminishing.
I've done some benchmarking
I've done some benchmarking of ICC and GCC and typically you find that recent GCC (4.3+) optimizes code at a high level better than ICC. With large messy codebases with lots of useless function calls with no side effects, GCC will often times reduce the code to a simple compile time constant. But when it comes to scheduling integer math mixed in with SSE intrinsics, the ICC compiler wins by 30% or more on the new Core 2 Quad on the old Northwood based Pentium 4. On AMD and Prescott based Pentium 4 processors the compilers are extremely close with GCC winning in some cases and ICC winning in others.
I'm sure there are some places in the kernel where ICC has a clear lead over GCC but I would be surpised if you don't also find places where ICC is slower as well. The real question is which compiler does a better job with the hot code paths.
Finding places where ICC does a better job than GCC is beneficial for GCC as well as it points to areas where effort to improve the GCC can be focused.
I do not think that CPU
I do not think that CPU usage optimizations are not that important for kernel.
Especially Linux, developers keep it clean from CPU intensive tasks, generally forcing them to user-space.
The problem with GCC (as kernels go) that it can sometimes produce code which is pretty fine for user space application, yet in kernel context can lead to silent corruptions or unexpected performance loss. As Linux went preemptive, with wider adoption of multi-core CPUs, the breakages were popping up quite often.
Lots of idiots around here
Reading the comments, it gets clear that there are way too many idiots (of the lowest /. class) around.
All it takes for ICC to generate better code than GCC is for it to address many of the GCC shortcomings. ICC does not care for anything but the Intel target archs, while GCC expends a lot of effort trying to target many arches. And ICC is a darn good compiler, Intel put a lot of effort in it.
And contributing to GCC on the very fundamental stuff that could give it ICC-like performance in Intel arches is NOT EASY. Not just due to technical reasons, there are political reasons too.
Besides, it is pretty clear that the claim is that the *KERNEL* ran up to 40% faster in certain places, not the whole userland workload!
That said, yes, performance numbers and profiling data would be nice.
What's wrong with this?
I personally don't have a dog in the ICC vs. GCC fight. I'm not a kernel dev nor do I do a whole lot of coding these days. My main use for a compiler is the occasional download of a source tarball for something I want to build from source instead of using a pre-built package (I don't like the maintainer's compile-time options, I need to mod it, whatever) - and then I use gcc.
As a Linux/*NIX admin, however, I really don't understand the hostility or derision coming from some respondents.
If these guys can come up with a reasonable and repeatable recipe for using a compiler environment other than GCC to compile a Linux kernel, more power to them. It validates the reason that OSS is better than CSS - OSS can be adapted to an organization - any organization - rather than forcing an organization to adapt to the software (as CSS does).
We should be cheering these guys on, not throwing tomatoes.
If you don't like ICC, don't use it.
If you think someone should be helping improve GCC's optimization capabilities, go contribute to the GCC project.
But don't bash these guys for not putting their time into what you think they ought to contribute to, and don't throw stones at them for showing yet another reason open-source is better than closed-source.
This is one reason many stay firmly away from Linux
There are many in the business world who agree with you. I'm one of them.
The most stupid argument you can make is that something better should be ridiculed because it isn't how you normally do things. That's about as far from the hacker ethic as I think you can get. I personally own ICC for Linux, OS X and Windows - and I'm a paying customer, not someone who just uses it for free. I'm also a big fan of Sun's Studio compiler on Solaris.
The problem with Linux isn't so much the technology itself (that has issues, just like every other technology out there), but the kinds of people it attracts. Discussions like this one do give the impression that some of you are infected with rabies - and you should get yourselves checked out by a doctor, pronto.
An alternative to GCC is a Good Thing. What is open source about, if it is not about choice?
Non free, not trustable
ICC is a non free program. It directly goes to the bin here.
Open source does not mean trustable, either.
There is an article by Ken Thomson in the ACM, entitled "Reflections on trusting trust" - I suggest you Google it, and read it. You will learn something.
It's a worthy argument as to why relying exclusively on GCC - even if it is open source - is extremely foolish. It only takes one "modified" compiler binary to creep into a distro like Red Hat in order for unseen code to propagate. If you rely only on that compiler, you are setting yourself up to be exploited.
By the way, so what if ICC is not free? Are you also willing to work for nothing, every day? Whoever said that nobody could make a living from writing software?
People like you make me glad I still use Windows.
Open source does not mean trustable, either.
There is an article by Ken Thomson in the ACM, entitled "Reflections on trusting trust" - I suggest you Google it, and read it. You will learn something.
It's a worthy argument as to why relying exclusively on GCC - even if it is open source - is extremely foolish. It only takes one "modified" compiler binary to creep into a distro like Red Hat in order for unseen code to propagate. If you rely only on that compiler, you are setting yourself up to be exploited.
By the way, so what if ICC is not free? Are you also willing to work for nothing, every day? Whoever said that nobody could make a living from writing software?
People like you make me glad I still use Windows.
icc is not faster than gcc
icc is only faster than gcc if you don't use the parameter -fomit-frame-pointer. With the parameter -fomit-frame-pointer gcc is slightly faster on my project.
Intel?
I haven't used an Intel based system for linux for several years. How will it work on an AMD based system?
Priorization
I am wondering why linux (and other open source) societies are not putting more effort to optimize GCC-compiler... Sounds pretty stupid that here we are using ridiculous amount of time optimizing every aspect of linux kernel when we could get 10% of more speed (on kernel level) with change of compiler. And we could say that if kernel level gives 10% more speed then higher runtime levels gives even more.
What we need is Univ. level projects to optimize GCC and implement perhaps new innovative optimizations to it. If GCC is now good it should be made better or even the best compiler.
> I am wondering why linux
> I am wondering why linux (and other open source) societies are not putting more effort to optimize GCC-compiler...
GCC stands for "GNU Compiler Collection." Notice that "GNU" part there. GNU and FSF (to which copyright of all GNU projects is assigned) are well known for their politically biased steering of the projects. They are not the nicest folks to work with.
Linus had created his own C parser to catch some common errors happening during kernel development. Though GCC already has the parsing functionality, it doesn't provide any interface to access the functionality and GNU in past rejected all proposals to add such interfaces. (But I heard that they finally caved in and started work on plug-in interface.)
s/recent/stale/
Don't make me laugh, 2.6.22 is ancient, the thing is almost 2 years old.
Full Ack!!!
Full Ack!!!
Maybe you mean Ingo A. Kubbilun
Anyway, if your application is spending so much time in the kernel that a 40% speed-up is possible only by changing the kernel's compiler, there's something wrong with your app. Even if the improvement is in the 10% range, something fishy is going on---for instance, if your scientific app is 90% user 10% kernel, then icc has to eliminate 100% of all the kernel time to get a 10% speedup. Magic pixie dust, anyone?
What Performance?
Are their figures, benchmarks to prove that linux 2.6.22 has performed 'better' or 'faster'. On what grounds? I don't see benchmark figures or benchmarks listed that can be used to test this. Does code execution path just improve in terms of Instruction Optimizations or Pipeline Optimizations.
To put it simply, where does the "Performance" improve specifically?
Intel compiler is great!
My company uses it to compile its Linux products because our customers demand performance.
We use compiler settings requiring SSE2 (Pentium IV or greater). Performance is outstanding. GCC is not even close.
Another Corporate Tool
Only a clueless person would believe that a new compiler would give even a 10% improvement without some proprietary information, like using undocumented processor instructions or understanding things about the processor that aren't widely known. C compilers have been around for a long time and are well understood. Every compiler that I've ever used besides a GCC compiler ended up falling short of GCC. Either it couldn't handle complicated pre-processor definitions or it just generated bad assembly code under certain conditions.
Also, I certainly wouldn't trust Intel to generate good AMD code so it'll be a cold day in hell before I use a Linux distro compiled with anything but GCC.
What proprietary
What proprietary instructions are you talking about?
Intel instructions are all documented, and Intel debugger can be used to check the assembly code generated by compiler. No secrets here.
http://www.intel.com/design/processor/manuals/253666.pdf
Intel knows their products well. This enables them to write a tightly optimized compiler. How is that a crime?
Intel offers a free version of its compiler to open source projects. So your anti corporate sentiment is completely misplaced.
GCC, on the other hand, is multi platform, which is a great bonus for some applications. But its optimizations for IA32/64 is not on pair with Intel's.
If you don't believe that Intel's compiler is 10% faster than GCC on average use, download it and try it. Demo license is free.
GPL?
How open is Intel's compiler?
It is good to have it as an additional, niche option.
However, unless I need a performance increase that can only be obtained by using a closed-source compiler, I'd rather stick with a GPLed one.
"There's more to life than living it fast." (Gandhi, I think)
Performance Boosts Are Alwasy Great
The nice quotes aside, lets get real. Any performance boost is a good thing in the world of computing. (Take that gained time to slowdown and spend it with people you care about.)
But, I'm a big fan of the GPL aspect of GCC. Are there any up and coming developments to boost the performance of GNU to be on par with Intel's compiler?
Also, the skeptic in me is forced to ask: Is GCC being used to it's fullest advantage when making these comparisons?
Why trust Chinese hackers?
Seems like Beijing is trying to sneak in some code into the Linux base.
Ridiculous!
You can't be serious...?
It has been done...
Why rediculous? Back in the the late 70s or early 80s one of the original Unix programmers (I think it was Ken Thompson himself) put some code in the C compiler that would add a back-door to the root account so that a special password known to him could log into any system. This code would place itself back in the C compiler also when recompiling the C compiler... so there was no way out.
He did this as a demonstration that in the end you have to trust someone, even if you have the source code. He then gave a talk about it and removed the backdoor code in the next version.
Countering "Trusting Trust"
No need to fear the code:
http://www.schneier.com/blog/archives/2006/01/countering_trus.html
No way... relax
The only code that's being modified is the Linux kernel. It would be crazy to think they could get away with putting a back door in to that!!!
Been done before
I hope you're that's just poor sarcasm there else you have a short memory. The linux kernel has been attacked before remember this root attack?
if ((options == (__WCLONE|__WALL)) && (current->uid = 0))
retval = -EINVAL;
The above rather nasty bit of code was only noticed because Larry McVoy spotted the CVS mirror revision didn't have a corresponding bitkeeper entry.
Wonderful!!
As already said: wonderful!!!!!!
I love the compiling field of study and it's always very interesting!!
This is a very good news!!!