Did Microsoft Buy Netcraft?

Okay, I'm not seriously suggesting Microsoft is paying off Netcraft to produce positive survey results (although this is certainly a standard operating procedure for Microsoft). But something is odd, if not rotten, in the state of Netcraft. I have often cited Netcraft web server surveys as evidence that open source beats closed source. The Netcraft surveys almost always showed Apache leading Microsoft IIS by a wide margin, and showed Apache growing as Microsoft IIS market share was shrinking. Lately, however, Netcraft began to claim that Apache market share has been shrinking rapidly while Microsoft IIS has been gaining the market share lost by Apache. Netcraft even proposed that, "Microsoft's recent gains raise the prospect that Windows may soon challenge Apache's leadership position." Microsoft IIS may displace Apache as the most-used web server? Could this really be true, or is this reporting from the Bizarro world? And does anyone else find the wording rather odd? "Windows" may challenge Apache? Huh?

If, as counterintuitive as it may seem and so contrary to "data by word-of-mouth", Microsoft IIS is actually challenging Apache in terms of market share, then so be it. But how does one explain why other web surveys do not detect this remarkable shift?

Here is the Netcraft survey and here is a Security Space survey. While Netcraft says Apache represents 51% market share and rapidly shrinking, Security Space puts Apache at 74% and growing! Netcraft says Microsoft IIS has 34% market share and is rapidly growing, Security Space pegs Microsoft IIS at 20% market share, as it continues to shrink.

Why the vast discrepancy? Does one or the other survey use a misleading polling technique (sites vs. domains vs. servers)? And which survey is misleading? Is Netcraft guilty of voodoo economics (perhaps we should start calling it Witchcraft)? Or is Security Space getting it wrong? I believe common sense favors Security Space, but what do you think?

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

How do I install netcraft toolbar in my firefox browser?

cris2per's picture

vigilon security

I downloaded netcraft toolbar but it was installed in my internet explorer browser on my laptop instead of the firefox browser. want to find out how to install in firefox.

NetCraft

Anonymous's picture

yes, it is true.

http://uptime.netcraft.com/up/graph/?host=www.thegatesnotes.com

running IIS on linux. what a weird arrangement.

Off topic- Question

Sargon's picture

Nick, as in Allentown Nick?

IIS is no longer winning!!

Anonymous's picture

Microsoft IIS dominates the Web server market (on Windows) because it jumped into the Windows kernel (while others have to cope with the Windows user-mode bloat), see:

http://trustleap.ch/

Now, for the first time, a user-mode Web not only server beats IIS in the kernel but also beats ASP.Net by offereing portable ANSI C scripts that are 5x faster than C#!

Bonus 1: TrustLeap has been under constant attacks since it was shipped 4 months ago -and no vulnerability was found.

Bonus 2: TrustLeap will ship Linux and Solaris versions by the end of year 2009.

Increase in MS tools usage

Don Flowers's picture

Most of the MS high end tools, like SQL Server and IIS, were not available until the past three years without purchase but MS has released their stuff free for non-commercial use and that has caused a huge increase in their use. They should have done this a long time ago for their developer base.

The advantage that Apache has over IIS was taken over by improvements in IIS but Apache being there for free is what drove MS to make their stuff better and improved over time here. MySQL is also not as good as MS SQL server but MS also released their database server management tools, with improvements, as well which are far superior to the others but the others are catching up in a hurry and this kind of competiton is quite good for us all.

Security Space Methodology

Terry's picture

Security Space has explained their methodology at

http://www.securityspace.com/s_survey/faq.html?mondir=/200707&domdir=&do...

Quote:

What sites do you visit?

We visit what we consider well-known sites. In our case, we define a well-known site as a site that had a link to it from at least one other site that we consider well-known. So, if we are visiting you, it means we know about you through a link from another site.

If a site stops responding to our request for 3 consecutive months, we automatically remove it from the survey. In this fashion, our list of known servers remains up to date.

Because of this technique, we find that we actually only visit about 10% of the web sites out on the web. This is because approximately 90% of all web sites are "fringe" sites, such as domain squatters, personal web sites, etc., that are considered unimportant by the rest of the web community (because no-one considers them important enough to link to.)

End Quote

Domain parkers and vanity homepages, which are often run on Microsoft IIS, are seldom counted by this subset of hosts.

Reasons for publication fraud?

Erkki Lintunen's picture

In the one possibility that Netcraft had sold the results/analysis, I'm just wondering why. Here Netcraft says where it is gaining its renevue.

http://news.netcraft.com/about-netcraft

How many are paying for freely published web-server statistics?

Is the web-server stats just a hook to get buyers into the shop for pricy Internet-analyse goodies (an analogy to warehouse marketing)?

Could this be a marketing stunt for regaining attention for web-server statistics, anylyse services and/or Netcraft name?

Or plain and simple could this be a paid analyse supporting an agenda of a significant player in the field?

In the last option I recently were just amazed that one major finnish financial newspaper published an article about collaboration solutions. The article was a normal article, but when read it turned out to an adverticement for Microsoft Exchange. I think normal practice is paid content are marked as such. The article wasn't even an editorial or a column.

The article had carefully chosen key phrases like "there is other solutions available" (no names of others), "...is the most used solution", "once deployed you can't live without" and some other bits.

This is just a simple sign that revenues from ads and subscriptions aren't enough for publications, but they sell everything: ad media, subscriber count, subscriptions and article content and what else.

I say all this not to blame them but just to pinpoint that in the world of business everything is for sale and it is consumer's responsibility to make a difference between fraud and real value. This fact is so easily forgotten with publications from newspapers to analysis. No matter, I could remember the fact whenever I read publications, I face it when next time I talk with my boss or my boss has talked with his colleague about the article. Do I start to educate them about readship or start shooting down the key pharases.

Finally, what is the value of the web-server statistics in the open and interoperable Internet world? Nothing. In the fenced proprietary world, it counts.

So, who needs the web-server statistics?

Old News

Roy Schestowitz's picture

Whether it's GoDaddy's promotional (for Microsoft) press release or even the many Web hosts that get free hardware to shift parked domains to Windows (stories I heard from Web hosts), there's clearly a pattern here. Here is an antitrust memo I'd like to share. It speaks volumes about such tactics that affect morale and perception.

PX03096 [PDF]

Enjoy.

China

Anonymous's picture

This came up at slashdot about a week ago. Someone went digging and found that China has about 60% IIS (mostly pirated). While the number at GoDaddy is large, China is about 10 times that many. They also found that North America and EU were still in 70% range for Apache.

In short, YES!

IG's picture

Although Microsoft didn't directly buy Netcraft, they did sort of indirectly buy themselves better numbers in the Netcraft survey. By simply making irresistible deals to the biggest domain parkers, Microsoft can skew the numbers by millions of servers at a time.

Not that it matters anymore anyway. The Netcraft numbers were overwhelmingly in Apache's favor at a time when we needed to prove that this open source thing was for real. Today, everyone knows that open source gets real work done in the real world. It's mainstream now and we don't need to make that point anymore. As a result, the Netcraft numbers really aren't important anymore.

NetCraft used to report Linux/Apache as MS/IIS

R Casha's picture

Once I checked my company's webserver, which uses Linux+Apache+JBoss, and it was reported as Windows/IIS. My guess then was that their spider was misled by the presence of a Microsoft ISA firewall/gateway. In any case I wrote to them and when I checked later the entry was corrected. I wonder whether there are any other cases where a gateway / proxy server / firewall causes the website to be tagged under a different platform.

Linux machines with IIS - is it a netcraft bug?

Steffen's picture

Indeed. Here we have a Linux machine running IIS:
http://uptime.netcraft.com/up/graph?site=www.uni-luebeck.de

Or would there be an advantage to give wrong information?

Lying about software

Phillep's picture

There's one obvious advantage: Throw off the script kiddies. They'll fart around all day, trying to find holes that are not there, instead of looking for the ones that are.

On Security Space's page

GeorgeA's picture

On Security Space's page they say that there are 24 million servers so I presume they count the servers; on Netcraft's site they say there are 128 million hostnames out of which about 60 million active, so they might consider different hostnames and this could show the difference.
From what I remember there was a shift in april 2006 when GoDaddy decided to go on IIS for it's parked domains.. 3.5 millions (but on recent news I read 4.5 million), and in Netcraft's news from that time (http://news.netcraft.com/archives/2006/04/06/april_2006_web_server_surve...) they also say "Microsoft's last major upward surge in market share in 2001-02 was boosted by migrations at Register.com and VeriSign." (that's when I decided to park my domains at opensourceparking.com).

And.. how many servers does it take to host several millions of parked domains ? I'd say probably less than 100 (but considering IIS could be a lot more :-)) ); so this could explain the difference between IIS and Apache.. few servers with IIS but with lots of hostnames.

George

There is another explaination of course....

JohnMc's picture

I hear this discussion from time to time. I always wonder why nobody else notices a link in the trend. As I recall the NetCraft survey also shows a chart. That chart shows a pronounced rise in use of IIS around late 2005 forward.

That is significant for this reason. That is also the time that M$ launched online BusinessServices and SharePoint. Why is that significant? What? You think M$ is going to put a customer on Apache? Every time they get a new customer for these services another IIS server goes up. IIS numbers are along for the ride on the popularity of SharePoint. This is the sole reason that IIS is on the rise.

Did Microsoft Buy Netcraft?

Jonathan Tappan's picture

I'm not in a position to verify this, but I have heard that Netcraft's results are skewed by a vast number of junk domains hosted by the largest domain parking services (e.g. GoDaddy, etc.) Supposedly Microsoft has been very successful at getting these services to use IIS for domain parking.

Did Microsoft Buy Netcraft?

Chiron's picture

The question boils down to how do we decide what survey is reliable (in general, not just with this particular question). Most of the time we simply believe the one that most agrees with our own prejudices. So I would tend to doubt Netcraft and favor Security Space. But this is just my personal prejudice, which is often just plain wrong.

What criteria do these two companies use? Who did the ask, or where did they get their data? Who, if anyone, commissioned the survey, or who's paying the bills? Is this an unlikely, but random fluctuation? Is there some unforeseen sample bias? Are the results rigged by one or the other (or both)?

We don't know any of these things. We have no way to judge what the truth is. Our impressions, while reasonable, may be a result of the company we choose to keep. We may gravitate towards Apache or IIS users, depending on what we use - possibly because the people who use our system are the ones who can best help us, best understand what difficulties we face. In other words, an Apache user might think Apache is more common, because all his colleagues use it; but that's because he has less in common with IIS users. And vice versa.

The fact is, most of us see only a tiny little sliver of the whole picture, and we have no way of reliably generalizing from that limited view. We don't know. We are at the mercy of those who purport to know. Not all of them are competent and unbiased. So - who *do* we believe?

Netcraft appears to be

Anonymous's picture

Netcraft appears to be hiding Microsoft's use of Linux servers.

This search: microsoft.com

is missing the .a.microsoft.com entries shown here:

.a.microsoft.com

The same with .msn.com and .search.msn.com

.msn.com

.search.msn.com

Maybe Akimai is using Linux?

Jerry R's picture

If you look at the site report, the netblock owner for each is Akamai (perhaps why the a is there in the subdomain).

Not exactly hiding it...

goblin's picture

Netcraft appears to be hiding Microsoft's use of Linux servers.

I wouldn't say they are hiding it. I'd rahter say that they are telling the world about it:
http://news.netcraft.com/archives/2003/08/17/wwwmicrosoftcom_runs_linux_...

why is LJ ignoring all the questions about the QSOL ad?

Carla Schroder's picture

and related issues? Ignoring problems doesn't make them go away, and ignoring all the people who have contacted LJ is not a wise policy. Yeah, it's bad form to post off-topic in a blog, but you're not responding to anything else.

Why Carla

20/20's picture

Please consider that this issue is not a show stopper and somewhat sexist on your part. You are applying reverse discrimination to readers who are NOT offended or bothered by an advertising page that is not what you say it is. You are attempting to impose your own personal views and beliefs to a population of both man and women that do not share your point of view.

Ive asked several women (> 10) and men (>15) to provide comments on the advertising, without informing them of why I was asking for an opinion. They read the article and found nothing out of the ordinary.
After they stated an opinion, I showed them your comments on the subject. They (some) felt you where out to lunch and that you purposely got your readers to be subjective on the article. In other words, you set the path for a negative view point.

Please consider the impact to your own columns readership, you may be viewed more bias than fair.

Paul G

QSOL ad and demographics

Clytie Siddall's picture

I seriously doubt if 20 people is an accurate representation of LJ's demographic. I have certainly talked to a great many more people, of both genders, who are disgusted by the QSOL advertisement. Then again, it's up to you: if you want to lose readers and advertisers with that sort of behaviour, and by ignoring customer feedback about it, then go ahead.

comments

Webmaster's picture

Out of respect to our authors, please try to keep comments on-topic.

Thank you,

Katherine
webmaster

Katherine Druckman
webmaster

please dont ignore me

Anonymous's picture

Hi. I have no job. I want to sit around and complain all day long. Why wont anyone listen to me? Seriously, can anyone see me? I exist, really I do.

Yeah, better stay at home

Pawel's picture

Yeah, better stay at home and let everyone else do what he want... Only idiots don't complain.

Btw. it seems that linuxjournal will be my favourite Linux site, because its authors aren't cowards. Thanks for such great articles.

You do not.

Anonymous's picture

You do not. Stop fooling you self.

Oh no something might not reflect my worldview

Jonathan Jesse's picture

I must accuse them of selling out to the "enemy." What happens if this is accurate? Will we see a blog entry stating the other survey site is in with Linux developers/comunnities/companies? I doubt that.

Let's look into the reasoning behind this apparent increase. Is IIS now more secure? Are more companies running IIS as they develop further sites via Silverlight, WCF, .NET 2.0+, Sharepoint, etc?? Or is Apache being replaced by a better Open Source web server? Or is the data flawed?

Jonathan

>> Are more companies

Jose's picture

>> Are more companies running IIS as they develop further sites via Silverlight, WCF, .NET 2.0+, Sharepoint, etc?? Or is Apache being replaced by a better Open Source web server? Or is the data flawed?

Just wanted to point out that FLOSS tools keep growing and being introduced very quickly (even commercial entities now seem to like Linux.. Vista wasn't very good to them). This explains why both net totals are growing, but it does not explain why Apache or IIS would gain percentage share on the other. The question there would be who is growing faster? Are more Windows devs defecting or are more *NIX devs defecting? Some studies indicate that more are now considering developing for Linux than in the past while fewer are targetting Windows than in the past (at least for desktop and for devices and in relative % terms compared to earlier years). I think Vista has something to do with this, and I have heard that Microsoft is still having quality control problems, especially as they try to fight more and more battles each day while having a tougher time attracting talent.

For these reasons and others (like MS's gaming of the numbers.. you don't do that if you are confident and are selling well), it seems to me to be much more likely that FLOSS servers are growing faster. It's easy to buy a few million domain names every now and then when you make over a billion dollars per month profit [that MSOffice monopoly has sure been useful.. and so have the accounting procedures where MS can book future deals quickly as necessary]. Netcraft does seem to record domain names. I also noticed that the lone Windows server than had a very high uptime according to Netcraft served a very simple static page (even simpler than the standard 404 error pages).

It's simple. Put the two products side by side and factor in the costs of each (for fun: to add insult to injury) and tell me which you would choose to host your site. Money not spent on hardware, licenses, and downtime is money in the pocket.

VS devs going web

Jeffs's picture

There is also a huge up swing of existing Visual Studio developers, in particular old VB6 devs, gradually shifting from desktop apps to web apps.

The vast majority of .Net development is with ASP.Net. And guess what? ASP.Net runs only on IIS (AFAIK).

In particular, when MS officially ended support for VB6 and VC6 last year, all the many old hold-outs who were resisting migrating their existing code to .Net (and why shouldn't they have been resistant? it was a huge cost with zero benefit) suddenly had to adapt or move on. This was also reflected in the big up swing in .Net book sales (particularly at O'Reilly).

So really, any big upswing in .Net or IIS usage is reflective of a big shift in Windows desktop devs going web - to the path of least resistence - ASP.Net, and to the long time hold-outs suddenly being forced to change.

And, as has already been pointed out, MS has greatly increased it's SAAS offerings, all of which, to be sure, will be running on top of IIS.

You guys may be correct.. OR

Jose's picture

You guys may be correct.. OR you may want us (and devs that are still "holding out" and considering a "difficult" move to a different platform) to believe you are correct.

If Microsoft weren't so good at FUD, I might just take what you two are saying at face value. After all if a few are known to fall into your group and others are vulnerable to reconsidering a prior decision to switch then clearly millions of others are too... Very clever. There may be something better on the other side as many claim .. or there may be evil snakes and many dangers just waiting. Oooooooo.

lies, damn lies... etc.

Anonymous's picture
White Paper
Linux Management with Red Hat Satellite: Measuring Business Impact and ROI

Linux has become a key foundation for supporting today's rapidly growing IT environments. Linux is being used to deploy business applications and databases, trading on its reputation as a low-cost operating environment. For many IT organizations, Linux is a mainstay for deploying Web servers and has evolved from handling basic file, print, and utility workloads to running mission-critical applications and databases, physically, virtually, and in the cloud. As Linux grows in importance in terms of value to the business, managing Linux environments to high standards of service quality — availability, security, and performance — becomes an essential requirement for business success.

Learn More

Sponsored by Red Hat

White Paper
Private PaaS for the Agile Enterprise

If you already use virtualized infrastructure, you are well on your way to leveraging the power of the cloud. Virtualization offers the promise of limitless resources, but how do you manage that scalability when your DevOps team doesn’t scale? In today’s hypercompetitive markets, fast results can make a difference between leading the pack vs. obsolescence. Organizations need more benefits from cloud computing than just raw resources. They need agility, flexibility, convenience, ROI, and control.

Stackato private Platform-as-a-Service technology from ActiveState extends your private cloud infrastructure by creating a private PaaS to provide on-demand availability, flexibility, control, and ultimately, faster time-to-market for your enterprise.

Learn More

Sponsored by ActiveState