Looking Past Search

Can we make search organic again? Or should we look past search completely?

Searching has become shopping, whether we like it or not. That's the assumption behind the results, and behind recent changes, at least to Google's search features and algorithms. I'm sure this isn't what Google thinks, even though it is a commercial enterprise and makes most of its money from advertising—especially on searches. Credit where due: from the moment it started adding advertising to search results, Google has been careful to distinguish between paid results and what have come to be called "organic".

But alas, the Web is now a vast urban scape of storefronts and pitch-mills. The ratio of commercial to noncommercial sites has become so lopsided that Tim Berners-Lee's library-like World Wide Web of linked documents has been reduced to a collection of city parks, the largest of which is Wikipedia (which, like Google, runs on Linux, according to Netcraft). Without Wikipedia, organic results for many subjects would be pushed off the front page entirely.

Let's say you want to know more about the common geometric figure called a square. (Or that you'd like to explore one of the other organic meanings of square, such as correct, level, honest or retro.) You can find your answers using Google or Bing, but none will be the top "I'm feeling lucky" result if you just search for "square" alone. Instead, you'll get the name of a company.

For example, when I look up square on my Chrome browser while also logged in to my Google account, the engine produces a crufty URL more than 200 characters long. The top result for this search is the company Square.com. With that result, Google also tells me Robert Scoble gives the company a "+1", and that I have "110 personal results and 296,000,000 other results". Google's engine is now tweaked to think I might want those results because I've shared a bunch of personal data in the course of whatever I do in Google's presence. But, while I think Square.com is a fine and interesting company, that's not the result I'm looking for.

When I carve the cruft out of that URL (so it just says https://www.google.com/search?en&q=square), the result is the same, without the bonus item from Scoble or the other personalized "social" stuff. When I search on Bing (with which I have no logged-in relationship), I also get long a crufty URL, again with Square.com as the top result. Likewise, if I reduce Bing's URL to just http://www.bing.com/search?q=square, I'm still at square zero (pun intended).

In both search engines, the #2 or #3 result is Wikipedia's one for the geometric square. So, you might say, the systems work in any case: they just happen to make Square.com the top result, because page ranking is based mostly on inbound links, and Square.com gets more of those than any page that describes the geometric shape. And hey, if you're looking for "taxi" on the streets of New York, you're probably looking for a business with four wheels, not for a definition of one.

Search engines operate today in the Web's urban environment, which is at least as commercial as any brick-and-mortar one. Search engines are also not libraries, even if libraries share Google's mission of "organizing the world's information and making it accessible and useful" (http://www.google.com/about/company). Fact is, the Web isn't organized by search engines at all. It's only indexed, which isn't the same thing.

See, the Web actually is organized, to some degree. That is, it has a directory, with paths to locations. Those paths can branch to infinity, but at least each one points somewhere. Search engines do crawl the Web's directory, and they do index its locations, but the index they search is a meta-version of the actual path-based Web. That meta-Web is organized by search engines into a giant haystack made entirely of needles. The job of the search engine is to make good guesses about the needles you might be looking for. At that they mostly succeed, which is why we have come to depend on them. But we're still dealing with a haystack of meta, rather than the real thing. This is important to bear in mind if we ever want to find a better way.

Lately Google has become convinced that the better way is to make searches more "social". Its latest move in that direction is Search, plus Your World—a non-memorable name even when abbreviated to the unpronounceable SpYW (http://googleblog.blogspot.com/2012/01/search-plus-your-world.html). The "Your World" of which Google speaks is what you've organized mostly through Google + and Picasa, though it also indexes the "public-facing" stuff on sites such as Quora. But, if you're one of the 3/4 billion persons who live in the Matrix-like Facebook, SpYW won't help you, because Facebook won't let Google index your Facebook-based social stuff. Nor will it provide you with the option to expose it to SpYW or anything like it outside Facebook itself. Whatever the reasons for that, the effect is to partition the social Web into commercial zones with exclusivities that resemble AOL, CompuServe, Prodigy and other silo'd "on-line services" that the Web obsoleted back in the last millennium.

Why should we think that searches need to be social at all? There are lots of reasons, but the main one is advertising. Google, along with everybody else in the advertising business, wants to get personal with you. It wants to make the ads you see more relevant to you, personally. If it didn't want to do that, SpYW wouldn't exist.

Two problems here. One is that much of what we might search for (such as the case above) does not benefit from "social" help. In those cases, the social stuff is just more noise. The other is that our worlds, even on-line, are not just about stores, shopping and sites that live off advertising. They are much bigger than that.

In his book, Too Big to Know: Rethinking Knowledge Now That the Facts Aren't the Facts, Experts Are Everywhere, and the Smartest Person in the Room Is the Room, David Weinberger writes, "Knowledge is now a property of the network, and the network embraces businesses, governments, media, museums, curated collections, and minds in communications." Google still embraces all those things, but in ways that enlarge business at the expense of everything else. Google might not want to do that, but its advertising business can't help rewarding countless sites that live off on-line advertising, as well as businesses that advertise—while doing little for all the other stuff that comprises the Web.

I don't think the way to solve this is with a better search engine. I think we need something completely different—something that isn't so meta that we lose track of what the Web was in the first place, and why it is still an ideal way to keep and to find knowledge, and not just a place to shop.

Search image via Shutterstock.com.


Doc Searls is the Editor in Chief of Linux Journal


Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.


sandra008's picture

It has been several years now that I am. Bravo for your meticulous work. I love your world. Nothing like coming here to enter fully into your world ... and assess the evolution of your work.

voyance gratuite par mail

No more different then real life

Shaka Wimber's picture

I agree in some way with you. But also think it's a bit hypocritical, because we all have made the internet exactly like real life. And it is just becoming it more and more with social media, crime, business, war, love, knowledge etc.
You can't blame the search engines to be bad. Or constantly incorrect. They change the overall algorithm every year, trying to keep commercialized material away from the top pages. But like in real life, they are hard to get rid off. Tv - bus terminals, streets, banners and poster all over the city and in you daily life. The main problem is that it is working for them. You'll need to answer a much larger philosophic question, if you dream is to change that.
Shaka / Writer at kindlefiresupport.net

Dreams and problems

Doc Searls's picture

The main problem is not just that advertising-funded search is working for Google, but that commercial activity is a subset of the Web, not vice versa. This in fact is also the case in the physical world. So the larger philosophical question is, "How do we subordinate commercial spaces to larger natural environments on the Web?" I believe that question is implicit in my post, but maybe it helps to make it explicit here.

Doc Searls is the Editor in Chief of Linux Journal

A vast urban scape of storefronts and pitch-mills

Nicolas's picture

"the Web is now a vast urban scape of storefronts and pitch-mills"

I cannot agree more with that and I feel guilty for it: after all even my own website is a front for my web services. But sometimes I wish the web was more like a place where you could walk freely and enlighten your mind and learn mindfull stuff for free, without getting thrown ads every corner of a page.

Well actually, it is already a bit like that but a lot of stuff you find on the web is sales pitch or if it's not it's there with the hope of selling you something in the end. Storefronts and pitch-mills are starting to overwhelm the library-like web.

I'm not sure the search engines will ever do the difference. In my job, I am creating websites for my clients and all they want is a storefront or something that will give them ROI. They are not interested in giving free library-like quality content as long as they won't get money out of it. We all want to make money and this is why most people are investing in a website. So in my opinion, this is the real concern: time and money is not invested in the web to give you great library-like content because it is not profitable. This is sad, but I don't see the day where it will change because it is not just good will that would be needed but a big deep cultural change.

Time, investment and ROI

Doc Searls's picture

FWIW, a great deal of time, if not money, is invested in a single library-like noncommercial environment called Wikipedia. Also in persistent noncommercial blogs such as mine, Dave Winer's and many others.

Doc Searls is the Editor in Chief of Linux Journal

DuckDuckGo and IP addresses

Magic Banana's picture

DuckDuckGo's privacy policy states:

When you access DuckDuckGo (or any Web site), your Web browser automatically sends information about your computer, e.g. your User agent and IP address.

Because this information could be used to link you to your searches, we do not log (store) it at all. This is a very unusual practice, but we feel it is an important step to protect your privacy.

How about Startpage (by Ixquick)?

Anonymous's picture

Startpage promises: "Startpage does NOT record your IP address!". I used to think that Ixquick itself guaranteed the same, but now I can't find it. DuckDuckGo does promise not to track you, but recent rumours say it does so all the same. And I must believe in rumours; what else is there? Of course, I do use Yahoo, Bing and even Google when I am desperate...

Our integrity is long gone.


Doc Searls's picture

Magic Banana, I do like DuckDuckGo a lot, for all the reasons you give.

FWIW, DuckDuckGo runs on Amazon Web Services, which runs on Linux, in addition to Linux for one of its URLs.

Doc Searls is the Editor in Chief of Linux Journal


Magic Banana's picture

https://duckduckgo.com/?q=square starts by listing 27 (!) meanings for the world "square" and you can click on one of them to restrict the search to a specific meaning. It also proposes "search suggestions". The commercial sites are the minority among the 15 first results. Those results are not personalized. More importantly: DuckDuckGo does not track its users. The #bang syntax is extremely useful and missing from Google or Bing. Time to switch! :-)

Searching "definition of a

coolsonh's picture

Searching "definition of a square" on Google gives the defintion, then links.

Problem solved.

Problem remains

Doc Searls's picture

True, but you missed my main point.

Doc Searls is the Editor in Chief of Linux Journal