EOF - The Limits of Scale
Linux is like limestone; you can build anything with it. So, while you find limestone in everything from lipstick to pyramids, you find Linux in everything from picture frames to Google.
What brings this analogy to mind is the matter of scale, long regarded as a virtue in the tech world. Getting to scale and staying there are both considered Good Things. But, as with other Good Things, is it possible to have too much? At what point do the biggest things we make with Linux risk turning into pyramids—that is, durable landmarks that are also dead?
These questions came up for me back in January, when two things happened. One was Larry Page replacing Eric Schmidt as Google's CEO. The other was mysterious account deletions at Flickr. Without Linux, there would be no Google or Flickr.
In Google's case, I saw the writing on the wall at the Techonomy conference in Lake Tahoe, August 2010. On stage was Eric Schmidt, amid four other panelists. In the Q&A, Eric said, “If we look at enough of your messaging and your location, and use artificial intelligence, we can predict where you are going to go....Show us 14 photos of yourself and we can identify who you are.” He added:
I would make a stronger point—that the only way to meet this set of challenges that we are facing is by much greater transparency and no anonymity. And the reason is that in a world of asymmetric threats, true anonymity is too dangerous....One of the errors that the Internet made a long time ago is that there was not an accurate and non-revocable identity management service....You need a name service for humans.....governments are going to require it at some point.
(You can follow along at wn.com/Eric_Schmidt_at_Techonomy, starting at 21:10. The first question is mine.)
I wanted to freeze time and say “Eric, no! Stop, big guy! Better to say nothing than this kind of stuff!” But I just sat and winced. Two months later in an interview with The Atlantic at the Washington Ideas Forum, Eric said, “We don't need you to type at all. We know where you are. We know where you've been. We can more or less know what you're thinking about.” Spoken like an eyeball on a pyramid.
At this point, it was just a matter of time before one of the founders would return, Steve Jobs-like (and hopefully not Jerry Yang-like) to bring the company back in alignment with Original Principles. That happened in January, followed quickly by a Bloomberg Businessweek cover story titled “Larry Page's Google 3.0”. Said the writers, “The unstated goal is to save the search giant from the ossification that can paralyze large corporations. It won't be easy, because Google is a tech conglomerate, an assemblage of parts that sometimes work at cross-purposes.” The piece goes on to profile a half-dozen “star deputies”. Of them, it says, “Together, their mandate is to help the company move more quickly and effectively—to keep it from becoming yet another once-dominant tech company that sees its mantle of innovation stolen away by upstarts.” Good luck with that.
Flickr's first pyramid moment was a report that photographer Deepa Praveen had her entire Pro account (the kind people pay for) deleted without explanation. The story broke first in Thomas Hawk's blog, and then the action moved to my own blog, with a post titled “What if Flickr fails?” That one racked up 107 comments, including a pair from Yahoo executives. (Flickr belongs to Yahoo.) Nowhere was there anything to relieve fears that an account deletion might come at any time, to anybody, with no chance of recovering whatever was lost. (My own exposure is about 50,000 photos.)
Then Mirco Wilhelm, another Flickr Pro photographer, had his 3,400 photos deleted, in what Flickr eventually admitted was its own error. These were later restored, with much apologizing by Flickr. Still, one had to wonder how much of the problem had to do with Flickr's size. According to the most recent reports at this writing, Flickr hosts more than 5,000,000,000 photos for 51,000,000 registered users, with new photos arriving at more than 3,000 per minute.
One of the best talks on Linux deployment was one given by Cal Henderson at the March 2006 O'Reilly Emerging Technology Conference. It was an all-day tutorial about “launching and scaling new Web services”. I remember being highly impressed at how well Linux allowed a fast-growing pile of digital goods to expand, while still providing near-instantaneous service to everybody who wanted it. I also remember wondering what would happen after Cal left—which he did in 2009.
The answer is workarounds and startups. Here are a few examples, just from the comments that followed my Flickr post: unhosted.org, couchapp.org, www.tonido.com, backupify.com, gallery.menalto.com, pix.am, status.net, thinkupapp.com, piwigo.org, www.zoofoo.com and https://pixi.me, in that order. None yet compete with Flickr, but maybe that's not the idea.
Nature's idea is to take its course. It's as much Linux's nature to start something as it is to grow to the limits of viability. It may help to remember that limestone is made from the corpses of once-living things. Without abundant endings, we wouldn't have beginnings.
Doc Searls is Senior Editor of Linux Journal. He is also a fellow with the Berkman Center for Internet and Society at Harvard University and the Center for Information Technology and Society at UC Santa Barbara.
Doc Searls is Senior Editor of Linux Journal
Practical Task Scheduling Deployment
July 20, 2016 12:00 pm CDT
One of the best things about the UNIX environment (aside from being stable and efficient) is the vast array of software tools available to help you do your job. Traditionally, a UNIX tool does only one thing, but does that one thing very well. For example, grep is very easy to use and can search vast amounts of data quickly. The find tool can find a particular file or files based on all kinds of criteria. It's pretty easy to string these tools together to build even more powerful tools, such as a tool that finds all of the .log files in the /home directory and searches each one for a particular entry. This erector-set mentality allows UNIX system administrators to seem to always have the right tool for the job.
Cron traditionally has been considered another such a tool for job scheduling, but is it enough? This webinar considers that very question. The first part builds on a previous Geek Guide, Beyond Cron, and briefly describes how to know when it might be time to consider upgrading your job scheduling infrastructure. The second part presents an actual planning and implementation framework.
Join Linux Journal's Mike Diehl and Pat Cameron of Help Systems.
Free to Linux Journal readers.Register Now!
- SUSE LLC's SUSE Manager
- My +1 Sword of Productivity
- Murat Yener and Onur Dundar's Expert Android Studio (Wrox)
- Managing Linux Using Puppet
- Non-Linux FOSS: Caffeine!
- Doing for User Space What We Did for Kernel Space
- SuperTuxKart 0.9.2 Released
- Google's SwiftShader Released
- Parsing an RSS News Feed with a Bash Script
- SourceClear Open
With all the industry talk about the benefits of Linux on Power and all the performance advantages offered by its open architecture, you may be considering a move in that direction. If you are thinking about analytics, big data and cloud computing, you would be right to evaluate Power. The idea of using commodity x86 hardware and replacing it every three years is an outdated cost model. It doesn’t consider the total cost of ownership, and it doesn’t consider the advantage of real processing power, high-availability and multithreading like a demon.
This ebook takes a look at some of the practical applications of the Linux on Power platform and ways you might bring all the performance power of this open architecture to bear for your organization. There are no smoke and mirrors here—just hard, cold, empirical evidence provided by independent sources. I also consider some innovative ways Linux on Power will be used in the future.Get the Guide