Why Big Compute and Big Storage will meet Big Pipe at the Last Mile
The next big frontier for Big Linux Build-out will be at a back end that's as close as anyone can get the front lines of big video production. That is, to consumers who are now also producers. And the parties in the best position to pioneer that frontier aren't in Seattle or Mountain view. They're in your home town.
Take a look at the Pew Internet report on Teens and Social Media. Then take a look at Terry Heaton's Study Teens, Study the Future. Then take a look at the large flat screens selling at every appliance store, or at the one already in your living room. Then look at all the HD video being shot, edited and put up on ...YouTube?.
TechCrunch reported last month on YouTube's plans for a higher-definition service at some point, but with Xtreme qualification:
Speaking at the NewTeeVee conference yesterday, YouTube co-founder Steve Chen confirmed that YouTube was testing HD video but qualified the statement by stating that YouTube is primarily focused on providing content to everyone, which doesn't necessarily facilitate a HD product.
Chen spoke on the difficulties on providing a watchable HD product to many, including buffering times that don't drive viewers away. Chen told CNet that the first
HDHigh Quality content on YouTube should be available within 3 months
Note the strike-through of HD. Also note the "difficulties" remark. Is he talking about storage, or the computing required for transcoding and the rest of it. Not likely. Google, which owns YouTube, is the biggest honking player at those games already. Though the biggest player at the back end these days is Amazon, with Amazon Web Services (AWS), which recently added DevPay and SimpleDB to EC2, S3 and the rest of the company's growing portfolio of back-end services. Let's run just those four down.
- DevPay is "a simple-to-use billing and account management service that makes it very easy for developers to get paid for applications they build on Amazon Web Services"
- SimpleDB runs structured queries on simple data in real time.
- EC2 (Elastic Compute Cloud) runs resizable compute capacity
- S3 (Simple Storage Service) provides cheap and easy storage of any size
What you're seeing here, at least partially (and ever more completely), is the new phone company business being re-invented from the back end forward. What makes AWS a phone company business is DevPay. Billing. Phone companies are basically billing engines. The difference is that phone companies have long been in the business of billing in monopoly conditions, often for scarcities that are essentially artificial. That is, created for the simple need to have something to bill.
The new phone company business, however, is one that's built around abundance. That's the clinic Amazon is holding for phone companies — and cable companies as well — with AWS.
Amazon is also setting itself up as an ideal partner for phone and cable companies, which bring several huge assets to the collective table: customers, local real estate, and pipes over the last mile to homes and businesses. Not to mention billing engines that can be repurposed for anything.
The Net may have an end-to-end architecture (back in '03 David Weinberger and I called it a World of Ends), but the realities of provisioning and latency cause "difficulties" of the sort Steve Chen hinted about. If you work or live in places where you get upwards of 20Mb/sec of upload and/or download speeds (as I do), you can see the picture when you run speed tests at distances upward of a couple hundred miles from your location. tend to go down. There are exceptions, but on the whole bit transport is faster locally than over long distances.
This, of course, is why Akamai is in business too. They provide servers, services and various kinds of localized optimizations. The result, Akamai brags, "has transformed the chaos of the Internet into a predictable, scalable, and secure platform for business and entertainment".
See the hole in that claim? It's you. The individual. The guy or gal or small business or school or church with a basic Internet connection — and a growing sum of "content" that's all yours, waiting for the Net to come through with its original promise of pure connected utility.
I believe the big driver here will be home-brewed video. The tipping point will come when there is sufficient storage and compute capacity on laptops and desktops for high-enough-quality home (and school, and church, and business) video production at qualities of 1080i, 1080p and up. Yes, digital movies today are recorded (no longer "filmed" in the literal sense) at higher resolutions, but it's significant that George Lucas shot some of his recent Star Wars movies with a 1080p/24fps camera — the likes of which will soon be affordable to ordinary "consumers" who are now also producers.
All this content will need to be transported, stored, streamed and otherwise shared. The customers needing to do all of that have already laid out good money for displays, cameras, computers and other related production gear and software. Price the services right (again, look to the Amazon models), and the business can come flowing in.
Telcos and cablecos have enormous advantages in a pile of critical areas:
- Existing relationships with customers
- Existing piping to homes and businesses
- Existing relationships with municipalities and others sharing poles and conduits
- Existing real estate
- Membership in local business and civic communities
- Ability to set up local "channels" for all kinds of stuff
The key will be switching off the scarcity model and switching on the abundance model. Make the transport cheap, symmetrical and open as possible — and make the big money on services. Do it by yourself or in partnership with others that are doing it already. One good example is outsourced IT companies such as MakeITWork, in Santa Barbara.
It's critical to get rid of prices that discriminate against businesses. Far as I know, all phone and cable companies in the U.S. still charge through the nose for "business" service that is hardly any different than home service. For example, Verizon's FiOS (fiber) for Business prices are far higher than for their ordinary consumer service. To the company's credit, it's beginning to offer symmetrical service for both consumers and businesses. But the consume/business distinction needs to go away. The sum of production coming out of homes will make the "consumer" label an archaic misnomer.
Concept is king. Phone and cable companies need to stop conceiving the Net as a tertiary service, subordinate to phone and television. Instead they need to conceive the Net as a vast and wide open transport system with enormous opportunities that will grow around maximizing services to local customers at its countless ends.
It won't be easy, but it will be necessary.
And filling that necessity will be Linux and open source deployments out the wazoo, of course. For models there, just look to Google, Amazon, Akamai and most of the reliable hosting services out there.
Doc Searls is Senior Editor of Linux Journal
Practical Task Scheduling Deployment
July 20, 2016 12:00 pm CDT
One of the best things about the UNIX environment (aside from being stable and efficient) is the vast array of software tools available to help you do your job. Traditionally, a UNIX tool does only one thing, but does that one thing very well. For example, grep is very easy to use and can search vast amounts of data quickly. The find tool can find a particular file or files based on all kinds of criteria. It's pretty easy to string these tools together to build even more powerful tools, such as a tool that finds all of the .log files in the /home directory and searches each one for a particular entry. This erector-set mentality allows UNIX system administrators to seem to always have the right tool for the job.
Cron traditionally has been considered another such a tool for job scheduling, but is it enough? This webinar considers that very question. The first part builds on a previous Geek Guide, Beyond Cron, and briefly describes how to know when it might be time to consider upgrading your job scheduling infrastructure. The second part presents an actual planning and implementation framework.
Join Linux Journal's Mike Diehl and Pat Cameron of Help Systems.
Free to Linux Journal readers.Register Now!
- SUSE LLC's SUSE Manager
- Murat Yener and Onur Dundar's Expert Android Studio (Wrox)
- My +1 Sword of Productivity
- Tech Tip: Really Simple HTTP Server with Python
- Non-Linux FOSS: Caffeine!
- Managing Linux Using Puppet
- Returning Values from Bash Functions
- Rogue Wave Software's Zend Server
- Doing for User Space What We Did for Kernel Space
- Google's SwiftShader Released
With all the industry talk about the benefits of Linux on Power and all the performance advantages offered by its open architecture, you may be considering a move in that direction. If you are thinking about analytics, big data and cloud computing, you would be right to evaluate Power. The idea of using commodity x86 hardware and replacing it every three years is an outdated cost model. It doesn’t consider the total cost of ownership, and it doesn’t consider the advantage of real processing power, high-availability and multithreading like a demon.
This ebook takes a look at some of the practical applications of the Linux on Power platform and ways you might bring all the performance power of this open architecture to bear for your organization. There are no smoke and mirrors here—just hard, cold, empirical evidence provided by independent sources. I also consider some innovative ways Linux on Power will be used in the future.Get the Guide