Why Big Compute and Big Storage will meet Big Pipe at the Last Mile

The next big frontier for Big Linux Build-out will be at a back end that's as close as anyone can get the front lines of big video production. That is, to consumers who are now also producers. And the parties in the best position to pioneer that frontier aren't in Seattle or Mountain view. They're in your home town.

Take a look at the Pew Internet report on Teens and Social Media. Then take a look at Terry Heaton's Study Teens, Study the Future. Then take a look at the large flat screens selling at every appliance store, or at the one already in your living room. Then look at all the HD video being shot, edited and put up on ...YouTube?.

TechCrunch reported last month on YouTube's plans for a higher-definition service at some point, but with Xtreme qualification:

Speaking at the NewTeeVee conference yesterday, YouTube co-founder Steve Chen confirmed that YouTube was testing HD video but qualified the statement by stating that YouTube is primarily focused on providing content to everyone, which doesn't necessarily facilitate a HD product.

Chen spoke on the difficulties on providing a watchable HD product to many, including buffering times that don't drive viewers away. Chen told CNet that the first HDHigh Quality content on YouTube should be available within 3 months

Note the strike-through of HD. Also note the "difficulties" remark. Is he talking about storage, or the computing required for transcoding and the rest of it. Not likely. Google, which owns YouTube, is the biggest honking player at those games already. Though the biggest player at the back end these days is Amazon, with Amazon Web Services (AWS), which recently added DevPay and SimpleDB to EC2, S3 and the rest of the company's growing portfolio of back-end services. Let's run just those four down.

  • DevPay is "a simple-to-use billing and account management service that makes it very easy for developers to get paid for applications they build on Amazon Web Services"
  • SimpleDB runs structured queries on simple data in real time.
  • EC2 (Elastic Compute Cloud) runs resizable compute capacity
  • S3 (Simple Storage Service) provides cheap and easy storage of any size

What you're seeing here, at least partially (and ever more completely), is the new phone company business being re-invented from the back end forward. What makes AWS a phone company business is DevPay. Billing. Phone companies are basically billing engines. The difference is that phone companies have long been in the business of billing in monopoly conditions, often for scarcities that are essentially artificial. That is, created for the simple need to have something to bill.

The new phone company business, however, is one that's built around abundance. That's the clinic Amazon is holding for phone companies — and cable companies as well — with AWS.

Amazon is also setting itself up as an ideal partner for phone and cable companies, which bring several huge assets to the collective table: customers, local real estate, and pipes over the last mile to homes and businesses. Not to mention billing engines that can be repurposed for anything.

The Net may have an end-to-end architecture (back in '03 David Weinberger and I called it a World of Ends), but the realities of provisioning and latency cause "difficulties" of the sort Steve Chen hinted about. If you work or live in places where you get upwards of 20Mb/sec of upload and/or download speeds (as I do), you can see the picture when you run speed tests at distances upward of a couple hundred miles from your location. tend to go down. There are exceptions, but on the whole bit transport is faster locally than over long distances.

This, of course, is why Akamai is in business too. They provide servers, services and various kinds of localized optimizations. The result, Akamai brags, "has transformed the chaos of the Internet into a predictable, scalable, and secure platform for business and entertainment".

See the hole in that claim? It's you. The individual. The guy or gal or small business or school or church with a basic Internet connection — and a growing sum of "content" that's all yours, waiting for the Net to come through with its original promise of pure connected utility.

I believe the big driver here will be home-brewed video. The tipping point will come when there is sufficient storage and compute capacity on laptops and desktops for high-enough-quality home (and school, and church, and business) video production at qualities of 1080i, 1080p and up. Yes, digital movies today are recorded (no longer "filmed" in the literal sense) at higher resolutions, but it's significant that George Lucas shot some of his recent Star Wars movies with a 1080p/24fps camera — the likes of which will soon be affordable to ordinary "consumers" who are now also producers.

All this content will need to be transported, stored, streamed and otherwise shared. The customers needing to do all of that have already laid out good money for displays, cameras, computers and other related production gear and software. Price the services right (again, look to the Amazon models), and the business can come flowing in.

Telcos and cablecos have enormous advantages in a pile of critical areas:

  • Existing relationships with customers
  • Existing piping to homes and businesses
  • Existing relationships with municipalities and others sharing poles and conduits
  • Existing real estate
  • Membership in local business and civic communities
  • Ability to set up local "channels" for all kinds of stuff

The key will be switching off the scarcity model and switching on the abundance model. Make the transport cheap, symmetrical and open as possible — and make the big money on services. Do it by yourself or in partnership with others that are doing it already. One good example is outsourced IT companies such as MakeITWork, in Santa Barbara.

It's critical to get rid of prices that discriminate against businesses. Far as I know, all phone and cable companies in the U.S. still charge through the nose for "business" service that is hardly any different than home service. For example, Verizon's FiOS (fiber) for Business prices are far higher than for their ordinary consumer service. To the company's credit, it's beginning to offer symmetrical service for both consumers and businesses. But the consume/business distinction needs to go away. The sum of production coming out of homes will make the "consumer" label an archaic misnomer.

Concept is king. Phone and cable companies need to stop conceiving the Net as a tertiary service, subordinate to phone and television. Instead they need to conceive the Net as a vast and wide open transport system with enormous opportunities that will grow around maximizing services to local customers at its countless ends.

It won't be easy, but it will be necessary.

And filling that necessity will be Linux and open source deployments out the wazoo, of course. For models there, just look to Google, Amazon, Akamai and most of the reliable hosting services out there.


Doc Searls is the Editor in Chief of Linux Journal


Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Toques para celular

Anonymous's picture


Hroberts's picture

When I first heard about wireless Internet service, I was a little bit skeptical. My son was begging me to get it. He thought the idea of wireless broadband Internet was so cool. You see, he is somewhat of a techno-Utopian. Having grown up with the Web, he has seen firsthand how much information is available and how easy communication is between different people. He writes two blogs and reads several more, talks to his friends online, And basically spend every spare moment surfing the web.

His argument for wireless Internet service is pretty ideological. You see, a few months ago we were going through hard times. One of the first things that we had to cut was Internet broadband service. We tried to use Internet dial up for awhile, but it was so unreliable that it didn't seem worth the 10 dollars a month. What I didn't know at the time was that my neighbor had wireless Internet services. She had made them free for anyone to use, not bothering to set up wireless Internet security. Whether or not she did this intentionally, I will never know, but my son set it up so that we could use her Internet.


Basically, she didn't really lose anything in the deal. Unless you are downloading or uploading massive amounts of data, you are rarely ever using your maximum bandwidth. Providing free wireless Internet service to anyone who might be passing by, looking for a link, does nothing bad to you. What it does do is help to provide more coverage in your area. If enough people are willing to plug in wireless Internet service without protecting it with a password, everywhere will be wired. Soon, anyone with a laptop will be able to access the Web from anywhere in the world. I can't figure out why this is such an important goal, but to my son it is of paramount importance.

Nonetheless, I can't agree to get wireless Internet service. You see, I simply don't trust it. I don't want strangers able to log into my Internet service provider anytime they want. I don't know enough about computers to know if my Internet connection is secure, and I'm not quite willing to take my son's word for it. For the time being, our cable modem works just fine, thank you very much. We will let someone else provide Internet service for the masses.

"make the big money on services."

Hamish MacEwan's picture

I have my doubts, you don't make big money in commodity utilities, you make reliable money and you have a reliable future, but the big, risky money is always further up the stack.

I've lost count of the number of times I have rolled out the pace layering diagram you used in an ancient presentation on civilisation moving at different speeds. Infrastructure doesn't make big money. Fashion and commerce do.

Now that we can have the structural separation that most other transport networks enjoy, even railways (in NZ & UK), that's what we need.

That's the trouble for the telcos, they have service return ambitions on infrastructural skills, which in the past, being bound together, meant the best of both worlds. Guaranteed monopoly granting high risk returns on low risk investment.

Or, put much better by Malcolm Matson:

"BT, like so many telcos faced with these shifting tectonic telecom plates, find themselves with assets supporting one [infrastructure], and earnings expectations based on the other [services]."

Attempting to lure telcos out of their troll mindset with the notion they can be TV or service providers and make "big money" is the bait others have tried for decades. And it doesn't work, because they're making "big money" now. Not forever, or even for long, but for now.


How "now"?

Doc Searls's picture

Hamish, fashion and commerce indeed do make money, and there is are countless opportunities for telcos and cablecos to make money supporting those things, rather than trying to own and control the whole stack.

Structural separation helps, which is why the European telcos are on the whole far ahead of those in the U.S., at least in terms of their consciousness of opportunities other than 'triple play". Not that they don't have a long way to go. They do. (Disclosure: i consult BT, which is one reason I won't remark on them. I'm also addressing this post to the U.S. situation, fwiw.)

It will be very hard for all the telcos to come out of their 'troll mindset", but all it takes is one.

I remember back in the 1980s, when every "local area network" (LAN) company other than Novell wanted to own the whole stack, from the wires upward. Magazines such as Data Communications were thick with editorial and ads that debated the relative merits of star, bus and ring topologies, of thin and thick Ethernet cabling, of twin-ax and co-ax, of six different kinds of Ethernet, and so on. Interoperability was zero. "Open" was blabbed everywhere and completely meaningless.

Then in stepped Novell, reconceiving the network as services, starting with file and print, and offering a "NOS" (network operating system) — NetWare — that ran on any hardware, any wiring, any way of moving data. And then supporting third parties higher up the stack. It wasn't as open as we now have with open source stacks, but it was an enormous advance.

I remember when Craig Burton told me he didn't want to brag about Novell selling more Ethernet cards than anybody else — a many-million dollar business — because in time these cards, which commonly sold for upwards of a thousand apiece, would be "a zero dollar business". He said Ethernet made far more business possible than it would ever make on its own. And that was good for both 3Com (founded in part by Ethernet dad Bob Metcalfe) and archrival Novell.

In time Net infrastructure plays will be close to a "zero dollar business" as well. Not right away, but in time. There will be far more leverage in ubiquitizing fat bandwidth than in selling it. "Because" will beat "With".

One of these companies is gonna roll. If they don't, some other company will. Or users will start building their own (ultimately) zero-dollar workaround from the edges in. The nothing they pay for their home networks, which are capable of 100Mb to 1Gb already, will be projected on the Net outside as well.

It's a matter of time. I'm just trying to shorten that.

Rock on,


Doc Searls is the Editor in Chief of Linux Journal

"Because" will beat "With". (please explain)

Anonymous's picture

Happy Holidays Doc.

" There will be far more leverage in ubiquitizing fat bandwidth than in selling it. "Because" will beat "With". "

Please explain.

Because vs. With

Doc Searls's picture

The short answer (because our plane is taking off shortly) is a question:

Which makes more money... the businesses selling bandwidth, or the businesses built on that bandwidth?

The first is with. The other is because of.

I'm not arguing against making money by selling pure bandwidth. I am arguing for finding other ways of making money. And I think there are lots.

Two pieces that explain "because effects" a bit:

"Greater Goods"

"Making a new world

Doc Searls is the Editor in Chief of Linux Journal

Because vs. With

Doc Searls's picture

The short answer (because our plane is taking off shortly) is a question:

Which makes more money... the businesses selling bandwidth, or the businesses built on that bandwidth?

The first is with. The other is because of.

I'm not arguing against making money by selling pure bandwidth. I am arguing for finding other ways of making money. And I think there are lots.

Two pieces that explain "because effects" a bit:

"Greater Goods"

"Making a new world

Doc Searls is the Editor in Chief of Linux Journal