Home › Blogs › Doc Searls's blog ›

What's the tweeting protocol?

Feb 17, 2009  By Doc Searls

Getting a fail whale this morning again on Twitter.

SMTP never gave me a whale. Nor has POP3, SSH, XMPP or any of the other protocols in the Internet Suite.

Twitter is a brilliant invention. It is also not quite a walled garden, which is good. But it's a private service with many single points of failure owned and run by one company. (Am I wrong? Could be. If so, tell me how. I'm here to re-learn.)

Being able to use other companies that do similar things (e.g. identi.ca and FriendFeed) is good, but not good enough.

If tweeting is something that ought to be Net-native, then we should find a free and open way to make it so.

Laconica, used by Identi.ca, is a free and open microblogging platform. But is it enough?

Not sure. Seems we still end up with dependencies that make services like Twitter (and even identi.ca) not quite what we need.

But, I dunno. What do ya'll think?

[Later...] Read down. of and explains how openmicroblogging is the protocol I'm asking about.

And read this post by Dave Winer. The money quote: "It's time for a thousand Twitters to bloom. The mother ship will do great, but we need a path that's independent of the corporate entity that runs Twitter."

______________________

Doc Searls is Senior Editor of Linux Journal

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

clear credit card debt

clear credit card debt's picture
Submitted by clear credit card debt (not verified) on Thu, 02/19/2009 - 02:01.

Till now we are only aware of Linux rules but thanks for giving the knowledge about exposure of Linux.

The protocol? HTTP, of course!

Jonathan Guthrie's picture
Submitted by Jonathan Guthrie (not verified) on Tue, 02/17/2009 - 23:29.

The protocol used by Twitter, and by just about everybody else that publishes an API, is a variant of XML with HTTP as the transport.

Protocol design, as such, is so 20th century. It's stuff that you have to hire old programmers (like me!) to do. Nowadays it makes more sense to send XML over HTTP to a Web framework like Rails or Django or whatever than it does to design a protocol from scratch. Communications technology has been heading this way for the twenty years I've been programming professionally, and I'm wondering where it's going next.

Twitterati

Paul McRae's picture
Submitted by Paul McRae on Tue, 02/17/2009 - 17:00.

I haven't quite figured out why Twitter is so popular. Dang kids, get off my lawn.

RSS is open

James Robertson's picture
Submitted by James Robertson (not verified) on Tue, 02/17/2009 - 12:07.

Here is how I think it should work:

  1. You update your microblog and press 'send'.
  2. Your server pings the server of all your friends to let them know you have updated.
    • if the server address of one of your friend's server changes it will be updated on your server database the next time they post a new update.
  3. Your friends servers fetch your latest update(s) from your RSS file.

In the event that your server goes down:

  1. Install a new instance of the microblogging system and update as normal.

My 2 cents

Freemor's picture
Submitted by Freemor (not verified) on Tue, 02/17/2009 - 10:16.

I love microblogging and think it will become a mainstay of the Internet. I however don't think Twitter will be that mainstay. I think Doc Searls was close to the point when he talked about smtp and pop3. Justin is right that Twitter is not a protocol, however if we go with the e-mail analogy for just a bit one can start to see where the problem lies. The current Twitter model is like having all e-mail, everywhere, run through one huge server. If that server goes down NOBODY has e-mail. When you compare this with the distributed nature of (most) other services on the net you can see the difference. If my ISP's mail server goes down it is a local problem not a global one. Also due to the prevelence and interoperability of smtp/pop servers I can still send e-mails to all my friends I just have to use a different server until my ISP gets their mail server back up. Turning the outage into a minor annoyance.

I think the Laconica model of microblogging, or some ancester of it will be what survives. The federated nature Laconica means that anyone can set up their own microblog. And their microblog will interact with all the other Laconica type micro blogs. So if I have an account on two Laconica type microblogs it's suddenly no biggie if one goes down I could still microblog from the other if it were a long outage I could send @ messages to friends informing them to follow my second feed.

I think it is this distributed nature that microblogging needs. Especially as it becomes more "important" in our lives the need for a faut tollerant system will pretty much demand it not be centralized.

Just my 2 cents
Freemor

Re: My 2 cents

RR's picture
Submitted by RR (not verified) on Fri, 02/20/2009 - 13:25.

"I can still send e-mails to all my friends I just have to use a different server until my ISP gets their mail server back up."

My ISP blocks port 25 to its residential customers, so if all of its SMTP servers are down (or unreachable from my home), I have no backup email service -except for Gmail or other webmail service.

In theory, I could use port 587 (mail submission service), but that requires I be authorized by the owner/operator of the server in question.

Give it a try and tell us!

Evan Prodromou's picture
Submitted by Evan Prodromou (not verified) on Tue, 02/17/2009 - 10:05.

Doc, you ask, "Laconica, used by Identi.ca, is a free and open microblogging platform. But is it enough? [...] Seems we still end up with dependencies that make services like Twitter (and even identi.ca) not quite what we need."

One thing that makes Laconica different than Twitter is that it supports a federated mesh of interconnected servers. Users on the Twit Army site, or Bleeper, or any of the other public servers that run Laconica, can subscribe to the notice streams of users on Identica (and each other, of course). A poster's service pushes their notices out to the servers of all subscribers to get near-real-time conversations.

The protocol we use is called OpenMicroBlogging. It has at least one other implementation, and we're actively lobbying Open Source and proprietary software developers in the µblog space to support it. Version 0.2 of the protocol is under active development.

If that sounds familiar -- it should be. We have a similar model with SMTP and other Internet email protocols -- people have one or more accounts on your own trusted servers, and those servers deal with the inter-site protocols.

Why isn't this more popular, then? Well, for the most part, because people who know better continue to use that monolithic SPOF system. What Identi.ca, and Laconica, need are advocates like you. You know that this is the Right Way to do it. And you know that walled gardens and a single points of failure aren't healthy for a growing network. You've been involved in FLOSS for a long time, so you know (and perhaps relish) the anguish of using an unpopular but superior program. You're ready to make the jump!

If you're not interested in having a presence on Identi.ca, we'd be happy to set up and maintain a dedicated Laconica server just for you. You'd be an awesome test case. If it makes it easier: we've built-in some tools to make it easy to maintain a presence on Twitter while doing the Right Thing.

Let me know your thoughts -- I'm very willing to help if you're willing to take the next steps.

A small FYI: Doc Searls at

Anonymous's picture
Submitted by Anonymous (not verified) on Tue, 03/23/2010 - 19:15.

A small FYI: Doc Searls at identi.ca is @dsearls (just like at Twitter)

oxymoron?

Anonymous's picture
Submitted by Anonymous (not verified) on Tue, 02/17/2009 - 09:54.

> But it's a private service with many single points of failure

That sounds kinda like an oxymoron...

Hmmmm.

Justin Ryan's picture
Submitted by Justin Ryan on Tue, 02/17/2009 - 08:07.

You make a good point - if the service is as all-pervasive as it's made out to be, it should be more stable.

However, you make a distinction as well that I think puts the issue into a clearer light: You've never gotten a fail whale from any of the protocols you listed, but surely you have from a service using those protocols. POP3 & SMTP might not fail us, but Gmail does sometimes. (As do all the other mail services at one time or another, regardless of whether they're free or pay, web-based or desktop.) SSH doesn't fail, but the specific server you're trying to sign into can and eventually, probably will. The protocol doesn't fail, the service using it does, and if that happens, you can choose to use a different service using that protocol.

Twitter is no different. Twitter isn't a protocol, it's a service. It's not SMTP or POP3 or SSH; it's a private business running a website that offers a service, and it's as vulnerable to failures as any other website. The protocol, if that dynamic is to be applied to Twitter, is "microblogging" itself. Microblogging didn't fail, just like POP3/SMTP/SSH didn't - a microblogging service did. Identi.ca and Plurk and FriendFeed and all the others are examples of other services using the microblogging "protocol" - they're the Hotmail/YahooMail/Mail.com to Twitter's Gmail, the server2/server3/server4 to Twitter's server1. You might go on (as though I haven't gone on enough...) and make the comparison to "regular" blogging - Twitter is to microblogging as Wordpress, for example, is to blogging. Wordpress.com might 404, but that doesn't mean blogging - as a protocol - failed.

I think Twitter should be more stable - and they do deserve credit for having stabilized the service a great deal. Still, they have millions in venture funding, some of which they just closed on, that could be being used to add infrastructure and manpower where necessary. If they're going to be the canonical example of microblogging, they've got to do more.

Justin Ryan is a Contributing Editor for Linux Journal.

I think you miss the point

Carl Meyer's picture
Submitted by Carl Meyer (not verified) on Tue, 02/17/2009 - 09:56.

"Microblogging" is a concept, not a protocol. The whole point is that there _ought_ to be a standard, free and open microblogging protocol equivalent to SMTP/POP3/SSH, and right now there isn't one. If there were, then all of the various microblogging services could be interoperable (like email servers), and it wouldn't matter nearly as much if a particular one goes down.

There is an open protocol

njg234908's picture
Submitted by njg234908 (not verified) on Tue, 02/17/2009 - 10:15.

There is an open protocol for Microblogging - see http://openmicroblogging.org as implemented by the open source microblogging platform laconica which powers http://identi.ca It allows for federated interoperable microblogging between different platforms. The problem is how do we get Twitter to give up their walled garden and come on board with the standard.

Webcast
More Resources
How to Build an Optimal Hadoop Cluster to Store and Maintain Unlimited Amounts of Data Using Microservers

Realizing the promise of Apache® Hadoop® requires the effective deployment of compute, memory, storage and networking to achieve optimal results. With its flexibility and multitude of options, it is easy to over or under provision the server infrastructure, resulting in poor performance and high TCO. Join us for an in depth, technical discussion with industry experts from leading Hadoop and server companies who will provide insights into the key considerations for designing and deploying an optimal Hadoop cluster.

Learn More

Sponsored by AMD

White Paper
More Resources
Red Hat White Paper: Using an Open Source Framework to Catch the Bad Guy

Built-in forensics, incident response, and security with Red Hat Enterprise Linux 6

Every security policy provides guidance and requirements for ensuring adequate protection of information and data, as well as high-level technical and administrative security requirements for a system in a given environment. Traditionally, providing security for a system focuses on the confidentiality of the information on it. However, protecting the data integrity and system and data availability is just as important. For example, when processing United States intelligence information, there are three attributes that require protection: confidentiality, integrity, and availability.

Learn more about catching the bad guy in this free white paper.

Learn More

Sponsored by DLT Solutions