At the Forge - Syndication with RSS

When I first started to use the Web, anyone who put up a site would send e-mail to Tim Berners-Lee, giving the URL and a brief description of what the site was about. Tim would respond with a brief personal note and would update his master list of Web sites, which anyone with a browser could retrieve. Active participants in the Web community would review that list regularly—and its successor, published by the same people who produced the Mosaic browser—for new and updated sites, so as not to miss a bit.

Fast-forward more than a decade, and the Web is obviously too large for anyone to maintain a list of new sites manually. And even if that were possible, no one can read more than a fraction of the new content that goes live each day. Add to this the fact that now there are hundreds of thousands of Weblogs, or blogs, many of which are frequently updated, and the task becomes even more difficult.

One solution is to use your browser's bookmarks. But after a while, it becomes a chore to check bookmarks each day, let alone several times a day. It would be nice if each site could indicate when its content has changed, so that you would visit only when necessary.

This insight is not new; the idea of announcing changes to Web content has existed for several years. But I must admit, it was only a few months ago when I began to realize how behind the times I was, when I would start each day by visiting a few of the sites in my bookmarks. By taking advantage of an RSS aggregator—namely, a program that looks at the RSS feeds from various sites and alerts me when there has been an update—I am able to do more in less time.

This month, we discuss the popular RSS (really simple syndication or RDF site summary) family of formats, looking at ways in which it might be useful and how it is created.

RSS began as the brainchild of Netscape, the Internet software company that has since been absorbed (and largely dismembered) by AOL. Netscape wanted to offer people news from multiple sources but on a single page. They accomplished this by publishing the specification for RSS 0.90. Anyone interested in publishing news through the Netscape portal needed to do so in RSS. Netscape's system would retrieve this RSS document from the Web site in question and publish the results.

Although RSS 0.90 sparked a revolution, it also was fairly complicated. Dave Winer, then the head of Userland Software, turned RSS into a simple specification, renamed it RSS 0.91 and began to talk about it on his Weblog, scripting.com. Suddenly, RSS 0.91 was everywhere; Dave's orange XML buttons, indicating that you could get an RSS feed from a site, became quite popular. Within a few years, RSS feeds sporting other versions were available as well. RSS 1.0 was developed by a group of developers on the Web, and RSS 2.0, coordinated by Dave, was seen as an upgrade to 0.9x.

If you have been following this history, you might have reached the conclusion that now there are three different syndication formats called RSS. Aside from the version numbers, and some obvious similarity between the different versions, these are three different formats.

In many ways, RSS resembles HTML and HTTP, which began as simple-to-understand, simple-to-implement standards written by a small group of people. All three of these standards have been forced to mature quite a bit in the past few years, losing some of their flexibility and simplicity in the process.

RSS 0.91 is the simplest of the bunch and is still rather popular. Everything sits within an <rss> element, which identifies its version and contains a single <channel> element. Several required tags (title, link, description, language and image) are followed by one or more <item> elements. Each item has its own title, link and description. For example, here is a simple RSS feed from my Weblog:

<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE rss PUBLIC
"-//Netscape Communications//DTD RSS 0.91//EN"
"http://my.netscape.com/publish/formats/rss-0.91.dtd">

<rss version="0.91">

<channel>
  <title>Altneuland</title>
  <link>http://altneuland.lerner.co.il/</link>
  <description>Reuven's Weblog</description>
  <item>
    <title>Independence Day</title>
    <link>http://altneuland.lerner.co.il//40</link>
  </item>
  <item>
    <title>Linux desktops for the masses?  Ha!</title>
    <link>http://altneuland.lerner.co.il//39</link>
  </item>
</channel>
</rss>

If you examine the above RSS feed, you can see it does not conform to the RSS 0.91 specification I described previously. Specifically, it lacks the required language and image elements within channel, and it lacks a description element within each item. Unfortunately, this comes as no surprise; as was the case with HTML in its earliest years, software authors often cut corners, producing output that was good enough for most purposes. And indeed, COREBlog (which, as of this writing, I am using to produce my Weblog) seems to have cut such a corner, producing a usable, but substandard, RSS 0.91 feed.