At the Forge - Issue 200

Reuven reminisces about the past of the Web, describes its present state and makes some predictions for the future.
The Present

Today, it's easy to create Web applications. Almost any part of the technological infrastructure you might need—including operating systems, databases, programming languages and frameworks—is available under an open-source license. Indeed, the problem is often not a matter of finding something that will be suitable, but rather sorting through the many competing open-source projects, each of which has its own advantages and disadvantages.

Open source is now the norm and even is expected in many places. I recently spoke about Ruby on Rails at a conference for Web developers in Israel, where one of the keynotes was given by a Microsoft employee. Every other sentence he uttered talked about open-source software, getting popular open-source packages to work under Microsoft technologies, and how small- and medium-size sites can get access to Microsoft products for free, until they achieve a certain level of success. In other words, Microsoft understands that the balance is shifting to the Open Source world and is competing by offering greater standards compliance and lower prices—something that open-source advocates can claim as a victory of sorts.

Modern Web development often takes place inside a “framework”, a collection of libraries that make the developer's life easier. Some of the most popular Web frameworks are Rails (Ruby), Django (Python), Symfony (PHP) and Catalyst (Perl), although there are dozens, and maybe hundreds, of others for these languages and others. By using a framework, developers can concentrate on their specific domains, rather than re-inventing the same infrastructure multiple times.

Most of these frameworks use the MVC (model-view-controller) paradigm pioneered more than 20 years ago by languages such as Smalltalk, reflecting not only the increasing complexity and sophistication of Web applications, but also the size and diversity of the teams needed to create such an application. Keeping things separate within an MVC framework ensures that a designer will probably not step on a developer's toes during the development process. By adopting the “convention over configuration” idea pioneered by Ruby on Rails, developers also can avoid discussions, arguments and consideration of where each file should be located.

Today, the question is not whether you want to use a database for data storage, but rather which one you want to use, whether it will be relational or non-relational (“NoSQL”), and what sort of interface you will use to communicate with it. Most modern frameworks handle relational databases seamlessly, often providing you with an ORM (object-relational mapper) that allows you to ignore the fact that you're actually using SQL to store information in two-dimensional tables. There also is growing support for non-relational databases in these Web frameworks, making it possible to choose what type of data storage is ideal for your particular application.

Not only have the frameworks changed, but the languages are starting to change too. Perl continues to be popular in some corners, and PHP still is hanging on, but the growth and action appears to be with Ruby and Python, as well as with many other newer languages. Indeed, I often say that Perl was perfectly suited to early Web applications, because its strengths were in text manipulation, networks and databases—precisely what you need for a Web application. As applications became larger, these strengths were less important than the ability to maintain code, something for which Ruby and Python are (in my opinion) better suited.

As we move into the future, we're seeing a need for functional and distributed programming, which has made languages such as Scala, Clojure and Erlang more popular. Scala and Clojure, although very different languages, are both built on top of the Java virtual machine (JVM), as is jRuby. The growing use of the JVM as the underlying infrastructure for a non-Java language continues to interest me, and it raises the question of what will happen to Java itself over time, as these languages become even more popular.

Perhaps the biggest surprise, to me at least, has been the growth of JavaScript during the past few years from a language that was barely used to animate some menus, to one that has led to the introduction of radically new JavaScript engines in all of the major browsers and to the creation of several high-quality, cross-platform libraries. I certainly tended to pooh-pooh JavaScript as a language. In many ways, the reason I now like working with JavaScript is because of the libraries (such as jQuery and Prototype) that insulate me from some of the problems with the language, rather than changes to the language itself.

JavaScript also continues to pop up in places other than browsers. JSON, the JavaScript object notion, has become a very popular, lightweight alternative to XML for transmitting data between computers. And Node.js, a JavaScript library for creating high-performance network and server applications using JavaScript, has begun to make serious inroads.

Once you've put together your application, where are you going to host it? You still can put it on a server that you own or on a fraction of a server that you rent, but cloud computing has taken hold of the industry, not only because it makes hosting so much easier, but also because it means you no longer need to hire a full IT staff to run the servers.

Finally, whereas we think of Web applications as having to do with people, the fact is that many applications are for machine-to-machine communication. The growth of various microformats, along with JSON and XML-based systems continues to rise. Moreover, the growth (and importance of) APIs has exploded during the past few years. Although it used to be a nice thing for a Web application to offer an API, it now is almost expected that everyone will offer an API, for use by desktop applications, mobile applications, aggregation systems or new uses that mix and match what already has been done.

Right now, we're enjoying what might seem to be the best of all possible worlds: easy, cheap and scalable hosting, programming languages and frameworks that lend themselves to rapid, maintainable development, and storage systems that are flexible, which connect seamlessly to our programming framework of choice. The main limits to creating Web applications today have more to do with skill and time than money, as we can see from the rapid growth of applications on Facebook, for example.