PyCon DC 2003

A wrapup of what happened at this year's Python conference, a DIY conference organized by Python enthusiasts to catch up on new projects and discuss the future.
The Hundred-Year Language

Paul Graham gave the keynote address on the programming language he envisions we'll be using in a hundred years. He says speculating this is not far-fetched, because programming already has existed for almost fifty years and has changed remarkably little. In a hundred years we'll still be writing programs in text files of some sort, and there will still be something called Fortran.

Graham says the languages that will survive the evolutionary change are those with the fewest number of axioms (primitives), because they are the most flexible. As hardware becomes ever more powerful, "inefficient" but flexible programming practices become feasible, and the languages that do this are thriving because they make work easier for programmers. For instance, Python's flexible data types are more flexible than C's or Java's strong types. Graham calls his search for flexibility a search for creative ways to "waste" CPU cycles in the future.

Graham recommends an even higher-level abstraction for data types. For instance, there's no need for a separate string primitive because a string can be represented as a character array (list). So let strings be dealt with through library routines rather than being built into the language. Arrays aren't necessary because they are simply a special case of mappings (dictionaries). His most radical suggestion is the elimination of numeric types, because a number N can be represented by an array of length N. There even are proofs that mathematical operations, such as adding and multiplying, can be performed on arrays as easily as on special numeric types--even negative and fractional numbers, I'm told. Don't worry about this type surgery infecting Python: Guido wasn't buying it. And Graham says Python already is closer to his ideal than most languages are.

Graham concluded by saying it's great that programming languages are being written now by programmers rather than by professors, because programmers have practical needs that professors tend to ignore. Somebody also asked what the killer application will be in a hundred years. The answer: games with skins, of course.

Twisted

Every Python conference offers one talk that has the biggest effect on me. At Python9 it was the Webware talk; at Python10 it was Tim Berners-Lee's talk on web-izing Python and RDF. Here at PyCon it was the Twisted tutorial. Twisted's developers were out in force, both at the sprint and with an entire track of refereed papers. Previously I had seen the Twisted web site and thought, "confusing, too big a learning curve, nothing I need", but hearing it explained in person really helped. Twisted is a modular platform for building internet applications, essentially an OO level that integrates Python's networking libraries and combines them with common internet services. It uses a single-threaded, non-blocking form of I/O, which they claim is significantly faster than multithreaded or multiprocess paradigms. Twisted has a few basic classes:

Reactor

The network event loop

Transport

TCP, SSL, UNIX domain sockets, etc.

Protocol

An interchange format for data (HTTP, SSH, DNS, etc.)

Factory

Creates a protocol-transport instance for a new client connection

Producer

Generates data at the rate a Consumer needs it

Service

Your program-specific logic; aka "business logic"

Application

Encapsulates all of the above

twistd

The dæmon that runs your application

One of Twisted's main features is the separation of transport from protocol. That makes changing a program's transport a simple two-line process, including one line for the import. It's useful for debugging programs from an alternate I/O source or for adapting to unforeseen circumstances.

As for the kinds of applications Twisted supports, somebody once said, "All applications eventually evolve to the point that you can read e-mail from them." Twisted can do that, as well as provide an SSH interface, a DNS server, a MUD game server and whatever else your application might want. The libraries are only an import statement away.

Why does Twisted have an SSH server? Why do people climb mountains? Because they're there; because they can. And it turns out that Twisted's SSH is faster than OpenSSH, because the latter forks twice for each connection. Why use Twisted's DNS rather than BIND? It's much smaller and simpler, so it's less prone to security alerts. Why use Twisted Web rather than Apache? The configuration file is much easier, employing ordinary Python method calls rather than a special configuration syntax.

For GUI programming, alternate reactors play well with various GUI event loops (GTK, Qt and so on). There are also libraries for accessing DBI-compliant databases (MySQL, etc); they are necessary because Twisted is single-threaded. So, if you use the standard database libraries, they will do blocking I/O and freeze all the other connections. There is also a library for running your business logic in a separate thread, in case you have to do blocking I/O for any other reason.

Twisted Web has a simple transaction model. A URL such as /foo/bar is translated to:

site.getChild('foo', request).getChild('bar', request).render()

foo and bar are twisted.resource.Resource objects. You define .getChild() and .render().

______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Links to the Refereed Papers - Updated

Anonymous's picture

Re: PyCon DC 2003

Anonymous's picture

Nice article, but two mistakes:

  • Zope3 does as yet does not use Twisted in any way. There are
    plans to this effect but nobody has yet stepped up to do the
    work.
  • PyCon while very nice was not Python's first amateur run
    conference. There were various smaller ones in the past,
    but EuroPython 2002 was entirely volunteer run and had
    in fact more attendees than PyCon. You can imagine that
    hearing this claim (now the second time) is rather frustrating
    to those volunteers (me included). I guess it
    could be taken as a compliment that so many people
    apparently think EuroPython was professionally organized. :)

Martijn Faassen

Webinar
One Click, Universal Protection: Implementing Centralized Security Policies on Linux Systems

As Linux continues to play an ever increasing role in corporate data centers and institutions, ensuring the integrity and protection of these systems must be a priority. With 60% of the world's websites and an increasing share of organization's mission-critical workloads running on Linux, failing to stop malware and other advanced threats on Linux can increasingly impact an organization's reputation and bottom line.

Learn More

Sponsored by Bit9

Webinar
Linux Backup and Recovery Webinar

Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs. However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.” After all, before data can be restored to a system, there must be a system to restore it to.

In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.

Learn More

Sponsored by Storix