PyCon DC 2003

A wrapup of what happened at this year's Python conference, a DIY conference organized by Python enthusiasts to catch up on new projects and discuss the future.
The Hundred-Year Language

Paul Graham gave the keynote address on the programming language he envisions we'll be using in a hundred years. He says speculating this is not far-fetched, because programming already has existed for almost fifty years and has changed remarkably little. In a hundred years we'll still be writing programs in text files of some sort, and there will still be something called Fortran.

Graham says the languages that will survive the evolutionary change are those with the fewest number of axioms (primitives), because they are the most flexible. As hardware becomes ever more powerful, "inefficient" but flexible programming practices become feasible, and the languages that do this are thriving because they make work easier for programmers. For instance, Python's flexible data types are more flexible than C's or Java's strong types. Graham calls his search for flexibility a search for creative ways to "waste" CPU cycles in the future.

Graham recommends an even higher-level abstraction for data types. For instance, there's no need for a separate string primitive because a string can be represented as a character array (list). So let strings be dealt with through library routines rather than being built into the language. Arrays aren't necessary because they are simply a special case of mappings (dictionaries). His most radical suggestion is the elimination of numeric types, because a number N can be represented by an array of length N. There even are proofs that mathematical operations, such as adding and multiplying, can be performed on arrays as easily as on special numeric types--even negative and fractional numbers, I'm told. Don't worry about this type surgery infecting Python: Guido wasn't buying it. And Graham says Python already is closer to his ideal than most languages are.

Graham concluded by saying it's great that programming languages are being written now by programmers rather than by professors, because programmers have practical needs that professors tend to ignore. Somebody also asked what the killer application will be in a hundred years. The answer: games with skins, of course.

Twisted

Every Python conference offers one talk that has the biggest effect on me. At Python9 it was the Webware talk; at Python10 it was Tim Berners-Lee's talk on web-izing Python and RDF. Here at PyCon it was the Twisted tutorial. Twisted's developers were out in force, both at the sprint and with an entire track of refereed papers. Previously I had seen the Twisted web site and thought, "confusing, too big a learning curve, nothing I need", but hearing it explained in person really helped. Twisted is a modular platform for building internet applications, essentially an OO level that integrates Python's networking libraries and combines them with common internet services. It uses a single-threaded, non-blocking form of I/O, which they claim is significantly faster than multithreaded or multiprocess paradigms. Twisted has a few basic classes:

Reactor

The network event loop

Transport

TCP, SSL, UNIX domain sockets, etc.

Protocol

An interchange format for data (HTTP, SSH, DNS, etc.)

Factory

Creates a protocol-transport instance for a new client connection

Producer

Generates data at the rate a Consumer needs it

Service

Your program-specific logic; aka "business logic"

Application

Encapsulates all of the above

twistd

The dæmon that runs your application

One of Twisted's main features is the separation of transport from protocol. That makes changing a program's transport a simple two-line process, including one line for the import. It's useful for debugging programs from an alternate I/O source or for adapting to unforeseen circumstances.

As for the kinds of applications Twisted supports, somebody once said, "All applications eventually evolve to the point that you can read e-mail from them." Twisted can do that, as well as provide an SSH interface, a DNS server, a MUD game server and whatever else your application might want. The libraries are only an import statement away.

Why does Twisted have an SSH server? Why do people climb mountains? Because they're there; because they can. And it turns out that Twisted's SSH is faster than OpenSSH, because the latter forks twice for each connection. Why use Twisted's DNS rather than BIND? It's much smaller and simpler, so it's less prone to security alerts. Why use Twisted Web rather than Apache? The configuration file is much easier, employing ordinary Python method calls rather than a special configuration syntax.

For GUI programming, alternate reactors play well with various GUI event loops (GTK, Qt and so on). There are also libraries for accessing DBI-compliant databases (MySQL, etc); they are necessary because Twisted is single-threaded. So, if you use the standard database libraries, they will do blocking I/O and freeze all the other connections. There is also a library for running your business logic in a separate thread, in case you have to do blocking I/O for any other reason.

Twisted Web has a simple transaction model. A URL such as /foo/bar is translated to:

site.getChild('foo', request).getChild('bar', request).render()

foo and bar are twisted.resource.Resource objects. You define .getChild() and .render().

______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Links to the Refereed Papers - Updated

Anonymous's picture

Re: PyCon DC 2003

Anonymous's picture

Nice article, but two mistakes:

  • Zope3 does as yet does not use Twisted in any way. There are
    plans to this effect but nobody has yet stepped up to do the
    work.
  • PyCon while very nice was not Python's first amateur run
    conference. There were various smaller ones in the past,
    but EuroPython 2002 was entirely volunteer run and had
    in fact more attendees than PyCon. You can imagine that
    hearing this claim (now the second time) is rather frustrating
    to those volunteers (me included). I guess it
    could be taken as a compliment that so many people
    apparently think EuroPython was professionally organized. :)

Martijn Faassen

White Paper
Linux Management with Red Hat Satellite: Measuring Business Impact and ROI

Linux has become a key foundation for supporting today's rapidly growing IT environments. Linux is being used to deploy business applications and databases, trading on its reputation as a low-cost operating environment. For many IT organizations, Linux is a mainstay for deploying Web servers and has evolved from handling basic file, print, and utility workloads to running mission-critical applications and databases, physically, virtually, and in the cloud. As Linux grows in importance in terms of value to the business, managing Linux environments to high standards of service quality — availability, security, and performance — becomes an essential requirement for business success.

Learn More

Sponsored by Red Hat

White Paper
Private PaaS for the Agile Enterprise

If you already use virtualized infrastructure, you are well on your way to leveraging the power of the cloud. Virtualization offers the promise of limitless resources, but how do you manage that scalability when your DevOps team doesn’t scale? In today’s hypercompetitive markets, fast results can make a difference between leading the pack vs. obsolescence. Organizations need more benefits from cloud computing than just raw resources. They need agility, flexibility, convenience, ROI, and control.

Stackato private Platform-as-a-Service technology from ActiveState extends your private cloud infrastructure by creating a private PaaS to provide on-demand availability, flexibility, control, and ultimately, faster time-to-market for your enterprise.

Learn More

Sponsored by ActiveState