Why Python?

Cardinal Biggles had Eric in the comfy chair for over four hours before wringing this confession from him...

My first look at Python was an accident, and I didn't much like what I saw at the time. It was early 1997, and Mark Lutz's book Programming Python from O'Reilly & Associates had recently come out. O'Reilly books occasionally land on my doorstep, selected from among the new releases by some mysterious benefactor inside the organization using a random process I've given up trying to understand.

One of them was Programming Python. I found this somewhat interesting, as I collect computer languages. I know over two dozen general-purpose languages, write compilers and interpreters for fun, and have designed any number of special-purpose languages and markup formalisms myself. My most recently completed project, as I write this, is a special-purpose language called SNG for manipulating PNG (Portable Network Graphics) images. Interested readers can surf to the SNG home page at http://www.catb.org/~esr/sng/. I have also written implementations of several odd general-purpose languages on my Retrocomputing Museum page, http://www.catb.org/retro/.

I had already heard just enough about Python to know that it is what is nowadays called a “scripting language”, an interpretive language with its own built-in memory management and good facilities for calling and cooperating with other programs. So I dived into Programming Python with one question uppermost in my mind: what has this got that Perl does not?

Perl, of course, is the 800-pound gorilla of modern scripting languages. It has largely replaced shell as the scripting language of choice for system administrators, thanks partly to its comprehensive set of UNIX library and system calls, and partly to the huge collection of Perl modules built by a very active Perl community. The language is commonly estimated to be the CGI language behind about 85% of the “live” content on the Net. Larry Wall, its creator, is rightly considered one of the most important leaders in the Open Source community, and often ranks third behind Linus Torvalds and Richard Stallman in the current pantheon of hacker demigods.

At that time, I had used Perl for a number of small projects. I'd found it quite powerful, even if the syntax and some other aspects of the language seemed rather ad hoc and prone to bite one if not used with care. It seemed to me that Python would have quite a hill to climb as yet another scripting language, so as I read, I looked first for what seemed to set it apart from Perl.

I immediately tripped over the first odd feature of Python that everyone notices: the fact that whitespace (indentation) is actually significant in the language syntax. The language has no analog of the C and Perl brace syntax; instead, changes in indentation delimit statement groups. And, like most hackers on first realizing this fact, I recoiled in reflexive disgust.

I am just barely old enough to have programmed in batch FORTRAN for a few months back in the 1970s. Most hackers aren't these days, but somehow our culture seems to have retained a pretty accurate folk memory of how nasty those old-style fixed-field languages were. Indeed, the term “free format”, used back then to describe the newer style of token-oriented syntax in Pascal and C, has almost been forgotten; all languages have been designed that way for decades now. Or almost all, anyway. It's hard to blame anyone, on seeing this Python feature, for initially reacting as though they had unexpectedly stepped in a steaming pile of dinosaur dung.

That's certainly how I felt. I skimmed through the rest of the language description without much interest. I didn't see much else to recommend Python, except maybe that the syntax seemed rather cleaner than Perl's and the facilities for doing basic GUI elements like buttons and menus looked fairly good.

I put the book back on the shelf, making a mental note that I should code some kind of small GUI-centered project in Python sometime, just to make sure I really understood the language. But I didn't believe what I'd seen would ever compete effectively with Perl.

A lot of other things conspired to keep that note way down on my priority list for many months. The rest of 1997 was eventful for me; it was, among other things, the year I wrote and published the original version of “The Cathedral and the Bazaar”. But I did find time to write several Perl programs, including two of significant size and complexity. One of them, keeper, is the assistant still used to file incoming submissions at the Metalab software archive. It generates the web pages you see at metalab.unc.edu/pub/Linux/!INDEX.html. The other, anthologize, was used to automatically generate the PostScript for the sixth edition of Linux from the Linux Documentation Project's archive of HOWTOs. Both programs are available at Metalab.

Writing these programs left me progressively less satisfied with Perl. Larger project size seemed to magnify some of Perl's annoyances into serious, continuing problems. The syntax that had seemed merely eccentric at a hundred lines began to seem like a nigh-impenetrable hedge of thorns at a thousand. “More than one way to do it” lent flavor and expressiveness at a small scale, but made it significantly harder to maintain consistent style across a wider code base. And many of the features that were later patched into Perl to address the complexity-control needs of bigger programs (objects, lexical scoping, “use strict”, etc.) had a fragile, jerry-rigged feel about them.

These problems combined to make large volumes of Perl code seem unreasonably difficult to read and grasp as a whole after only a few days' absence. Also, I found I was spending more and more time wrestling with artifacts of the language rather than my application problems. And, most damning of all, the resulting code was ugly—this matters. Ugly programs are like ugly suspension bridges: they're much more liable to collapse than pretty ones, because the way humans (especially engineer-humans) perceive beauty is intimately related to our ability to process and understand complexity. A language that makes it hard to write elegant code makes it hard to write good code.

With a baseline of two dozen languages under my belt, I could detect all the telltale signs of a language design that had been pushed to the edge of its functional envelope. By mid-1997, I was thinking “there has to be a better way” and began casting about for a more elegant scripting language.

One course I did not consider was going back to C as a default language. The days when it made sense to do your own memory management in a new program are long over, outside of a few specialty areas like kernel hacking, scientific computing and 3-D graphics—places where you absolutely must get maximum speed and tight control of memory usage, because you need to push the hardware as hard as possible.

For most other situations, accepting the debugging overhead of buffer overruns, pointer-aliasing problems, malloc/free memory leaks and all the other associated ills is just crazy on today's machines. Far better to trade a few cycles and a few kilobytes of memory for the overhead of a scripting language's memory manager and economize on far more valuable human time. Indeed, the advantages of this strategy are precisely what has driven the explosive growth of Perl since the mid-1990s.

I flirted with Tcl, only to discover quickly that it scales up even more poorly than Perl. Old LISPer that I am, I also looked at various current dialects of Lisp and Scheme—but, as is historically usual for Lisp, lots of clever design was rendered almost useless by scanty or nonexistent documentation, incomplete access to POSIX/UNIX facilities, and a small but nevertheless deeply fragmented user community. Perl's popularity is not an accident; most of its competitors are either worse than Perl for large projects or somehow nowhere near as useful as their theoretically superior designs ought to make them.

My second look at Python was almost as accidental as my first. In October 1997, a series of questions on the fetchmail-friends mailing list made it clear that end users were having increasing trouble generating configuration files for my fetchmail utility. The file uses a simple, classically UNIX free-format syntax, but can become forbiddingly complicated when a user has POP3 and IMAP accounts at multiple sites. As an example, see Listing 1 for a somewhat simplified version of mine.

Listing 1

I decided to attack the problem by writing an end-user-friendly configuration editor, fetchmailconf. The design objective of fetchmailconf was clear: to completely hide the control file syntax behind a fashionable, ergonomically correct GUI interface replete with selection buttons, slider bars and fill-out forms.

The thought of implementing this in Perl did not thrill me. I had seen GUI code in Perl, and it was a spiky mixture of Perl and Tcl that looked even uglier than my own pure-Perl code. It was at this point I remembered the bit I had set more than six months earlier. This could be an opportunity to get some hands-on experience with Python.

Of course, this brought me face to face once again with Python's pons asinorum, the significance of whitespace. This time, however, I charged ahead and roughed out some code for a handful of sample GUI elements. Oddly enough, Python's use of whitespace stopped feeling unnatural after about twenty minutes. I just indented code, pretty much as I would have done in a C program anyway, and it worked.

That was my first surprise. My second came a couple of hours into the project, when I noticed (allowing for pauses needed to look up new features in Programming Python) I was generating working code nearly as fast as I could type. When I realized this, I was quite startled. An important measure of effort in coding is the frequency with which you write something that doesn't actually match your mental representation of the problem, and have to backtrack on realizing that what you just typed won't actually tell the language to do what you're thinking. An important measure of good language design is how rapidly the percentage of missteps of this kind falls as you gain experience with the language.

When you're writing working code nearly as fast as you can type and your misstep rate is near zero, it generally means you've achieved mastery of the language. But that didn't make sense, because it was still day one and I was regularly pausing to look up new language and library features!

This was my first clue that, in Python, I was actually dealing with an exceptionally good design. Most languages have so much friction and awkwardness built into their design that you learn most of their feature set long before your misstep rate drops anywhere near zero. Python was the first general-purpose language I'd ever used that reversed this process.

Not that it took me very long to learn the feature set. I wrote a working, usable fetchmailconf, with GUI, in six working days, of which perhaps the equivalent of two days were spent learning Python itself. This reflects another useful property of the language: it is compact--you can hold its entire feature set (and at least a concept index of its libraries) in your head. C is a famously compact language. Perl is notoriously not; one of the things the notion “There's more than one way to do it!” costs Perl is the possibility of compactness.

But my most dramatic moment of discovery lay ahead. My design had a problem: I could easily generate configuration files from the user's GUI actions, but editing them was a much harder problem. Or, rather, reading them into an editable form was a problem.

The parser for fetchmail's configuration file syntax is rather elaborate. It's actually written in YACC and Lex, two classic UNIX tools for generating language-parsing code in C. In order for fetchmailconf to be able to edit existing configuration files, I thought it would have to replicate that elaborate parser in Python. I was very reluctant to do this, partly because of the amount of work involved and partly because I wasn't sure how to ascertain that two parsers in two different languages accept the same. The last thing I needed was the extra labor of keeping the two parsers in synchronization as the configuration language evolved!

This problem stumped me for a while. Then I had an inspiration: I'd let fetchmailconf use fetchmail's own parser! I added a --configdump option to fetchmail that would parse .fetchmailrc and dump the result to standard output in the format of a Python initializer. For the file above, the result would look roughly like Listing 2 (to save space, some data not relevant to the example is omitted).

Listing 2

Python could then evaluate the fetchmail --configdump output and have the configuration available as the value of the variable “fetchmail”.

This wasn't quite the last step in the dance. What I really wanted wasn't just for fetchmailconf to have the existing configuration, but to turn it into a linked tree of live objects. There would be three kinds of objects in this tree: Configuration (the top-level object representing the entire configuration), Site (representing one of the sites to be polled) and User (representing user data attached to a site). The example file describes five site objects, each with one user object attached to it.

I had already designed and written the three object classes (that's what took four days, most of it spent getting the layout of the widgets just right). Each had a method that caused it to pop up a GUI edit panel to modify its instance data. My last remaining problem was somehow to transform the dead data in this Python initializer into live objects.

I considered writing code that would explicitly know about the structure of all three classes and use that knowledge to grovel through the initializer creating matching objects, but rejected that idea because new class members were likely to be added over time as the configuration language grew new features. If I wrote the object-creation code in the obvious way, it would be fragile and tend to fall out of sync when either the class definitions or the initializer structure changed.

What I really wanted was code that would analyze the shape and members of the initializer, query the class definitions themselves about their members, and then adjust itself to impedance-match the two sets.

This kind of thing is called metaclass hacking and is generally considered fearsomely esoteric—deep black magic. Most object-oriented languages don't support it at all; in those that do (Perl being one), it tends to be a complicated and fragile undertaking. I had been impressed by Python's low coefficient of friction so far, but here was a real test. How hard would I have to wrestle with the language to get it to do this? I knew from previous experience that the bout was likely to be painful, even assuming I won, but I dived into the book and read up on Python's metaclass facilities. The resulting function is shown in Listing 3, and the code that calls it is in Listing 4.

Listing 3

Listing 4

That doesn't look too bad for deep black magic, does it? Thirty-two lines, counting comments. Just from knowing what I've said about the class structure, the calling code is even readable. But the size of this code isn't the real shocker. Brace yourself: this code only took me about ninety minutes to write—and it worked correctly the first time I ran it.

To say I was astonished would have been positively wallowing in understatement. It's remarkable enough when implementations of simple techniques work exactly as expected the first time; but my first metaclass hack in a new language, six days from a cold standing start? Even if we stipulate that I am a fairly talented hacker, this is an amazing testament to Python's clarity and elegance of design.

There was simply no way I could have pulled off a coup like this in Perl, even with my vastly greater experience level in that language. It was at this point I realized I was probably leaving Perl behind.

This was my most dramatic Python moment. But, when all is said and done, it was just a clever hack. The long-term usefulness of a language comes not in its ability to support clever hacks, but from how well and how unobtrusively it supports the day-to-day work of programming. The day-to-day work of programming consists not of writing new programs, but mostly reading and modifying existing ones.

So the real punchline of the story is this: weeks and months after writing fetchmailconf, I could still read the fetchmailconf code and grok what it was doing without serious mental effort. And the true reason I no longer write Perl for anything but tiny projects is that was never true when I was writing large masses of Perl code. I fear the prospect of ever having to modify keeper or anthologize again—but fetchmailconf gives me no qualms at all.

Perl still has its uses. For tiny projects (100 lines or fewer) that involve a lot of text pattern matching, I am still more likely to tinker up a Perl-regexp-based solution than to reach for Python. For good recent examples of such things, see the timeseries and growthplot scripts in the fetchmail distribution. Actually, these are much like the things Perl did in its original role as a sort of combination awk/sed/grep/sh, before it had functions and direct access to the operating system API. For anything larger or more complex, I have come to prefer the subtle virtues of Python—and I think you will, too.

Resources

All listings referred to in this article are available by anonymous download in the file ftp.linuxjournal.com/pub/lj/listings/issue73/3882.tgz.

Eric Raymond is a Linux advocate and the author of The Cathedral & The Bazaar . He can be reached via e-mail at (esr@thyrsus.com).

______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Re: After using python cross-plat

Anonymous's picture

> "After using python cross-platform in a real world project I can tell you that perl is much better."

I came to the Python world just because I noticed "real world projects" in Python were really good. I'm telling you about Plone, Twisted, ReportLabs, pysim,...

> "If you have a large class and at some line you accidentally add a blank line or you end up chopping a line, then your program will act busted - when it was just a silly extra line."

No at all. You just need a good editor. There are many of them. I just use Scite that visually let you see such mistakes while typing code. There are other much more advanced tools like http://pychecker.sourceforge.net/ that check the code for common scripting languages bugs.

> "Multiple versions(old/new) of python running together on the same linux box causes problems that you shouldn't have to deal with but since python doesn't have a plan for dealing with multiple version of the same library file then I guess you're screwed. try blender which uses a old version of python, get a python plugin for blender and then put it on a system with a new version of python - ouch the pains.
"
That's not the case for moderm Python versions (2.x series). You are speaking about very old code (Python 1.x). Obviosly when Python was born a not widely used it was a good idea not to keep compatibility with previous versions and clarify syntax and code in newer versions. Larry Wall even noticed that at some time you have to break compatibility and next Perl version will do so trying to solve the Perl syntax mess. And was't more important moderm virtualization technologies (Xen) solve such versioning problems and many others so language designers have not to worry to bear with design pitfalls for years.

Next generation languages for quick development

Anonymous's picture

For the Mono and .NET runtime:
http://boo.codehaus.org/

For the Java VM:
http://groovy.codehaus.org/

The both take the best from ruby and python, and throw out the crappy stuff that never was fixed in python or ruby.

This looks very cool.

Dave O'Hearn's picture

Thanks. In my view, it's a shame SUN and Microsoft don't put this kind of support directly into their frameworks. Scripting is such a perfect match for their languages, as they already have a runtime to support it.

I would be happier with these Codehaus technologies if they had the kind of industry support suggesting they'll still be around in 10 years. Still, it's definately an improvement over giving a scripting language its own runtime, a la Python and Ruby.

I think that Python/Ruby are

Nazgob's picture

I think that Python/Ruby are the future. Personaly I'm going to use Python as a tool for my Ubuntu administration and try to use it together with my C++ programms.

Ruby is Worth Looking at Too.

John Stevens's picture

I work at a Fortune 100 company, and I use Python in
my work. I can pretty much give a hearty second to
Eric's article, as I too started out in Perl, but quickly
discovered it's five biggest draw backs:

1) A short hiatus meant relearning the language, and almost
NOBODY knows the whole language.
2) While there are many free, third party Perl modules, many
of them are of very low quality.
3) Perl programs are harder to write, harder to debug, and
can have unexpected, hidden bugs that don't show themselves
for years.
4) Perl 5 can be used in a semi-OO way, but the result
is almost unrecognizable by even experienced Perl
programmers.
5) Perl is nearly unmaintainable, and almost impossible to
scale to large projects.

But as much as I use and like Python, Ruby is also an OO
programming language, and it has some advantages over
Python.

To summarize some of the differences:

It is more free form than Python, more regular, and
has better support for mixins (extensions), sub-classing
the built-in (standard) classes, has regular expressions
as a part of the language, has iteration and block
support . . . in short, think of Ruby as Python++.

This is not to say that Ruby is superior to Python
in every way. As of yet, Ruby has less third party
support, and while it is growing quickly, it has yet
to aquire the following that Python has.

I will be continuing to use Python for commercial, production
work, but I look forward to a time when I can start using
Ruby for commericial products.

agreed

Ian Monroe's picture

Unlike ESR, my initial reaction to python was positive. I always indent stuff in the fashion that python requires, so that didn't put me off.

Then I wrote a KDE application in python. The syntax can just get messy. Whats the point of having OOP if you have to specify the (self) agrument explicitly... I don't know how many times I had to go back to the code to add those.

I then wrote a KDE app in Ruby. Ruby is just a joy to program. You think of how you want to solve the problem, Ruby lets you solve it. You can write really elegant code.

Ruby does suffer somewhat in being a little too flexible (eval() belongs next goto as a no-no in all but the most limited uses IMO). It also lacks a proper VM so system threads are not possible, but will have one for Ruby 2.0.

OOP is NOT about omitting sel

Jaroslav's picture

OOP is NOT about omitting self or this. The real thing (called Smalltalk) uses it explicitly again and again. Even in C++ it is a very good habit to type *this* explicitly. I don't want to have to remember every time what an object is and where it belongs. The gold Python rule is "explicit is better than implicit" - it is IMO very wise and I like it very much.

Ruby is fairly good language but its author(s) really should have been more disciplined when designing it. IMO it has a lot of good features but not too good syntax. But I of course respect different feelings from the others.

Even better under the hood.

Mike's picture

You can appreciate the elegant engineering of the language by looking
at it's interface to C. I was able to integrate a C++ based parsing library we use in our company, into a python extension module in an afternoon. Elegant structure and documentation to go along with the common sense syntax.

Perl, Python, what's next?

Neal's picture

My main issue with these nifty new languages that pop up is wondering how long before language XYZ becomes passe and is replaced by the next big thing. Python is still pretty esoteric, and I'm pretty sure I don't know anyone who is handy with it. Also, I haven't heard much about it being used in platforms other than Linux. I am probably displaying my ignorance here -- but I am very conservative, maybe paranoid, in hooking on to any trend that requires me to re-tool my thinking about code. Put another way, I have no interest in being a leader in the promotion of the coolest language of the moment. I am a follower. I tend to prefer least-common-denominator languages like C, sh, sed, and awk. These languages, at least, are likely to work almost anywhere and survive over the long haul. I would rather continue to get better at bread and butter languages such as these (especially C) than learn the flavor of the week. I guess I put maintainability and consistency above all else.
I spent some time studying Perl, but couldn't muster the effort to get good with it, because it is very much not "compact," it's hard to learn, and for simple tasks it seems redundant with respect to shell scripting.
I had an enjoyable affair with Scheme, but in my view, there is little point in training myself to think functionally, when, for all practical purposes, Scheme will never be anything more than an academic curiosity (ditto SML, Lisp, etc).

These are my thoughts.

Cross platform programming.

Anonymous's picture

Try writing a program that workd on MAC OS X, Linux and Windows with the same code base. I have done it in Python.

Perl, Python, what's next?

Greg Smith's picture

Allow me to address a few misconceptions you have.
(1) Re windows: A lot of those Linux python applications will work under Windows as-is. Paint Shop Pro 8 has python embedded, as a scripting language. The mail client Mahogany (win/linux, at least) allows you to write python code to screen/sort your mail. So, even though it's unusual to have whole windows apps written in python, there's a lot of stuff using it. It's a chicken/egg thing; nobody wants to ship an app that requires you to have Python to run it (but the same doesn't seem to apply to Java). There are ways around that, though.

(2) Lots of folks are using it. Google uses it. many wiki, and blog systems, and similar, are entirely in Python. I just did a search for 'python' on Google, and 18 of the first 20 hits were about the language (the other 2 were about Monty Python; no actual snakes).
(Maybe this wasn't true when you posted in March).

(3) Re python being the 'flavor of the week': I've been using Python for about 7 years; it's been around in some form for at least 13. It's been much more visible lately though. It's been standard on Red Hat linux since (at least) 7.1, and the red hat install program (anaconda) is actually a python script. If you start using python now, you can easily be a follower rather than a leader. You could start by customizing one of the zillions of open-source projects which use python, for instance.

(4) Re: perl being redundant vs. shell scripting: If you spend the time to learn python, you may decide that shell scripting, perl, tcl, awk, even 'make' (except for very simple cases of any of these) are all redundant with respect to Python. Really. It can do all of these well enough that it's far easier to apply your knowledge of Python rather than spending time learning the strange behaviour (especially, all the arcane lexical rules) of some other language that was designed for a narrow application scope. Very complex makefiles for instance, can be horrendous to write and debug; it's much easier to write a python program which does all the complex stuff and generates a 'flat' makefile, and then let 'make' do its real job of checking timestamps etc.

You (like many others) do Python a great disservice by lumping it with Perl, bash,awk, tcl, and not C/C++. Unlike those languages, it doesn't have weird lexical conventions which help for simple cases and get ugly very quickly for more complex cases. Python has a very clean tokenized syntax like C,C++,pascal, etc.

You can write a 50,000 line program in Python and it will be maintainable. It may take 200,000+ lines of C++ to do the same thing, and the result will be probably be less flexible and harder to maintain.

BTW, I can identify with your viewpoint, since 6 or 7 years ago, I felt exactly the same way, and thought I would always use C or C++ for any 'real' work. And I would have made a similar assumption, that you need to stick with a 'real' language in order to keep 'maintainability and consistency'. OK, but python is a 'real' language.

Now I use Python for *everything* unless there's a reason I have to use C or C++. I find I can write code about 5 times faster in Python, and it's shorter, more intuitive, more reliable and more maintainable. And oddly, often faster; because higher level optimizations (those which reduce overall work, rather than make inner loops go faster) are often too much trouble in C, and very easy in Python.

Some of the things I've used python for: a custom binary relocator/linker; a custom network API; a network protocol logger; multidimensional curve-fitting (numerical computation); a ray tracing program for optical analysis of a sytem with a curved mirror; automatic generation of C/C++ source code (and python code); image processing; web scripting (with image processing); automation of backup operations, and operations related to version control; automation of one-off huge multi-file operations; automation of test-case generation and sequencing; automation of (and components of) complex build processes for embedded code; implementation of various declarative languages (there is an excellent lex/yacc for python - PLY); gui control panels for all kinds of things. I don't know of any other language which I would want to use for all of those (and I've been using C since 1980).
Quite a few of these projects would never have been done if I had had to do them in C/C++, since the time taken to code the project would have rendered the project's application infeasible; I.e. the near-term benefits would not have justified the schedule cost. So, any long-term benefits of having done these are an additional bonus from using Python.

ABOVE ALL, don't let your experience with trying to learn Perl (which sounds like it was the same as mine) prevent you from spending the 2 or 3 hours it will take you to learn a critical mass of Python. It's not the same beast.

Perl, Python, what's next Indeed

Charles Burnaford's picture

How about Zope, a very nice Object oriented database system with the Silva content management system on top of it.

FYI, Zope is written in Python.

Iron Python

manx's picture

Have a look at Iron Python (http://www.ironpython.com/), which is a .NET implementation of Python and claims to be faster than Python-2.3.

IronPython is still very earl

Anonymous's picture

IronPython is still very early early bleeding alpha. It can hardly do anything.

See instead boo: http://boo.codehaus.org/
It is much more stable than IronPython, is much faster (static typing with type inference), and has many more features than python (anonymous closures, interfaces, extensible compiler architecture,...).

Next Big Thing vs. (R)evolution

ChoyLeeFut's picture

I'll qualify my response by saying I'm a sysadmin with a heavy bent for infrastructure administration. I'll use (ba)sh scripts for some tasks, Perl for others, and at times a hybrid of both, depending on what's needed. I've tried to learn Python, but other than working through books, until I can work on a real problem to solve with it, I may not learn it. And getting Python installed on all systems requires the buy-in from the rest of my team. And given that the generic response is "We can do what we need with Perl/sh", that's not likely to change. Maybe in the next gig. ;-)

Having said that, I've also been hearing good things about Ruby on and off for the past couple years. When I first heard about it, I threw my hands in the air, thinking, "Oh great... now there's another language being touted as the Next Big Thing? Pft...." But more recently I've taken another look at it, and Rails in particular. I've seen the demo movie which shows someone building a web-based blog from scratch (with MySQL on the back end) in 10 mins. Basic functionality, but a working one.

So would I abandon Perl for Python/Ruby? Unlikely. But this is only because of familiarity. It's possible that once I get into Python, I'll drop Perl.

As far as maintainability, you only need to look back as far as COBOL to see what can happen if you staunchly refuse to look at other options. Some would argue that COBOL is the best for what it does, but if you take someone straight out of university, do you think they'll know COBOL?

Hmm... trying to think of a better example. How about having an auto mechanic come out of tech school and being forced to service coal-burning trains, simply because "Well we've always done it this way"? Sure, coal-burning trains are quaint and nostalgic, but there are better technologies now. And noone ever said that replacing a coal-burning train with a highway infrastructure and petroleum fuel delivery system would be an overnight success.

Certainly if you have a black box system which just works and never needs to be updated, then there's no reason to change the language in use. But that sort of scenario is the exception, not the rule.

Anyway, I'm rambling... and my comments won't enlighten the minds of those who are steadfast in their beliefs. :-)

I am using python to write systems here...

doug's picture

FWIW, I am the computer geek for a business that needs 3 separate significant (lots of records (> 1 mil), lots of moolah involved) database systems...the old ones were written in various versions of foxpro, and prior to that, as400...after a LOT of testing various languages etc...and even half writing one whole system in Delphi, I discovered python, and have used it along with wxWindows (now called wxWidgits) for gui, and an api to access ms-sql server with GREAT results on Windows platform.

My love for python knows no bounds currently...I was able to get these systems up and running sweetly in a very short time compaired to the other system I had worked with (VFP, Delphi, VP)...ok, and this is WITHOUT A SERIOUS GUI SCREEN DESIGNER TOO!

I never would have believed it myself either...but just using the stock wxDesigner that comes with wxWindows as a resource editor, and using a simple but nice programmers editor (pspad) I have been able to create nice, full featured, easy to maintain systems.

All tools I have used with creating these sytems were available FREE off the inet...even though where I work would have sprung for whatever expensive development system I desired, I choose this becouse I found it to be the best...and why...becouse of the ease of use of the python language (wxWindows/wxWidgits gets great cudos too!).

The only drag has been report writing...since there was no gui report designer available...I first started generating reports in HTML, but finally figuered out the wxWindows wxPrintout class, and now have a nice report writer class as well.

Anyhoo...my 2 cents from nashville, tn

Boa Constructor

Anonymous's picture

If you are using WX widgets you really should look into Boa Constructor for python. You will find it at sourceforge. Boa is and IDE written in python that will give some great tools, including a GUI designer. It also hooks into Zope.

Cheers!
Ed.

Python has been around since

Anonymous's picture

Python has been around since 1990, much longer than Java and is still gaining mommentum. Read some history before you worry about the future.

re:Python has been around since

Jack Marder's picture

You are right, but let's have a look on hotscripts.com ... alone the sheer amount of scripts available speak for a language itself 8-)

re:Python has been around since

Greg Smith's picture

What does the volume say? Maybe that the python community has a lot of other places to host open source projects, rather than at hotscripts.com? I'd never heard of it. If that site's full of perl, perl programmers will go there (at least until they try Python :-) )

I can remember when all the magazines were full of BASIC programs you could type into your Apple ][ or Apple /// or Commodore 64 or whatever (this was before there was a www in any form we know it). Did the staggering popularity of BASIC mean it was a good language? No, it was the only language everybody had access to.

I've read several testimonials written by perl programmers who tried python, and preferred it; and they always make that same point, that you can't go back to a perl program and make sense of it 3 months later. Perl users who haven't tried python doubtless see this unreadability as the inevitable cost of using a powerful language, or something; just as advanced C++ programmers shrug off the forests of nested < >. I've yet to hear of anyone switching from python to perl.

Unfortunately, I think, the perl-vs-python issue casts python in a bad light, since it distracts from the fact that you can use python for so many things which perl can't really touch. If you keep watching the perl-vs-python scuffle, you may miss noticing that a lot of projects not falling into that space are being done in Python, rather than, say , C++ or Java.

Keep Python alive

Joseph Q.'s picture

QBASIC is almost a dead language. Yet there are people who won't let it die. Some python fanatic (maybe me) will do his best to make a comunity that will keep python alive when a new language comes out that threatens pythons throne.

As for Cross-Platform usabili

Anonymous's picture

As for Cross-Platform usability, Python comes with OpenOffice.org for Windows. OpenOffice also provides a Python-UNO bridge which allows Python to access any number of Object Models (COM, CORBA, Java, etc). Python builds on any POSIX-Compliant system, and many Non-POSIX systems.

Python doesn't come with Open

Anonymous's picture

Python doesn't come with OpenOffice. OpenOffice is a free, open source office software suite with word processing, presentation software, spreadsheet tool, etc. It runs on Linux, Windows, or Mac. And it has bridges to allow you to code extensions in python OR any java VM language or any .NET/Mono language.

Experience Python and other languages look promising

Anonymous's picture

After looking at Python I believe that COBOL will make it big again,
especially if they call keep on calling it Object Oriented COBOL
force you to interpret, not compile it.
Let me say that I always recommend interpreted languages
to prospective software developers.
One big advantage of such languages is, of course, that if you (or your customer) runs into a problem you/he will do so at runtime,
which may be quite a while after you have sold your code (and cashed
the check).
Also, since your application will be about ten- to a hundredfold slower than a compiled version, you may profitably hook up with a
friend selling the newest and fastest 128-bit hardware with
more than 16GB of RAM or so, so that your customer will actually
be able to run your 'application' with tolerable performance.
At times it will still feel slow though, since the average
interpreter has the habit of allowing your app to eat up all of
that memory, at which point it will get an out of memory hint
and will a process called garbage collecting, which will take
about a minute or two for 16GB.
Other advantage is that even if you write a proggie that is about
five lines you can convince the customer you've done major software
engineering by installing the colossal interpreter, 'optimized'
to do all things from serving secure HTTP pages through 3D graphics processing and UI functionality to scientific SIMD calculations.
To do all of this it also comes with about half a terabyte
of semi-documented 'libraries'without which your program will
utterly fail irrespective of its requirements, and do so with
incomprehensible error messages that are fully out of your control.
If you make sure that your customer gets face-to-face with these
messages a number of times early on in the lifecycle it may even
land you a support contract. Thus all of this should be
considered beneficial to the engineer.

But perhaps the best part of an interpreted language is that if your
customer notices your app's lousy performance you can always
blame it on the interpreter. In fact you can probably blame the
stupidest of your programming errors on the interpreter. And then
the real fun starts. You mutter "yes but PERLTHON 98.3 is now out,
and it is free". Your customer will be fascinated.

Not for long because he will be forced to fork over a whole lot
more money on you rewriting your application after he accidentally
agrees to upgrade his system to this 'new, better' interpreter.
This without you having to add any new functionality whatsoever!
Rewriting each time some unknown, outside force decides to release a
new 'sorta backwards compatible' interpreter is yet another bigtime
advantage of the interpreted language to the software engineer
in search of cash.
If you haven't screwed up too badly earlier on in the product
cycle (ALWAYS blame 'the interpreter', never yourself!) you can
really make big bucks during this stage.

Readability, intelligibility and maintainability are key words
for any sofware project. The three words conspire to keep a sofware
engineer out of a higly paid and irreplacable position.
This brings me to free-form languages. They are known to allow you
to create programs that are virtually meaningless to anybody but
yourself - if you have a photographic memory - and should
considered as prime choices for any engineer trying to secure
future employment. Some may construe this to mean that I prefer
PERL over Python. While I do agree that PERL has a plethora of
built-in features to allow even unskilled engineers to craft
absolutely unreadable code, I see no reason why you could
not create similarly unreadable code with Python! My advise to
you is: apply yourself, be creative and you may be surpised.
Remember that the Python code may well be slower, too!

For big projects with many tens of thousands of lines of code only
the best structured object-oriented languages will lead to a
manageable project. Thus, for such projects Python or PERL are both
excellent candidates to maximize cost and man-power required.
Allow me to introduce yet another option though. You may have heard
of COBOL. Back in the sixties I did some punchcard programming
and was forced to look at COBOL. I was not impressed at the time.
You could create a working program with just a dozen punchcards.
However time has passed.
We are 2005 today. We can now have all the disadvantages of COBOL
without any of the benefits. Try interpreted COBOL. You will fill
half a moderate-sized harddisk with just the libraries. With
COBOL, as with Python, you can enter keystrokes as fast as
you can type them - just make sure that you are not forced
to debug them. Alternatively, an arcane sequence of rarely
executed if-then-else statements will allow you to get away
with an almost limitless quantity of erroneous code, an especially
attractive characteristic for those paid by lines of code written.

THE great advantage of COBOL is that is a brand name. Your client
will have heard of it. Just like Coca Cola, it may have a bad taste
but it rings a bell. Quite unlike Python which sounds as if it
could strangle, or at least bite, a prospective client later on
during the product lifecyle. Just ask your client, I suggest COBOL
for this, but if would you rather have me implement this in PYTHON...
Ten-to one he will agree to let you do it in COBOL.

The kicker here is that some engineer here or there might still be
interested to look into Perl or Forth or LISP or Python, but
there ain't a soul in the world looking forward to doing some
COBOL. Meaning that you will be quite sure of prolonged employment
if you convince somebody to implement anything critical in COBOL.

Ask for very little money up-front. Your customer is bound to come
back to you. Do it in interpreted COBOL and your pay will go
through the roof!

Conclusion.

I applaud Eric Raymond on his article. When it comes to writing
interpreted and unmaintainable code, Python like PERL has lot
going for it. Now with Python we have the choice.
But when it comes to get money off of prospective customers I
think you will soon realize the value of Brand Names!
IMHO Object Oriented Interpreted COBOL is the way forward.

Thank you.

The *real* way forward

franzg's picture

Hi,

IMHO the writers opinion is quite acceptable. However instead of COBOL I would choose a more modern yet stable development tool: INTERCAL for instance has been around since 1972, and its development is rather slow - so that a programmer should not be concerned about possible API changes or sudden twists in supported features or functionalities. ESR himself was so fascinated, that he maintains a port of the tool.

I think that the real future is to foster a group to enhance INTERCAL without modifying its underlying structure, possibly developing an Object Oriented version (I'd suggest OO-INTERCAL as a name) along with an appropriate framework or standard library, implemented as labeled statements to be appended to the program. From the hired-coder's point of view, this would also increase the possibilities of a higher income when a pay-per-line policy is applied.

Going on in this advocacy, the "brand" name INTERCAL would surely remind a possible customer of those good old massive supercomputers that were so popular about 30 years ago, thus giving him an impression of stability and reliability.

Cheers,

F.

Brush up on JIT

Anonymous's picture

Dear poster,

Before you lament for so long about interpreters, may I advise that you brush up on bytecode, just-in-time (JIT) compilers, say, in JVM (Sun) or CLR (Microsoft) implementations, and modern garbage collection strategies? I hope Java and C# both ring your bell here.

Thankful reader.

Can even i can do it ??

agent 258's picture

Hi, i recently turned 15 i i was browsing the web and found ure website ... i don`t know that much of computers do you think I can hack or not. Please reply ,im waiting for an answer ...

signed : agent 258 ( nickname)

WTF??

Anonymous's picture

This is the most astonishing blather I've ever read.

Object Oriented Interpreted COBOL is the way forward, for reasons of brand recognition?!?!?!?

Two questions:

1. Do you do programming, or just market research?
2. Did you write your comment before or after the 1980's?

Man, do you know at least who

Anonymous's picture

Man, do you know at least who wrote this article?
"Market Research"? No, I don't think so.

Visual COBOL 2005 Express Edition Beta

Anonymous's picture

Shocking.

Because I have been using the new Visual COBOL EE beta for months now. Imagine COBOL with Intellisense and Code Completion. Even has a real-time punchcard visual realizer!

Perl. Let me think. Perl is like a mental patient on LSD.

The author of the long-winded article is simply upset that anyone can be moderately productive with Perl. This threatens his way of programming and he is simply ranting about it.

Let him have his say......

listing 2

Anonymous's picture

I hope those passwords in Listing 2 aren't supposed to be secret anymore.

The other comments

VoidEngineer's picture

This comment; which isn't common for me, is in response to the alarming lack of intelligence in the previous comments. At first, upon reading the first one or two comments, I found myself just going, "oh jeez, what a dolt!" But it continued for FOUR PAGES. For f**k sake, did all the idiots of the web gather round to parade their new skills in HalfWit Retoric? Get a clue people; if you have to ask what python is, then there is NO HOPE for you. Maybe try custodial engineering; I don't care. I just can't believe how f**king DUMB you f**ks are. If you cannot figure out what www.fsckingoogleit.com is, then I emplore you to seek employment at your local McDonalds, cause at this rate there is only failure ahead of you. I so need a drink after this. @_@

re: SurfControl

Anonymous's picture

as i understand SurfControl runs more often then not on the proxy servers of school networks, unless you know how to do some tunnelling you got no hope of getting past it

knoppix 4.0.2

Anonymous's picture

If you have a DVD Rom drive you can do a hard reboot
Knoppix will load and give you your own Linux live
Distro. This will allow you to investigate the network.
Surf control is OK but I think you will find it is not able
deal with this situation. You can Tunnel OK from this

goto www.python.org/download/

jeo's picture

goto www.python.org/download/ download the language and guide there... learn it yourself my friend...

Just go to Python.com for tut

Neotheone123's picture

Just go to Python.com for tutorial and the program.

python very easy

mayat_python's picture

python very easy to learn but you must know algorima and logic

What is Bill Gates doing read

nighthawk808's picture

What is Bill Gates doing reading Linux Journal?

Asswad

U-no-hoo's picture

Nighthawk ... you are such an asswad.... I`m loving IT!

I need help

Old Man's picture

I seldom actually laugh out loud.

Yes!

Mason's picture

I concur.

Python

PythonJohn's picture

Python is new to me, well not that new. But so far after using ZOPE and the ZWiki I have to say out of all the languages out there Python just intrigues me to no end. MySql and php in my opinion are so confusing I gave up trying to comprehend them. Python on the other hand makes so much sense just the way it is set up and this is coming from a novice-like mindset.

At any rate I just wanted to plug my take on this issue, for me Python and nothing else if this is possible.

PythonJohn

hi

franco's picture

i will like to know more

I want to learn

radu's picture

I want to learn about IP and mirc

I want to learn how to make p

TEACH ME PLZ !'s picture

I want to learn how to make programs and the "hacking" language i guess you could say :s Anyone who is willing to help plz email me!!

RTFM

Anonymous's picture

RTFM

Why Python - Please Not .NET

Anonymous's picture

Great Article, years later still valid. I have programmed for years in c vb powerbuilder msaccess(vba) html/php and until recently never tried the 'scrippy' languages. A few days and Python was a dream come true, why hadn't I found it years ago! Sorry, I loved vb3,vb4,vb5,vb6 as design environments and a language that was(I thought then) easy and fast to develop in. I even wrote intensive mapping software that ran very fast in vb4 and won an international award!

VB7 (.NET) has nothing to do with VB! Its a new language designed for Accountant styled programmers (straight line boring) I am an Artist styled programmer (lots of curves and imagination). Many limitations on what you can do compared to VB6, and a 300 line VB6 Majhong game I rewrote in .NET required over 900 lines and more hours to modify than the original program took to write, with less functionality. .NET is anti-productive, I needed a new direction quickly.

Please forget .NET - it locks you into a single design choice that is MICROSOFT! They have a Plan. M$ is not open and is only there to assimilate or obliterate competition. Believing MONO and IronPython is not part of their plan would be foolish. M$ is NOT foolish.

Pythons design and evolution is perhaps a perfect living demonstration of the success of the philosophy of Open Source. FREE and OPEN and ORGANIC. Oh, the sensation, I feel dizzy.

I used to teach 'beginning in program design' with vb, as soon as I get a little more Python under my belt I plan to start teaching again - come on people, lets let everyone know about better options than M$. Get our bosses approval to use it in the work place (or not-fait-accomplices is nice too).

For disillusioned VBer's looking for a better VB solution - try boa-constructor or spe (free ide's & gui's) and wxWidgetswxPython (cross platform GUI). Python has people writing incredible open books and references. (http://diveintopython.org http://rgruet.free.fr/PQR2.3.html)

God, I feel good that is off my chest.

by a (VB Converted to) Python Evangelist.

Re: Why Python - Please Not .NET

Andrés Meza's picture

Greetings.
I agree with you. M$ don't embrace open source philosophy just because. I think they made their .NET movement in order to make their competitors weaker (open source and free software solutions).
I used to be an entusiastic Java programmer but since I tried Python the first time i love it.

Re: Why Python?

Anonymous's picture

quiero ser el mejor en la compu queiro hacer programas nuevos
quiero hacer como tuuuuuu

Re: Why Python?

Anonymous's picture

hola soy de chile quero ser hacker no se mucho ayudarme plis

Webinar
One Click, Universal Protection: Implementing Centralized Security Policies on Linux Systems

As Linux continues to play an ever increasing role in corporate data centers and institutions, ensuring the integrity and protection of these systems must be a priority. With 60% of the world's websites and an increasing share of organization's mission-critical workloads running on Linux, failing to stop malware and other advanced threats on Linux can increasingly impact an organization's reputation and bottom line.

Learn More

Sponsored by Bit9

Webinar
Linux Backup and Recovery Webinar

Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs. However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.” After all, before data can be restored to a system, there must be a system to restore it to.

In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.

Learn More

Sponsored by Storix