ODF, Python and XML

Combine Python with the open format of ODF files to manipulate fine details.
A Real-Life Example

That exercise proved the concept, so now we can get to work. My wife's poetry book was about 60 pages long, and it needed these issues addressed:

  1. Those straight quotes, which came from plain-text e-mail messages or other word processors.

  2. Apostrophes (or single quotes), which also were straight rather than curled the right way.

  3. Double hyphens and shorter dashes (the en dash), which should all be changed into the longer em dash. Writer has keystroke sequences for creating the en dash as well as the longer em dash. Sometimes the wrong sequence was typed, so an en dash appeared instead of the desired em dash. Plain text imported from e-mail messages sometimes had double hyphens (that is, --).

Concretely, we want to transform what's shown in Figure 5 into what's shown in Figure 6.

Figure 5. Before...

Figure 6. ...and After

Let's develop the automated script in two pieces, and let's do it top-down. The top layer will create a temporary directory, unpack the original document and then run the bottom layer, a program (designated to modify content.xml. Afterward, it will pack up the files into the new document and clean up.

The Top Layer: a Shell Script

I want to use the highest-level language reasonable for each task; for this top layer, that's probably the shell. This script, called, turned out to be longer than I thought it would be, mostly because of all the error checking:

# Script to fix up OpenDocument Text (.odt) files
# "cd" to the directory containing "".

# Make $TMPDIR, a new temporary directory

TMPDIR=/tmp/ODFfixit.$(date +%y%m%d.%H%M%S).$$
if rm -rf $TMPDIR && mkdir $TMPDIR; then
   : # Be happy
   echo >&2 "Can't (re)create $TMPDIR; aborting"
   exit 1


# Check number of parameters.
# Ensure $NEWFILE's dir exists and is writable.
# Quietly Unzip $OLDFILE. Whine and abort on error.

if [[ $# -eq 2 ]] &&
   touch $NEWFILE && rm -f $NEWFILE &&
                  unzip -q $OLDFILE -d $TMPDIR ; then
   : # All good; be happy.

   # Trouble! Print usage message, clean up, abort.

   echo >&2 "Usage: $0 OLDFILE NEWFILE"
   echo >&2 "  ... both OpenDocument Text (odt) files"
   echo >&2 "Note: 'OLDFILE' must already exist."
   rm -rf $TMPDIR
   exit 1

# Save file list in $F; is content.xml there?

F=$(unzip -l $OLDFILE |
       sed -n '/:[0-9][0-9]/s|^.*:.. *||p')
if echo "$F" | grep -q '^content\.xml$'; then
   : # Good news; we have content.xml
   echo >&2 "content.xml not in $OLDFILE; aborting"
   echo >&2 TMPDIR is $TMPDIR
   exit 1

# Now invoke the Python program to fix content.xml

mv $TMPDIR/content.xml $TMPDIR/OLDcontent.xml
if ./ $TMPDIR/OLDcontent.xml > \
                  $TMPDIR/content.xml; then
   : # It worked.
   echo >&2 " failed in $TMPDIR; aborting"
   exit 1

if (cd $TMPDIR; zip -q - $F) | cat > $NEWFILE; then
   # Everything worked! Clean up $TMPDIR
   rm -rf $TMPDIR
else # something Bad happened.
   echo >&2 "zip failed in $TMPDIR on $F"
   exit 1

It's long but straightforward, so I explain only a few things here.

First, the temporary directory name includes the date and time (the date +% stuff), and the shell's process ID (the $$) prevents name collisions.

Second, the grep line looks the way it does because I want it to accept content.xml but not something like discontent.xml or content-xml.

Finally, we clean up the temporary directory ($TMPDIR) except in some error cases, where we leave it intact for debugging and tell the user where it is.

We can't run this script yet, because we don't yet have actually modify content.xml. But, we can use a stub to validate what we have so far. The script assumes will take one parameter (the original content.xml's pathname) and put the result onto stdout. This just happens to match the calling sequence for /bin/cat with one parameter; hence, if we use /bin/cat as our, should give us a new document with the same content as the old. So, let's give it a whirl:

% ln -s /bin/cat
% ./ ex1.odt foo.odt
% ls -l ex1.odt foo.odt
-rw-r--r--  1 collin users 7839 2006-11-14 17:50 ex1.odt
-rw-r--r--  1 collin users 7900 2006-11-14 19:45 foo.odt
% oowriter foo.odt

The new file, foo.odt, is slightly larger than ex1.odt, but when I looked at it with Writer, it had the right stuff.

As far as writing a program for manipulating content.xml—well, back in the 1990s, I probably would have spent many hours with yacc (or bison)—but today, Python with its XML libraries is a more natural choice.


White Paper
Linux Management with Red Hat Satellite: Measuring Business Impact and ROI

Linux has become a key foundation for supporting today's rapidly growing IT environments. Linux is being used to deploy business applications and databases, trading on its reputation as a low-cost operating environment. For many IT organizations, Linux is a mainstay for deploying Web servers and has evolved from handling basic file, print, and utility workloads to running mission-critical applications and databases, physically, virtually, and in the cloud. As Linux grows in importance in terms of value to the business, managing Linux environments to high standards of service quality — availability, security, and performance — becomes an essential requirement for business success.

Learn More

Sponsored by Red Hat

White Paper
Private PaaS for the Agile Enterprise

If you already use virtualized infrastructure, you are well on your way to leveraging the power of the cloud. Virtualization offers the promise of limitless resources, but how do you manage that scalability when your DevOps team doesn’t scale? In today’s hypercompetitive markets, fast results can make a difference between leading the pack vs. obsolescence. Organizations need more benefits from cloud computing than just raw resources. They need agility, flexibility, convenience, ROI, and control.

Stackato private Platform-as-a-Service technology from ActiveState extends your private cloud infrastructure by creating a private PaaS to provide on-demand availability, flexibility, control, and ultimately, faster time-to-market for your enterprise.

Learn More

Sponsored by ActiveState