OpenOffice.org ODF, Python and XML
Let's take fix1.py and make an easy modification. Whenever two hyphens appear, replace them with the em dash. Then, when we're done, write the XML to stdout—that's exactly what the shell script (fixit.sh) expects.
We'll specify the em dash by giving its hex value; to find it, locate the em dash in OpenOffice.org Writer's Insert→Special Character dialog (Figure 8).
When I select the long dash (the em dash), its Unicode value appears in the lower-right corner, where I've put a purple ellipse; that's the value to put into the string in place of the double hyphens. Let's call this script fix2.py:
#!/usr/bin/python -tt import xml.dom.minidom import sys DEBUG = 1 def dprint(what): if DEBUG == 0: return sys.stderr.write(what.encode('ascii', 'replace') + '\n') emDash=u'\u2014' def fixdata(td, depth): dprint("depth=%d: childNode: %s" % (depth, td.data)) # OK, so '--' becomes em dash everywhere td.data = td.data.replace('--', emDash) def handle_xml_tree(aNode, depth): if aNode.hasChildNodes(): for kid in aNode.childNodes: handle_xml_tree(kid, depth+1) else: if 'data' in dir(aNode): fixdata(aNode, depth) def doit(argv): doc = xml.dom.minidom.parse(argv) handle_xml_tree(doc, 0) sys.stdout.write(doc.toxml('utf-8')) if __name__ == "__main__": doit(sys.argv)
Notice how easy Python makes it to replace a pattern in a string. Strings in recent Python versions have a built-in method, replace, that causes one substring to be replaced by another:
td.data = td.data.replace('--', emDash)
Let's plug fix2.py into fixit.sh to see how well it works:
% ln -sf fix2.py fixit.py % ./fixit.sh ex3.odt ex3-1.odt depth=5: childNode: The ?en? dash ? is too short depth=5: childNode: The ?em? dash ? which is longer ? is what we need. depth=5: childNode: And two hyphens -- ugly -- should be turned into ?em? dashes. depth=5: childNode: This line has "straight double quotes" depth=5: childNode: These 'single quotes' aren't pretty. % oowriter ex3-1.odt %
Success! Now for the rest. Besides the double hyphen, we want to change the en dash into an em dash. That syntax is just like the double hyphen replacement.
Replacing straight quotes with curly ones is more complicated though, because we have to decide between a starting double quote and an ending double-quote character. How to tell? Well, if the quote character is at the start of the string, and there's a nonspace character afterward, it's a left (or start of quote) curly quote. Ditto if there's a blank before it and a nonspace afterward.
That's the easy way to describe it. We could code it like that, or we could simply write a regular expression. I looked at the section titled “re -- Regular expression operations” in Chapter 4 of Python's library documentation and eventually came up with this:
sDpat = re.compile(r'(\A|(?<=\s))"(?=\S)', re.U)
Let me explain this left to right. We are creating sDpat, the pattern for a starting double quote or Starting Double-quote PATtern. We do that by calling the method compile in the re module (for regular expressions). That analyzes the pattern once and creates a regular expression object. We'll use sDpat to match straight double quotes that should be turned into nice curly quotes at the start of a quotation.
Now, about the pattern—the pattern contains a double-quote character (") so we delimit it with single quotes, 'like this'. Also, we'll pass some escapes (such as \A and \s) to re.compile, so let's make this a raw string by putting an r in front of it.
(A little explanation for Perl users: in Python, \ escapes are interpolated except in raw strings, whether single-quoted or double-quoted; the delimiters don't affect interpolation as they do in Perl.)
We can see how raw strings work by using Python's shell:
>>> print 'normal string: \n is a newline' normal string: is a newline >>> print r'raw string: \n is not a newline' raw string: \n is not a newline >>>
So, what's in that raw string? It consists of three parts:
The part before the quote character (\A|(?<=\s)). What we are doing is matching something (the '"' in this case), but only if it occurs at the beginning of the string or if it's preceded by a whitespace character. The \A means “match beginning of the string”, the | means “or” and (?<=\s) means “match if immediately preceded by whitespace (a blank, tab or newline), but don't include that whitespace itself in the match”. The enclosing parentheses denote grouping.
The straight double quote itself: ". That's what we're matching.
The part after the '"': (?=\S). What we're doing is adding another condition—that the quote character be followed by a non-whitespace character.
If all three conditions are met—that is, if a quote is there (condition 2), and it's either at the start of the string or preceded by whitespace (condition 1), and it's followed by some non-whitespace character (condition 3), we want to replace it by an opening double-quote character.
Besides the pattern, you also can pass flags to re.compile. We pass re.U to make certain escapes dependent on the Unicode character database. Because we're parsing a Unicode string, I think we want that.
Let's call this fix3.py:
#!/usr/bin/python -tt import xml.dom.minidom import sys import re # new in fix3.py DEBUG = 1 def dprint(what): if DEBUG == 0: return sys.stderr.write(what.encode('ascii', 'replace') + '\n') emDash=u'\u2014' enDash=u'\u2013' # new in fix3.py sDquote=u'\u201c' # new in fix3.py # sDpat: pattern for starting dbl quote, as # "Go! <-- the quote there # We look for it either at start (\A) or # after a space (\s), and we want it to be # followed by a non-space sDpat = re.compile(r'(\A|(?<=\s))"(?=\S)', re.U) # new in fix3.py def fixdata(td, depth): dprint("depth=%d: childNode: %s" % (depth, td.data)) # OK, so '--' becomes em dash everywhere td.data = td.data.replace('--', emDash) # Change 'en' dash to 'em' dash td.data = td.data.replace(enDash , emDash) # new in fix3.py # Make a nice starting curly-quote # new in fix3.py td.data = sDpat.sub(sDquote, td.data) # new in fix3.py def handle_xml_tree(aNode, depth): if aNode.hasChildNodes(): for kid in aNode.childNodes: handle_xml_tree(kid, depth+1) else: if 'data' in dir(aNode): fixdata(aNode, depth) def doit(argv): doc = xml.dom.minidom.parse(argv) handle_xml_tree(doc, 0) sys.stdout.write(doc.toxml('utf-8')) if __name__ == "__main__": doit(sys.argv)
Note that the syntax for replacing a regular expression differs from that of substring replacement: we use the sub (substitute) method of the regular expression object (sDpat in this case):
td.data = sDpat.sub(sDquote, td.data)
Here we're taking td.data, the data in this particular node in the XML tree, looking for the regular expression specified by sDpat, and replacing whatever matched it (the straight " character in the appropriate context) with the starting double quote, sDquote.
Now, if we try fixit.sh with fix3.py as the lower-level program:
% ln -sf fix3.py fixit.py % ./fixit.sh ex3.odt ex3-2.odt depth=5: childNode: The ?en? dash ? is too short depth=5: childNode: The ?em? dash ? which is longer ? is what we need. depth=5: childNode: And two hyphens -- ugly -- should be turned into ?em? dashes. depth=5: childNode: This line has "straight double quotes" depth=5: childNode: These 'single quotes' aren't pretty. % oowriter ex3-2.odt %
Practical Task Scheduling Deployment
July 20, 2016 12:00 pm CDT
One of the best things about the UNIX environment (aside from being stable and efficient) is the vast array of software tools available to help you do your job. Traditionally, a UNIX tool does only one thing, but does that one thing very well. For example, grep is very easy to use and can search vast amounts of data quickly. The find tool can find a particular file or files based on all kinds of criteria. It's pretty easy to string these tools together to build even more powerful tools, such as a tool that finds all of the .log files in the /home directory and searches each one for a particular entry. This erector-set mentality allows UNIX system administrators to seem to always have the right tool for the job.
Cron traditionally has been considered another such a tool for job scheduling, but is it enough? This webinar considers that very question. The first part builds on a previous Geek Guide, Beyond Cron, and briefly describes how to know when it might be time to consider upgrading your job scheduling infrastructure. The second part presents an actual planning and implementation framework.
Join Linux Journal's Mike Diehl and Pat Cameron of Help Systems.
Free to Linux Journal readers.Register Now!
- Paranoid Penguin - Building a Secure Squid Web Proxy, Part IV
- SUSE LLC's SUSE Manager
- Google's SwiftShader Released
- Managing Linux Using Puppet
- My +1 Sword of Productivity
- Murat Yener and Onur Dundar's Expert Android Studio (Wrox)
- Non-Linux FOSS: Caffeine!
- SuperTuxKart 0.9.2 Released
- Parsing an RSS News Feed with a Bash Script
- SourceClear Open
With all the industry talk about the benefits of Linux on Power and all the performance advantages offered by its open architecture, you may be considering a move in that direction. If you are thinking about analytics, big data and cloud computing, you would be right to evaluate Power. The idea of using commodity x86 hardware and replacing it every three years is an outdated cost model. It doesn’t consider the total cost of ownership, and it doesn’t consider the advantage of real processing power, high-availability and multithreading like a demon.
This ebook takes a look at some of the practical applications of the Linux on Power platform and ways you might bring all the performance power of this open architecture to bear for your organization. There are no smoke and mirrors here—just hard, cold, empirical evidence provided by independent sources. I also consider some innovative ways Linux on Power will be used in the future.Get the Guide