Python Scripts as a Replacement for Bash Utility Scripts

To demonstrate the power of combining Python scripts in a modular and piped fashion, let's expand further on the problem space. Let's find the top five users of the service. head is a command that allows you to specify a certain number of lines to display of the standard input it is given. Adding this to the command chain gives the following:

$ cat names.log | python namescount.py | sort -rn | head -n 5

This prints only the top five users and ignores the rest. Similarly, to get the five users who use the service least, you can use the tail command, which takes the same arguments. The result of the Python command being printed to standard output allows you to build and extend upon its functionality.

To demonstrate the modularity of this script, let's once again change the problem space. The service also generates a comma-separated value (CSV) log file that contains a list of e-mail addresses and the comments that each e-mail address made about the service. Here's an example entry:

"email@example.com", "This service is great."

The task is to provide a way for the service to send a thank-you message to the top ten users in terms of comment frequency. First, you need a script that can read and print a certain column of CSV data. The standard library of Python provides a CSV reader. The Python script below completes this goal:

#!/usr/bin/env python
# CSV module that comes with the Python standard library
import csv
import sys


if __name__ == "__main__":
    # The CSV module exposes a reader object that takes
    # a file object to read. In this example, sys.stdin.
    csvfile = csv.reader(sys.stdin)

    # The script should take one argument that is a column number.
    # Command-line arguments are accessed via sys.argv list.
    column_number = 0
    if len(sys.argv) > 1:
            column_number = int(sys.argv[1])

    # Each row in the CSV file is a list with each 
    # comma-separated value for that line.
    for row in csvfile:
            print row[column_number]

This script can parse the CSV data and return in plain text the column that is supplied as a command-line argument. It uses print instead of sys.stdout.write, as print, by default, uses standard out as its output file.

Let's add this script to the chain. The new script is chained with the others to print out a list of e-mail addresses and their comment frequencies using the command listed below (the .csv log file is assumed to be called emailcomments.csv and the new Python script, csvcolumn.py):

$ cat emailcomments.csv | python csvcolumn.py | 
 ↪python namescount.py | sort -rn | head -n 5

Next, you need a way to send an e-mail. In the Python standard library of functions, you can import smtplib, which is a module that allows you to connect to an SMTP server to send mail. Let's write a simple Python script that uses this library to send a message to each of the top ten e-mail addresses found already:

#!/usr/bin/env python
import smtplib
import sys


GMAIL_SMTP_SERVER = "smtp.gmail.com"
GMAIL_SMTP_PORT = 587

GMAIL_EMAIL = "Your Gmail Email Goes Here"
GMAIL_PASSWORD = "Your Gmail Password Goes Here"


def initialize_smtp_server():
    '''
    This function initializes and greets the smtp server.
    It logs in using the provided credentials and returns 
    the smtp server object as a result.
    '''
    smtpserver = smtplib.SMTP(GMAIL_SMTP_SERVER, GMAIL_SMTP_PORT)
    smtpserver.ehlo()
    smtpserver.starttls()
    smtpserver.ehlo()
    smtpserver.login(GMAIL_EMAIL, GMAIL_PASSWORD)
    return smtpserver


def send_thank_you_mail(email):
    to_email = email
    from_email = GMAIL_EMAIL
    subj = "Thanks for being an active commenter"
    # The header consists of the To and From and Subject lines
    # separated using a newline character
    header = "To:%s\nFrom:%s\nSubject:%s \n" % (to_email,
            from_email, subj)
    # Hard-coded templates are not best practice.
    msg_body = """
    Hi %s,

    Thank you very much for your repeated comments on our service.
    The interaction is much appreciated.

    Thank You.""" % email
    content = header + "\n" + msg_body
    smtpserver = initialize_smtp_server()
    smtpserver.sendmail(from_email, to_email, content)
    smtpserver.close()


if __name__ == "__main__":
    # for every line of input.
    for email in sys.stdin.readlines():
            send_thank_you_mail(email)

This Python script supports contacting any SMTP server, whether local or remote. For ease of use, I have included Gmail's SMTP server, and it should work, provided you give the scripts the correct Gmail credentials. The script uses the functions provided to send mail in smtplib. This again demonstrates the power of using Python at this level. Something like SMTP interaction is easy and readable in Python. Equivalent shell scripts are messy, and such libraries are not as easily accessible, if they exist at all.

In order to send the e-mails to the top ten users sorted by comment frequency, first you must isolate only the e-mail column of the output of column names. To isolate a certain column in Linux, you use the cut command. In the example below, the commands are given in two separate chains. For ease of use, I wrote the output into a temporary file, which can be loaded into the second chain. This simply makes the process more readable (the Python script for sending mail is referred to as sendemail.py):

$ cat emailcomments.csv | python csvcolumn.py | 
 ↪python namescount.py | sort -rn > /tmp/comment_freq
$ cat /tmp/comment_freq | head -n 10 | cut -f2 | 
 ↪python sendemail.py

This shows the real power of Python as a utility in a chain of bash commands such as this. Writing scripts that accept input from standard input and write any data out to standard out, allows the developer to chain commands such as these together quickly and easily with a link in the chain often being a Python program. This philosophy of designing a small application that services one purpose fits nicely with the flow of commands being used here.