Python Scripts as a Replacement for Bash Utility Scripts
To demonstrate the power of combining Python scripts in a modular and
piped fashion, let's expand further on the problem space. Let's find
the top five users of the service.
head is a command
that allows you to
specify a certain number of lines to display of the standard input it
is given. Adding this to the command chain gives the following:
$ cat names.log | python namescount.py | sort -rn | head -n 5
This prints only the top five users and ignores the rest. Similarly, to
get the five users who use the service least, you can use the
tail command, which
takes the same arguments. The result of the Python command
being printed to standard output allows you to build and extend upon
To demonstrate the modularity of this script, let's once again change the problem space. The service also generates a comma-separated value (CSV) log file that contains a list of e-mail addresses and the comments that each e-mail address made about the service. Here's an example entry:
"firstname.lastname@example.org", "This service is great."
The task is to provide a way for the service to send a thank-you message to the top ten users in terms of comment frequency. First, you need a script that can read and print a certain column of CSV data. The standard library of Python provides a CSV reader. The Python script below completes this goal:
#!/usr/bin/env python # CSV module that comes with the Python standard library import csv import sys if __name__ == "__main__": # The CSV module exposes a reader object that takes # a file object to read. In this example, sys.stdin. csvfile = csv.reader(sys.stdin) # The script should take one argument that is a column number. # Command-line arguments are accessed via sys.argv list. column_number = 0 if len(sys.argv) > 1: column_number = int(sys.argv) # Each row in the CSV file is a list with each # comma-separated value for that line. for row in csvfile: print row[column_number]
This script can parse the CSV data and return in plain text the
column that is supplied as a command-line argument. It uses
Let's add this script to the chain. The new script is chained with the others to print out a list of e-mail addresses and their comment frequencies using the command listed below (the .csv log file is assumed to be called emailcomments.csv and the new Python script, csvcolumn.py):
$ cat emailcomments.csv | python csvcolumn.py | ↪python namescount.py | sort -rn | head -n 5
Next, you need a way to send an e-mail. In the Python standard library of functions, you can import smtplib, which is a module that allows you to connect to an SMTP server to send mail. Let's write a simple Python script that uses this library to send a message to each of the top ten e-mail addresses found already:
#!/usr/bin/env python import smtplib import sys GMAIL_SMTP_SERVER = "smtp.gmail.com" GMAIL_SMTP_PORT = 587 GMAIL_EMAIL = "Your Gmail Email Goes Here" GMAIL_PASSWORD = "Your Gmail Password Goes Here" def initialize_smtp_server(): ''' This function initializes and greets the smtp server. It logs in using the provided credentials and returns the smtp server object as a result. ''' smtpserver = smtplib.SMTP(GMAIL_SMTP_SERVER, GMAIL_SMTP_PORT) smtpserver.ehlo() smtpserver.starttls() smtpserver.ehlo() smtpserver.login(GMAIL_EMAIL, GMAIL_PASSWORD) return smtpserver def send_thank_you_mail(email): to_email = email from_email = GMAIL_EMAIL subj = "Thanks for being an active commenter" # The header consists of the To and From and Subject lines # separated using a newline character header = "To:%s\nFrom:%s\nSubject:%s \n" % (to_email, from_email, subj) # Hard-coded templates are not best practice. msg_body = """ Hi %s, Thank you very much for your repeated comments on our service. The interaction is much appreciated. Thank You.""" % email content = header + "\n" + msg_body smtpserver = initialize_smtp_server() smtpserver.sendmail(from_email, to_email, content) smtpserver.close() if __name__ == "__main__": # for every line of input. for email in sys.stdin.readlines(): send_thank_you_mail(email)
This Python script supports contacting any SMTP server, whether local or remote. For ease of use, I have included Gmail's SMTP server, and it should work, provided you give the scripts the correct Gmail credentials. The script uses the functions provided to send mail in smtplib. This again demonstrates the power of using Python at this level. Something like SMTP interaction is easy and readable in Python. Equivalent shell scripts are messy, and such libraries are not as easily accessible, if they exist at all.
In order to send the e-mails to the top ten users sorted by comment
frequency, first you must isolate only the e-mail column of the output of
column names. To isolate a certain column in Linux, you use the
command. In the example below, the commands are given in two
separate chains. For ease of use, I wrote the output into a temporary
file, which can be loaded into the second chain. This simply makes the
process more readable (the Python script for sending mail is referred
to as sendemail.py):
$ cat emailcomments.csv | python csvcolumn.py | ↪python namescount.py | sort -rn > /tmp/comment_freq $ cat /tmp/comment_freq | head -n 10 | cut -f2 | ↪python sendemail.py
This shows the real power of Python as a utility in a chain of bash commands such as this. Writing scripts that accept input from standard input and write any data out to standard out, allows the developer to chain commands such as these together quickly and easily with a link in the chain often being a Python program. This philosophy of designing a small application that services one purpose fits nicely with the flow of commands being used here.
Richard Delaney is a software engineer with Demonware Ireland. Richard works on back-end Web services using Python and the Django Web framework. He has been an avid Linux user and evangelist for the past five years.
Fast/Flexible Linux OS Recovery
On Demand Now
In this live one-hour webinar, learn how to enhance your existing backup strategies for complete disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible full-system recovery solution for UNIX and Linux systems.
Join Linux Journal's Shawn Powers and David Huffman, President/CEO, Storix, Inc.
Free to Linux Journal readers.Register Now!
- The Italian Army Switches to LibreOffice
- Download "Linux Management with Red Hat Satellite: Measuring Business Impact and ROI"
- Linux Mint 18
- Oracle vs. Google: Round 2
- The FBI and the Mozilla Foundation Lock Horns over Known Security Hole
- Varnish Software's Varnish Massive Storage Engine
- Petros Koutoupis' RapidDisk
- Devuan Beta Release
- Privacy and the New Math
Until recently, IBM’s Power Platform was looked upon as being the system that hosted IBM’s flavor of UNIX and proprietary operating system called IBM i. These servers often are found in medium-size businesses running ERP, CRM and financials for on-premise customers. By enabling the Power platform to run the Linux OS, IBM now has positioned Power to be the platform of choice for those already running Linux that are facing scalability issues, especially customers looking at analytics, big data or cloud computing.
￼Running Linux on IBM’s Power hardware offers some obvious benefits, including improved processing speed and memory bandwidth, inherent security, and simpler deployment and management. But if you look beyond the impressive architecture, you’ll also find an open ecosystem that has given rise to a strong, innovative community, as well as an inventory of system and network management applications that really help leverage the benefits offered by running Linux on Power.Get the Guide