Home ›

Tech Tip: Fun With Gawk

Feb 15, 2010  By James Hinnant
 in

When grep and sed aren't enough, gawk may provide the extra horsepower that you need. The following tip contains a sampling of some of the things one might do with gawk.

Extract the last column from a text file, whitespace-separated: 

cat myfile | gawk '{print $NF}'

or: 

gawk '{print $NF}' myfile

List counts of files owned by each user in the current directory: 

/bin/ls -l | \
    gawk 'NR > 1 {counts[$3]++;}
          END {for (s in counts) {
                   printf("  %-15s : % 5d\n",
                          s, counts[s]);}}' | \
        sort

Kill your processes (one use is to kill a hung login if you can remotely log in to the workstation from another machine): 

ps -elf | \
    gawk -v me="$USER" '$3 == me {print $4}' | \
        egrep -v $$ | \
            xargs -i@@ kill -9 @@; kill -9 $$
______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

thanks

James Hinnant's picture
Submitted by James Hinnant on Tue, 02/23/2010 - 22:15.

Thanks for the tips, especially pkill (which is new with Solaris 7; 1998! :) ).

The file count routine I use for counting source files in the current directory, so strange filenames are not an issue (also I find find to be kind of slow).

Here is the online manual for gawk: http://www.gnu.org/software/gawk/manual/

Here is a little program people might remember coding from FORTRAN class; maybe it can help put the baby to sleep:

echo 100 1 10 0.01 | gawk '{amp = $1; freq = $2; cycles = $3; tdelta = $4; for (i = 0; i <= 2 * 3.14159 * cycles; i += 2 * 3.14159 / 360) {printf("%*s\n", int((amp/2.0) * sin(i * freq) + (amp/2.0)), "*"); system("sleep " tdelta); }}'

What does this have to do

Anonymous's picture
Submitted by Anonymous (not verified) on Thu, 03/25/2010 - 19:00.

What does this have to do with Fortran? I use Fortran every day, and have no idea what you're talking about. Also, I was born in '84, long after punch cards and FORTRAN IV etc., which may explain my confusion.

don't parse output of ls

chattr's picture
Submitted by chattr (not verified) on Mon, 02/15/2010 - 23:58.

Too many odd things can be in file names, spaces, newlines, etc.

It's a rare case that all you want is a list of file names. In most cases you want to do something to the files based on their names, and if that's the case, 'find' can be a better tool, or maybe ' stat -c %n [glob]'.

Start learning basic unix commands before playing with awk

przemoc's picture
Submitted by przemoc (not verified) on Mon, 02/15/2010 - 15:17.

Sorry, but there is no need to use (g)awk here. I really like (g)awk, but your examples are just poor. Ok, maybe first one is a typical case for awk.

You forgot about dot files, but second one can be replaced with:

ls -Al | sed '1d;s/\([^ \t]\+[ \t]\+\)\{2\}//;s/[ \t].*//' | sort | uniq -c

or even better:

find . -printf "%u\n" -mindepth 1 -maxdepth 1 | sort | uniq -c

Third one (really badly and also wrongly implemented) is a perfect case for pkill:

pkill -9 -U "$UID"

I can agree that your second example might be faster than mine, but it's irrelevant here.

Regards.

White Paper
More Resources
Fabric-Based Computing Enables Optimized Hyperscale Data Centers

Today’s modular x86 servers are compute-centric, designed as a least common denominator to support a wide range of IT workloads. Those generic, virtualized IT workloads have much different resource optimization requirements than hyperscale and cloud applications. They have resulted in a “one size fits all” enterprise IT architecture that is not optimized for a specific set of IT workloads, and especially not emerging hyperscale workloads, such as web applications, big data, and object storage. In this report, you will learn how shifting the focus from traditional compute-centric IT architectures to an innovative disaggregated fabric-based architecture can optimize and scale your data center.

Learn More

Sponsored by AMD

White Paper
More Resources
Red Hat White Paper: Using an Open Source Framework to Catch the Bad Guy

Built-in forensics, incident response, and security with Red Hat Enterprise Linux 6

Every security policy provides guidance and requirements for ensuring adequate protection of information and data, as well as high-level technical and administrative security requirements for a system in a given environment. Traditionally, providing security for a system focuses on the confidentiality of the information on it. However, protecting the data integrity and system and data availability is just as important. For example, when processing United States intelligence information, there are three attributes that require protection: confidentiality, integrity, and availability.

Learn more about catching the bad guy in this free white paper.

Learn More

Sponsored by DLT Solutions