Real Programming with AWK

 in
AWK may not be the first language that comes to mind when you need to write a program, but its flexibility may surprise you.
Conclusion

If you have experience with C programming, you probably will find it quite easy to generate programs in the AWK programming language. I hope this has given you some idea about what AWK is capable of doing. The next time you sit down to write a program, give AWK a chance--it may surprise you yet.

Acknowledgements

Thanks to Ian, Lailaa, Darren and Vinesh for proofreading this article. The references used were the GNU AWK Manual and The AWK Programming Language by Aho, Weinberger and Kernighan.

Alan Bradley works as a Senior Systems Engineer for a large ICT company in South Africa. In between writing AWK programs, he plays chess, fly fishes and makes knives.

______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

AWK Cheat Sheet

Peteris Krumins's picture

Hey, this is a great post for beginners on AWK.

For AWK learners I'd like to suggest the AWK cheat sheet I have made and put on my blog:
AWK Cheat Sheet.

This cheat sheet contains:
* Predefined Variable Summary, which lists all the predefined variables and which awk versions (original awk, nawk or gawk) have it built in,
* GNU awk’s command line argument summary,
* I/O statements,
* Numeric functions,
* Bit manipulation functions,
* I18N (internatiolization) functions,
* String functions, and finally,
* Time functions.

For more advanced users, I made a youtube video downloader in GAWK (which supports networking):
Read how it was made (download link on the page) on my blog:
GAWK Youtube Video Downloader

Sincerely,
Peteris Krumins

Re: Real Programming with AWK

Anonymous's picture

Awk does initiate the Nostalgic old man in me.

And it's a great resource for small tasks, also for those small
tasks that might need to be managed in another language afterwards (almost zero learning curve *is* a boost for
project jumpstarting).

Awk is also more available in every kind of setup.

I used to write notes at university at my calculator, then
I transferred to my system and used a pseudoheader
analyzed by AWK in my notes, to redirect them to
the proper file in my system. Clever, compact and thus
quite Unix-like.

Yet I didn't start doing it in Unix, but rather in Windows
using Cygwin: That distribution (b20) lacked perl.

Even at my first fulltime job in a Telephone Company,
AIX and SunOS installations lacked perl, but had awk
readily available.

E.G.
..---------`````\
) :/ \ :/\ // \ )
( || || || || (
\........------``
E-mail: garciag@ieee.org
Cellphone: MEX-55-9198-7119
Web: http://garcia.d2g.biz/garcia/
Messenger: ernestogarciagossio
PGP: http://pgpkeys.mit.edu/ - garcia@prousa.net

Just use Perl, please...

Anonymous's picture

As someone who used to write a lot of shell and awk (I once wrote a 4GL compiler using this...) before Perl was available, I just have to urge anyone reading this article to not bother with awk.

Perl is very easy to get used to, and unlike awk can be used for virtually any programming task, as well as being good for whipping up simple scripts. You can even use its bundled 'a2p' tool to convert scripts from awk to Perl. Try taking some awk scripts and feeding them into a2p to get an idea of how Perl works.

Perl?

Anonymous's picture

Perl is a horrid language, its bloated and extemely un-intuitve and very hard to 'code-read'. Even good Perl programmers find themselves saying WTF when comming back to their own code after a couple of months. Bash, python, php,there are lots of other better choices

Choice

KP's picture

The keywords are choice, time, knowledge and availibility. If you have a few minutes, a some small Linux distribution or old *nix and only awk, the choice is determined: awk. awk is a standard programming language; it's small and practical. Perl has more features; it's more than programming language - it's a technology. So, you can make choice, if you can.

KP.

Re: Just use Perl, please...

Anonymous's picture

Please...

I used a screwdriver to mine marble. It wasn't fun. Therefore, all screwdrivers suck in all contexts.

BigGiantClue: Awk isn't a particularly good tool to impliment "a 4GL complier", whatever that buzzword means this week...

Awk is ruthlessly efficient in its problem space; it's inefficient in all others. Perl tries to be all things to all people, with all the apropos tradeoffs, with the bonus of apropriately byzantine syntax!

Those who can't tell the difference have bigger issues.

Re: Real Programming with AWK

Anonymous's picture

I can't agree more how powerful AWK could be, I've just used awk to complement a shell script to automate duplicating files from a list and setting the proerties as needed, It's is also very simple to learn (for the record i am a 3 weeks old linux user (and i wrote the script 2 weeks ago).. It is simple and usable for freshies

Re: Real Programming with AWK

Anonymous's picture

I use it to grind log files and generate tabular data (spam rejects, naughty hosts from ids) for display as web pages. BEGIN and END are great for appending html headers and footers.

Re: Real Programming with AWK

Anonymous's picture

Well, awk is a fairly opaque language. Why bother learning awk when you can learn perl and do so much more? And please don't tell me that perl is a ressource hog compared to awk! That doesn't wash any more. Forget awk and sed, use perl for goodness sake!

Re: Real Programming with AWK

Anonymous's picture

Absolute crap. Awk is an order of magnatude faster than Perl for anything other than the most complicated awk scripts. Sed is vastly more efficient than perl for what it does.

And awk is "opaque" but perl isn't...oh, I get it...this is a troll! Sorry...I got completely taken...

Re: Real Programming with AWK

Anonymous's picture

- for the same reason that some times is better to do something in perl rather than C or C++.
- for the same reason that other times is better to do something in a shell script rather than awk
- because AWK is learnt in 5 minutes (totally) so there are no cost
I recomend you to learn AWK and you will find several other reasons.

I'm learning Awk at the moment

Anonymous's picture

Hi, Thanks for the artical. I'm learning awk at the moment & its a very nice language to learn, its great for Admin & programming for fields. Its also a nice way to wet your feet for programming.

Re: I'm looking for Alan Bradley

Anonymous's picture

I am not a programmer and I need a script (possibily with awk) to read a .txt file and convert it to html or xml.
The .txt file has a structure which must be known starting with the headlines and titles of the file.
To html the file has to have a file with the title, the headline and the content.
The same wiht xml files.
Best regards.
Marcio@ViaBsb.com

Webinar
One Click, Universal Protection: Implementing Centralized Security Policies on Linux Systems

As Linux continues to play an ever increasing role in corporate data centers and institutions, ensuring the integrity and protection of these systems must be a priority. With 60% of the world's websites and an increasing share of organization's mission-critical workloads running on Linux, failing to stop malware and other advanced threats on Linux can increasingly impact an organization's reputation and bottom line.

Learn More

Sponsored by Bit9

Webinar
Linux Backup and Recovery Webinar

Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs. However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.” After all, before data can be restored to a system, there must be a system to restore it to.

In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.

Learn More

Sponsored by Storix