Using grep

New Linux users unfamiliar with this standard Unix tool may not realize how useful it is. In this tutorial for the novice user, Eric demonstrates grep techniques.
Special Characters

Many Unix utilities use regular expressions to specify patterns. Before we go into actual examples of regular expressions, let's define a few terms and explain a few conventions that I will use in the exercises.

  • Character any printable symbol, such as a letter, number, or punctuation mark.

  • String a sequence of characters, such as cat or segment (sometimes referred to as a literal).

  • Expression also a sequence of characters. The difference between a string and an expression is that while strings are to be taken literally, expressions must be evaluated before their actual value can be determined. (The manual page for GNU grep compares regular expressions to mathematical expressions.) An expression usually can stand for more than one thing, for example the regular expression th[ae]n can stand for then or than. Also, the shell has its own type of expression, called globbing, which is usually used to specify file names. For example, *.c matches any file ending in the characters .c.

  • Metacharacters the characters whose presence turns a string into an expression. Metacharacters can be thought of as the operators that determine how expressions are evaluated. This will become more clear as we work through the examples below.

Interference

You have probably entered a shell command like

$ ls -l *.c

at some time. The shell “knows” that it is supposed to replace *.c with a list of all the files in the current directory whose names end in the characters .c.

This gets in the way if we want to pass a literal * (or ?, |, $, etc.) character to grep. Enclosing the regular expression in `single quotes' will prevent the shell from evaluating any of the shell's metacharacters. When in doubt, enclose your regular expression in single quotes.

Basic Searches

The most basic regular expression is simply a string. Therefore a string such as foo is a regular expression that has only one match: foo.

We'll continue our examples with another file in the same directory, so make sure you are still in the /usr/src/linux directory:

$ grep Linus CREDITS
Linus
N: Linus Torvalds
E: Linus.Torvalds@Helsinki.FI
D: Personal information about Linus

This quite naturally gives the four lines that have Linus Torvalds' name in them.

As I said earlier, the Unix shells have different metacharacters, and use different kinds of expressions. The metacharacters . and * cause the most confusion for people learning regular expression syntax after they have been using shells (and DOS, for that matter).

In regular expressions, the character . acts very much like the ? at the shell prompt: it matches any single character. The *, by contrast, has quite a different meaning: it matches zero or more instances of the previous character.

If we type

$ grep tha. CREDITS

we get this (partial listing only):

S: Northampton
E: Hein@Informatik.TU-Clausthal.de

As you can see, grep printed every instance of tha followed by any character. Now try

$ grep 'tha*' CREDITS
S: Northampton
D: Author of serial driver
D: Author of the new e2fsck
D: Author of loopback device driver

We received a much larger response with “*”. Since “*” matches zero or more instances of the previous character (in this case the letter “a”), we greatly increase our possibility of a match because we made th a legal match!

Character Classes

One of the most powerful constructs available in regular expression syntax is the character class. A character class specifies a range or set of characters to be matched. The characters in a class are delineated by the [ and ] symbols. The class [a-z] matches the lowercase letters a through z, the class [a-zA-Z] matches all letters, uppercase or lowercase, and [Lh] would match upper case L or lower case h.

$ grep 'sm[ai]' CREDITS
E: csmith@convex.com
D: Author of several small utilities

since our expression matches sma or smi. The command

$ grep '[a-z]' CREDITS

gives us most of the file. If you look at the file closely, you'll see that a few lines have no lowercase letters; these are the only lines that grep does not print.

Now since we can match a set of characters, why not exclude them instead? The circumflex, ^, when included as the first member of a character class, matches any character except the characters specified in the class.

$ grep Sm CREDITS

gives us three lines:

D: Small patches for kernel, libc
D: Smail binary packages for Slackware and Debian
N: Chris Smith
$ grep 'Sm[^i]' CREDITS

gives us two

D: Small patches for kernel, libc
D: Smail binary packages for Slackware and Debian

because we excluded i as a possible letter to follow Sm.

To search for a class of characters including a literal ^ character, don't place it first in the class. To search for a class including a literal -, place it the very last character of the class. To search for a class including the literal character ], place it the first character of the class.

Often it is convenient to base searches on the position of the characters on a line. The ^ character matches the beginning of a line (outside of a character class, of course) and the $ matches the end. (Users of vi may recognize these metacharacters as commands.) Earlier, searching for Linus gave us four lines. Let's change that to:

grep 'Linus$' CREDITS

which gives us

Linus
D: Personal information about Linus

two lines, since we specified that Linus must be the last five characters of the line. Similarly,

grep - CREDITS

produces 99 lines, while

grep '^-' CREDITS

produces only one line:

----------

In some circumstances you may need to match a metacharacter. Inside a character class set all characters are taken as literals (except ^, -, and ], as shown above). However, outside of classes we need a way to turn a metacharacter into a literal character to match.

______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Some confuusion while searching using grep

Vinod Semwal's picture

I hav a dir and on that file present like this

-rw-r--r-- 1 vs users 1390254 Jun 2 01:00 abcdefg.1006010100.dat
-rw-r--r-- 1 vs users 1388800 Jun 3 01:00 abcdefg.1006020100.dat
-rw-r--r-- 1 vs users 1388555 Jun 4 01:00 abcdefg.1006030100.dat
-rw-r--r-- 1 vs users 1392184 Jun 5 01:00 abcdefg.1006040100.dat
-rw-r--r-- 1 vs users 1391747 Jun 6 01:00 abcdefg.1006050100.dat
-rw-r--r-- 1 vs users 1392099 Jun 7 01:00 abcdefg.1006060100.dat
-rw-r--r-- 1 vs users 1389362 Jun 8 01:00 abcdefg.1006070100.dat
-rw-r--r-- 1 vs users 1392676 Jun 9 01:00 abcdefg.1006080100.dat
-rw-r--r-- 1 vs users 1436696 Jun 10 01:00 abcdefg.1006090100.dat
-rw-r--r-- 1 vs users 1060539 Jun 10 18:39 abcdefg.1006100100.dat

please check output while i am using grep command in below manner ..

vs@vodalksmvs2 /var/opt/nokia/smvs/tmp > ls -ltr | grep abc*
vs@vodalksmvs2 /var/opt/nokia/smvs/tmp > ls -ltr | grep "abc*"| tail
-rw-r--r-- 1 vs users 1390254 Jun 2 01:00 abcdefg.1006010100.dat
-rw-r--r-- 1 vs users 1388800 Jun 3 01:00 abcdefg.1006020100.dat
-rw-r--r-- 1 vs users 1388555 Jun 4 01:00 abcdefg.1006030100.dat
-rw-r--r-- 1 vs users 1392184 Jun 5 01:00 abcdefg.1006040100.dat
-rw-r--r-- 1 vs users 1391747 Jun 6 01:00 abcdefg.1006050100.dat
-rw-r--r-- 1 vs users 1392099 Jun 7 01:00 abcdefg.1006060100.dat
-rw-r--r-- 1 vs users 1389362 Jun 8 01:00 abcdefg.1006070100.dat
-rw-r--r-- 1 vs users 1392676 Jun 9 01:00 abcdefg.1006080100.dat
-rw-r--r-- 1 vs users 1436696 Jun 10 01:00 abcdefg.1006090100.dat
-rw-r--r-- 1 vs users 1059029 Jun 10 18:37 abcdefg.1006100100.dat
vs@vodalksmvs2 /var/opt/nokia/smvs/tmp > ls -ltr | grep abc| tail
-rw-r--r-- 1 vs users 1390254 Jun 2 01:00 abcdefg.1006010100.dat
-rw-r--r-- 1 vs users 1388800 Jun 3 01:00 abcdefg.1006020100.dat
-rw-r--r-- 1 vs users 1388555 Jun 4 01:00 abcdefg.1006030100.dat
-rw-r--r-- 1 vs users 1392184 Jun 5 01:00 abcdefg.1006040100.dat
-rw-r--r-- 1 vs users 1391747 Jun 6 01:00 abcdefg.1006050100.dat
-rw-r--r-- 1 vs users 1392099 Jun 7 01:00 abcdefg.1006060100.dat
-rw-r--r-- 1 vs users 1389362 Jun 8 01:00 abcdefg.1006070100.dat
-rw-r--r-- 1 vs users 1392676 Jun 9 01:00 abcdefg.1006080100.dat
-rw-r--r-- 1 vs users 1436696 Jun 10 01:00 abcdefg.1006090100.dat
-rw-r--r-- 1 vs users 1060539 Jun 10 18:39 abcdefg.1006100100.dat
vs@vodalksmvs2 /var/opt/nokia/smvs/tmp > ls -ltr | grep abc.*| tail -2
-rw-r--r-- 1 vs users 1436696 Jun 10 01:00 abcdefg.1006090100.dat
-rw-r--r-- 1 vs users 1060539 Jun 10 18:39 abcdefg.1006100100.dat
vs@vodalksmvs2 /var/opt/nokia/smvs/tmp > ls -ltr | grep *abc| tail -2
vs@vodalksmvs2 /var/opt/nokia/smvs/tmp > ls -ltr | grep *.abc| tail -2
vs@vodalksmvs2 /var/opt/nokia/smvs/tmp >

I want to know in what manner grep runs and giving output for above commands.

grep (-A|-B|-C)

Anonymous's picture

I use linux as my development environment. Grep with -A, -B and -C option provides me with context for what I'm searching. These options print out lines either above or below the target line.

For example I search for a function call and get grep to display the lines above and/or below the said line.

grep -p

iisdjp's picture

I come from the AIX world. We have many scripts that use grep -p to get the paragraph containing a search string. I cannot find anything comparable to grep -p in Red Hat Linux. Any ideas?

grep -p

iisdjp's picture

Never mind, I found this code. (Shoulda done more googling first! :-) )

#!/bin/sh

# usage: pargrep

inFile="$1"
searchString="$2"

awk '
BEGIN {
FS="\n"
RS=""
}
/'"$searchString"'/ { print }
' ${inFile}

non-standard args - could be made less confusing

Anonymous's picture

That's good as far as it goes, but you should reverse $1 and $2. NORMAL grep is:

grep

That pargrep is

grep

At least you should minimize the differences. Also, if you go with the "standard" way, a very small change to that script could be made to search across multiple files, just like grep would.

[sigh] stupid html. Ok, that

Anonymous's picture

[sigh] stupid html.

Ok, that should have read:

...
grep pattern filename

that pargrep is:

grep filename pattern
...

Question

Anonymous's picture

Dear Sir,

How can i search in my lunix server for a word in the sources code for any file that is located in the vhost. for example:

I need to search for the word "iframe" in all the "vhosts" folder on my server. this word is in the "Sources code for all my files for my websites"

Can u help me in this please ?
i will too much greatfull for you
regards
samer

The fastest method is...

Anonymous's picture

The following, replacing "*.php" by your source files' extension:

$ find . -name "*.php" -print0 | xargs -0 grep iframe

Search

Mitch Frazier's picture

There are a number of ways of doing that, try:

  $ find /path/to/vhosts -type f -exec grep --with-filename iframe {} \;

or try:

  $ grep -r iframe /path/to/vhosts/*

Mitch Frazier is an Associate Editor for Linux Journal.

Webinar
One Click, Universal Protection: Implementing Centralized Security Policies on Linux Systems

As Linux continues to play an ever increasing role in corporate data centers and institutions, ensuring the integrity and protection of these systems must be a priority. With 60% of the world's websites and an increasing share of organization's mission-critical workloads running on Linux, failing to stop malware and other advanced threats on Linux can increasingly impact an organization's reputation and bottom line.

Learn More

Sponsored by Bit9

Webinar
Linux Backup and Recovery Webinar

Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs. However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.” After all, before data can be restored to a system, there must be a system to restore it to.

In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.

Learn More

Sponsored by Storix