Kernel Korner - Intro to inotify

From Issue #139
November 2005

Sep 28, 2005 By Robert Love

Linux Journal

Applications that watch thousands of files for changes, or that need to know when a storage device gets disconnected, need a clean, fast solution to the file change notification problem. Here it is.

On Unmount

One of the biggest issues with dnotify (aside from the signals and basically everything else) is that a dnotify watch on a directory requires that said directory remain open. Consequently, watching a directory on, say, a USB keychain drive prevents the drive from unmounting. inotify solves this problem by not requiring that any file be open.

inotify takes this one step further, though, and sends out the IN_UNMOUNT event when the filesystem on which a file resides is unmounted. It also automatically destroys the watch and cleanup.

Moves

Move events are complicated because inotify may be watching the directory that the file is moved to or from, but not the other. Because of this, it is not always possible to alert the user of the source and destination of a file involved in a move. inotify is able to alert the application to both only if the application is watching both directories.

In that case, inotify emits an IN_MOVED_FROM from the watch descriptor of the source directory, and it emits an IN_MOVED_TO from the watch descriptor of the destination directory. If watching only one or the other, only the one event will be sent.

To tie together two disparate moved to/from events, inotify sets the cookie field in the inotify_event structure to a unique nonzero value. Two events with matching cookies are thus related, one showing the source and one showing the destination of the move.

Obtaining the Size of the Queue

The size of the pending event queue can be obtained via FIONREAD:


unsigned int queue_len;
int ret;

ret = ioctl (fd, FIONREAD, &queue_len);
if (ret < 0)
        perror ("ioctl");
else
        printf ("%u bytes pending in queue\n", queue_len);

This is useful to implement throttling: reading from the queue only when the number of events has grown sufficiently large.

Configuring inotify

inotify is configurable via procfs and sysctl.

/proc/sys/filesystem/inotify/max_queued_events is the maximum number of events that can be queued at once. If the queue reaches this size, new events are dropped, but the IN_Q_OVERFLOW event is always sent. With a significantly large queue, overflows are rare even if watching many objects. The default value is 16,384 events per queue.

/proc/sys/filesystem/inotify/max_user_instances is the maximum number of inotify instances that a given user can instantiate. The default value is 128 instances, per user.

/proc/sys/filesystem/inotify/max_user_watches is the maximum number of watches per instance. The default value is 8,192 watches, per instance.

These knobs exist because kernel memory is a precious resource. Although any user can read these files, only the system administrator can write to them.

Conclusion

inotify is a simple yet powerful file change notification system with an intuitive user interface, excellent performance, support for many different events and numerous features. inotify is currently in use in various projects, including Beagle, an advanced desktop indexing system, and Gamin, a FAM replacement.

What application will use inotify next?

Resources for this article: /article/8534.

Robert Love is a senior kernel hacker in Novell's Ximian Desktop group and the author of Linux Kernel Development (SAMS 2005), now in its second edition. He holds degrees in CS and Mathematics from the University of Florida. Robert lives in Cambridge, Massachusetts.

______________________

Comments

support on 64-bit machines

Submitted by vishalj on Thu, 07/22/2010 - 03:36.

Do i need to make some changes to get it working on 64 bit machines .

read() length, alignment

Submitted by Anonymous (not verified) on Fri, 07/02/2010 - 16:15.

fyi, something that wasn't clear in the article is the requirements on the read() call.

I was playing with inotify, and I tried to call read an event in 2 parts:
struct inotify_event evt_hdr;
ssize_t nr = read(fd, &evt_hdr, sizeof(evt_hdr));

read() returned EINVAL
( at least on 2.6.23.17-88.fc7 )

When I restructured the code to read the entire event,
(similar to the article example) it works.

It would seem also that the read buffer must have proper alignment for a
struct inotify_event as well, so just declaring a char buffer on the stack
for the read() destination isn't necessarily going to work, either.

otherwise, nice article, nice feature!
Thanks

Inotify does not work

Submitted by Inotify user (not verified) on Mon, 06/14/2010 - 04:59.

I have registered Inotify to a file for listening IN_MODIFY | IN_DELETE | IN_DELETE_SELF. But it does not work when I fwrite() to the file. This doest not happen always.

Seems to be a bug in iNotify

changes

Submitted by Anonymous on Tue, 02/09/2010 - 06:57.

Seems changes are coming, not primary in inotify and the old dnotify, but in the whole system of fs/notify.
After a long search I think there is a guy called Eric Paris (RedHat) that is currently in charge of notify structures in kernel.
He is actively commiting to the kernel git repository, which can be followed always at http://git.kernel.org/?p=linux/kernel/git/next/linux-next.git.

As Eric Paris claims he is preparing the whole thing for upcoming "fanotify" system, which will be more powerfull, sending the possibility to decide which file accessse should be granted to the userspace (if I got right the idea).

http://lwn.net/Articles/311350/
http://lwn.net/Articles/339253/

Which is of course fantastic, but for many cases we dont need this power, and the inotify does the job at right level. I just needed a little add-on to provide the uid in inotify_event, but now it seems we will have to wait for the fanotify to be introduced first, and after to be decided the future of inotify ...

Inotify

Submitted by aveugle (not verified) on Tue, 02/02/2010 - 09:12.

Is there any possibility to obtain the uid of the trigger for certain inotify_event?

P.S. in general - is inotify a living project or it is shut-down?

how to check for status of the inotify

Submitted by Anonymous (not verified) on Tue, 08/25/2009 - 05:28.

i have a problem and hope i get a response from you.
i have a script that will create an sql file to the watch directory of my inotify. upon detection, my C program then executes the sql statment using OCI functions. is there any way i can throw back the result/status to my script if the execution was successful or not?

i have a problem and hope i

Submitted by Anonymous (not verified) on Tue, 08/25/2009 - 05:34.

i have a problem and hope i get a response from you.

currently, my C program with the inotify is already running. this C program will wait base on its watch directory.

i then have a script that will create an sql file to the watch directory of my inotify. upon detection, my C program then executes the sql statment using OCI functions. is there any way i can throw back the result/status to my script if the execution was successful or not?

*** note: my script did not call my C program which is already running from the start

Not able to notify changes apart from /tmp

Submitted by Anonymous (not verified) on Sun, 08/16/2009 - 23:54.

Hi,

I am facing some different behaviour using inotify for notifying file changes.

I added file notification for some file in /tmp directory with IN_MODIFY flag.
when i modified contents in this file, i am able to read the notifications of this change and the mask value for this notification is IN_MODIFY(2).

If i use same file with same source code in other directory(ex : /home), iam getting mask value as IN_IGNORED flag(3276. After this any modifications to the file, i am not getting any notifications.

This is some strange behaviour.

Kernel : 2.6.15-1.2054_FC5

Thank you for this, it was

Submitted by Anonymous on Wed, 03/18/2009 - 02:42.

Thank you for this, it was very useful.

One annoyance for someone coming across this: while I created the inotify using
IN_MODIFY | IN_CREATE | IN_DELETE

When the file was deleted, it returned only IN_IGNORED (because the file was deleted, the inotify was removed), not IN_DELETED

inotify for /proc

Submitted by madhav (not verified) on Wed, 02/04/2009 - 02:38.

Hi,
There is a problem in my application. I am trying to monitor /proc for process creation/deletion, but i am not able to monitor the /proc directory in particular. The inotify will monitor all the sub directories in the /proc and other directories, but not in /proc, i.e. for process creation/deletion. i am using linux kernel-2.6.20-hardened. There was a bug previously reported in v2.6.16 regarding this issue, but i think it is fixed in further versions. if not please may i know so i will be able to patch it myself.
Please help..
Waiting for the reply

How to install inotify

Submitted by Anonymous (not verified) on Fri, 11/14/2008 - 03:31.

Hi..
can u pls send me the full procedure to download as wel as how to install inotify... it s very userful for me ...

Thanks

Refer this on how to install

Submitted by AG (not verified) on Wed, 12/10/2008 - 17:58.

Refer this on how to install and how to use for various scenarios. http://inotify.aiken.cz/?section=inotify&page=faq&lang=en

Monitoring whole directory tree

Submitted by Anonymous (not verified) on Mon, 09/29/2008 - 07:24.

Can you please help me?
I want to monitor the whole directory tree, using inotify.
I have done monitoring directory which notify me changes in file within that perticular directory, creation of directories into it, butI am unable to get the changes done within its subdirectories.
Please help me for this.

How would one get the user associated with an event.....

Submitted by rdifalco (not verified) on Mon, 06/25/2007 - 19:07.

If I wanted to write a utility that would list each time a file I was interested in was modified and by who, how would I do that? Could I do that with inotify? I'm guessing no.

INOTIFY a very helpful tool

Submitted by Ioan PREDESCU (not verified) on Wed, 10/04/2006 - 04:36.

I tested INOTIFY in conjunction with POSIX queues (mq_open…) on my CentOS 4.3 – 2.6.18 and it seems to work OK, I am really impressed by this ‘new’ feature:

Just a small ‘observation’ as the code piece from below is going to drastically decrease the performances (stack is exhausted as ‘event’ is an in loop declaration) :-)) :

while (i < len) {
struct inotify_event *event;
…………………

Thanks to INOTIFY creators and contributors,
John

you're flat out wrong about the stack exhausted <EOM>

Submitted by Anonymous (not verified) on Sat, 05/05/2007 - 18:54.

Code Feedback from gcc and a.out

Submitted by Anonymous (not verified) on Wed, 08/30/2006 - 08:55.

Thanks for posting this information and the examples. Was very stuck trying to convert to inotify without this article, and now have a working program thanks to your help. Feedback on this article accumulated from the process:

Unbalanced parenthesis in the #define for BUF_LEN
In that same code snippet, the while loop is conditional, based on (!len). That loop needs to be invoked when len is true and non-negative. As originally posted, the loop isn't invoked when an event is received from the read().
IN_CREATE is missing from Table 1, although it is referenced earlier in the article.

Thanks! inotify ROCKS!

wrong /proc path

Submitted by Anonymous (not verified) on Mon, 08/14/2006 - 07:47.

As of 2.6.17, it is NOT

/proc/sys/filesystem/inotify/max_user_watches

but

/proc/sys/fs/inotify/max_user_watches

Nice and helpful article

Submitted by Manoj Awasthi (not verified) on Tue, 04/04/2006 - 00:45.

Thanks for the same.

Special files

Submitted by Alberto (not verified) on Tue, 10/24/2006 - 14:55.

Hello!

Is inotify supposed to work on special files in /sys ?

I tried monitoring normal files and it works great, but when i try to monitor files on sysfs, strace shows that the executable is blocked on the read() call.

Thanks!
Alberto

Webcast

More Resources

How to Build an Optimal Hadoop Cluster to Store and Maintain Unlimited Amounts of Data Using Microservers

Realizing the promise of Apache® Hadoop® requires the effective deployment of compute, memory, storage and networking to achieve optimal results. With its flexibility and multitude of options, it is easy to over or under provision the server infrastructure, resulting in poor performance and high TCO. Join us for an in depth, technical discussion with industry experts from leading Hadoop and server companies who will provide insights into the key considerations for designing and deploying an optimal Hadoop cluster.

Learn More


Non-Linux FOSS: libnotify, OS X Style	Jun 18, 2013
Containers—Not Virtual Machines—Are the Future Cloud	Jun 17, 2013
Lock-Free Multi-Producer Multi-Consumer Queue on Ring Buffer	Jun 12, 2013
Weechat, Irssi's Little Brother	Jun 11, 2013
One Tail Just Isn't Enough	Jun 07, 2013
Introduction to MapReduce with Hadoop on Linux	Jun 05, 2013

Featured Jobs


Linux Systems Administrator \| Houston and Austin, Texas	Host Gator
Senior Perl Developer \| Austin, Texas	Host Gator
Technical Support Rep \| Houston and Austin, Texas	Host Gator
UX Designer \| Austin, Texas	Host Gator
Web & UI Developer (JavaScript & j Query) \| Austin, Texas	Host Gator

Free Webinar: Hadoop

How to Build an Optimal Hadoop Cluster to Store and Maintain Unlimited Amounts of Data Using Microservers

Some of key questions to be discussed are:

What is the “typical” Hadoop cluster and what should be installed on the different machine types?
Why should you consider the typical workload patterns when making your hardware decisions?
Are all microservers created equal for Hadoop deployments?
How do I plan for expansion if I require more compute, memory, storage or networking?

Register now for this free webinar.

Developer Poll

Already a subscriber? Click here for subscriber services.

First Name	Last Name

Address 1	Address 2

City	State	Zip

Email