Loading
Kernel Korner - Intro to inotify
Applications that watch thousands of files for changes, or that need to know when a storage device gets disconnected, need a clean, fast solution to the file change notification problem. Here it is.
One of the biggest issues with dnotify (aside from the signals and basically everything else) is that a dnotify watch on a directory requires that said directory remain open. Consequently, watching a directory on, say, a USB keychain drive prevents the drive from unmounting. inotify solves this problem by not requiring that any file be open.
inotify takes this one step further, though, and sends out the IN_UNMOUNT event when the filesystem on which a file resides is unmounted. It also automatically destroys the watch and cleanup.
Move events are complicated because inotify may be watching the directory that the file is moved to or from, but not the other. Because of this, it is not always possible to alert the user of the source and destination of a file involved in a move. inotify is able to alert the application to both only if the application is watching both directories.
In that case, inotify emits an IN_MOVED_FROM from the watch descriptor of the source directory, and it emits an IN_MOVED_TO from the watch descriptor of the destination directory. If watching only one or the other, only the one event will be sent.
To tie together two disparate moved to/from events, inotify sets the cookie field in the inotify_event structure to a unique nonzero value. Two events with matching cookies are thus related, one showing the source and one showing the destination of the move.
The size of the pending event queue can be obtained via FIONREAD:
unsigned int queue_len;
int ret;
ret = ioctl (fd, FIONREAD, &queue_len);
if (ret < 0)
perror ("ioctl");
else
printf ("%u bytes pending in queue\n", queue_len);
This is useful to implement throttling: reading from the queue only when the number of events has grown sufficiently large.
inotify is configurable via procfs and sysctl.
/proc/sys/filesystem/inotify/max_queued_events is the maximum number of events that can be queued at once. If the queue reaches this size, new events are dropped, but the IN_Q_OVERFLOW event is always sent. With a significantly large queue, overflows are rare even if watching many objects. The default value is 16,384 events per queue.
/proc/sys/filesystem/inotify/max_user_instances is the maximum number of inotify instances that a given user can instantiate. The default value is 128 instances, per user.
/proc/sys/filesystem/inotify/max_user_watches is the maximum number of watches per instance. The default value is 8,192 watches, per instance.
These knobs exist because kernel memory is a precious resource. Although any user can read these files, only the system administrator can write to them.
inotify is a simple yet powerful file change notification system with an intuitive user interface, excellent performance, support for many different events and numerous features. inotify is currently in use in various projects, including Beagle, an advanced desktop indexing system, and Gamin, a FAM replacement.
What application will use inotify next?
Resources for this article: /article/8534.
- « first
- ‹ previous
- 1
- 2
- 3
- 4
______________________
Webcast
How to Build an Optimal Hadoop Cluster to Store and Maintain Unlimited Amounts of Data Using Microservers
Realizing the promise of Apache® Hadoop® requires the effective deployment of compute, memory, storage and networking to achieve optimal results. With its flexibility and multitude of options, it is easy to over or under provision the server infrastructure, resulting in poor performance and high TCO. Join us for an in depth, technical discussion with industry experts from leading Hadoop and server companies who will provide insights into the key considerations for designing and deploying an optimal Hadoop cluster.
Sponsored by AMD
White Paper
Red Hat White Paper: Using an Open Source Framework to Catch the Bad Guy
Built-in forensics, incident response, and security with Red Hat Enterprise Linux 6
Every security policy provides guidance and requirements for ensuring adequate protection of information and data, as well as high-level technical and administrative security requirements for a system in a given environment. Traditionally, providing security for a system focuses on the confidentiality of the information on it. However, protecting the data integrity and system and data availability is just as important. For example, when processing United States intelligence information, there are three attributes that require protection: confidentiality, integrity, and availability.
Learn more about catching the bad guy in this free white paper.
Sponsored by DLT Solutions
- Dynamic DNS—an Object Lesson in Problem Solving
- Making Linux and Android Get Along (It's Not as Hard as It Sounds)
- Using Salt Stack and Vagrant for Drupal Development
- New Products
- A Topic for Discussion - Open Source Feature-Richness?
- Drupal Is a Framework: Why Everyone Needs to Understand This
- Validate an E-Mail Address with PHP, the Right Way
- RSS Feeds
- Readers' Choice Awards
- Tech Tip: Really Simple HTTP Server with Python
- BASH script to log IPs on public web server
1 hour 15 min ago - DynDNS
4 hours 51 min ago - Reply to comment | Linux Journal
5 hours 23 min ago - All the articles you talked
7 hours 47 min ago - All the articles you talked
7 hours 50 min ago - All the articles you talked
7 hours 51 min ago - myip
12 hours 16 min ago - Keeping track of IP address
14 hours 7 min ago - Roll your own dynamic dns
19 hours 20 min ago - Please correct the URL for Salt Stack's web site
22 hours 32 min ago
Enter to Win an Adafruit Pi Cobbler Breakout Kit for Raspberry Pi
It's Raspberry Pi month at Linux Journal. Each week in May, Adafruit will be giving away a Pi-related prize to a lucky, randomly drawn LJ reader. Winners will be announced weekly.
Fill out the fields below to enter to win this week's prize-- a Pi Cobbler Breakout Kit for Raspberry Pi.
Congratulations to our winners so far:
- 5-8-13, Pi Starter Pack: Jack Davis
- 5-15-13, Pi Model B 512MB RAM: Patrick Dunn
- 5-21-13, Prototyping Pi Plate Kit: Philip Kirby
- Next winner announced on 5-27-13!
Free Webinar: Hadoop
How to Build an Optimal Hadoop Cluster to Store and Maintain Unlimited Amounts of Data Using Microservers
Realizing the promise of Apache® Hadoop® requires the effective deployment of compute, memory, storage and networking to achieve optimal results. With its flexibility and multitude of options, it is easy to over or under provision the server infrastructure, resulting in poor performance and high TCO. Join us for an in depth, technical discussion with industry experts from leading Hadoop and server companies who will provide insights into the key considerations for designing and deploying an optimal Hadoop cluster.
Some of key questions to be discussed are:
- What is the “typical” Hadoop cluster and what should be installed on the different machine types?
- Why should you consider the typical workload patterns when making your hardware decisions?
- Are all microservers created equal for Hadoop deployments?
- How do I plan for expansion if I require more compute, memory, storage or networking?
Developer Poll
Comments
support on 64-bit machines
Do i need to make some changes to get it working on 64 bit machines .
read() length, alignment
fyi, something that wasn't clear in the article is the requirements on the read() call.
I was playing with inotify, and I tried to call read an event in 2 parts:
struct inotify_event evt_hdr;
ssize_t nr = read(fd, &evt_hdr, sizeof(evt_hdr));
read() returned EINVAL
( at least on 2.6.23.17-88.fc7 )
When I restructured the code to read the entire event,
(similar to the article example) it works.
It would seem also that the read buffer must have proper alignment for a
struct inotify_event as well, so just declaring a char buffer on the stack
for the read() destination isn't necessarily going to work, either.
otherwise, nice article, nice feature!
Thanks
Inotify does not work
I have registered Inotify to a file for listening IN_MODIFY | IN_DELETE | IN_DELETE_SELF. But it does not work when I fwrite() to the file. This doest not happen always.
Seems to be a bug in iNotify
changes
Seems changes are coming, not primary in inotify and the old dnotify, but in the whole system of fs/notify.
After a long search I think there is a guy called Eric Paris (RedHat) that is currently in charge of notify structures in kernel.
He is actively commiting to the kernel git repository, which can be followed always at http://git.kernel.org/?p=linux/kernel/git/next/linux-next.git.
As Eric Paris claims he is preparing the whole thing for upcoming "fanotify" system, which will be more powerfull, sending the possibility to decide which file accessse should be granted to the userspace (if I got right the idea).
http://lwn.net/Articles/311350/
http://lwn.net/Articles/339253/
Which is of course fantastic, but for many cases we dont need this power, and the inotify does the job at right level. I just needed a little add-on to provide the uid in inotify_event, but now it seems we will have to wait for the fanotify to be introduced first, and after to be decided the future of inotify ...
Inotify
Is there any possibility to obtain the uid of the trigger for certain inotify_event?
P.S. in general - is inotify a living project or it is shut-down?
how to check for status of the inotify
i have a problem and hope i get a response from you.
i have a script that will create an sql file to the watch directory of my inotify. upon detection, my C program then executes the sql statment using OCI functions. is there any way i can throw back the result/status to my script if the execution was successful or not?
i have a problem and hope i
i have a problem and hope i get a response from you.
currently, my C program with the inotify is already running. this C program will wait base on its watch directory.
i then have a script that will create an sql file to the watch directory of my inotify. upon detection, my C program then executes the sql statment using OCI functions. is there any way i can throw back the result/status to my script if the execution was successful or not?
*** note: my script did not call my C program which is already running from the start
Not able to notify changes apart from /tmp
Hi,
I am facing some different behaviour using inotify for notifying file changes.
I added file notification for some file in /tmp directory with IN_MODIFY flag.
when i modified contents in this file, i am able to read the notifications of this change and the mask value for this notification is IN_MODIFY(2).
If i use same file with same source code in other directory(ex : /home), iam getting mask value as IN_IGNORED flag(3276. After this any modifications to the file, i am not getting any notifications.
This is some strange behaviour.
Kernel : 2.6.15-1.2054_FC5
Thank you for this, it was
Thank you for this, it was very useful.
One annoyance for someone coming across this: while I created the inotify using
IN_MODIFY | IN_CREATE | IN_DELETE
When the file was deleted, it returned only IN_IGNORED (because the file was deleted, the inotify was removed), not IN_DELETED
inotify for /proc
Hi,
There is a problem in my application. I am trying to monitor /proc for process creation/deletion, but i am not able to monitor the /proc directory in particular. The inotify will monitor all the sub directories in the /proc and other directories, but not in /proc, i.e. for process creation/deletion. i am using linux kernel-2.6.20-hardened. There was a bug previously reported in v2.6.16 regarding this issue, but i think it is fixed in further versions. if not please may i know so i will be able to patch it myself.
Please help..
Waiting for the reply
How to install inotify
Hi..
can u pls send me the full procedure to download as wel as how to install inotify... it s very userful for me ...
Thanks
Refer this on how to install
Refer this on how to install and how to use for various scenarios. http://inotify.aiken.cz/?section=inotify&page=faq&lang=en
Monitoring whole directory tree
Can you please help me?
I want to monitor the whole directory tree, using inotify.
I have done monitoring directory which notify me changes in file within that perticular directory, creation of directories into it, butI am unable to get the changes done within its subdirectories.
Please help me for this.
How would one get the user associated with an event.....
If I wanted to write a utility that would list each time a file I was interested in was modified and by who, how would I do that? Could I do that with inotify? I'm guessing no.
INOTIFY a very helpful tool
I tested INOTIFY in conjunction with POSIX queues (mq_open…) on my CentOS 4.3 – 2.6.18 and it seems to work OK, I am really impressed by this ‘new’ feature:
Just a small ‘observation’ as the code piece from below is going to drastically decrease the performances (stack is exhausted as ‘event’ is an in loop declaration) :-)) :
while (i < len) {
struct inotify_event *event;
…………………
Thanks to INOTIFY creators and contributors,
John
you're flat out wrong about the stack exhausted <EOM>
Code Feedback from gcc and a.out
Thanks for posting this information and the examples. Was very stuck trying to convert to inotify without this article, and now have a working program thanks to your help. Feedback on this article accumulated from the process:
Thanks! inotify ROCKS!
wrong /proc path
As of 2.6.17, it is NOT
/proc/sys/filesystem/inotify/max_user_watches
but
/proc/sys/fs/inotify/max_user_watches
Nice and helpful article
Thanks for the same.
Special files
Hello!
Is inotify supposed to work on special files in /sys ?
I tried monitoring normal files and it works great, but when i try to monitor files on sysfs, strace shows that the executable is blocked on the read() call.
Thanks!
Alberto