Kernel Korner - Intro to inotify
John McCutchan and I had been working on inotify for about a year when it was finally merged into Linus' kernel tree and released with kernel version 2.6.13. Although a long struggle, the effort culminated in success and was ultimately worth every rewrite, bug and debate.
inotify is a file change notification system—a kernel feature that allows applications to request the monitoring of a set of files against a list of events. When the event occurs, the application is notified. To be useful, such a feature must be simple to use, lightweight with little overhead and flexible. It should be easy to add new watches and painless to receive notification of events.
To be sure, inotify is not the first of its kind. Every modern operating system provides some sort of file notification system; many network and desktop applications require such functionality—Linux too. For years, Linux has offered dnotify. The problem was, dnotify was not very good. In fact, it stank.
dnotify, which ostensibly stands for directory notify, was never considered easy to use. Sporting a cumbersome interface and several painful features that made life arduous, dnotify failed to meet the demands of the modern desktop, where asynchronous notification of events and a free flow of information rapidly are becoming the norm. dnotify has, in particular, several problems:
dnotify can watch only directories.
dnotify requires maintaining an open file descriptor to the directory that the user wants to watch. First, this open file descriptor pins the directory, disallowing the device on which it resides from being unmounted. Second, watching a large number of directories requires too many open file descriptors.
dnotify's interface to user space is signals. Yes, seriously, signals!
dnotify ignores the issue of hard links.
The goal, therefore, was twofold: design a first-class file notification system and ensure that all of the deficiencies of dnotify were addressed.
inotify is an inode-based file notification system that does not require a file ever be opened in order to watch it. inotify does not pin filesystem mounts—in fact, it has a clever event that notifies the user whenever a file's backing filesystem is unmounted. inotify is able to watch any filesystem object whatsoever, and when watching directories, it is able to tell the user the name of the file inside of the directory that changed. dnotify can report only that something changed, requiring applications to maintain an in-memory cache of stat() results and compare for any changes.
Finally, inotify is designed with an interface that user-space application developers would want to use, enjoy using and benefit from using. Instead of signals, inotify communicates with applications via a single file descriptor. This file descriptor is select-, poll-, epoll- and read-able. Simple and fast—the world is happy.
inotify is available in kernel 2.6.13-rc3 and later. Because some bugs were found and subsequently fixed right after that release, kernel 2.6.13 or later is recommended. The inotify system calls, being the new kids on the block, might not yet be supported in your system's version of the C library, in which case the header files listed in the on-line Resources will provide the necessary C declarations and system call stubs.
If your C library supports inotify, all you should need is the following:
If not, grab the two header files, stick them in the same directory as your source files, and use the following:
#include "inotify.h" #include "inotify-syscalls.h"
The following examples are in straight C. You can compile them the same as any other C application.
inotify is initialized via the inotify_init() system call, which instantiates an inotify instance inside the kernel and returns the associated file descriptor:
int inotify_init (void);
On failure, inotify_init() returns minus one and sets errno as appropriate. The most common errno values are EMFILE and ENFILE, which signify that the per-user and the system-wide open file limit was reached, respectively.
Usage is simple:
int fd; fd = inotify_init (); if (fd < 0) perror ("inotify_init");
|Non-Linux FOSS: libnotify, OS X Style||Jun 18, 2013|
|Containers—Not Virtual Machines—Are the Future Cloud||Jun 17, 2013|
|Lock-Free Multi-Producer Multi-Consumer Queue on Ring Buffer||Jun 12, 2013|
|Weechat, Irssi's Little Brother||Jun 11, 2013|
|One Tail Just Isn't Enough||Jun 07, 2013|
|Introduction to MapReduce with Hadoop on Linux||Jun 05, 2013|
- Containers—Not Virtual Machines—Are the Future Cloud
- Non-Linux FOSS: libnotify, OS X Style
- Linux Systems Administrator
- Validate an E-Mail Address with PHP, the Right Way
- Lock-Free Multi-Producer Multi-Consumer Queue on Ring Buffer
- Senior Perl Developer
- Technical Support Rep
- UX Designer
- Introduction to MapReduce with Hadoop on Linux
- RSS Feeds
Free Webinar: Hadoop
How to Build an Optimal Hadoop Cluster to Store and Maintain Unlimited Amounts of Data Using Microservers
Realizing the promise of Apache® Hadoop® requires the effective deployment of compute, memory, storage and networking to achieve optimal results. With its flexibility and multitude of options, it is easy to over or under provision the server infrastructure, resulting in poor performance and high TCO. Join us for an in depth, technical discussion with industry experts from leading Hadoop and server companies who will provide insights into the key considerations for designing and deploying an optimal Hadoop cluster.
Some of key questions to be discussed are:
- What is the “typical” Hadoop cluster and what should be installed on the different machine types?
- Why should you consider the typical workload patterns when making your hardware decisions?
- Are all microservers created equal for Hadoop deployments?
- How do I plan for expansion if I require more compute, memory, storage or networking?