The Lustre Filesystem Dropped from the Linux 4.18 Kernel

It's now official: the latest RC1 pull request for the Linux 4.18 will not host the nearly 15-year-old Lustre filesystem.

Greg Kroah-Hartman has been growing weary of the team developing its source code not pushing cleaner and fixed code to the staging tree. The removal was committed on June 5, 2018: with the following notes:

The Lustre filesystem has been in the kernel tree for over 5 years now. While it has been an endless source of enjoyment for new kernel developers learning how to do basic coding style cleanups, as well as a semi-entertaining source of bewilderment from the vfs developers any time they have looked into the codebase to try to figure out how to port their latest api changes to this filesystem, it has not really moved forward into the "this is in shape to get out of staging" despite many half-completed attempts.

And getting code out of staging is the main goal of that portion of the kernel tree. Code should not stagnate, and it feels like having this code in staging is only causing the development cycle of the filesystem to take longer than it should. There is a whole separate out-of-tree copy of this codebase where the developers work on it, and then random changes are thrown over the wall at staging at some later point in time. This dual-tree development model has never worked, and the state of this codebase is proof of that.

So, let's just delete the whole mess. Now the lustre developers can go off and work in their out-of-tree codebase and not have to worry about providing valid changelog entries and breaking their patches up into logical pieces. They can take the time they have spent doing those types of housekeeping chores and get the codebase into a much better shape, and it can be submitted for inclusion into the real part of the kernel tree when ready.

Honestly, I do not blame him. The staging tree is primarily intended for unstable and less than mature code, which ideally should move to the mainline within a short time of further development. It's a temporary (that is, staging) location. It's not that I don't appreciate the Lustre filesystem. In fact, I once wrote about it for Linux Journal in the past.

For those who are less familiar with this filesystem: Lustre (or Linux Cluster) is a distributed filesystem typically deployed in large-scale cluster computing environments. Lustre is designed to be both performant and to scale to tens of thousands of nodes and to petabytes of storage. And as what may have just been alluded to already, a distributed filesystem allows access to files from multiple hosts sharing a computer network.

Through the years, ownership of the filesystem has jumped around from Sun Microsystems to Oracle (via its Sun acquisition), to even Xyratex (later acquired by Seagate). It was between the years 2014–2015, that Seagate donated the project back to the Open Source community. Development has since continued, and releases are still quite frequent. The community surrounding the project continues to remain strong. That's why I am somewhat surprised that their contributions back to the kernel are not where they need to be. I hope the actions of Greg Kroah-Hartman light a fire underneath the filesystems developers. It's a great filesystem! Maybe one day in the near future, we'll finally find it merged into the kernel's mainline tree.

You can learn more about the Lustre filesystem by visiting the project's official website.

Petros Koutoupis, LJ Editor at Large, is currently a senior performance software engineer at Cray for its Lustre High Performance File System division. He is also the creator and maintainer of the RapidDisk Project. Petros has worked in the data storage industry for well over a decade and has helped pioneer the many technologies unleashed in the wild today.

Load Disqus comments