The Kernel Hacker's Guide to Source Code Control

 in
Greg explains how to use patch and diff or BitKeeper for kernel development.
Source Code Control

The process of kernel development using patch and diff generally works quite well. But after a while, most people grow tired of it and look for a different way to work that does not involve so much tedious patching and merging.

A few years ago I discovered BitKeeper (available at www.bitmover.com) and have been using it ever since for kernel development. It originally enabled me to track easily external changes to the kernel tree and allowed me to forward port my kernel changes with almost no effort. Now that Linus Torvalds and Marcelo Tosatti are using BitKeeper for their kernel development, it also allows me to send patches to them easily for inclusion into the main kernel tree.

The use of BitKeeper as a kernel development tool is one that a lot of people find contentious, given BitKeeper's licensing strategy. Read over the license and decide for yourself if you should use it. You also should go through the tutorial on the BitMover web site to familiarize yourself with the tool and some of the different commands.

To do kernel work with BitKeeper, you can base your kernel off Linus' or Marcelo's kernel tree, or you can create your own, with all of the different versions. However, unless you are planning on using BitKeeper to send your patches to Linus or Marcelo, I recommend creating your own kernel tree. That way you are not buried in the vast number of different changesets that all of the different kernel developers are creating, and you can focus on your work.

Two Trees

Again, with BitKeeper you end up creating two different trees (or repositories as I will now call them) to do kernel work: a clean tree and a working tree.

To create a clean BitKeeper repository, start with a released kernel in your working directory:

$ ls
linux-2.4.18.tar.gz

Uncompress this kernel:

$ tar -zxf linux-2.4.18.tar.gz
$ ls
linux  linux-2.4.18.tar.gz
Now create a BitKeeper project called linux-2.4:
$ bk setup linux-2.4
BitKeeper will ask you a few questions and then provide a file to edit where you should describe your project. Fill this out with your favorite editor, and save it.

You will now have a directory called linux-2.4, which is where your project will be held. Now import the original kernel version into the new repository:

$ ls
linux  linux-2.4  linux-2.4.18.tar.gz
$ bk import -tplain linux linux-2.4

This will take some time. After BitKeeper is finished importing all of the files, I recommend tagging this point with the kernel version number. This will allow you to find the different kernel versions more easily in the future:

$ cd linux-2.4
$ bk tag LINUX_2.4.18
Now make a clone of that repository, which is a clean kernel tree, in a different directory so you can make your own changes:
$ bk clone linux-2.4 greg-2.4
All of our kernel work will be done in the greg-2.4 directory.

You can use the -l option to bk clone. That will use a lot less disk space and go faster by creating hard links to the metadata files. If a file is modified, BitKeeper will break the link and create a new one where needed. If you end up creating a lot of different repositories on the same disk, you should use this option.

After we are finished with our work, creating changesets by checking in our changes all during the development process (see the BitKeeper tutorial for more details of this), we would like to create a patch to show our changes. This can be done with a simple command from within the greg-2.4 directory:

$ bk export -tpatch -rLINUX_2.4.18..+ -h \
> ../my_patch

This will create a patch showing all of the changes from the tagged version (LINUX_2.4.18) up to the current changeset and save it in the my_patch file. This patch can then be sent to other people through e-mail, just like any patch created with diff. You will notice that creating this patch was a much shorter process than the previous method of using diff and patch.

Submitting Kernel Patches

New Kernel Versions

When a new kernel version is released, you will want to forward port your changes to the new version. This is where BitKeeper really shines over the previous patch and diff method.

First, go to the original, clean kernel tree and import the new patch:

$ ls
greg-2.4 linux-2.4 patch-2.4.19
$ cd linux-2.4
$ bk import -tpatch -SLINUX_2.4.19 ../patch-2.4.19 .

If BitKeeper thinks any files that the patch file shows as created and deleted might actually be files that were renamed or moved around the tree, it will pop up a GUI tool that you can use to show manually which files were renamed, which files simply were deleted and which ones simply were created. Figure 1 shows an example of this dialog box.

Figure 1. BitKeeper Example Dialog Box

Now go back to your working repository and pull the new changes into it:

$ cd ../greg-2.4
$ bk pull

BitKeeper will then merge all of the changes between kernels 2.4.18 and 2.4.19 into your working repository. If there are any merge conflicts between any changes you have made and changes that have showed up in the new kernel version, it will report this and ask you what you want to do. I suggest using the graphical three-way merge tool to help resolve these conflicts. This tool shows the original file with the changes that you have made and the changes that the patch (or someone else) has made. It then lets you pick which change you want to accept, or you can hand-edit the file, merging both changes together. Figure 2 shows an example of a change that I made to a file that conflicts with a change that happened in the main kernel.

Figure 2. A Merge Conflict

After you are finished resolving any conflicts (and wasn't that much easier than manually looking through .rej files?), you can continue working in your updated kernel. Again, to export a patch with all of the changes you have created, use the following command within the greg-2.4 directory:

$ bk export -tpatch -rLINUX_2.4.19..+ -h \
> ../my_patch

______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Your article helped

Satish Muniyappa's picture

Hey Greg!

We use bitkeeper here. Your article on how to resolve merge conflicts helped a great deal. Just needed to thank you for posting this article.

Great job!

Warm Regards,
Satish

Cool! I can't read the artic

Anonymous's picture

Cool! I can't read the article, but I can leave a comment -- well, I think the article lacks depth, insight, and accuracy. It is very concise though, consisting as it does, simply of a screen saying REGISTER!

Webinar
One Click, Universal Protection: Implementing Centralized Security Policies on Linux Systems

As Linux continues to play an ever increasing role in corporate data centers and institutions, ensuring the integrity and protection of these systems must be a priority. With 60% of the world's websites and an increasing share of organization's mission-critical workloads running on Linux, failing to stop malware and other advanced threats on Linux can increasingly impact an organization's reputation and bottom line.

Learn More

Sponsored by Bit9

Webinar
Linux Backup and Recovery Webinar

Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs. However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.” After all, before data can be restored to a system, there must be a system to restore it to.

In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.

Learn More

Sponsored by Storix