Virtualize a Server with Minimal Downtime

When it's time to convert a physical machine to a virtual one, use these steps to make the move safely and with a small maintenance window.
Tweak Boot Settings

Even though the files on the virtual machine are identical to the physical machine, the virtual machine will not boot correctly at this point until you make some changes to the boot settings. This works best from within a chroot environment, so type:

$ sudo chroot /mnt/sda1

before you run the rest of the commands. Be sure to replace /mnt/sda1 with the mountpoint for your root partition if it is different.

GRUB or LILO Changes

The first change you need to make within the chroot environment is to restore your bootloader. If you use GRUB, look at /boot/grub/menu.lst or /boot/grub/grub.conf. If you use LILO, look at /etc/lilo.conf. Check for any devices that may have changed. In particular, if you switched from an IDE to a SCSI device, from a RAID to a non-RAID or changed the root partition order, be sure to make changes here to reflect that. Next, if you use GRUB, type:

# grub-install /dev/sda

Change /dev/sda to match the primary disk device from which you will boot. If you use LILO, type:

# /sbin/lilo

After your bootloader has been installed, check /etc/fstab and confirm that any drive, partition or device changes you made in your bootloader config file also were changed here.

Re-create the initrd

Many servers these days use an initrd file to load modules that are essential for the boot process but that don't necessarily fit in the kernel image. Often, this initrd file contains only the modules that suit your hardware, so when you make the switch to new hardware, such as is the case with VMware's virtual SCSI controllers, you need to create a fresh initrd that has these new modules in it.

On a Red Hat system, edit either /etc/modules.conf or /etc/modprobe.conf for RHEL 4, and remove any references to scsi_hostadapter you find there. If you configured your virtual machine to use VMware's virtual BusLogic SCSI controller, replace those references with the following:

alias scsi_hostadapter BusLogic

If you chose VMware's LSI Logic SCSI controller, add the following lines instead:

alias scsi_hostadapter mptbase
alias scsi_hostadapter1 mptscsih

Obviously, these modules are specific to VMware virtualization, so if you want to attempt this with another virtualization technology, you will need to look up which SCSI modules it uses and make sure they are referenced here.

Now, you are ready to create a new initrd. Find the location of the initrd your server last used from your /boot/grub/menu.lst, /boot/grub/grub.conf or /etc/lilo.conf file, and then move it out of the way so you can create a new one safely. Then, run mkinitrd with the path to the initrd file to create and the name of the current kernel. For my example server, I am using the Red Hat 2.4.21-32.0.1.ELsmp kernel, so I would type:

# mv /boot/initrd-2.4.21-32.0.1.ELsmp.img 
 ↪/boot/initrd-2.4.21-32.0.1.ELsmp.img.bak
# mkinitrd /boot/initrd-2.4.21-32-0.1.ELsmp 
 ↪2.4.21-32-0.1.ELsmp

As I said before, this is the method Red Hat uses to create initrd files. Unfortunately, different distributions use different methods. For instance, Debian's mkinitrd stores configuration files under /etc/mkinitrd, and the mkinitrd command uses slightly different options, so you might need to do some extra research to create a new initrd for your server's distribution.

At this point, you can reboot the virtual machine. Confirm that your physical machine no longer has its original IP address, or otherwise, simply power it off to be safe. If your server runs a hardware configuration service like kudzu, it most likely will prompt you at boot time because it has detected changes in the server's hardware. Be sure to select Keep Configuration for any old SCSI or network hardware it mentions, and select Ignore for any new SCSI or network hardware; however, you safely can remove old video, sound, USB and similar hardware if you are prompted.

Once the machine has booted completely, confirm that all system services have started and that you are connected to the network. I have noticed on some Red Hat systems that the network card's MAC address has been hard-coded into the configuration file, and as that has changed on the new virtual hardware, the network won't resume. In this case, simply edit the configuration file for your network card under /etc/sysconfig/network-scripts/ (often ifcfg-eth0), and either remove the reference to the MAC address or change it to reflect the new MAC address. Then, restart the networking service.

Practice this procedure on a few test machines to be sure you have all the steps down for your particular network before attempting it on a live production machine. Nothing is worse than scrambling to fix strange initrd issues on a virtual machine while the physical server is down and your maintenance window is ticking away. You will find that the more often you perform these migrations, the faster you can do them—you even might be able to stagger them and complete a few at the same time.

Kyle Rankin is a Senior Systems Administrator in the San Francisco Bay Area and the author of a number of books, including Knoppix Hacks and Ubuntu Hacks for O'Reilly Media. He is currently the president of the North Bay Linux Users' Group.

______________________

Kyle Rankin is a director of engineering operations in the San Francisco Bay Area, the author of a number of books including DevOps Troubleshooting and The Official Ubuntu Server Book, and is a columnist for Linux Journal.

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Older Linux ext2/3 filesystem

Paul Schilling's picture

I was able to virtualize two older RedHat system using this method though I did run into ext2/3 filesystem issues. One system was an old RedHat 5.2 (circa 1998). I was booting the VM with Knoppix V6.0 live CD and then creating the partitions and ext2 filesystem. After rsync'ing files I found I couldn't boot. This turned out to be ext2 filesystem features. The Knoppix OS created an ext2 with all the latest features but the old RedHat 5.2 predated many of these features so it didn't know how to handle them.

I rebuilt the ext2 FS with options to turn off just about every feature.
mkfs -t ext2 -O none -r0 -O ^dir_index /dev/sdxx

Then I was able to rsync and complete all steps ending with a bootable RedHat 5.2 guest VM

Thanks for the great article.

Ubuntu 8 2.6.24 kernel I had

Pkilpo's picture

Ubuntu 8 2.6.24 kernel

I had to use "mount --bind /dev /mnt/sda1"
then chroot /mnt/sda1
then mount /proc /proc -t proc
and used mkinitramfs -o /boot/initrd.img

Filesystem has unsupported features

Len Kranendonk's picture

After migrating an old RedHat 9 server to VMWare using this howto, I got an error saying fsck.ext3: Filesystem has unsupported features.

I've posted a description on how to solve this issue on my blog: fsck: Filesystem has unsupported features.

I hope this might be helpful for people encountering the same issue.

nodev

Johannes Wolter's picture

I ran into problems because Knoppix mounted my root partition with the nodev option, which made the grub-install and mkinitrd calls fail, because the devices were not accessible. I had to change the mount-options with mount -oremount,dev,setuid /dev/sda1.
Furthermore I had a problem because /var is on a different partition (/dev/sda3)in my setup. I had to unmount this partition before chroot-ing and had to (re-)mount in the chroot-environment.

grub install issue

gc's picture

I'm having some issue with grub-install, mainly the fact that the system I'm virtualizing has a raid device for the hard drive. I've tried the grub install using some soft links to map the device names and the setup (hd0) command appears to complete but the virt-manager states the disk is not bootable

any suggestions

I had similar issues because

spingary's picture

I had similar issues because my set up has a /boot partition:

/dev/sda1 -> /boot
/dev/sda2 -> /
/dev/sda3 -> (swap)

so chroot'ing into my root, /dev/sda2, won't exactly work right since /boot is not there. Here's what I did to get it to work, using manual grub instead of grub-install:

Before you chroot into anything (while in Knoppix shell):

$ grub
grub> root (hd0,0)
grub> setup (hd0)
grub> exit

The 1st line points root to my first partition (/dev/sda1)
2nd line sets it up. What this does is set up grub on the MBR of the virtual disk.

Then, to rebuild the initrd correctly, I had to do this:

$ umount /mnt/sda1

$ mount -o remount,dev /mnt/sda2
(assuming sda2 was already mounted)
This mount command helps eliminate /dev/null problems when using the mkinitrd command later.

$ mount /dev/sda1 /mnt/sda2/boot
(This puts the boot partition into my sda2 root mount)

Now we can chroot:
$ chroot /dev/sda2
$ ls /boot
(should see the /boot partition stuff)

I had to add these to /etc/modprobe.conf in my install (I am using VMWare server 2.0.0 RC1):

alias scsi_hostadapter mptbase
alias scsi_hostadapter1 mptscsih

Now I rebuilt initrd:

Now we do the equivalent of this:

# mv /boot/initrd-2.4.21-32.0.1.ELsmp.img /boot/initrd-2.4.21-32.0.1.ELsmp.img.bak
# mkinitrd -v /boot/initrd-2.4.21-32-0.1.ELsmp.img 2.4.21-32-0.1.ELsmp
(Notice that the orig article was missing the .img in the second line)
(Also use the -v flag to see any useful errors)

I had one last problem - when I rebooted it complained the my e2fsck was too old and keep dumping me into repair mode. My install is a real old FC2, and when I used Knoppix to create the ext3 file systems, there are "new features" that the older e2fsck didn't support. So I followed these instructions and all is good:

http://www.mail-archive.com/trilug%40trilug.org/msg26479.html

Good luck everyone. Thank you for a great guide!

I had a problem...

Enrique Garcia's picture

Hi,

Before the second and final rsync, I wanted to test if the machines goes alive, but when I do the chroot /mnt/sda, and then
# grub-install /dev/sda
it gives:
/usr/share/grub/i386-redhat/stage1: Not found

And tried:
fdisk -l (without result)

It seems that I can't have access to /var /usr /home which are other partitions

What I tried next, was to cp the files in /mnt/sda3 (/usr), to /mnt/sda1 (/) just the files in the corresponding /usr/share/grub/i386-redhat/

And the output of that command was:
/sbin/grub-install: line 501: uniq: command not found
/dev/sda: Not found or not a block device.

any advice will be greatly appreciated.

Thanks in advance.

Even though

Enrique Garcia's picture

Even though the missing -H option, this is a great HOWTO, include I will be using this to migrate from one server to other and another, obviously I had never figure it out by myself... Thank you very much, And if possible, I would like to have your agreement to translate it, (and include fdisk steps) to Spanish.

Great!!!

rsync and hardlinks

Thomas Mueller's picture

the rsync options "-avx --numeric-ids --progress" IMHO doesn't include "-H" (hardlinks). So if a systems uses many hardlinks there will be many duplicated files.

Webinar
One Click, Universal Protection: Implementing Centralized Security Policies on Linux Systems

As Linux continues to play an ever increasing role in corporate data centers and institutions, ensuring the integrity and protection of these systems must be a priority. With 60% of the world's websites and an increasing share of organization's mission-critical workloads running on Linux, failing to stop malware and other advanced threats on Linux can increasingly impact an organization's reputation and bottom line.

Learn More

Sponsored by Bit9

Webinar
Linux Backup and Recovery Webinar

Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs. However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.” After all, before data can be restored to a system, there must be a system to restore it to.

In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.

Learn More

Sponsored by Storix