Hack and / - Migrate to a New Hard Drive

 in
Storage needs always seem to grow, but when you are ready to upgrade to a new hard drive, how do you transfer all those files? Read on for a tried-and-true method to migrate all your old files to a new drive.

In another article in this issue of Linux Journal [page 84], I talk about my experiences with the new solid state drive (SSD) I installed on my laptop. One of the things I didn't mention in the article was how I transferred all my data and settings to the new drive. There are a number of ways to solve this problem. For instance, you could image the old drive onto the new one and then grow the last partition to fill up the presumably larger disk (which wouldn't work for me, as my new SSD actually was substantially smaller than the old drive). Other people just re-install their OS every time they get a new drive and then transfer their /home directory and other settings, but I've always had just enough custom programs and settings on my laptop for that method to be a pain. You also could use rsync with certain flags to migrate the files, and although I do like that method for network transfers, for local transfers, it can be a hassle, because it first must scan through the entire drive before it begins.

I've done many hard drive migrations during the years with a tried-and-true combination of find piped to cpio. I like this method because it uses common tools that are sure to be installed, it starts immediately and doesn't need to scan the drive, and with the right flags, it correctly can handle (and avoid) special filesystems, such as /proc, /sys and so on. So far, it hasn't failed me, and this migration was no exception. However, this time, I did run into a few gotchas that I will talk about shortly. First, onto the basic steps.

1. Move to a Safe State

You don't want files to be changed as you are copying them, so you don't want to do this migration from your normal desktop environment. Typically, I boot in to a rescue disk like Knoppix, so that the filesystem stays frozen. Other times, I simply switch to single-user mode, so most files won't change. For desktop systems, I generally just connect both drives directly to the system, and for laptops, I use a USB hard drive adapter, so that both can be connected at the same time. For my last migration, I didn't happen to have a USB adapter for a 1.8" drive, so I transferred the data to an intermediate drive first, then installed the new drive and transferred again.

2. Partition Your New Drive and Format the Filesystems

You can use any partitioning tool that works for you—from fdisk to qtparted. This may sound obvious, but make sure that you allocate plenty of room to fit your existing data, and if you move to a larger hard drive, plenty of room to grow. Once you partition the drive, use mkfs or your preferred formatting tool to write a filesystem to each partition (or mkswap for the swap partition).

3. Mount the New Partitions

Create mountpoints under /mnt for the new partitions you have created. For my example, I have a root partition at /dev/sdb1 and a home partition at /dev/sdb3, so I would type as root:

mkdir /mnt/sdb1
mkdir /mnt/sdb3
mount /dev/sdb1 /mnt/sdb1
mount /dev/sdb3 /mnt/sdb3

If you run this from a rescue disk, you also need to make sure your source partitions are mounted as well.

4. Run the find | cpio Spell

Now this spell doesn't have a lot to it, but it's funny how you memorize scripts like this over the years after using them and passing them along to friends. First, change to the root level of the partition you want to copy and then execute the command as root. So, to migrate my root partition from single-user mode, I did the following:

cd /
find ./ -xdev -print0 | cpio -pa0V /mnt/sdb1

To migrate from a rescue disk, the command is almost identical, but you change to the mountpoint of the source partition instead (I mounted it at /dev/sda1):

cd /mnt/sda1
find ./ -xdev -print0 | cpio -pa0V /mnt/sdb1

The find command searches through the entire root partition for files and directories. The -xdev flag tells find to stay within the current mounted filesystem. Otherwise, when find gets to /home, it would copy the contents of that directory as well and potentially fill up the new partition. It then passes the files to cpio, which places them under my new mountpoint while preserving permissions, symlinks and other settings. The cpio command also outputs one dot for each file it copies, so you can have some sense of its progress. However, what I typically do is go to another terminal and monitor the output of df so I can watch it grow:

watch df

Once the first find | cpio command completes, repeat it for each of your other partitions. In my example, if I were in single-user mode, I'd do the following:

cd /home
find ./ -xdev -print0 | cpio -pa0V /mnt/sdb3

If I were using a rescue disk, I'd do this:

cd /mnt/sda3
find ./ -xdev -print0 | cpio -pa0V /mnt/sdb3

______________________

Kyle Rankin is a director of engineering operations in the San Francisco Bay Area, the author of a number of books including DevOps Troubleshooting and The Official Ubuntu Server Book, and is a columnist for Linux Journal.

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

More complete guide

Anonymous's picture

There's a more complete guide at http://bremford.org/tips/MigrateDiskInUbuntu.html which uses the CPIO approach but also includes details on how to configure grub.

Didn't work for me first time

Benny's picture

My "new" drive (external USB) was mounted read-only. This led to a cryptic stream of messages from CPIO complaining that the file being copied was not found on the destination drive.
Remounted as RW and all is good :-)

Kyle's Disk Migration

Theo Schmidt's picture

This is a super method which I've used many times with good success, although the correct GRUB update remains a challenge.

Webinar
One Click, Universal Protection: Implementing Centralized Security Policies on Linux Systems

As Linux continues to play an ever increasing role in corporate data centers and institutions, ensuring the integrity and protection of these systems must be a priority. With 60% of the world's websites and an increasing share of organization's mission-critical workloads running on Linux, failing to stop malware and other advanced threats on Linux can increasingly impact an organization's reputation and bottom line.

Learn More

Sponsored by Bit9

Webinar
Linux Backup and Recovery Webinar

Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs. However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.” After all, before data can be restored to a system, there must be a system to restore it to.

In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.

Learn More

Sponsored by Storix