Copying a Filesystem between Computers

 in

If you need to transfer an entire filesystem from one machine to another, for example, when you get a new computer, do the following steps.

1) Boot both PCs with any Linux live CD (for example, Knoppix), and make sure they can access each other via the network.

2) On the source machine, mount the partition containing the filesystem to be copied, and start the transfer using netcat and tar:

cd /mnt/sda1
tar -czpsf - . | pv -b | nc -l 3333

3) On the destination machine, mount the partition to receive the filesystem, and start the process:

cd /mnt/sda1
nc 192.168.10.101 3333 | pv -b | tar -xzpsf -

The nc (netcat) command is used for any kind of TCP connections between two hosts. The pv (progress viewer) command is used to display the progress of the transfer. tar is used to archive the files on the source machine and un-archive them on the destination.

______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

nc may need -p

D. Joe's picture

I found it necessary to invoke netcat on the source side of the connection with the -p switch to specify the port, that is

tar cvf - . | nc -l -p 3333

instead of

tar cvf - . | nc -l 3333

This may be due to version differences--I'm using Ubuntu 8.10's "netcat-traditional" package, the description for which notes that it is different from the more featureful "netcat-openbsd" package. YMMV.

cat /dev/hda1 >

Anonymous's picture

cat /dev/hda1 > /dev/hdb1

That's all you need for a perfect clone of your file system. Of course you can pipe it through netcat/ssh/whatever if you need to do it across the network.

My recent experience of

Peter L's picture

My recent experience of migrating data was when updating PC due to over zealous cleaning of the old box's cpu fan with a vacuum cleaner...
Hence no original PC to connect to. I put the original hard drives in an enclosure with usb connection and then used rsync to copy the data.
Our household computer has several users. An important thing to remember to do when setting up the users on a new system is to specify the same numeric user and group IDs as on the previous system (write them down!). This ensures the permissions preserved by "rsync -a" make sense.
Finally, any suggestions on life cycle management of user data? The way "home" is used for application and user data, and lack of robust migration tools for data, results in an eclectic mess. I copied the old hard drives' data to "/archive" for now....ouch. Manage a home network over a period of a decade or more and the real issues are way deeper than just copying the files!

file system

Mohsen's picture

WoW!
very nice

Copying a Filesystem between Computers, tar + nc vs rsync

Paul Bartholdi's picture

I see many drawbacks to the use of tar+nc+pv:
- hard links are not conserved;
- it is difficult to filter out some files (exclude file) or add extra files (include file);
- connection is not encryted, nor compressed (default with rsync, but not necessary);
- rsync can also be used for updating a [sub]partition, without having to copy everything, just the modified files.
- rsync is used without connecting to both computers, either in sink or feed direction.

The only drawback of rsync is the large number of options. The documentation is well done, rather clear, but most options are never used. A typical use of rsync would not take more space than the introduction to tar+nc+pv above, and would be more usefull, in particular to newcomers. Some thing like:

rsync -alvH source_path [user@]host:target_path
in your case:
rsync -alvH /mnt/sda1 192.168.10.101:/mnt/sda1

On the other hand, I agree that nc and in particular pv can be nice little tools, worth knowing about.

Regards, Paul

rsync

Felipe Alvarez's picture

rsync + ssh = really easy, fast, and saves time.

If you want to actually copy

Theo Parmakis's picture

If you want to actually copy the *filesystem* (as opposed to simply the files_, you could do something like:

first on machine 2:
nc -l -p 7777 | dd of=PC_one.img

and then on machine 1:
dd if=/dev/hda1 | nc pc2 7777

Where machine 1 is the source and machine 2 is the destination.

I used to use tar, it's

Anonymous's picture

I used to use tar, it's good, but it doesn't maintain hard links. Now I've switched to rsync and I am using rsync -avHhx --progress to backup and/or duplicate whole filesystems.

-z option for tar

catclub's picture

I disagree with the -z option.

Given two machines essentially bonded together, isn't the compression a time waster in this situation?

Especially so if you have lots of mostly compressed files - such as jpegs or .tgz files.

Host keys

Anonymous's picture

Having just performed one of these mirrorings (personally, I used 'tar ... | ssh', yet another way of accomplishing it ;) ) I stumbled across one more little "gotcha!" -- if you don't generate new host keys on the mirrored machine, you will have two machines sharing one set of access tokens. That's probably not desireable...

another way

jeffc's picture

Albeit more indepth: I married a geek so now when Jeff gets a new laptop, the missus gets the same one. For the longest time I would get new hardware, drop Linux on it, spend time figuring out all the stuff to install, hardware tweaks, etc, then have to repeat the process on her box. Now, I get the first computer, burn it with whatever distro I want, trick it out so everything is cool, then add her account. Then I boot each machine off of a pair of el-cheapo 256M usb keys, basically do the netcat trick to move it over the gigabit network but I us dd as the in and out on each side. The drives are the same since we always buy two of the same (well, the odds are in our favor). I start the process before we crash for the night and in the morning she reboots into her new system. The one thing you want to remember to do is set the new hostname to something else.

Happy Cloning.

JeffC

pv ?

goll's picture

Hi

Where can I find this 'pv' tool ?

PV:

Andreas Schamanek's picture

PV: http://www.ivarch.com/programs/pv.shtml
It's also available as packages in Debian and Debian based systems such as Ubuntu. HTH.

ntfsclone

Simon Faulkner's picture

Don't forget ntfsclone for all those Windows boxes out there...

alternative way

Anonymous's picture

Why not just use rysnc after booting the target from a livecd (assuming it's a new install), enabling ssh and mounting the target partition?

original:
mkdir /mnt/fake
mount -o bind / /mnt/fake
cd /mnt/fake
rsync -avz --progress ./* user@other.systems.ip.address:/some/mount

/boot /home or /var other other partitions can be done the same way, and attributes are preserved. Bind mounting will keep rsync from trying to copy /proc /dev or /sys from your original / partition, too. Only / has to be bind mounted before copying.

Extended attributes

Anonymous's picture

This method won't preserve any extended attributes of the filesystem in question. For that matter, dump can be used, although not supported by all filesystems. For ext2/3, there is dumpe2fs which works just fine.

Webinar
One Click, Universal Protection: Implementing Centralized Security Policies on Linux Systems

As Linux continues to play an ever increasing role in corporate data centers and institutions, ensuring the integrity and protection of these systems must be a priority. With 60% of the world's websites and an increasing share of organization's mission-critical workloads running on Linux, failing to stop malware and other advanced threats on Linux can increasingly impact an organization's reputation and bottom line.

Learn More

Sponsored by Bit9

Webinar
Linux Backup and Recovery Webinar

Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs. However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.” After all, before data can be restored to a system, there must be a system to restore it to.

In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.

Learn More

Sponsored by Storix