Re: [Nolug] Fastest way to create a local mirror of a huge tree?

From: Jerry Wilborn <jerrywilborn_at_gmail.com>
Date: Tue, 6 Sep 2011 22:28:59 -0500
Message-ID: <CAK2QZfQb5PNVLhKOzdsHwdrS3K-KF1wTg8Dp=A2mPwOVX=fd9A@mail.gmail.com>

If /data is a separate partition you could use dd to clone it to your
destination.

http://www.linuxquestions.org/linux/answers/Applications_GUI_Multimedia/How_To_Do_Eveything_With_DD

Jerry Wilborn
jerrywilborn@gmail.com

On Tue, Sep 6, 2011 at 10:21 PM, Ron Johnson <ron.l.johnson@cox.net> wrote:

> The mirror doesn't exist yet, so rsync's clever data-minimizing algorithms
> aren't valid. (Also, there are lots of symlinks that need be preserved.)
>
> 1. cp -av /data /mnt/backups/data
> 2. cd /data && tar -cvf – . | (cd /mnt/backups/data && tar -xpvf -)
> 3. rsync -avz --stats --progress /data /mnt/backups/data
>
> In a long test, I got 32MB/s from the cp when copying files in the 200MB to
> 4GB range, but 4MB/s for files in the 100KB range.
>
> In a short test only 6MB/s from tar; maybe it was scanning through the list
> of files?.
>
> rsync bounces around from 25MB/s to 4MB/s.
>
> So, what's your experience in the best way to create an initial mirror?
>
> --
> Supporting World Peace Through Nuclear Pacification
> ___________________
> Nolug mailing list
> nolug@nolug.org
>

___________________
Nolug mailing list
nolug@nolug.org
Received on 09/06/11

This archive was generated by hypermail 2.2.0 : 09/07/11 EDT