Re: [Nolug] Fastest way to create a local mirror of a huge tree?

On Tue, Sep 06, 2011 at 10:21:33PM -0500, Ron Johnson wrote:
> The mirror doesn't exist yet, so rsync's clever data-minimizing
> algorithms aren't valid. (Also, there are lots of symlinks that need be
> preserved.)

Not sure what data mining rsync does; and it has options to preserve
symlinks.

>
> 1. cp -av /data /mnt/backups/data
> 2. cd /data && tar -cvf ? . | (cd /mnt/backups/data && tar -xpvf -)
> 3. rsync -avz --stats --progress /data /mnt/backups/data
>
> In a long test, I got 32MB/s from the cp when copying files in the 200MB
> to 4GB range, but 4MB/s for files in the 100KB range.
>
> In a short test only 6MB/s from tar; maybe it was scanning through the
> list of files?.
>
> rsync bounces around from 25MB/s to 4MB/s.
>
> So, what's your experience in the best way to create an initial mirror?

rsync

...are you looking for something like TSM?

Brett

>
> --
> Supporting World Peace Through Nuclear Pacification
> ___________________
> Nolug mailing list
> nolug@nolug.org

-- 
B. Estrade <estrabd@gmail.com>
___________________
Nolug mailing list
nolug@nolug.org