Re: [Nolug] Fastest way to create a local mirror of a huge tree?

On Wed, Sep 07, 2011 at 06:34:34AM -0500, Ron Johnson wrote:
> If I wanted to make *bad* choices, I'd go back to Windows!

Why is dd a bad choice?

And rsync doesn't "datamine". It uses rolling checksums and other
heuristics (with an incredibly high probability of success) to determine
what bits of a file to transmit, thus making it very efficient for
r"sync"'ing. That you have to initially transfer all files the first
time is a consequence of the case where the target mirror is in no way
similar to the source. Maybe there is a "do initial copy" mode, but I
doubt it.

Maybe you want to ghost your machine, if so check out g4u. You won't
get any speed benefits from it, though.

Bret

>
> On 09/07/2011 06:21 AM, Brad Bendily wrote:
>>Because you can?
>>Isn't that what Linux is all about, choices!
>>
>>
>>
>>On Sep 7, 2011, at 5:54 AM, Ron Johnson<ron.l.johnson@cox.net> wrote:
>>
>>>Why in God's name would I dd-over-ssh on a single machine?
>>>
>>>On 09/07/2011 05:46 AM, B. Estrade wrote:
>>>>I like the dd (over ssh) idea.
>>>>
>>>>B. Estrade<estrabd@gmail.com>
>>>>On Sep 7, 2011 12:23 AM, "Jimmy Hess"<mysidia@gmail.com> wrote:
>>>>>On Tue, Sep 6, 2011 at 10:21 PM, Ron Johnson<ron.l.johnson@cox.net>
>>>>wrote:
>>>>>>The mirror doesn't exist yet, so rsync's clever data-minimizing
>>>>algorithms
>>>>>>aren't valid. (Also, there are lots of symlinks that need be
>>>>>>preserved.)
>>>>>>1. cp -av /data /mnt/backups/data
>>>>>>2. cd /data&& tar -cvf ??? . | (cd /mnt/backups/data&& tar -xpvf -)
>>>>>>3. rsync -avz --stats --progress /data /mnt/backups/data
>>>>>>
>>>>>Netcat + XZ or gzip + CPIO or tar.
>>>>>
>>>>>nc otherhost portnumber | xz -d | cpio -idm
>>>>>find pathname -print | cpio -o -Hnewc | xz -1 | nc -l portnumber
>>>>>
>>>>>Then rsync to reconcile.
>>>>>
>>>>>
>>>>>If on a dedicated partition, consider a block-based tool that won't
>>>>>need to traverse the filesystem
>>>>>directory structure and won't need to copy unused disk blocks, e.g.
>>>>partimage.
>>>>>
>
> --
> Supporting World Peace Through Nuclear Pacification
> ___________________
> Nolug mailing list
> nolug@nolug.org

-- 
B. Estrade <estrabd@gmail.com>
___________________
Nolug mailing list
nolug@nolug.org