Re: [Nolug] Fastest way to create a local mirror of a huge tree?

From: Ron Johnson <ron.l.johnson_at_cox.net>
Date: Wed, 07 Sep 2011 08:55:45 -0500
Message-ID: <4E6777E1.3090009@cox.net>

dd a live fs from one lvm2 pseudo-device to another lvm2 pseudo-device?

Do they both have to be dismounted?

Should the target device be unformatted so that the dd formats the
target as it dd's over everything including inodes?

On 09/07/2011 07:50 AM, Jerry Wilborn wrote:
> This is something I use from time-to-time to sync up large amounts of data
> (millions of files consuming hundreds of gigs). If you have large numbers
> of sub directories this script may help you copy them in parallel. It's
> still not as fast as a dd, but you should be able to crank up the
> throughput. The /tmp/maxchildren file can be tuned as its running, but at
> some point you reach diminishing returns.
>
> Jerry Wilborn
> jerrywilborn@gmail.com
>
>
> On Wed, Sep 7, 2011 at 7:43 AM, B. Estrade<estrabd@gmail.com> wrote:
>
>> On Wed, Sep 07, 2011 at 06:34:34AM -0500, Ron Johnson wrote:
>>> If I wanted to make *bad* choices, I'd go back to Windows!
>>
>> Why is dd a bad choice?
>>
>> And rsync doesn't "datamine". It uses rolling checksums and other
>> heuristics (with an incredibly high probability of success) to determine
>> what bits of a file to transmit, thus making it very efficient for
>> r"sync"'ing. That you have to initially transfer all files the first
>> time is a consequence of the case where the target mirror is in no way
>> similar to the source. Maybe there is a "do initial copy" mode, but I
>> doubt it.
>>
>> Maybe you want to ghost your machine, if so check out g4u. You won't
>> get any speed benefits from it, though.
>>

-- 
Supporting World Peace Through Nuclear Pacification
___________________
Nolug mailing list
nolug@nolug.org
Received on 09/07/11

This archive was generated by hypermail 2.2.0 : 09/07/11 EDT