Re: [Nolug] Fastest way to create a local mirror of a huge tree?

From: Jerry Wilborn <jerrywilborn_at_gmail.com>
Date: Wed, 7 Sep 2011 09:46:11 -0500
Message-ID: <CAK2QZfSmr9rvb2kconipuCX0moPZqvv54PY5Z4rdj6rcV-DqbA@mail.gmail.com>

I don't have any experience dd'ing a live (read-write) partition as a
source, could be an interesting exercise. At a minimum I would mount the
/data partition as ro.

The only thing you should need to do to the destination will be to make sure
it's partitioned with the same number of blocks as the source, dd will take
care of the rest.

Jerry Wilborn
jerrywilborn@gmail.com

On Wed, Sep 7, 2011 at 8:55 AM, Ron Johnson <ron.l.johnson@cox.net> wrote:

> dd a live fs from one lvm2 pseudo-device to another lvm2 pseudo-device?
>
> Do they both have to be dismounted?
>
> Should the target device be unformatted so that the dd formats the target
> as it dd's over everything including inodes?
>
>
>
> On 09/07/2011 07:50 AM, Jerry Wilborn wrote:
>
>> This is something I use from time-to-time to sync up large amounts of data
>> (millions of files consuming hundreds of gigs). If you have large numbers
>> of sub directories this script may help you copy them in parallel. It's
>> still not as fast as a dd, but you should be able to crank up the
>> throughput. The /tmp/maxchildren file can be tuned as its running, but at
>> some point you reach diminishing returns.
>>
>> Jerry Wilborn
>> jerrywilborn@gmail.com
>>
>>
>> On Wed, Sep 7, 2011 at 7:43 AM, B. Estrade<estrabd@gmail.com> wrote:
>>
>> On Wed, Sep 07, 2011 at 06:34:34AM -0500, Ron Johnson wrote:
>>>
>>>> If I wanted to make *bad* choices, I'd go back to Windows!
>>>>
>>>
>>> Why is dd a bad choice?
>>>
>>> And rsync doesn't "datamine". It uses rolling checksums and other
>>> heuristics (with an incredibly high probability of success) to determine
>>> what bits of a file to transmit, thus making it very efficient for
>>> r"sync"'ing. That you have to initially transfer all files the first
>>> time is a consequence of the case where the target mirror is in no way
>>> similar to the source. Maybe there is a "do initial copy" mode, but I
>>> doubt it.
>>>
>>> Maybe you want to ghost your machine, if so check out g4u. You won't
>>> get any speed benefits from it, though.
>>>
>>>
>
> --
> Supporting World Peace Through Nuclear Pacification
> ___________________
> Nolug mailing list
> nolug@nolug.org
>

___________________
Nolug mailing list
nolug@nolug.org
Received on 09/07/11

This archive was generated by hypermail 2.2.0 : 09/07/11 EDT