Path: csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!goblin1!goblin2!goblin.stu.neva.ru!newsfeed.xs4all.nl!newsfeed6.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
MIME-Version: 1.0
Sender: chris@rebertia.com
In-Reply-To: <4F381DA0.4020505@syslang.net>
References: <4F381DA0.4020505@syslang.net>
Date: Sun, 12 Feb 2012 13:09:51 -0800
Subject: Re: Need help with shutils.copytree
From: Chris Rebert <clp2@rebertia.com>
To: "Steven W. Orr" <steveo@syslang.net>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Cc: python list <python-list@python.org>
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.5736.1329080995.27778.python-list@python.org>
Lines: 82
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: x330-a1.tempe.blueboxinc.net comp.lang.python:20294

On Sun, Feb 12, 2012 at 12:14 PM, Steven W. Orr <steveo@syslang.net> wrote:
> I have a 'master' directory and a collection of 'slave' dirs. I want the
> master to collect all of the stuff in the slave dirs.
>
> The slaves all look like this,
>
> .
> |-- slaveX
> | =C2=A0 `-- archI
> | =C2=A0 | =C2=A0 `-- distJ
> | =C2=A0 | =C2=A0 | =C2=A0 ` -- FILE
>
> Where the different slaveX dirs may contain multiple occurrences of archI
> and distJ, but across all slaveX dirs, there will only be one *unique*
> instance of FILE in archI and distJ.
>
> Here's an example: Given slave[1234], arch1 and arch2, and dist1 and dist=
2,
> I want master to end up looking like this:
>
> .
> |-- master
> | =C2=A0 `-- arch1
> | =C2=A0 | =C2=A0 ` -- dist1
> | =C2=A0 | =C2=A0 | =C2=A0 =C2=A0` -- FILE
> | =C2=A0 `-- arch1
> | =C2=A0 | =C2=A0 ` -- dist2
> | =C2=A0 | =C2=A0 | =C2=A0 =C2=A0` -- FILE
> | =C2=A0 `-- arch2
> | =C2=A0 | =C2=A0 ` -- dist1
> | =C2=A0 | =C2=A0 | =C2=A0 =C2=A0` -- FILE
> | =C2=A0 `-- arch2
> | =C2=A0 | =C2=A0 ` -- dist2
> | =C2=A0 | =C2=A0 | =C2=A0 =C2=A0` -- FILE
>
> etc...

You have multiple directories at the same level in the hierarchy with
identical names (e.g. two "arch1"s), which is invalid. I assume you
meant for them to be combined?

> In bash, I might use cpio passthrough mode and say something like:
>
> master=3D$path_to_master
> for slave in ${slaves}
> do
> =C2=A0 =C2=A0pushd $slave
> =C2=A0 =C2=A0find . -print | cpio -pdum $master
> =C2=A0 =C2=A0popd
> done
>
> but I'm having a hard time trying to get this functionality in python. (I=
'm
> trying to avoid writing a subprocess.)
>
> I tried using shutil.copytree with a try / except that does a pass on
> OSError (which is what gets raised when trying to create a dir that alrea=
dy
> exists). No joy there.

Right; the stack has already been unwound by the time your `except`
clause is reached.
You just need to recover by instead copying the children of the
subtree individually yourself when their parent already exists.
For example: master/arch1 already exists? Then copy the
slave/arch1/distN-s individually.

Or alternately, abandon copytree() entirely: you just LYBL and check
if the parent directories already exist; if not, you try to create the
directories yourself; and finally, you copy the individual files.
[Useful funcs: os.listdir(), os.path.exists(), os.mkdir() /
os.mkdirs()]

> I also tried an ignore function that always returns ().

That's an effective no-op which doesn't alter copytree()'s behavior whatsoe=
ver.

Cheers,
Chris
--
http://rebertia.com