Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.os.linux.misc > #8579

Re: rsync: exclude all, then include only specified, wildcard problems..

Cross-posted to 2 groups.

Show all headers | View raw

Dave Gibson wrote, on 2013-06-20:
> [ Followup-To: set to colm ]

[Noted, but I have un-set the followup, as I don't see any harm in 
continuing the thread in all of the small number of groups that it was 
posted to. The original crosspost was to try to maximise replies in 
case some people only regularly read one group or the other, and I 
believe that _limited_ sensible crossposting is not against the Spirit 
of Usenet [1] :-)  Apologies if we differ on this!]

[1] In true usenet fashion, an off-topic debate on this matter will then
ensue.. ;-D 



> In comp.os.linux.misc, David <david@55952163-3189045.bogus.domain.invalid>
>> wrote:
>> I've been struggling with rsync (and its helpfully voluminous, but
>> somewhat confusing, man page) all day.. :-(
>
>> explicitly allow only the folder hierarchies that are supposed to be 
>> synced.
>
> You want to rsync a few directories from a hierarchy while retaining
> the directory structure?

Basically, yes. :-)
If we incorporate further files into our repository in the future, then
there may be a few further folders that also are to be added to the list
of what-to-be-synced..



>> SRCROOT="/data/import/svn-test/svn-checkedout/$REPONAME/data"
>> #SRCROOT="/data/import/svn-holding/svn-checkedout/$REPONAME/data"
>> 
>> DSTROOT="/data/import/svn-test/data-test"
>> DSTROOT_LIVE="/data"
>
> [snip]
>
>> RS_INCLUDE="
>> --include='/webs/web1/php_includes/'
>> --include='/webs/web1/php_snippets/'
>> --include='/webs/web2/php/'
>> "
>
> [snip: building up a command line in a string]
>
> You have
>
>  "$SRCROOT"/webs/web1/php_includes/
>  "$SRCROOT"/webs/web1/php_snippets/
>  "$SRCROOT"/webs/web1/irrelevant_things
>  "$SRCROOT"/webs/web2/php/
>  "$SRCROOT"/webs/web2/something_to_be_ignored
>  "$SRCROOT"/webs/other/stuff
>
> You want
>
>  "$DSTROOT"/webs/web1/php_includes/
>  "$DSTROOT"/webs/web1/php_snippets/
>  "$DSTROOT"/webs/web2/php/


Sort of :-)

There should (for now), in $SRCROOT, only exist the three folders listed 
immediately above, and I want to be sure to sync *only* those into $DSTROOT.

There may (probably unlikely, but possibly), inadvertently, be other 
files/folders that have found their way into $SRCROOT, and if so, these must
somehow expressly *not* be synced.

However, in the future, there may/will also be (eg)
"$SRCROOT"/webs/web2/something_to_be_ignored/something_to_be_synced/

and so something_to_be_synced/ should be synced across, without "damaging"
anything else further "up" in "$DSTROOT"/webs/web2/something_to_be_ignored/


> Right?  Use -R (note use of '/./' to strip preceding path elements).
>
> rsync -av -R                            \
>   "$SRCROOT/./webs/web1/php_includes/"  \
>   "$SRCROOT/./webs/web1/php_snippets/"  \
>   "$SRCROOT/./webs/web2/php/"           \
>   "$DSTROOT"

I have to say, I didn't really understand the section of the manual about
-R!! My previous uses of rsync have been rather simpler, and I didn't 
need to worry about it, one way or another.. :-(


> Referring back to the attempt to have embedded quotes within a string,
> when the shell expands RS_INCLUDE it will treat the embedded single
> quotes as ordinary characters.  To have the behaviour you were aiming
> for 

..I'm not quite sure what I was aiming for! ;-/
I was getting thoroughly confused as to where a wildcard character would 
be being interpreted by rsync, and where by the shell. Not knowing how 
the respective command interpreters really do their stuff, I am probably 
guilty of trying to do something too clever for my own good.. :-(


> (are there spaces in there?) you'd have to use eval.  Don't use
> eval.  Use arrays.

There shouldn't be any spaces in the file paths that would be being 
specified explicitly in the lists of what to sync/not-sync (and 
hopefully not in any of the actual files in the repository, assuming 
sensible naming conventions by all of the handful of developers), but 
you are right that we should consider the possibility..


>
>   RS_INCLUDE=(
>     --include='/webs/we 1/php includes/'
>     --include='/webs/we 1/php snippets/'
>     --include='/webs/web2/php/'
>   )
>   printf '%s\n' "${RS_INCLUDE[@]}"
>
> Contrast the output from that with the following
>
>   RS_INCLUDE="
>     --include='/webs/we 1/php includes/'
>     --include='/webs/we 1/php snippets/'
>     --include='/webs/web2/php/'
>   "
>   printf '%s\n' $RS_INCLUDE


[goes away and tries the exercise..]

Thanks. I wasn't aware of shell arrays. Is this bash-specific or
applicable to sh as well? (I try to avoid bashisms just in case I should
ever need to write something portable (no, the need hasn't actually
arisen yet..))

So, if I understand that correctly, each line inside the ( ..... ) is
a separate entry in the array?



Thanks,


David.

Back to comp.os.linux.misc | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

rsync: exclude all, then include only specified, wildcard problems.. David <david@55952163-3189045.bogus.domain.invalid> - 2013-06-19 15:17 +0000
  Re: rsync: exclude all, then include only specified, wildcard problems.. Rikishi42 <skunkworks@rikishi42.net> - 2013-06-20 00:49 +0200
    Re: rsync: exclude all, then include only specified, wildcard problems.. David <david@55952163-3189045.bogus.domain.invalid> - 2013-06-25 11:50 +0000
      Re: rsync: exclude all, then include only specified, wildcard problems.. Rikishi42 <skunkworks@rikishi42.net> - 2013-06-26 01:59 +0200
  Re: rsync: exclude all, then include only specified, wildcard problems.. Richard Kettlewell <rjk@greenend.org.uk> - 2013-06-20 10:16 +0100
    Re: rsync: exclude all, then include only specified, wildcard problems.. The Natural Philosopher <tnp@invalid.invalid> - 2013-06-20 14:19 +0100
      Re: rsync: exclude all, then include only specified, wildcard problems.. David <david@55952163-3189045.bogus.domain.invalid> - 2013-06-25 11:17 +0000
  Re: rsync: exclude all, then include only specified, wildcard problems.. Chris Davies <chris-usenet@roaima.co.uk> - 2013-06-20 10:57 +0100
    Re: rsync: exclude all, then include only specified, wildcard problems.. David <david@55952163-3189045.bogus.domain.invalid> - 2013-06-25 11:40 +0000
      Re: rsync: exclude all, then include only specified, wildcard problems.. Chris Davies <chris-usenet@roaima.co.uk> - 2013-06-25 21:00 +0100
  Re: rsync: exclude all, then include only specified, wildcard problems.. dave.gma+news002@googlemail.com.invalid (Dave Gibson) - 2013-06-20 18:37 +0100
    Re: rsync: exclude all, then include only specified, wildcard problems.. David <david@55952163-3189045.bogus.domain.invalid> - 2013-06-25 14:00 +0000
      Re: rsync: exclude all, then include only specified, wildcard problems.. Richard Kettlewell <rjk@greenend.org.uk> - 2013-06-25 15:24 +0100
      Re: rsync: exclude all, then include only specified, wildcard problems.. dave.gma+news002@googlemail.com.invalid (Dave Gibson) - 2013-06-25 19:23 +0100
  Re: rsync: exclude all, then include only specified, wildcard problems.. Chick Tower <c.tower@deadspam.com> - 2013-06-20 18:04 +0000
  Re: rsync: exclude all, then include only specified, wildcard problems.. dave.gma+news002@googlemail.com.invalid (Dave Gibson) - 2013-06-25 23:37 +0100
    Re: rsync: exclude all, then include only specified, wildcard problems.. dave.gma+news002@googlemail.com.invalid (Dave Gibson) - 2013-06-26 01:54 +0100
    Re: rsync: exclude all, then include only specified, wildcard problems.. David <david@55952163-3189045.bogus.domain.invalid> - 2013-06-28 16:12 +0000
      Re: rsync: exclude all, then include only specified, wildcard problems.. dave.gma+news002@googlemail.com.invalid (Dave Gibson) - 2013-06-29 01:30 +0100
      Re: rsync: exclude all, then include only specified, wildcard problems.. Chris Davies <chris-usenet@roaima.co.uk> - 2013-07-01 09:51 +0100

csiph-web