Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.os.linux.misc > #8597

Re: rsync: exclude all, then include only specified, wildcard problems..

Cross-posted to 2 groups.

Followups directed to: comp.os.linux.misc

Show all headers | View raw

[ Followup-To set ]

In comp.os.linux.misc, David
> <david@55952163-3189045.bogus.domain.invalid> wrote:

> I am trying to write a shell script to use rsync to sync certain files to 
> another location on the same computer. (The files are checked out from a
> subversion repository into the source location and need to be transferred 
> to their final location, and have ownership and permission changes.)

> My script is included below. Any help in trying to diagnose what's wrong
> with it gratefully received! (And if you want to pick it apart for 
> inelegance too, feel free..)

> #!/bin/sh
> #
> # rsync-webs: rsync the checked-out svn files into the webserver files area
> 
> 
> ############################################################
> # Config
> ############################################################
> 
> # The repository being used
> REPONAME=web-test
> #REPONAME=web
> 
> 
> # Make sure src & dest "match" (ie, both paths end in 'data')
> SRCROOT="/data/import/svn-test/svn-checkedout/$REPONAME/data"
> #SRCROOT="/data/import/svn-holding/svn-checkedout/$REPONAME/data"
> 
> DSTROOT="/data/import/svn-test/data-test"
> DSTROOT_LIVE="/data"
> 
> 
> # DRYRUN is the rsync option for running in dryrun mode 
> # (no files actually transferred)
> DRYRUN="-n"
> MODE='dryrun'

[snip: --include=... --exclude=... ]

> # rsync options:

> RS_OPTS="-vviaz --no-perms --chmod=u=rx,g=rx,o-rwx
> $RS_INCLUDE
> $RS_EXCLUDE
> --cvs-exclude
> "

> if [ "$USER" != "root" ]; then
>    echo 'You need to run this script as root, exiting..'
>    exit
> fi

Use id -u rather than testing $USER.

> # Add group perms to the src files first 
> # (so that they can be svn update'd later by another dev)
> echo 'Adding group permissions to source files..'
> chmod -R g+rwx "$SRCROOT"

The comment gives the impression that the hierarchy having the
correct/desired permissions is a side-effect of this script.

If that's the case, the modes should be set in a separate script.

> 
> # chown the src files first 
> # (so that the dest files will have the correct u+g ownership)
> echo 'chowning source files with required ownership for destination..'
> chown -R root:apache "$SRCROOT"

Use newgrp to change root's primary group to apache, then run rsync
with --no-g.

> 
> 
> # run rsync in dry-run mode (no changes actually made) UNLESS 
> # 'test' or 'commit' option explicitly specified
> 
> # test = copy to test destination
> if [ "$1" == "-test" ]; then
>    unset DRYRUN
>    MODE='test'
> fi
> 
> # commit = copy to real destination
> if [ "$1" == "-commit" ]; then
>    unset DRYRUN
>    DSTROOT="$DSTROOT_LIVE"
>    MODE='commit'
> fi

"==" is (currently) non-standard.

When testing a single parameter against several strings, use case.

> 
> 
> # Set up the rsync command
> # src/ = trailing slash means sync *contents* of src folder 
> # (do not create 'src' folder on dest)
> RS_CMD="rsync $DRYRUN  $RS_OPTS  $SRCROOT/  $DSTROOT"

That approach often creates problems related to word splitting, quoting
and filename generation.

> 
> 
> 
> # Sync the files..
> echo '** Running in mode: ' "$MODE"
> echo '** (specify: -test to sync to test dir, -commit to sync to live dir).'
> 
> echo 'Source:'  $SRCROOT
> echo '  Dest:'  $DSTROOT
> echo 'rsync command:'
> echo $RS_CMD
> 
> echo
> echo '** OK to sync files? [yN]'
> read CONFIRM
> if [ "$CONFIRM" == "y" ]; then
>    echo 'Syncing files..'
>    $RS_CMD
> fi
> 
> 
> echo 'chgrping source files back again..'
> chgrp -R staff "$SRCROOT"
> echo 'End of script.'

You're dealing with the sort of task often run as a cron job.  The
interactive confirmation makes the current script unsuitable for
such use.


I *think* the following script does more or less what you want but
consider it untested.  You have known-good backups, right?



#! /bin/sh -

if [ `id -u` -ne 0 ]; then
  echo "$0: administrator privileges required." 1>&2
  exit 1
fi

REPONAME=web-test
#REPONAME=web

SRCROOT="/data/import/svn-test/svn-checkedout/$REPONAME/data"
#SRCROOT="/data/import/svn-holding/svn-checkedout/$REPONAME/data"

DSTROOT="/data/import/svn-test/data-test"
DSTROOT_LIVE="/data"

# DRYRUN is the rsync option for running in dryrun mode 
DRYRUN="-n"
MODE='dryrun'

synclist="
$SRCROOT/./webs/web1/php_includes
$SRCROOT/./webs/web1/php_snippets
$SRCROOT/./webs/web2/php
"

# destination group
dgrp=apache

debugging=no

# "yes" to prompt for confirmation.
pause=no

opts="
-vviaz
-R
--no-g
--no-p
--chmod=u=rx,g=rx,o-rwx
--cvs-exclude
"

while [ $# -gt 0 ]; do
  case $1 in
    -test )    unset DRYRUN ; MODE=test ;;
    -commit )  unset DRYRUN ; MODE=commit ; DSTROOT=$DSTROOT_LIVE ;;
    -debug )   debugging=yes ;;
    -auto )    pause=no ;;
    --help )   echo '***FIXME*** help not implemented' ; exit 0 ;;
    -- )       shift ; break ;;
    * )        break ;;
  esac
  shift
done

cmd="
rsync
$DRYRUN
$opts
$synclist
$DSTROOT
"

# Restrict word splitting and disable globbing.
# There is one newline between the single quotes -- "IFS='<newline>'"
IFS='
'
set -f

if [ "$debugging" = yes ] || [ "$pause" = yes ]; then
  printf '** Running in mode: %s\n' "$MODE"
  printf 'Source: %s\n  Dest: %s\n' "$SRCROOT" "$DSTROOT"
  printf 'rsync command: '
  printf ' %s' $cmd
  printf '\n'
fi

if [ "$pause" = yes ]; then
  printf '\n** OK to sync files? [yN] '
  read CONFIRM
  case $CONFIRM in
    [yY] | [yY][eE][sS] ) true ;;
    * )                   exit 0 ;;
  esac
fi

[ -n "$dgrp" ] && newgrp "$dgrp"

$cmd

Back to comp.os.linux.misc | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

rsync: exclude all, then include only specified, wildcard problems.. David <david@55952163-3189045.bogus.domain.invalid> - 2013-06-19 15:17 +0000
  Re: rsync: exclude all, then include only specified, wildcard problems.. Rikishi42 <skunkworks@rikishi42.net> - 2013-06-20 00:49 +0200
    Re: rsync: exclude all, then include only specified, wildcard problems.. David <david@55952163-3189045.bogus.domain.invalid> - 2013-06-25 11:50 +0000
      Re: rsync: exclude all, then include only specified, wildcard problems.. Rikishi42 <skunkworks@rikishi42.net> - 2013-06-26 01:59 +0200
  Re: rsync: exclude all, then include only specified, wildcard problems.. Richard Kettlewell <rjk@greenend.org.uk> - 2013-06-20 10:16 +0100
    Re: rsync: exclude all, then include only specified, wildcard problems.. The Natural Philosopher <tnp@invalid.invalid> - 2013-06-20 14:19 +0100
      Re: rsync: exclude all, then include only specified, wildcard problems.. David <david@55952163-3189045.bogus.domain.invalid> - 2013-06-25 11:17 +0000
  Re: rsync: exclude all, then include only specified, wildcard problems.. Chris Davies <chris-usenet@roaima.co.uk> - 2013-06-20 10:57 +0100
    Re: rsync: exclude all, then include only specified, wildcard problems.. David <david@55952163-3189045.bogus.domain.invalid> - 2013-06-25 11:40 +0000
      Re: rsync: exclude all, then include only specified, wildcard problems.. Chris Davies <chris-usenet@roaima.co.uk> - 2013-06-25 21:00 +0100
  Re: rsync: exclude all, then include only specified, wildcard problems.. dave.gma+news002@googlemail.com.invalid (Dave Gibson) - 2013-06-20 18:37 +0100
    Re: rsync: exclude all, then include only specified, wildcard problems.. David <david@55952163-3189045.bogus.domain.invalid> - 2013-06-25 14:00 +0000
      Re: rsync: exclude all, then include only specified, wildcard problems.. Richard Kettlewell <rjk@greenend.org.uk> - 2013-06-25 15:24 +0100
      Re: rsync: exclude all, then include only specified, wildcard problems.. dave.gma+news002@googlemail.com.invalid (Dave Gibson) - 2013-06-25 19:23 +0100
  Re: rsync: exclude all, then include only specified, wildcard problems.. Chick Tower <c.tower@deadspam.com> - 2013-06-20 18:04 +0000
  Re: rsync: exclude all, then include only specified, wildcard problems.. dave.gma+news002@googlemail.com.invalid (Dave Gibson) - 2013-06-25 23:37 +0100
    Re: rsync: exclude all, then include only specified, wildcard problems.. dave.gma+news002@googlemail.com.invalid (Dave Gibson) - 2013-06-26 01:54 +0100
    Re: rsync: exclude all, then include only specified, wildcard problems.. David <david@55952163-3189045.bogus.domain.invalid> - 2013-06-28 16:12 +0000
      Re: rsync: exclude all, then include only specified, wildcard problems.. dave.gma+news002@googlemail.com.invalid (Dave Gibson) - 2013-06-29 01:30 +0100
      Re: rsync: exclude all, then include only specified, wildcard problems.. Chris Davies <chris-usenet@roaima.co.uk> - 2013-07-01 09:51 +0100

csiph-web