Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #223964

Re: Looping Shell Scripts and System Load

Show all headers | View raw

On Wed, Jun 24, 2020 at 12:19:30PM -0500, Martin McCormick wrote:
> #!/bin/sh

Why?  Use bash.

> unarchive ()  {
>  unzip $1

Quotes.  <https://mywiki.wooledge.org/Quotes>

> MEDIADIR=`pwd`

Don't use all caps variable names.

Don't use backticks.  Use $() for command substitution.

Don't use $(pwd) to get the current directory.  It's in the PWD variable
already.

> mountpoint /mags >/dev/null  ||mount /mags
> mountpoint /mags >/dev/null || exit 1
> cd /mags

Check the result of cd.  Exit if it fails.  cd /mags || exit 1

> #rm -r -f *
>      for MEDIAFILE in `ls $MEDIADIR/*`; do

Do not use ls.  <https://mywiki.wooledge.org/ParsingLs>

Quotes again.  <https://mywiki.wooledge.org/Quotes>

What you want is:   for mediafile in "$mediadir"/*; do

> dirname=`basename $MEDIAFILE`
> mkdir $dirname
> cd $dirname

Quotes, quotes, quotes.  <https://mywiki.wooledge.org/Quotes>

Always check the result of a cd.  cd "$dirname" || exit 1

> unarchive $MEDIAFILE &

Quotes!  <https://mywiki.wooledge.org/Quotes>

> 	If there are 3 zipped files, it's probably going to be ok
> and start 3 unzip processes.  This directory had 13 zip files and
> the first 2 or 3 roared to life and then things slowed down as
> they all tried to run.

<https://mywiki.wooledge.org/ProcessManagement> has some examples
for writing "run n jobs at a time".  We found some newer ways as well,
and those haven't all made it to the wiki yet.

One of the better ones is:

13:40 =greybot> Run N processes in parallel (bash 4.3): i=0 n=5; for elem in 
                "${array[@]}"; do if (( i++ >= n )); then wait -n; fi; my_job 
                "$elem" & done; wait

In your script, that would be something like:

#!/bin/bash
# Requires bash 4.3 or higher.

# cd and mount and stuff

i=0 n=3
for f; do
  if ((i++ >= n)); then wait -n; fi
  unarchive "$f" &
done
wait


If you have to target older versions of bash, see the ProcessManagement
page on the wiki for alternatives.

Back to linux.debian.user | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Looping Shell Scripts and System Load "Martin McCormick" <martin.m@suddenlink.net> - 2020-06-24 19:20 +0200
  Re: Looping Shell Scripts and System Load Roberto C. Sánchez <roberto@debian.org> - 2020-06-24 19:30 +0200
    Re: Looping Shell Scripts and System Load Greg Wooledge <wooledg@eeg.ccf.org> - 2020-06-24 19:50 +0200
  Re: Looping Shell Scripts and System Load Greg Wooledge <wooledg@eeg.ccf.org> - 2020-06-24 19:50 +0200
    Re: Looping Shell Scripts and System Load Roger Price <debian@rogerprice.org> - 2020-06-24 20:40 +0200
      Re: Looping Shell Scripts and System Load Greg Wooledge <wooledg@eeg.ccf.org> - 2020-06-24 21:00 +0200
        Re: Looping Shell Scripts and System Load "Martin McCormick" <martin.m@suddenlink.net> - 2020-06-24 22:00 +0200
    Re: Looping Shell Scripts and System Load Tixy <tixy@yxit.co.uk> - 2020-06-25 11:30 +0200
      Re: Looping Shell Scripts and System Load David <bouncingcats@gmail.com> - 2020-06-25 15:40 +0200
  Re: Looping Shell Scripts and System Load David Christensen <dpchrist@holgerdanske.com> - 2020-06-24 21:40 +0200

csiph-web