Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #212650 > unrolled thread

Re: Strange Bash Script Problem

Back to article view | Back to linux.debian.user

Contents

  Re: Strange Bash Script Problem "Stephen P. Molnar" <s.molnar@sbcglobal.net> - 2019-08-11 20:20 +0200
    Re: Strange Bash Script Problem Greg Wooledge <wooledg@eeg.ccf.org> - 2019-08-12 15:40 +0200
      Re: Strange Bash Script Problem "Stephen P. Molnar" <s.molnar@sbcglobal.net> - 2019-08-12 16:30 +0200
      Re: Strange Bash Script Problem Greg Wooledge <wooledg@eeg.ccf.org> - 2019-08-12 19:30 +0200
        Re: Strange Bash Script Problem Lee <ler762@gmail.com> - 2019-08-12 19:40 +0200
          Re: Strange Bash Script Problem Greg Wooledge <wooledg@eeg.ccf.org> - 2019-08-12 19:50 +0200
            Re: Strange Bash Script Problem Lee <ler762@gmail.com> - 2019-08-12 20:00 +0200
              Re: Strange Bash Script Problem Greg Wooledge <wooledg@eeg.ccf.org> - 2019-08-12 20:20 +0200
                Re: Strange Bash Script Problem Lee <ler762@gmail.com> - 2019-08-12 21:10 +0200
                  Re: Strange Bash Script Problem David <bouncingcats@gmail.com> - 2019-08-13 02:00 +0200
      Re: Strange Bash Script Problem Lee <ler762@gmail.com> - 2019-08-12 19:30 +0200

#212650 — Re: Strange Bash Script Problem

On 08/11/2019 11:56 AM, Lee wrote:
> On 8/11/19, David <bouncingcats@gmail.com> wrote:
>> On Mon, 12 Aug 2019 at 01:07, Stephen P. Molnar <s.molnar@sbcglobal.net>
>> wrote:
>>> Thanks for the suggestion.
>>> However,
>>> comp@AbNormal:~/Apps/Models/1-PhosphorusLigands/Acetylcholinesterases/3K9B/Results$ 
>>>
>>> ./Run.ligand.list.sh
>>> ./Run.ligand.list.sh: line 4: cd: $'Acetylcholine\r_apo-3k9b': No such
>>> file or directory
>> The form of that error message indicates that
>> the directory name attempted as a result of
>> this substitution
>> cd "${d}_apo-3k9b"
>>
>> contains a carriage return (\r) character.
>>
>> Here's a demonstration:
>>
>> [david@kablamm junk]$ cd foo$'\r'foo
>> bash: cd: $'foo\rfoo': No such file or directory
>>
>> The $'\r' in my input command is a carriage return
>> character.
>> Note the identical form of quoting in my error message
>> and your error message.
>> The $'foo\rfoo' in the error message is a carriage
>> return character between two strings 'foo'.
>>
>> If the mysterious carriage return character is not
>> in your ligand.list file, then it must be in
>> your Run.ligand.list.sh file just before the underscore
>> in the line
>> cd "${d}_apo-3k9b"
>>
>> Have a look at your script file using
>> 'cat -v Run.ligand.list.sh'
>> The effect of cat -v is to show control characters.
>>
>> If possible, use an editor that can show control characters.
>> If not available, you can edit that line in your script file in a way
>> that would remove an invisible character in the suspect position.
> Or you could try doing
>    dos2unix ligand.list
>    dos2unix Run.ligand.list.sh
> and see if that gets rid of the embedded \r
>
> Lee
>
>

Thanks to all that shared their expertise.

#!/bin/bash
while IFS= read -r d
do
     cd "${d}_apo-3k9b"
     echo "${d}_apo-3k9b"
     echo "${d}_apo-3k9b.dpf"
     /home/comp/Apps/Autodock/autodock4 -p "${d}_apo-3k9b.dpf" -l 
"${d}_apo-3k9b.dlg"
     cd ..
done <ligand.list

and dos2unix solved the problem!!!!!!

-- 
Stephen P. Molnar, Ph.D.            Life is a fuzzy set
http://www.Molecular-Modeling.net   Multivariate and stochastic
614.312.7528 (c)
Skype:  smolnar1

[toc] | [next] | [standalone]

#212681

On Sun, Aug 11, 2019 at 02:10:06PM -0400, Stephen P. Molnar wrote:
> Thanks to all that shared their expertise.
> 
> #!/bin/bash
> while IFS= read -r d
> do
>     cd "${d}_apo-3k9b"
>     echo "${d}_apo-3k9b"
>     echo "${d}_apo-3k9b.dpf"
>     /home/comp/Apps/Autodock/autodock4 -p "${d}_apo-3k9b.dpf" -l
> "${d}_apo-3k9b.dlg"
>     cd ..
> done <ligand.list
> 
> and dos2unix solved the problem!!!!!!

There's still one more problem you want to fix: the exit status of cd
isn't being checked.  If the cd fails, your script still continues on
to run the two echo commands and the autodock4 command.  I don't know
what autodock4 does, but running it in the wrong directory is probably
not helpful.

Here's how I would solve that (there are other ways too):

#!/bin/bash
while IFS= read -r d
do
    ( cd "${d}_apo-3k9b" || exit
      echo "${d}_apo-3k9b"
      echo "${d}_apo-3k9b.dpf"
      exec /home/comp/Apps/Autodock/autodock4 -p "${d}_apo-3k9b.dpf" -l "${d}_apo-3k9b.dlg"
    )
done <ligand.list


P.S. it would also have been possible to work around the carriage return
issues with IFS, but your dos2unix approach is perfectly valid as well.

[toc] | [prev] | [next] | [standalone]

#212687

On 08/12/2019 09:39 AM, Greg Wooledge wrote:
> On Sun, Aug 11, 2019 at 02:10:06PM -0400, Stephen P. Molnar wrote:
>> Thanks to all that shared their expertise.
>>
>> #!/bin/bash
>> while IFS= read -r d
>> do
>>      cd "${d}_apo-3k9b"
>>      echo "${d}_apo-3k9b"
>>      echo "${d}_apo-3k9b.dpf"
>>      /home/comp/Apps/Autodock/autodock4 -p "${d}_apo-3k9b.dpf" -l
>> "${d}_apo-3k9b.dlg"
>>      cd ..
>> done <ligand.list
>>
>> and dos2unix solved the problem!!!!!!
> There's still one more problem you want to fix: the exit status of cd
> isn't being checked.  If the cd fails, your script still continues on
> to run the two echo commands and the autodock4 command.  I don't know
> what autodock4 does, but running it in the wrong directory is probably
> not helpful.
>
> Here's how I would solve that (there are other ways too):
>
> #!/bin/bash
> while IFS= read -r d
> do
>      ( cd "${d}_apo-3k9b" || exit
>        echo "${d}_apo-3k9b"
>        echo "${d}_apo-3k9b.dpf"
>        exec /home/comp/Apps/Autodock/autodock4 -p "${d}_apo-3k9b.dpf" -l "${d}_apo-3k9b.dlg"
>      )
> done <ligand.list
>
>
> P.S. it would also have been possible to work around the carriage return
> issues with IFS, but your dos2unix approach is perfectly valid as well.
>
>

Many thanks for the update. I checked it and it works perfectly.

-- 
Stephen P. Molnar, Ph.D.            Life is a fuzzy set
http://www.Molecular-Modeling.net   Multivariate and stochastic
614.312.7528 (c)
Skype:  smolnar1

[toc] | [prev] | [next] | [standalone]

#212701

On Mon, Aug 12, 2019 at 01:19:45PM -0400, Lee wrote:
> On 8/12/19, Greg Wooledge <wooledg@eeg.ccf.org> wrote:
> > P.S. it would also have been possible to work around the carriage return
> > issues with IFS, but your dos2unix approach is perfectly valid as well.
> 
> Just out of curiosity - how?

while IFS=$' \t\r\n' read -r d; do ...

[toc] | [prev] | [next] | [standalone]

#212703

On 8/12/19, Greg Wooledge <wooledg@eeg.ccf.org> wrote:
> On Mon, Aug 12, 2019 at 01:19:45PM -0400, Lee wrote:
>> On 8/12/19, Greg Wooledge <wooledg@eeg.ccf.org> wrote:
>> > P.S. it would also have been possible to work around the carriage
>> > return
>> > issues with IFS, but your dos2unix approach is perfectly valid as well.
>>
>> Just out of curiosity - how?
>
> while IFS=$' \t\r\n' read -r d; do ...

Doesn't work for me.  What am I doing wrong?

$ sh test
/tmp/mydir1_apo
_apo: 2: cd: can't cd to mydir2
/tmp/mydir3_apo

$ cat test
while IFS=$' \t\r\n' read -r d; do
  ( cd ${d}_apo || exit
    pwd
  )
done < /tmp/input

$ od -c input
0000000   m   y   d   i   r   1  \n   m   y   d   i   r   2  \r  \n   m
0000020   y   d   i   r   3  \n
0000026

Thanks,
Lee

[toc] | [prev] | [next] | [standalone]

#212704

On Mon, Aug 12, 2019 at 01:37:16PM -0400, Lee wrote:
> On 8/12/19, Greg Wooledge <wooledg@eeg.ccf.org> wrote:
> > On Mon, Aug 12, 2019 at 01:19:45PM -0400, Lee wrote:
> >> On 8/12/19, Greg Wooledge <wooledg@eeg.ccf.org> wrote:
> >> > P.S. it would also have been possible to work around the carriage
> >> > return
> >> > issues with IFS, but your dos2unix approach is perfectly valid as well.
> >>
> >> Just out of curiosity - how?
> >
> > while IFS=$' \t\r\n' read -r d; do ...
> 
> Doesn't work for me.  What am I doing wrong?
> 
> $ sh test

You're running sh.  The $'...' feature I'm using is bash only.

> $ cat test
> while IFS=$' \t\r\n' read -r d; do
>   ( cd ${d}_apo || exit
>     pwd
>   )
> done < /tmp/input

Try "bash test" instead of "sh test".  Also, you want double quotes
around "${d}_apo", or at least around the "${d}" part.


wooledg:~$ printf 'foo\r\nbar\r\n' > foo
wooledg:~$ while IFS=$' \t\r\n' read -r d; do printf %s "$d" | od -tx1 -An; done < foo
 66 6f 6f
 62 61 72

[toc] | [prev] | [next] | [standalone]

#212705

On 8/12/19, Greg Wooledge <wooledg@eeg.ccf.org> wrote:
> On Mon, Aug 12, 2019 at 01:37:16PM -0400, Lee wrote:
>> On 8/12/19, Greg Wooledge <wooledg@eeg.ccf.org> wrote:
>> > On Mon, Aug 12, 2019 at 01:19:45PM -0400, Lee wrote:
>> >> On 8/12/19, Greg Wooledge <wooledg@eeg.ccf.org> wrote:
>> >> > P.S. it would also have been possible to work around the carriage
>> >> > return
>> >> > issues with IFS, but your dos2unix approach is perfectly valid as
>> >> > well.
>> >>
>> >> Just out of curiosity - how?
>> >
>> > while IFS=$' \t\r\n' read -r d; do ...
>>
>> Doesn't work for me.  What am I doing wrong?
>>
>> $ sh test
>
> You're running sh.  The $'...' feature I'm using is bash only.
>
>> $ cat test
>> while IFS=$' \t\r\n' read -r d; do
>>   ( cd ${d}_apo || exit
>>     pwd
>>   )
>> done < /tmp/input
>
> Try "bash test" instead of "sh test".

yup -- that was it
$ bash test
/tmp/mydir1_apo
/tmp/mydir2_apo
/tmp/mydir3_apo

>  Also, you want double quotes
> around "${d}_apo", or at least around the "${d}" part.

What's the difference between ${d} and "${d}"?  Or is that a bashism
also? (all my scripts use /bin/sh so I'm pretty clueless wrt bash)

Thanks
Lee

[toc] | [prev] | [next] | [standalone]

#212706

On Mon, Aug 12, 2019 at 01:56:46PM -0400, Lee wrote:
> What's the difference between ${d} and "${d}"?  Or is that a bashism
> also? (all my scripts use /bin/sh so I'm pretty clueless wrt bash)

This applies to both sh and bash.

An unquoted substitution, like $d or ${d}, undergoes several steps.  The
first step is actually copying the contents of the variable.  After that
comes word splitting (dividing the content into words/fields using IFS),
and then pathname expansion ("globbing").

I have a helper script called "args" which I use to illustrate this stuff.

wooledg:~$ cat bin/args
#!/bin/sh
printf "%d args:" "$#"
printf " <%s>" "$@"
echo

Using that, we can demonstrate:

wooledg:~$ d="a variable"
wooledg:~$ args "$d"
1 args: <a variable>
wooledg:~$ args $d
2 args: <a> <variable>

The curly braces don't matter in this case, because there's nothing after
the $d for it to matter.

wooledg:~$ args ${d}
2 args: <a> <variable>

The curly braces are only needed because of the _stuff after the d.
Without them, d_stuff is treated as a variable name.

wooledg:~$ args "$d_stuff"
1 args: <>
wooledg:~$ args "${d}_stuff"
1 args: <a variable_stuff>

The quotes are still needed.  Without them, we still get word splitting
and pathname expansion.

wooledg:~$ args ${d}_stuff
2 args: <a> <variable_stuff>


For more details, see <https://mywiki.wooledge.org/Quotes>.

[toc] | [prev] | [next] | [standalone]

#212709

On 8/12/19, Greg Wooledge <wooledg@eeg.ccf.org> wrote:
> On Mon, Aug 12, 2019 at 01:56:46PM -0400, Lee wrote:
>> What's the difference between ${d} and "${d}"?  Or is that a bashism
>> also? (all my scripts use /bin/sh so I'm pretty clueless wrt bash)
>
> This applies to both sh and bash.
>
> An unquoted substitution, like $d or ${d}, undergoes several steps.  The
> first step is actually copying the contents of the variable.  After that
> comes word splitting (dividing the content into words/fields using IFS),
> and then pathname expansion ("globbing").
>
> I have a helper script called "args" which I use to illustrate this stuff.
>
> wooledg:~$ cat bin/args
> #!/bin/sh
> printf "%d args:" "$#"
> printf " <%s>" "$@"
> echo
>
> Using that, we can demonstrate:
>
> wooledg:~$ d="a variable"
> wooledg:~$ args "$d"
> 1 args: <a variable>
> wooledg:~$ args $d
> 2 args: <a> <variable>
>
> The curly braces don't matter in this case, because there's nothing after
> the $d for it to matter.
>
> wooledg:~$ args ${d}
> 2 args: <a> <variable>
>
> The curly braces are only needed because of the _stuff after the d.
> Without them, d_stuff is treated as a variable name.
>
> wooledg:~$ args "$d_stuff"
> 1 args: <>
> wooledg:~$ args "${d}_stuff"
> 1 args: <a variable_stuff>
>
> The quotes are still needed.  Without them, we still get word splitting
> and pathname expansion.
>
> wooledg:~$ args ${d}_stuff
> 2 args: <a> <variable_stuff>
>
>
> For more details, see <https://mywiki.wooledge.org/Quotes>.

Wow!  It's going to take me a while to understand the implications of
all that, so I'll just stop here with a
Thank you so much for the detailed explanation!

Lee

[toc] | [prev] | [next] | [standalone]

#212719

On Tue, 13 Aug 2019 at 05:04, Lee <ler762@gmail.com> wrote:
> On 8/12/19, Greg Wooledge <wooledg@eeg.ccf.org> wrote:
> > On Mon, Aug 12, 2019 at 01:56:46PM -0400, Lee wrote:

>> What's the difference between ${d} and "${d}"?  Or is that a bashism
>> also? (all my scripts use /bin/sh so I'm pretty clueless wrt bash)
> >
> > For more details, see <https://mywiki.wooledge.org/Quotes>.
>
> Wow!  It's going to take me a while to understand the implications of
> all that, so I'll just stop here with a
> Thank you so much for the detailed explanation!

Don't stop!! :)

Quoting is the #1 priority thing you must understand for
shell scripting, until then any script you write is
relying on good fortune rather than good practice.

Here is the absolute minimum to know (it's only 3 lines):
https://mywiki.wooledge.org/Quotes#I.27m_Too_Lazy_to_Read.2C_Just_Tell_Me_What_to_Do

The #2 thing to know is where to go to learn
good practice:
https://mywiki.wooledge.org/BashGuide

[toc] | [prev] | [next] | [standalone]

#212702

On 8/12/19, Greg Wooledge <wooledg@eeg.ccf.org> wrote:
> On Sun, Aug 11, 2019 at 02:10:06PM -0400, Stephen P. Molnar wrote:
>> Thanks to all that shared their expertise.
>>
>> #!/bin/bash
>> while IFS= read -r d
>> do
>>     cd "${d}_apo-3k9b"
>>     echo "${d}_apo-3k9b"
>>     echo "${d}_apo-3k9b.dpf"
>>     /home/comp/Apps/Autodock/autodock4 -p "${d}_apo-3k9b.dpf" -l
>> "${d}_apo-3k9b.dlg"
>>     cd ..
>> done <ligand.list
>>
>> and dos2unix solved the problem!!!!!!
>
  <.. snip ..>
> P.S. it would also have been possible to work around the carriage return
> issues with IFS, but your dos2unix approach is perfectly valid as well.

Just out of curiosity - how?

Thanks,
Lee

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.user

csiph-web