Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.unix.shell > #14023 > unrolled thread

variable IFS

Back to article view | Back to comp.unix.shell

Contents

  variable IFS JaceK <JaceK@wu_pe.pe_el> - 2016-10-04 14:09 +0200
    Re: variable IFS Kaz Kylheku <221-501-9011@kylheku.com> - 2016-10-04 12:50 +0000

#14023 — variable IFS

When default value of IFS is set and variable x is declared as:

x="a   b  c"

then the following loop gives the result showing that there are 3 tokens 
to be looped over:

$ for i in $x; do echo $i; done
a
b
c

When IFS and variable x are changed to:

IFS="^"
x="a^^^b^^c"

similar loop shows that there were more than 3 tokens:

$ for i in $x; do echo $i; done
a
	# <- "empty line"
	# <- "empty line"
b
	# <- "empty line"
c

Could you please explain why the "empty spaces instead of ^" are 
preserved (ie. they are not compacted as with default IFS and multiple 
spaces inside x variable) and why one is missing, ie. three ^ 
"generated" two empty lines, two ^ "generated" one empty line.

I suppose that if IFS plays any role in both examples then it just plays 
its role differently. I cannot find explanation for this and I would 
appreciate your explanation of this behaviour.

JaceK

[toc] | [next] | [standalone]

#14024

On 2016-10-04, JaceK <JaceK@wu_pe.pe_el> wrote:
> When default value of IFS is set and variable x is declared as:

Invitation to Fuck your Shell: IFS.

:)

> x="a   b  c"
>
> then the following loop gives the result showing that there are 3 tokens 
> to be looped over:
>
> $ for i in $x; do echo $i; done
> a
> b
> c
>
> When IFS and variable x are changed to:
>
> IFS="^"
> x="a^^^b^^c"
>
> similar loop shows that there were more than 3 tokens:
>
> $ for i in $x; do echo $i; done
> a
> 	# <- "empty line"
> 	# <- "empty line"
> b
> 	# <- "empty line"
> c

Also note the approximately parallel behavior in Awk: in Awk, the
default value of FS is single space " ". But that has the meaning
of not separating fields, but actually extracting tokens consisting of
non-blank characters, so "  a      b   c " produces the fields
"a" "b" "c". However, if you set FS to anything else, like "^", you get
field separation: the record "^a^^b^c" turns into the fields "" "a" "b"
"c".

This is required for sane processing of delimited formats which can have
empty fields.

> I suppose that if IFS plays any role in both examples then it just plays 
> its role differently. I cannot find explanation for this and I would 
> appreciate your explanation of this behaviour.

Like Awk's FS, the shell IFS covers requirements for loose tokenizing
of whitespace-delimited words in a "free format", as well as
requirements for strict delimiting.

For the exact rules, which are somewhat curious, see:

http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_05

Basically IFS seems to be overloaded with a number of behaviors for the
sake of avoiding a proliferation of mode flag variables.

If you set it to a mixture of (IFS-) whitespace and non-whitespace characters,
you get a mixed delimiting: leading and trailing spaces are skipped,
and runs of such spaces act as a single separator; whereas the
non-whitespace characters in IFS are treated as strict separators.
That allows for a format like "   a ,  b, , c ,  " to be parsed into
the fields "a" "b" "" "c" "":

  $ s="  a , , b ,,c"; IFS=" ,"; for x in $s ; do echo "<$x>"; done
  <a>
  <>
  <b>
  <>
  <c>

[toc] | [prev] | [standalone]

Back to top | Article view | comp.unix.shell

csiph-web