Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.unix.shell > #25973 > unrolled thread

a sed question

Back to article view | Back to comp.unix.shell

Contents

  a sed question Salvador Mirzo <smirzo@example.com> - 2024-12-18 16:46 -0300
    Re: a sed question John-Paul Stewart <jpstewart@personalprojects.net> - 2024-12-18 15:12 -0500
    Re: a sed question Ralf Damaschke <rwspam@gmx.de> - 2024-12-19 01:14 +0000
      Re: a sed question Salvador Mirzo <smirzo@example.com> - 2024-12-19 09:05 -0300
        Re: a sed question Ralf Damaschke <rwspam@gmx.de> - 2024-12-20 00:55 +0000
          Re: a sed question gazelle@shell.xmission.com (Kenny McCormack) - 2024-12-20 12:44 +0000
            Re: a sed question Ralf Damaschke <rwspam@gmx.de> - 2024-12-21 00:17 +0000
              Re: a sed question Kaz Kylheku <643-408-1753@kylheku.com> - 2024-12-21 03:09 +0000
                sed... (Was: a sed question) gazelle@shell.xmission.com (Kenny McCormack) - 2024-12-21 03:36 +0000
                  Re: sed... (Was: a sed question) Kaz Kylheku <643-408-1753@kylheku.com> - 2024-12-21 03:57 +0000
                    Re: sed... (Was: a sed question) Janis Papanagnou <janis_papanagnou+ng@hotmail.com> - 2024-12-21 15:38 +0100
                      Re: sed... (Was: a sed question) Janis Papanagnou <janis_papanagnou+ng@hotmail.com> - 2024-12-21 17:29 +0100
                        Re: sed... Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2024-12-21 14:23 -0800
                          Re: sed... Janis Papanagnou <janis_papanagnou+ng@hotmail.com> - 2024-12-22 00:33 +0100
                    Re: sed... (Was: a sed question) Lars Poulsen <lars@cleo.beagle-ears.com> - 2024-12-21 21:46 +0000
                      Re: sed... (Was: a sed question) Kaz Kylheku <643-408-1753@kylheku.com> - 2024-12-22 21:22 +0000
                Re: a sed question Janis Papanagnou <janis_papanagnou+ng@hotmail.com> - 2024-12-21 15:35 +0100
                Re: a sed question Ralf Damaschke <rwspam@gmx.de> - 2024-12-22 00:43 +0000
    Re: a sed question Janis Papanagnou <janis_papanagnou+ng@hotmail.com> - 2024-12-20 15:55 +0100
      Checking for right # of args in a shell script (Was: a sed question) gazelle@shell.xmission.com (Kenny McCormack) - 2024-12-20 15:11 +0000
        Re: Checking for right # of args in a shell script (Was: a sed question) Janis Papanagnou <janis_papanagnou+ng@hotmail.com> - 2024-12-20 16:49 +0100
          Re: Checking for right # of args in a shell script (Was: a sed question) gazelle@shell.xmission.com (Kenny McCormack) - 2024-12-20 17:43 +0000
      Re: a sed question Salvador Mirzo <smirzo@example.com> - 2024-12-21 09:17 -0300
        Re: a sed question Janis Papanagnou <janis_papanagnou+ng@hotmail.com> - 2024-12-21 16:19 +0100
          Re: a sed question Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2024-12-21 13:41 -0800
            Re: a sed question Janis Papanagnou <janis_papanagnou+ng@hotmail.com> - 2024-12-22 00:50 +0100
              Re: a sed question Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2024-12-21 16:26 -0800
                Re: a sed question Janis Papanagnou <janis_papanagnou+ng@hotmail.com> - 2024-12-22 01:41 +0100
              Re: a sed question Lawrence D'Oliveiro <ldo@nz.invalid> - 2024-12-22 00:31 +0000
                Re: a sed question Janis Papanagnou <janis_papanagnou+ng@hotmail.com> - 2024-12-22 02:06 +0100
        Re: a sed question Andy Walker <anw@cuboid.co.uk> - 2024-12-21 15:34 +0000
          Re: a sed question Janis Papanagnou <janis_papanagnou+ng@hotmail.com> - 2024-12-21 17:14 +0100
          Re: a sed question Salvador Mirzo <smirzo@example.com> - 2024-12-21 15:21 -0300
            Re: a sed question Janis Papanagnou <janis_papanagnou+ng@hotmail.com> - 2024-12-21 20:48 +0100
        Re: a sed question Helmut Waitzmann <nn.throttle@xoxy.net> - 2024-12-21 19:20 +0100
      Re: a sed question Ordatious <order@order.invalid> - 2024-12-22 09:19 +0000
        Re: a sed question Janis Papanagnou <janis_papanagnou+ng@hotmail.com> - 2024-12-22 19:23 +0100
    Re: a sed question Ed Morton <mortonspam@gmail.com> - 2024-12-21 08:13 -0600
      Re: a sed question Lawrence D'Oliveiro <ldo@nz.invalid> - 2024-12-21 21:09 +0000
        Re: a sed question Janis Papanagnou <janis_papanagnou+ng@hotmail.com> - 2024-12-22 01:02 +0100
          Re: a sed question Lawrence D'Oliveiro <ldo@nz.invalid> - 2024-12-22 00:28 +0000
            Re: a sed question Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2024-12-21 16:36 -0800
              Re: a sed question Lawrence D'Oliveiro <ldo@nz.invalid> - 2024-12-22 01:52 +0000
                Re: a sed question Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2024-12-21 21:09 -0800
                  Re: a sed question Lawrence D'Oliveiro <ldo@nz.invalid> - 2024-12-22 05:56 +0000
                    Re: a sed question Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2024-12-21 22:55 -0800
                      Re: a sed question Eric Pozharski <apple.universe@posteo.net> - 2024-12-23 10:09 +0000
                        Re: a sed question gazelle@shell.xmission.com (Kenny McCormack) - 2024-12-23 12:41 +0000
                          Re: a sed question Kaz Kylheku <643-408-1753@kylheku.com> - 2024-12-23 18:47 +0000
                          Re: a sed question Eric Pozharski <apple.universe@posteo.net> - 2024-12-24 11:20 +0000
            Re: a sed question Janis Papanagnou <janis_papanagnou+ng@hotmail.com> - 2024-12-22 02:22 +0100
        Re: a sed question gazelle@shell.xmission.com (Kenny McCormack) - 2024-12-22 01:09 +0000
        Re: a sed question Kaz Kylheku <643-408-1753@kylheku.com> - 2024-12-22 20:03 +0000
        Re: a sed question Ed Morton <mortonspam@gmail.com> - 2024-12-23 07:26 -0600
          How to solve The Miracle (was Re: a sed question) Janis Papanagnou <janis_papanagnou+ng@hotmail.com> - 2024-12-23 21:09 +0100
          Re: a sed question Lawrence D'Oliveiro <ldo@nz.invalid> - 2024-12-23 22:57 +0000
            Re: a sed question Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2024-12-23 15:06 -0800
              Re: a sed question Lawrence D'Oliveiro <ldo@nz.invalid> - 2024-12-24 00:28 +0000
                Re: a sed question anthk <anthk@openbsd.home> - 2025-03-23 09:45 +0000
                  Re: a sed question Lawrence D'Oliveiro <ldo@nz.invalid> - 2025-03-23 23:41 +0000
            Re: a sed question Ed Morton <mortonspam@gmail.com> - 2024-12-24 06:20 -0600
              Dealing with four-year-olds... (Was: a sed question) gazelle@shell.xmission.com (Kenny McCormack) - 2024-12-24 12:58 +0000
              Re: a sed question Lawrence D'Oliveiro <ldo@nz.invalid> - 2024-12-24 20:57 +0000
                Re: a sed question Kaz Kylheku <643-408-1753@kylheku.com> - 2024-12-24 21:46 +0000
                  Arguing with a four-year-old (Was: a sed question) gazelle@shell.xmission.com (Kenny McCormack) - 2024-12-24 23:53 +0000

Page 2 of 4 — ← Prev page 1 [2] 3 4  Next page →

#25983 — Re: Checking for right # of args in a shell script (Was: a sed question)

On 20.12.2024 16:11, Kenny McCormack wrote:
> In article <vk40gi$3g9sm$1@dont-email.me>,
> Janis Papanagnou  <janis_papanagnou+ng@hotmail.com> wrote:
>> On 18.12.2024 20:46, Salvador Mirzo wrote:
>>> [...]
>>
>> First (before I forget it) change your string comparison '<' to the
>> numerical comparison operator '-lt' as in:   test $# -lt 2 && usage
>> Otherwise, if you get used to using the wrong operator, you may get
>> subtle errors in future if you continue that habit.
> 
> Agreed, in general, but in practice, the need rarely arises.

I certainly disagree on this; if you have 10..19 (or 100..199 etc.)
arguments the '<' test just doesn't trigger but '-lt' does. I mean,
why use a wrong operator. If it will only in specific cases produce
correct results, or if it produced in most cases correct results;
it's just the wrong thing.

> 
> The idiomatic way to do this is just:
> 
>     [ $# = 2 ] || usage()

Yes, but I don't use that but prefer (like you) [[...]], an in, say,

      [[ $# == 2 ]] || usage

(since I use Kornshell's [[...]] I also use its "modern" '==').

> 
> Also, when I need to do more complex arg verification, I use bash's [[ ]]
> mechanism (Yes, I know OP is using /bin/sh, but there is no reason nowadays
> not to use bash). 

If the OP is on Linux the 'sh' might actually be a Bash. If he's,
say, on an AIX the 'sh' may actually be a 'ksh'. - So in any case
there's a good chance to have [[...]] supported. - But it's true
that if he specifies /bin/sh he should not rely on something else
than a POSIX shell (at least).

> Say I want there to be 2 or 3 args (no other # is
> acceptable and the 2nd arg must be numeric.  Like this:
> 
>     [[ $#,$2 =~ ^[23],[0-9]+$ ]] || { echo "Arg error!"; exit; }

An interesting pattern!

Usually I'm spoiled by Kornshell's 'getopts' feature. As I'm using,
say "s:" in an optstring for an option '-s' with a string argument
I'm just using "i#" for an option '-i' with a numeric argument.
(For simple cases with few options I typically don't use 'getopts',
though; I'm lazy.)

Janis

[toc] | [prev] | [next] | [standalone]

#25984 — Re: Checking for right # of args in a shell script (Was: a sed question)

In article <vk43n0$3gtg6$1@dont-email.me>,
Janis Papanagnou  <janis_papanagnou+ng@hotmail.com> wrote:
...
>> Agreed, in general, but in practice, the need rarely arises.
>
>I certainly disagree on this; if you have 10..19 (or 100..199 etc.)
>arguments the '<' test just doesn't trigger but '-lt' does. I mean,
>why use a wrong operator. If it will only in specific cases produce
>correct results, or if it produced in most cases correct results;
>it's just the wrong thing.

We're not talking about the same thing.

>> 
>> The idiomatic way to do this is just:
>> 
>>     [ $# = 2 ] || usage()
>
>Yes, but I don't use that but prefer (like you) [[...]], an in, say,

[ ] is easier in the simple cases.  But, whatever, either way is fine.

>> Also, when I need to do more complex arg verification, I use bash's [[ ]]
>> mechanism (Yes, I know OP is using /bin/sh, but there is no reason nowadays
>> not to use bash). 
>
>If the OP is on Linux the 'sh' might actually be a Bash. If he's,

I assume Linux unless/until I hear otherwise.  And I tend to also assume
some Debian-based Linux (again, unless/until ...).  In Debian-based
Linuxes, sh is "dash", which is pretty much a minimal
subset/POSIX-compliant version of the shell.  So, [[ ]] isn't available
there.

-- 
The randomly chosen signature file that would have appeared here is more than 4
lines long.  As such, it violates one or more Usenet RFCs.  In order to remain
in compliance with said RFCs, the actual sig can be found at the following URL:
	http://user.xmission.com/~gazelle/Sigs/GodDelusion

[toc] | [prev] | [next] | [standalone]

#25991

Janis Papanagnou <janis_papanagnou+ng@hotmail.com> writes:

> On 18.12.2024 20:46, Salvador Mirzo wrote:
>> (*) Summary
>> 
>> I wrote a sed script that makes a line replacement after it finds the
>> right spot.  So far so good.  Then I added quit command after the
>> change, but the quit does not seem to take effect---violating my
>> expectation.  I'll appreciate any help on understanding what's going on.
>
> First (before I forget it) change your string comparison '<' to the
> numerical comparison operator '-lt' as in:   test $# -lt 2 && usage
> Otherwise, if you get used to using the wrong operator, you may get
> subtle errors in future if you continue that habit.

Changed.  Why is it the wrong operator?  It seems it's not the standard
one---checking just now on the POSIX.1 spec.  I think I just tried it
out and given it worked as expected, I didn't think of checking it.  I'm
new to the whole thing.

> Also note that using $* may not work correctly (e.g. depending on
> filenames [containing spaces] used). The safe form is a quoted  "$@"

I learned something here.

--8<-------------------------------------------------------->8---
$ cat script.sh
#!/bin/sh
echo dollar-star $*
./script dollar-star $*

echo quoted-dollar-star "$*"
./script "$*"

echo dollar-at $@
./script $@

echo quoted-dollar-at "$@"
./script "$@"

$ cat script.c
#include <stdio.h>

int main(int argc, char *argv[]) {
  printf("\nfull cmdline: ");
  for (int i = 0; i < argc; ++i) {
    printf("%s ", argv[i]);
  }
  printf("\nargs: %d\n", argc);
  for (int i = 0; i < argc; ++i) {
    printf("arg %d: %s\n", i, argv[i]);
  }
  return 0;
}
--8<-------------------------------------------------------->8---

$ ./script.sh 1 2 "th ree"
dollar-star 1 2 th ree

full cmdline: ./script dollar-star 1 2 th ree
args: 6
arg 0: ./script
arg 1: dollar-star
arg 2: 1
arg 3: 2
arg 4: th
arg 5: ree
quoted-dollar-star 1 2 th ree

full cmdline: ./script 1 2 th ree
args: 2
arg 0: ./script
arg 1: 1 2 th ree
dollar-at 1 2 th ree

full cmdline: ./script 1 2 th ree
args: 5
arg 0: ./script
arg 1: 1
arg 2: 2
arg 3: th
arg 4: ree
quoted-dollar-at 1 2 th ree

full cmdline: ./script 1 2 th ree
args: 4
arg 0: ./script
arg 1: 1
arg 2: 2
arg 3: th ree
$

> (Then I was tempted to make a similar comment as Kenny. But...)

I'm studying and I often go back to the past to see what life was I
like.  I initially tried to solve the problem with /ed/, but did not
find a way to insert a string coming from the a shell script's cmdline.
Then I thought that /sed/ was there to make /ed/ more scriptable.

> WRT your question I'd be interested to understand more about the
> intention of your original question...

The intention is mostly in the paragraph above, but the way I study is
to put things in real-world practice as much as possible.  (When I
realize the solution is indeed too old to make sense, I replace it.
Otherwise I stick with it.)  I have a literate programming file that
contains a chunk that's the version of the program I'm writing.  So when
you ask the program its version, the information is included in the
executable, an idea which I like.  I get the version with a git command
such as

$ git log --oneline | head -1 | awk '{print $1}'
2566d31

So I wanted to include such string in the literate programming file.  At
first I wrote a solution in the programming language I'm using, but I
remember seeing many older software using sed for something like that,
so I decided to study sed a bit.  I read the sed part of Dale Dougherty
and Arnold Robbins's book "sed & awk", second edition, and I thought sed
was quite neat and sensible.  What I'm noticing now is that there are
too many different sed behavior out there to make it sensible to use.
UNIX systems as a whole are like that, so I'm used to reminding myself
of using the common subset of everything.  Perhaps the common subset of
sed is too small.  If I have to use it just for search and replace, then
perhaps it's not really worth it.

> I mean if you don't trust your 'sed' command just pipe it though
> 'less'; there's no need to change the 'sed' program just for that.
>
> Personally I'd try whether it works (by adding "something" before
> and also after the desired place in your sample.txt to be sure the
> other occurrences were not changed), and then just call
>
>   sed -e '/<<Release>>=/,+1s/something/sth else/'  sample.txt
>
> to see it working.

Thanks for the excellent instruction!

[toc] | [prev] | [next] | [standalone]

#25995

On 21.12.2024 13:17, Salvador Mirzo wrote:
> Janis Papanagnou <janis_papanagnou+ng@hotmail.com> writes:
>> On 18.12.2024 20:46, Salvador Mirzo wrote:
>>> (*) Summary
>>>
>>> I wrote a sed script that makes a line replacement after it finds the
>>> right spot.  So far so good.  Then I added quit command after the
>>> change, but the quit does not seem to take effect---violating my
>>> expectation.  I'll appreciate any help on understanding what's going on.
>>
>> First (before I forget it) change your string comparison '<' to the
>> numerical comparison operator '-lt' as in:   test $# -lt 2 && usage
>> Otherwise, if you get used to using the wrong operator, you may get
>> subtle errors in future if you continue that habit.
> 
> Changed.  Why is it the wrong operator?  It seems it's not the standard
> one---checking just now on the POSIX.1 spec.  I think I just tried it
> out and given it worked as expected, I didn't think of checking it.  I'm
> new to the whole thing.

If you're new be very cautious with Shell; it looks easy to handle
but has a lot subtleties!

You have two sets of comparison operators in Unix shell; strangely
the ones we'd expect to compare numerically (=, !=, <, <=, >, >=)
instead do string comparisons (order of characters). And for the
numeric comparison there's other (-eq, -ne, -lt, -le, -gt, -ge)
operators.

The difference can be observed when comparing numbers in strings
like, say, 11 and 2.
[ 11 -lt 2 ]    # returns false (numeric 11 is greater than 2)
[ 11 '<' 2 ]    # returns true (string 11 comes before string 2)
The first is a comparison [of strings] interpreted as a numeric
entity, the second is a comparison [of strings] as a string with
lexicographic ordering.

Note also that in modern shells there's also $((...)) and ((...))
(arithmetic expansion and the [non-standard] arithmetic command)
available which support a syntax that is less surprising; e.g.

$ echo $(( 11 < 2 ))
$ (( 11 < 2 )) && ...

(without quoted operator and with numerical semantics).

> 
>> Also note that using $* may not work correctly (e.g. depending on
>> filenames [containing spaces] used). The safe form is a quoted  "$@"
> 
> I learned something here.

The point with spaces in filenames is also something elementary
to know; that's why the "$@" needs quoting (and quoted variables
is the rule of thumb for variable expansions [for most cases]).

> [snip code]
> 
>> (Then I was tempted to make a similar comment as Kenny. But...)
> 
> I'm studying and I often go back to the past to see what life was I
> like.  I initially tried to solve the problem with /ed/, but did not
> find a way to insert a string coming from the a shell script's cmdline.
> Then I thought that /sed/ was there to make /ed/ more scriptable.

I very very rarely use 'ed' but I recall to have used it feeding
input from stdin to it, so it is in principle possible to also
feed in expanded shell variables as part of the 'ed' input.

> 
>> WRT your question I'd be interested to understand more about the
>> intention of your original question...
> 
> The intention is mostly in the paragraph above, but the way I study is
> to put things in real-world practice as much as possible.  (When I
> realize the solution is indeed too old to make sense, I replace it.
> Otherwise I stick with it.)  I have a literate programming file that
> contains a chunk that's the version of the program I'm writing.  So when
> you ask the program its version, the information is included in the
> executable, an idea which I like.  I get the version with a git command
> such as

As previously mentioned, 'sed' might not be the best choice for
developing such scripts; you might want to consider to learn 'awk'.

> $ git log --oneline | head -1 | awk '{print $1}'
> 2566d31

With Awk you don't need 'head', it can be done like this

$ git log --oneline | awk 'NR==1 {print $1}'

(For long input files you may want an early exit
  ...| awk 'NR==1 { print $1 ; exit(0) }'
but that just as an aside.)

> 
> So I wanted to include such string in the literate programming file.  At
> first I wrote a solution in the programming language I'm using, but I
> remember seeing many older software using sed for something like that,
> so I decided to study sed a bit.  I read the sed part of Dale Dougherty
> and Arnold Robbins's book "sed & awk", second edition, and I thought sed
> was quite neat and sensible.  What I'm noticing now is that there are
> too many different sed behavior out there to make it sensible to use.
> UNIX systems as a whole are like that, so I'm used to reminding myself
> of using the common subset of everything.  Perhaps the common subset of
> sed is too small.  If I have to use it just for search and replace, then
> perhaps it's not really worth it.

Awk has a powerful large common subset and its base is standardized.
(GNU Awk as a prevalent variant has a couple interesting extensions.)
Since your book covers Awk you should have a look into it. (Arnold
has also another book solely on Awk published, the GNU Awk Manual,
which is also available online.)

> 
>> I mean if you don't trust your 'sed' command just pipe it though
>> 'less'; there's no need to change the 'sed' program just for that.
>>
>> Personally I'd try whether it works (by adding "something" before
>> and also after the desired place in your sample.txt to be sure the
>> other occurrences were not changed), and then just call
>>
>>   sed -e '/<<Release>>=/,+1s/something/sth else/'  sample.txt
>>
>> to see it working.
> 
> Thanks for the excellent instruction!

You're welcome.

Janis

[toc] | [prev] | [next] | [standalone]

#26003

Janis Papanagnou <janis_papanagnou+ng@hotmail.com> writes:
> On 21.12.2024 13:17, Salvador Mirzo wrote:
[...]
> As previously mentioned, 'sed' might not be the best choice for
> developing such scripts; you might want to consider to learn 'awk'.
>
>> $ git log --oneline | head -1 | awk '{print $1}'
>> 2566d31
>
> With Awk you don't need 'head', it can be done like this
>
> $ git log --oneline | awk 'NR==1 {print $1}'
>
> (For long input files you may want an early exit
>   ...| awk 'NR==1 { print $1 ; exit(0) }'
> but that just as an aside.)
[...]

This raises another issue: it's often possible to replace a command in a
pipeline that filters output with an option to the command that does the
same thing.  There's no general rule for how to do this, since different
commands do things differently, but for the example above:

    git log --oneline -n 1 | awk '{print $1}'

or even:

    git log -n 1 --format=%h

I haven't memorized the "--format" option, so I don't generally us it in
ad-hoc one-liners, but I do use it in scripts.  Note both of the above
commands avoid generating the entire list of log entries, which could
save significant time on a large repo.

Using unnecessary commands in pipelines is Mostly Harmless, but IMHO
it's good to think about how to do things more efficiently.  See also
"Useless use of cat" (UUOC).

-- 
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
void Void(void) { Void(); } /* The recursive call of the void */

[toc] | [prev] | [next] | [standalone]

#26007

On 21.12.2024 22:41, Keith Thompson wrote:
> Janis Papanagnou <janis_papanagnou+ng@hotmail.com> writes:
>> On 21.12.2024 13:17, Salvador Mirzo wrote:
> [...]
>> As previously mentioned, 'sed' might not be the best choice for
>> developing such scripts; you might want to consider to learn 'awk'.
>>
>>> $ git log --oneline | head -1 | awk '{print $1}'
>>> 2566d31
>>
>> With Awk you don't need 'head', it can be done like this
>>
>> $ git log --oneline | awk 'NR==1 {print $1}'
>>
>> (For long input files you may want an early exit
>>   ...| awk 'NR==1 { print $1 ; exit(0) }'
>> but that just as an aside.)
> [...]
> 
> This raises another issue: it's often possible to replace a command in a
> pipeline that filters output with an option to the command that does the
> same thing.  There's no general rule for how to do this, since different
> commands do things differently, but for the example above:
> 
>     git log --oneline -n 1 | awk '{print $1}'

Yes. - I just used the OP's presented sample to show the principle
(and not make up an own example to illustrate the case).

In practice it goes even farther; with Awk typical pipeline command
sequences that use utilities like cat, head, tail, grep, cut, sed,
tr, wc, seq, tee, etc. can typically all be represented and combined
by Awk. There's also the additional effect that if you want to pass
some context information from a tool near the front of the pipe to
a tool near the other end it's possible to maintain arbitrary state
information within the Awk program.

Of course, if you can _reduce_ the amount of data at an early stage
(like in your 'git -n 1' sample) the earlier the better! (My 'git',
BTW, doesn't seem to support an option '-n'; which might be another
reason to let a standard tool like Awk do the task for which it has
been defined, text-processing.)

> 
> or even:
> 
>     git log -n 1 --format=%h
> 
> I haven't memorized the "--format" option, so I don't generally us it in
> ad-hoc one-liners, but I do use it in scripts.  Note both of the above
> commands avoid generating the entire list of log entries, which could
> save significant time on a large repo.
> 
> Using unnecessary commands in pipelines is Mostly Harmless, but IMHO
> it's good to think about how to do things more efficiently.  See also
> "Useless use of cat" (UUOC).

Janis

[toc] | [prev] | [next] | [standalone]

#26009

Janis Papanagnou <janis_papanagnou+ng@hotmail.com> writes:
[...]
> Of course, if you can _reduce_ the amount of data at an early stage
> (like in your 'git -n 1' sample) the earlier the better! (My 'git',
> BTW, doesn't seem to support an option '-n'; which might be another
> reason to let a standard tool like Awk do the task for which it has
> been defined, text-processing.)

What version of git are you using?

Note that "-n 1" is an argument to "git log", not to "git".
"git -n 1" is an error, as is "git -n 1 log".  "git log -n 1" should work.

The "-n" option can also be given as just the number:

    git log -1

or as a long option:

    git log --max-count=1

-- 
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
void Void(void) { Void(); } /* The recursive call of the void */

[toc] | [prev] | [next] | [standalone]

#26013

On 22.12.2024 01:26, Keith Thompson wrote:
> Janis Papanagnou <janis_papanagnou+ng@hotmail.com> writes:
> [...]
>> Of course, if you can _reduce_ the amount of data at an early stage
>> (like in your 'git -n 1' sample) the earlier the better! (My 'git',
>> BTW, doesn't seem to support an option '-n'; which might be another
>> reason to let a standard tool like Awk do the task for which it has
>> been defined, text-processing.)
> 
> What version of git are you using?
> 
> Note that "-n 1" is an argument to "git log", not to "git".

Stupid me! - Of course. I need some coffee...

Janis

> [...]

[toc] | [prev] | [next] | [standalone]

#26011

On Sun, 22 Dec 2024 00:50:45 +0100, Janis Papanagnou wrote:

> In practice it goes even farther; with Awk typical pipeline command
> sequences that use utilities like cat, head, tail, grep, cut, sed, tr,
> wc, seq, tee, etc. can typically all be represented and combined by Awk.

Another counterexample to the hoary old “do one thing and do it well” 
trope ...

[toc] | [prev] | [next] | [standalone]

#26015

On 22.12.2024 01:31, Lawrence D'Oliveiro wrote:
> On Sun, 22 Dec 2024 00:50:45 +0100, Janis Papanagnou wrote:
> 
>> In practice it goes even farther; with Awk typical pipeline command
>> sequences that use utilities like cat, head, tail, grep, cut, sed, tr,
>> wc, seq, tee, etc. can typically all be represented and combined by Awk.
> 
> Another counterexample to the hoary old “do one thing and do it well” 
> trope ...

I'm not quite sure what you're aiming at with your statement.

My own stance on that is that this "do one thing and do it well”
is okay in principle, but it's also no dogma for me. - It still
holds in many places.

There's some tools, though, that stuff all sorts of functions
into its own feature set that aren't appropriate. (I had a good
example but it evades me at the moment. - Where's my coffee?! -
Anyway, you may have observed your own examples.)

For the given discussion, Awk's "one thing" is text-processing.
OTOH, 'cut', for example, does one [very primitive] thing, but
it _completely_ *fails* to do it well.

The point is that a pipeline is a very restricted way of data
processing and it quickly turns out in non-trivial applications
that you have your data-processing task split into "elementary
operations" that you clumsily have to glue together with shell,
and some things just can't be reasonably done that way.

I still use 'head' or 'tail' (or even 'cat') if I want to get
a quick look onto some data - I certainly don't use Awk for
that. But as soon as many of your "elementary operations" get
combined (which often can be anticipated) using a better tool
might be appropriate (Awk, Perl, Python, whatever one thinks
fits best).

It's good that we have tools like, e.g., 'sort'. And I'm not
very excited about, say, GNU Awk's various sort() functions.
(But they might come in handy in cases.)

In short; it's good to use the appropriate tools, notice if
tools get worse at certain jobs, and change if necessary.

Janis

[toc] | [prev] | [next] | [standalone]

#25996

On 21/12/2024 12:17, Salvador Mirzo wrote:
> I'm studying and I often go back to the past to see what life was I
> like.  I initially tried to solve the problem with /ed/, but did not
> find a way to insert a string coming from the a shell script's cmdline.
> Then I thought that /sed/ was there to make /ed/ more scriptable.

	I think the other contributors are somewhat harsh on Sed.  For
those who started on V6 Unix, there was just Ed, and, as you thought,
Sed was added in V7 as a scripting improvement to Ed.  Awk also came
in with V7.  Some people adopted Awk with enthusiasm, but the early
versions were quite limited/buggy, partly thanks to the limitations of
the PDP-11;  Sed was pretty reliable even in those days.  So at least
some users tried and failed with Awk, but found Sed usable with very
little to learn, thanks to the relationship with Ed.  The arcana of
Sed are much easier to understand if you are/were a regular user of Ed,
whereas Awk requires you to learn a whole new language [I'm not in any
way suggesting that that is unusually difficult].

	So students will normally no longer learn or use Ed/Sed, apart
perhaps from "s/foo/bar/" and similar;  it makes more sense to learn a
visual editor and Awk.  But for older hands, and for those interested
in the history, there is still a use for Sed.  Personally, I know my
way around Sed much better than Awk.  So, again personally, Sed is my
stream editor of choice for tasks somewhat harder than "s ...", but not
so hard that I need to do some serious programming and checking of the
documentation.  [YMMV, and I'm certainly not trying to persuade any Awk
users that they should learn to use Sed instead.]

-- 
Andy Walker, Nottingham.
    Andy's music pages: www.cuboid.me.uk/andy/Music
    Composer of the day: www.cuboid.me.uk/andy/Music/Composers/Paderewski

[toc] | [prev] | [next] | [standalone]

#25997

On 21.12.2024 16:34, Andy Walker wrote:
> On 21/12/2024 12:17, Salvador Mirzo wrote:
>> I'm studying and I often go back to the past to see what life was I
>> like.  I initially tried to solve the problem with /ed/, but did not
>> find a way to insert a string coming from the a shell script's cmdline.
>> Then I thought that /sed/ was there to make /ed/ more scriptable.
> 
>     I think the other contributors are somewhat harsh on Sed.  For
> those who started on V6 Unix, there was just Ed, and, as you thought,
> Sed was added in V7 as a scripting improvement to Ed. 

If all you have is Sed, Sed may look fine.

If you had invested a lot time to learn or master Sed you're of course
less objecting to Sed.

> Awk also came
> in with V7.  Some people adopted Awk with enthusiasm, but the early
> versions were quite limited/buggy, partly thanks to the limitations of
> the PDP-11;  Sed was pretty reliable even in those days. 

You should mention that you are speaking of the 1977 version of AT&T
Awk (released in 1979); that [buggy] version appeared 45 years ago!

The Awk standard (1992) is based on the 1985 version (released 1987);
yet also already 37 years ago, the Awk POSIX standard 32 years ago.

The GNU Awk version was introduced 1986; it is continuously supported
(and is meanwhile available in version 5.x).

> So at least some users tried and failed with Awk,

(There was obviously plenty of time to not need to stick to Sed.)

> but found Sed usable with very
> little to learn, thanks to the relationship with Ed.  The arcana of
> Sed are much easier to understand if you are/were a regular user of Ed,
> whereas Awk requires you to learn a whole new language [I'm not in any
> way suggesting that that is unusually difficult].

Spreading FUD (bugs in the first release, whole new language) is not
appropriate here! - Awk is stable now since decades and the language
is extremely terse and with an excellent power/complexity ratio!)

A few more things should be pointed out in that respect...
At Washington University St. Louis there was an examination made how
long it requires to learn Awk; the result was that in one laboratory
50% of the students considered themselves being confident with Awk,
after a week it were 90% of the students.
Awk's syntax is extremely simple. The imperative actions resemble the
"C" language, so they're easy for the many users knowing any language
that was [syntactically] derived from "C" (but far more simple).
The few syntax elements are also readable - no cryptic abbreviations
as in Sed - so they are easily memorable.

I'm sure if one is constantly working with Sed that at some point he
will also have the [cryptic] Sed language incorporated.

> 
>     So students will normally no longer learn or use Ed/Sed, apart
> perhaps from "s/foo/bar/" and similar;  it makes more sense to learn a
> visual editor and Awk.  But for older hands, and for those interested
> in the history, there is still a use for Sed.  Personally, I know my
> way around Sed much better than Awk.  So, again personally, Sed is my
> stream editor of choice for tasks somewhat harder than "s ...", but not
> so hard that I need to do some serious programming and checking of the
> documentation.  [YMMV, and I'm certainly not trying to persuade any Awk
> users that they should learn to use Sed instead.]

Fair enough.

For newbies, as the OP seems to be, the personal historic experience
with Sed is not that useful, though. Since he now has the choice he
should take the most appropriate tool.

I also don't think that any of the negative responses concerning Sed
had meant to express any disrespect of Sed users.

Janis

[toc] | [prev] | [next] | [standalone]

#25999

Andy Walker <anw@cuboid.co.uk> writes:

> On 21/12/2024 12:17, Salvador Mirzo wrote:
>> I'm studying and I often go back to the past to see what life was I
>> like.  I initially tried to solve the problem with /ed/, but did not
>> find a way to insert a string coming from the a shell script's cmdline.
>> Then I thought that /sed/ was there to make /ed/ more scriptable.
>
> 	I think the other contributors are somewhat harsh on Sed.  For
> those who started on V6 Unix, there was just Ed, and, as you thought,
> Sed was added in V7 as a scripting improvement to Ed.  Awk also came
> in with V7.  Some people adopted Awk with enthusiasm, but the early
> versions were quite limited/buggy, partly thanks to the limitations of
> the PDP-11;  Sed was pretty reliable even in those days.  So at least
> some users tried and failed with Awk, but found Sed usable with very
> little to learn, thanks to the relationship with Ed.  The arcana of
> Sed are much easier to understand if you are/were a regular user of Ed,
> whereas Awk requires you to learn a whole new language [I'm not in any
> way suggesting that that is unusually difficult].

It doesn't seem difficult to learn, but there could be so many different
implementations now that it could be a challenge to write a script and
distribute it around.

> 	So students will normally no longer learn or use Ed/Sed, apart
> perhaps from "s/foo/bar/" and similar;  it makes more sense to learn a
> visual editor and Awk.  But for older hands, and for those interested
> in the history, there is still a use for Sed.

History is my main thing here.  I like to know how life was like.  I
recently read ``Hackers'' by Steven Levy, 1984.  (I was sad it ended.)
I have an open mind about software.  I'm here on the USENET because I
discovered it in non-standard media.  It turns out I find NNTP a much
better medium of discourse than any other.  (I'm writing this from a GNU
EMACS buffer and will be send out using Gnus.  Gnus can be dramatically
uninuitive, but there seems to be no real replacement for it when it
comes to USENET and perhaps mail.)  It is noticeable that newer
generations are not really better at writing software compared to
previous generations.  I wouldn't change make, for example, for newer
ones.  Have you seen Gradle?  What was that for?  There's a website
whose slogan is ``newer is not always better''.

> Personally, I know my way around Sed much better than Awk.  So, again
> personally, Sed is my stream editor of choice for tasks somewhat
> harder than "s ...", but not so hard that I need to do some serious
> programming and checking of the documentation.  [YMMV, and I'm
> certainly not trying to persuade any Awk users that they should learn
> to use Sed instead.]

I actually know awk a little bit.  I've read the ``AWK Programming
Language'' by Aho, Kernighan and Weinberger, 1988.  I loved it.  It
replaced pretty everything I used when shell scripting something.  But
it doesn't feel right to ignore ed and sed.  I need to know at least how
to use them for the essential cases.  These tools incorporate a certain
way of thinking that's very valuable and knowing where they come from is
very valuable.

[toc] | [prev] | [next] | [standalone]

#26001

On 21.12.2024 19:21, Salvador Mirzo wrote:
>
> [...] I'm here on the USENET because I
> discovered it in non-standard media.  It turns out I find NNTP a much
> better medium of discourse than any other.  (I'm writing this from a GNU
> EMACS buffer and will be send out using Gnus.  Gnus can be dramatically
> uninuitive, but there seems to be no real replacement for it when it
> comes to USENET and perhaps mail.)  [...]

There's tools like Thunderbird where you can work with Usenet similarly
to using a typical GUI-based email client (it is actually also an email
client). If you like historic tools I've heard that there's still 'nn'
around, a text-oriented newsreader that I used in the 1990's (loved it).

> 
> I actually know awk a little bit.  I've read the ``AWK Programming
> Language'' by Aho, Kernighan and Weinberger, 1988.  I loved it.  [...]

This is an excellent source! So you might want to look into the GNU Awk
Manual just for a contemporary Awk variant, to see a couple more useful
features supported in an efficient Awk implementation.

Janis

[toc] | [prev] | [next] | [standalone]

#26000

 Salvador Mirzo <smirzo@example.com>:
> Janis Papanagnou <janis_papanagnou+ng@hotmail.com> writes:
>
>> On 18.12.2024 20:46, Salvador Mirzo wrote:
>>
>> First (before I forget it) change your string comparison '<' to the
>> numerical comparison operator '-lt' as in:   test $# -lt 2 && usage
>> Otherwise, if you get used to using the wrong operator, you may get
>> subtle errors in future if you continue that habit.
>
> Changed.  Why is it the wrong operator?


 '<' compares two strings lexically, whereas '-lt' compares two 
 numbers numerically. 


 Some examples:

 1 < 2 ==> true

 2 < 2 ==> false

 3 < 2 ==> false

 10 < 2 ==> true, probably not your intention.

 1 -lt 2 ==> true

 2 -lt 2 ==> false

 3 -lt 2 ==> false

 10 -lt 2 ==> false



> It seems it's not the standard one---checking just now on the
> POSIX.1 spec. 

 The recent POSIX standard knows of the '<' operator, see
 <https://pubs.opengroup.org/onlinepubs/9799919799/utilities/test.html#tag_20_121_05>.

>> Also note that using $* may not work correctly (e.g. depending on
>> filenames [containing spaces] used). The safe form is a quoted  "$@"
>
> I learned something here.
>
> --8<-------------------------------------------------------->8---
> $ cat script.sh
> #!/bin/sh
> echo dollar-star $*
> ./script dollar-star $*
>
> echo quoted-dollar-star "$*"
> ./script "$*"
>
> echo dollar-at $@
> ./script $@
>
> echo quoted-dollar-at "$@"
> ./script "$@"
>
> $ cat script.c
> #include <stdio.h>
>
> int main(int argc, char *argv[]) {
>   printf("\nfull cmdline: ");
>   for (int i = 0; i < argc; ++i) {
>     printf("%s ", argv[i]);
>   }
>   printf("\nargs: %d\n", argc);
>   for (int i = 0; i < argc; ++i) {
>     printf("arg %d: %s\n", i, argv[i]);
>   }
>   return 0;
> }
> --8<-------------------------------------------------------->8---
>
> $ ./script.sh 1 2 "th ree"
> dollar-star 1 2 th ree
>
> full cmdline: ./script dollar-star 1 2 th ree
> args: 6
> arg 0: ./script
> arg 1: dollar-star
> arg 2: 1
> arg 3: 2
> arg 4: th
> arg 5: ree
> quoted-dollar-star 1 2 th ree
>
> full cmdline: ./script 1 2 th ree
> args: 2
> arg 0: ./script
> arg 1: 1 2 th ree
> dollar-at 1 2 th ree
>
> full cmdline: ./script 1 2 th ree
> args: 5
> arg 0: ./script
> arg 1: 1
> arg 2: 2
> arg 3: th
> arg 4: ree
> quoted-dollar-at 1 2 th ree
>
> full cmdline: ./script 1 2 th ree
> args: 4
> arg 0: ./script
> arg 1: 1
> arg 2: 2
> arg 3: th ree
> $

 So using "$@" is the only way to have the shell pass its 
 positional parameters unmodified (i. e. neither glued together 
 nor split at internal white space (more precise: split according 
 to the contents of the IFS shell parameter)) to commands it 
 invokes. 

[toc] | [prev] | [next] | [standalone]

#26022

On Fri, 20 Dec 2024 15:55:12 +0100, Janis Papanagnou wrote:

> you may get subtle errors in future if you continue that habit.

For us newbies, do you have an example?

-- 
Order! Order in the group!

[toc] | [prev] | [next] | [standalone]

#26023

On 22.12.2024 10:19, Ordatious wrote:
> On Fri, 20 Dec 2024 15:55:12 +0100, Janis Papanagnou wrote:
> 
>> you may get subtle errors in future if you continue that habit.
> 
> For us newbies, do you have an example?

(I had to look up what in my post you were referring to here...)

It was about -lt vs '<' in comparisons of strings and numbers
respectively. - And I had already answered that elsethread.

Depending on the actual value in a string variable you may get
_correct looking_ result for many values but since you're not
using the correct operator you "unexpectedly" may see errors
that you "can't explain".

The OP was comparing the number of arguments against a value
of 2 with the string comparison. Try (for example) the code

for i in a b c d e f g h i j k l m n o p q r s t u v w x y z
do
    set $@ $i
    if [ $# '<' 2 ]
    then echo "$# < 2"
    else echo "$# >= 2"
    fi
done

and observe the output

1 < 2
2 >= 2
3 >= 2
4 >= 2
5 >= 2
6 >= 2
7 >= 2
8 >= 2
9 >= 2
10 < 2
11 < 2
12 < 2
13 < 2
14 < 2
15 < 2
16 < 2
17 < 2
18 < 2
19 < 2
20 >= 2
21 >= 2
22 >= 2
23 >= 2
24 >= 2
25 >= 2
26 >= 2

So for "typical" argument numbers <=9 you get the correct result,
but once you have more arguments with this program, or if you
have wrongly used '<' in another program that expects "naturally"
more arguments than the OP's sample program you'll get erroneous
behavior (and might not be aware of that if you're used to using
'<').

It makes thus no sense to use the wrong operator only because it
*sometimes* works correctly even for numeric arguments.

HTH.

Janis

[toc] | [prev] | [next] | [standalone]

#25992

On 12/18/2024 1:46 PM, Salvador Mirzo wrote:
> (*) Summary
> 
> I wrote a sed script that makes a line replacement after it finds the
> right spot.  So far so good.  Then I added quit command after the
> change, but the quit does not seem to take effect---violating my
> expectation.  I'll appreciate any help on understanding what's going on.
> 
> (*) A detailed description
> 
> I wrote this program:
> 
> --8<-------------------------------------------------------->8---
> %cat make-release
> #!/bin/sh
> usage()
> {
>    printf '%s tag file\n' $0
>    exit 1
> }
> test $# '<' 2 && usage
> tag="$1"
> shift
> sed "/<<Release>>=/ {
>   n;
>   c\\
> $tag
> }" $*
> --8<-------------------------------------------------------->8---
> 
> Here's how I use it.  My objective with it is to replace that
> /something/ in the text file with a new argument.
> 
> --8<-------------------------------------------------------->8---
> %cat sample.txt
> Lorem ipsum dolor...
> 
> <<Release>>=
> something
> @
> 
> ... sit a met [...]
> %
> --8<-------------------------------------------------------->8---
<snip>

I'm not going to get into what might be wrong in your sed script 
because, while sed is great for doing s/old/new/ on individual lines, 
for anything else you should just use awk for some combination of 
robustness, clarity, portability, maintainability, and all of the other 
desirable attributes of good software.

For example, if there's always just 1 line of text under `<<Release>>=` 
then using any POSIX awk you could do:

tag="$tag" awk '
     !f { print }
     { f = 0 }
     $0 == "<<Release>>=" {
         print ENVIRON["tag"]
         f = 1
     }
' "${@:--}"

or if it can be multiple lines ending with `@`:

tag="$tag" awk '
     $0 == "@" { f = 0 }
     !f { print }
     $0 == "<<Release>>=" {
         print ENVIRON["tag"]
         f = 1
     }
' "${@:--}"

If you want to temporarily exit after printing the replaced value just 
add `exit` after `f = 1`.

Note that `tag="$tag" awk` have to be on a single line exactly as shown. 
See 
https://stackoverflow.com/questions/19075671/how-do-i-use-shell-variables-in-an-awk-script 
for more info on using shell variables values in an awk script and 
https://stackoverflow.com/questions/29613304/is-it-possible-to-escape-regex-metacharacters-reliably-with-sed 
for more info on using shell variables values in a sed script.

     Ed.

[toc] | [prev] | [next] | [standalone]

#26002

On Sat, 21 Dec 2024 08:13:52 -0600, Ed Morton wrote:

> for anything else you should just use awk ...

If you want to suggest Awk, you might as well use Perl. That does 
everything Awk does, just as concisely, and plenty more besides.

[toc] | [prev] | [next] | [standalone]

#26008

On 21.12.2024 22:09, Lawrence D'Oliveiro wrote:
> On Sat, 21 Dec 2024 08:13:52 -0600, Ed Morton wrote:
> 
>> for anything else you should just use awk ...

(I wouldn't formulate it that way. - Just for the record.)

> If you want to suggest Awk, you might as well use Perl.

(Unfortunately no.)

> That does 
> everything Awk does, just as concisely, and plenty more besides.

(Features are not the whole story.)

The advantage of Awk is that it's standard on Unix systems and can
thus also be used in restricted professional environments (that are
not uncommon as I experienced).

There's yet more advantages, like (compared to Perl) legible syntax
and being a terse language that you can quickly learn (and master).
(I've elsethread mentioned R.Loui's examination of learning curves;
he compared Perl and Awk in that respect, and Perl was, expectedly,
much worse concerning that.)

I know you're a fan of Perl, so I finish my post with the comment
that Perl has other strengths. (Or Python, or <insert your favorite
tool here>, etc.)

Janis

[toc] | [prev] | [next] | [standalone]

Page 2 of 4 — ← Prev page 1 [2] 3 4  Next page →

Back to top | Article view | comp.unix.shell

csiph-web