Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.unix.shell > #21464 > unrolled thread

Extended pattern syntax extension

Back to article view | Back to comp.unix.shell

Contents

  Extended pattern syntax extension Janis Papanagnou <janis_papanagnou@hotmail.com> - 2021-05-13 10:34 +0200
    Re: Extended pattern syntax extension Spiros Bousbouras <spibou@gmail.com> - 2021-05-13 13:22 +0000
      Re: Extended pattern syntax extension Janis Papanagnou <janis_papanagnou@hotmail.com> - 2021-05-13 16:59 +0200
        Re: Extended pattern syntax extension Spiros Bousbouras <spibou@gmail.com> - 2021-05-13 18:34 +0000
          Re: Extended pattern syntax extension Janis Papanagnou <janis_papanagnou@hotmail.com> - 2021-05-15 11:54 +0200
    Re: Extended pattern syntax extension Kaz Kylheku <563-365-8930@kylheku.com> - 2021-05-13 16:04 +0000
      Re: Extended pattern syntax extension Janis Papanagnou <janis_papanagnou@hotmail.com> - 2021-05-13 20:18 +0200
    Re: Extended pattern syntax extension Janis Papanagnou <janis_papanagnou@hotmail.com> - 2021-07-04 00:56 +0200
      Re: Extended pattern syntax extension Spiros Bousbouras <spibou@gmail.com> - 2021-07-04 14:32 +0000
        Re: Extended pattern syntax extension Janis Papanagnou <janis_papanagnou@hotmail.com> - 2021-07-04 16:52 +0200
          Re: Extended pattern syntax extension Spiros Bousbouras <spibou@gmail.com> - 2021-07-04 15:07 +0000
            Re: Extended pattern syntax extension Janis Papanagnou <janis_papanagnou@hotmail.com> - 2021-07-04 17:36 +0200
              Re: Extended pattern syntax extension William Unruh <unruh@invalid.ca> - 2021-07-04 17:03 +0000
                Re: Extended pattern syntax extension "David W. Hodgins" <dwhodgins@nomail.afraid.org> - 2021-07-04 15:32 -0400
                  Re: Extended pattern syntax extension "David W. Hodgins" <dwhodgins@nomail.afraid.org> - 2021-07-04 15:40 -0400
                Re: Extended pattern syntax extension joerg@schily.net - 2021-07-05 10:43 +0000
              Re: Extended pattern syntax extension Spiros Bousbouras <spibou@gmail.com> - 2021-07-04 17:36 +0000
            Re: Extended pattern syntax extension William Unruh <unruh@invalid.ca> - 2021-07-04 15:57 +0000
              Re: Extended pattern syntax extension Spiros Bousbouras <spibou@gmail.com> - 2021-07-04 17:17 +0000
                Re: Extended pattern syntax extension William Unruh <unruh@invalid.ca> - 2021-07-04 19:43 +0000
          Re: Extended pattern syntax extension William Unruh <unruh@invalid.ca> - 2021-07-04 15:33 +0000

Page 1 of 2  [1] 2  Next page →

#21464 — Extended pattern syntax extension

Kornshell introduced patterns (called extended patterns in bash) that
allowed, for example, pattern expressions like @(A|B|C), matching any
of the sub-patterns A, B, or C.  I was under the impression that I've
somewhere - maybe in some intermediate or experimental ksh version -
seen an extension of those patterns allowing expressions like @(A&B&C)
with the semantics of matching all sub-patterns (in arbitrary order).

One can think of various work-arounds for that like the equivalents

[[ $1 == *@(A*B*C|A*C*B|B*A*C|B*C*A|C*A*B|C*B*A)* ]]

[[ $1 == *A* && $1 == *B* && $1 == *C* ]]

((c=0))
case $1 in
(*A*) ((c++)) ;&
(*B*) ((c++)) ;&
(*C*) ((c++)) ;&
(*) ((c==3))
esac

none of them really satisfying (if compared to @(A&B&C) ).

Has someone seen such an extension in a shell (or other programming
language's regexp library)?  I'm quite sure I've seen it somewhere
but don't recall where it was.

Thinking of an implementation, though, I'm not sure how to implement
that efficiently based on an FSM. (OR-branches are simple, but ANDs?)

Janis

[toc] | [next] | [standalone]

#21467

[ Crossposting to  comp.programming ]

On Thu, 13 May 2021 10:34:04 +0200
Janis Papanagnou <janis_papanagnou@hotmail.com> wrote:
> Kornshell introduced patterns (called extended patterns in bash) that
> allowed, for example, pattern expressions like @(A|B|C), matching any
> of the sub-patterns A, B, or C.  I was under the impression that I've
> somewhere - maybe in some intermediate or experimental ksh version -
> seen an extension of those patterns allowing expressions like @(A&B&C)
> with the semantics of matching all sub-patterns (in arbitrary order).

Does  @(ab&bc)  match the string   abc ? In other words , are the
matches allowed to overlap ?

> One can think of various work-arounds for that like the equivalents
> 
> [[ $1 == *@(A*B*C|A*C*B|B*A*C|B*C*A|C*A*B|C*B*A)* ]]

For this the matches cannot overlap.

> [[ $1 == *A* && $1 == *B* && $1 == *C* ]]
> 
> ((c=0))
> case $1 in
> (*A*) ((c++)) ;&
> (*B*) ((c++)) ;&
> (*C*) ((c++)) ;&
> (*) ((c==3))
> esac

For both of these alternatives the matches can overlap.

> none of them really satisfying (if compared to @(A&B&C) ).
> 
> Has someone seen such an extension in a shell (or other programming
> language's regexp library)?  I'm quite sure I've seen it somewhere
> but don't recall where it was.

The closest I can think of is the  \&  operator in  vim  regular
expressions for which the  vim  documentation says :

                                                    */branch* */\&*
    2. A branch is one or more concats, separated by "\&".  It matches the last
       concat, but only if all the preceding concats also match at the same
       position.  Examples:
            "foobeep\&..." matches "foo" in "foobeep".
            ".*Peter\&.*Bob" matches in a line containing both "Peter" and "Bob"

I'm surprised it's not more common. Personally I often need such an idiom. On
the shell command line I usually do   grep A | grep B | grep ...  .Obviously
it can be done with AWK too.

> Thinking of an implementation, though, I'm not sure how to implement
> that efficiently based on an FSM. (OR-branches are simple, but ANDs?)

Create finite automata M1 , M2 , M3 corresponding to A , B , C .For each
symbol read , update each automaton. If any automaton reaches an accepting
state , stop updating it. If at some point all 3 automata have reached an
accepting state then the expression has matched ; if you reach the end of the
input and at least 1 automaton has not reached an accepting state , then the
expression has not matched.

-- 
I thought we were the better team. In the first half our keeper didn't
have a save to make apart from the two goals.
  Dean Saunders , football coach.

[toc] | [prev] | [next] | [standalone]

#21472

On 13.05.2021 15:22, Spiros Bousbouras wrote:
> [ Crossposting to  comp.programming ]
> 
> On Thu, 13 May 2021 10:34:04 +0200
> Janis Papanagnou <janis_papanagnou@hotmail.com> wrote:
>> Kornshell introduced patterns (called extended patterns in bash) that
>> allowed, for example, pattern expressions like @(A|B|C), matching any
>> of the sub-patterns A, B, or C.  I was under the impression that I've
>> somewhere - maybe in some intermediate or experimental ksh version -
>> seen an extension of those patterns allowing expressions like @(A&B&C)
>> with the semantics of matching all sub-patterns (in arbitrary order).
> 
> Does  @(ab&bc)  match the string   abc ? In other words , are the
> matches allowed to overlap ?

I would say so (but don't recall the implementation I thought to have
seen).

> 
>> One can think of various work-arounds for that like the equivalents
>>
>> [[ $1 == *@(A*B*C|A*C*B|B*A*C|B*C*A|C*A*B|C*B*A)* ]]
> 
> For this the matches cannot overlap.

You are right.

> 
>> [[ $1 == *A* && $1 == *B* && $1 == *C* ]]
>>
>> ((c=0))
>> case $1 in
>> (*A*) ((c++)) ;&
>> (*B*) ((c++)) ;&
>> (*C*) ((c++)) ;&
>> (*) ((c==3))
>> esac
> 
> For both of these alternatives the matches can overlap.
> 
>> none of them really satisfying (if compared to @(A&B&C) ).
>>
>> Has someone seen such an extension in a shell (or other programming
>> language's regexp library)?  I'm quite sure I've seen it somewhere
>> but don't recall where it was.
> 
> The closest I can think of is the  \&  operator in  vim  regular
> expressions for which the  vim  documentation says :
[...]
> 
> I'm surprised it's not more common. Personally I often need such an idiom. On
> the shell command line I usually do   grep A | grep B | grep ...  .

I needed it as well in a couple cases.

(It could as well have been the case that I haven't seen that feature
in some tool, but just thought it would be an useful and in Kornshell
a lexically coherent extension.)

> Obviously it can be done with AWK too.

Yes, and quite trivially (in Awk I'm using that pattern quite often)

/A/&&/B/&&/C/

> 
>> Thinking of an implementation, though, I'm not sure how to implement
>> that efficiently based on an FSM. (OR-branches are simple, but ANDs?)
> 
> Create finite automata M1 , M2 , M3 corresponding to A , B , C .For each
> symbol read , update each automaton. If any automaton reaches an accepting
> state , stop updating it. If at some point all 3 automata have reached an
> accepting state then the expression has matched ; if you reach the end of the
> input and at least 1 automaton has not reached an accepting state , then the
> expression has not matched.

Well, okay. I was more thinking (initially) about a method that will
stay within the Regular Expression (Chomsky 3) class. As I understand
it this method (like a counting automaton, etc.) would not qualify for
that. On the other hand, Kornshell's patterns are also already beyond
Chomsky 3 since they support backreferences in their patterns, so it's
okay, I guess, to think about non-regular implementations.

Would that be a candidate for ksh93u+m ?

Janis

[toc] | [prev] | [next] | [standalone]

#21476

On Thu, 13 May 2021 16:59:05 +0200
Janis Papanagnou <janis_papanagnou@hotmail.com> wrote:
> On 13.05.2021 15:22, Spiros Bousbouras wrote:
> > [ Crossposting to  comp.programming ]
> > 
> > On Thu, 13 May 2021 10:34:04 +0200
> > Janis Papanagnou <janis_papanagnou@hotmail.com> wrote:

[...]

> >> Thinking of an implementation, though, I'm not sure how to implement
> >> that efficiently based on an FSM. (OR-branches are simple, but ANDs?)
> > 
> > Create finite automata M1 , M2 , M3 corresponding to A , B , C .For each
> > symbol read , update each automaton. If any automaton reaches an accepting
> > state , stop updating it. If at some point all 3 automata have reached an
> > accepting state then the expression has matched ; if you reach the end of the
> > input and at least 1 automaton has not reached an accepting state , then the
> > expression has not matched.
> 
> Well, okay. I was more thinking (initially) about a method that will
> stay within the Regular Expression (Chomsky 3) class. As I understand
> it this method (like a counting automaton, etc.) would not qualify for
> that.

It can be realised with a regular expression like the  \{n,m\}  operator can.

Anyway , I did some digging and I saw a better way to do conjunction (it's
the same idea really but all the details are presented) : you take the
product of 2 deterministic finite automata. Details are on pages 30-32 of
http://www.cse.chalmers.se/~coquand/AUTOMATA/o2.pdf .And right after , the
notes describe a modification of the product which works to get disjunction.

> On the other hand, Kornshell's patterns are also already beyond
> Chomsky 3 since they support backreferences in their patterns, so it's
> okay, I guess, to think about non-regular implementations.

-- 
I was once seriously berated for revealing the ending of
"The Passion of the Christ."
  Roger Ebert

[toc] | [prev] | [next] | [standalone]

#21481

On 13.05.2021 20:34, Spiros Bousbouras wrote:
> On Thu, 13 May 2021 16:59:05 +0200
> Janis Papanagnou <janis_papanagnou@hotmail.com> wrote:
>> On 13.05.2021 15:22, Spiros Bousbouras wrote:
>>> [ Crossposting to  comp.programming ]
>>>
>>> On Thu, 13 May 2021 10:34:04 +0200
>>> Janis Papanagnou <janis_papanagnou@hotmail.com> wrote:
> 
> [...]
> 
>>>> Thinking of an implementation, though, I'm not sure how to implement
>>>> that efficiently based on an FSM. (OR-branches are simple, but ANDs?)
>>>
>>> Create finite automata M1 , M2 , M3 corresponding to A , B , C .For each
>>> symbol read , update each automaton. If any automaton reaches an accepting
>>> state , stop updating it. If at some point all 3 automata have reached an
>>> accepting state then the expression has matched ; if you reach the end of the
>>> input and at least 1 automaton has not reached an accepting state , then the
>>> expression has not matched.
>>
>> Well, okay. I was more thinking (initially) about a method that will
>> stay within the Regular Expression (Chomsky 3) class. As I understand
>> it this method (like a counting automaton, etc.) would not qualify for
>> that.
> 
> It can be realised with a regular expression like the  \{n,m\}  operator can.

Hmm.. - I've considered the range terminology (and a few other syntaxes)
always like an abbreviation, still within FSA/Chomsky-3, if linearized.
(While for large values of 'm' this is practically not a sophisticated
implementation, of course; complexity would be depending on 'm', as the
expanded form of the FSM would be that huge.)

(With "counting automaton" I rather meant syntaxes like, for example,
bracket counting.)

> 
> Anyway , I did some digging and I saw a better way to do conjunction (it's
> the same idea really but all the details are presented) : you take the
> product of 2 deterministic finite automata. Details are on pages 30-32 of
> http://www.cse.chalmers.se/~coquand/AUTOMATA/o2.pdf .And right after , the
> notes describe a modification of the product which works to get disjunction.

I'll have a look into that. Thanks!

Janis

[toc] | [prev] | [next] | [standalone]

#21473

On 2021-05-13, Janis Papanagnou <janis_papanagnou@hotmail.com> wrote:
> Kornshell introduced patterns (called extended patterns in bash) that
> allowed, for example, pattern expressions like @(A|B|C), matching any
> of the sub-patterns A, B, or C.  I was under the impression that I've
> somewhere - maybe in some intermediate or experimental ksh version -
> seen an extension of those patterns allowing expressions like @(A&B&C)
> with the semantics of matching all sub-patterns (in arbitrary order).
>
> One can think of various work-arounds for that like the equivalents
>
> [[ $1 == *@(A*B*C|A*C*B|B*A*C|B*C*A|C*A*B|C*B*A)* ]]
>
> [[ $1 == *A* && $1 == *B* && $1 == *C* ]]
>
> ((c=0))
> case $1 in
> (*A*) ((c++)) ;&
> (*B*) ((c++)) ;&
> (*C*) ((c++)) ;&
> (*) ((c==3))
> esac
>
> none of them really satisfying (if compared to @(A&B&C) ).
>
> Has someone seen such an extension in a shell (or other programming
> language's regexp library)?  I'm quite sure I've seen it somewhere
> but don't recall where it was.
>
> Thinking of an implementation, though, I'm not sure how to implement
> that efficiently based on an FSM. (OR-branches are simple, but ANDs?)

Regex derivatives can be used; the & operator has a simple
interpretation under derivatives.  Regex derivatives were discovered by
Janusz Brzozowski, who published about them in 1964.

Derivatives can be interpreted directly, and that's how they were
originally described.

The basic idea behind this is that given a regular expression R,
we can calculate a "derivative" of R with respect to an input symbol s,
or R' = d(R, s). Maybe dR/ds, haha.

The derivative R' is a regular expression which matches the rest of the
input after s has been consumed.

Matching a string is performed by repeatedly calculating a chain of
derivatives, for each symbol in the string. If the end of the string is
reached, and the remaining regular expression is nullable (capable of
matching the empty string), then the string has been matched.  If at any
time a derivative is produced which doesn't match any strings, then the
match has failed.

Some more recent work describes ways of using derivatives to compile
to DFA.  The gist is to chase the possible derivatives at compile time
and calculate the resulting transition graph statically.
The trick, while doing that, is to recognize the identical states to
condense the graph and recognize loops. Sometimes the derivative of
a regex will yield the same regex, or a regex that was seen before.
Repetition does this, for instance. The derivative of a* with respect
to a is a*.  A notion of "sameness" is required, in consideration of
the issue that sometimes regexes have the same meaning, but look
different.

About the and operator: given R1&R2, the derivative has an easy
interpretation.  d(R1&R2, s) is just d(R1, s)&d(R2, s).  The derivative
operation parallelizes into the branches of the and, and the results are
combined with & again. If either derivative is the empty regex [],
then there is no match since R&[] = [].

-- 
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal

[toc] | [prev] | [next] | [standalone]

#21475

On 13.05.2021 18:04, Kaz Kylheku wrote:
> On 2021-05-13, Janis Papanagnou <janis_papanagnou@hotmail.com> wrote:
>> Kornshell introduced patterns (called extended patterns in bash) that
>> allowed, for example, pattern expressions like @(A|B|C), matching any
>> of the sub-patterns A, B, or C.  I was under the impression that I've
>> somewhere - maybe in some intermediate or experimental ksh version -
>> seen an extension of those patterns allowing expressions like @(A&B&C)
>> with the semantics of matching all sub-patterns (in arbitrary order).
[...]
>> Thinking of an implementation, though, I'm not sure how to implement
>> that efficiently based on an FSM. (OR-branches are simple, but ANDs?)
> 
> Regex derivatives can be used; the & operator has a simple
> interpretation under derivatives.  Regex derivatives were discovered by
> Janusz Brzozowski, who published about them in 1964.
> 
> Derivatives can be interpreted directly, and that's how they were
> originally described.
> 
> The basic idea behind this is that given a regular expression R,
> we can calculate a "derivative" of R with respect to an input symbol s,
> or R' = d(R, s). Maybe dR/ds, haha.
> 
> The derivative R' is a regular expression which matches the rest of the
> input after s has been consumed.
> 
> Matching a string is performed by repeatedly calculating a chain of
> derivatives, for each symbol in the string. If the end of the string is
> reached, and the remaining regular expression is nullable (capable of
> matching the empty string), then the string has been matched.  If at any
> time a derivative is produced which doesn't match any strings, then the
> match has failed.
> 
> Some more recent work describes ways of using derivatives to compile
> to DFA.  The gist is to chase the possible derivatives at compile time
> and calculate the resulting transition graph statically.
> The trick, while doing that, is to recognize the identical states to
> condense the graph and recognize loops. Sometimes the derivative of
> a regex will yield the same regex, or a regex that was seen before.
> Repetition does this, for instance. The derivative of a* with respect
> to a is a*.  A notion of "sameness" is required, in consideration of
> the issue that sometimes regexes have the same meaning, but look
> different.
> 
> About the and operator: given R1&R2, the derivative has an easy
> interpretation.  d(R1&R2, s) is just d(R1, s)&d(R2, s).  The derivative
> operation parallelizes into the branches of the and, and the results are
> combined with & again. If either derivative is the empty regex [],
> then there is no match since R&[] = [].

This is very interesting.  What would be the complexity e.g. of the
recognition of "identical states to condense the graph and loops"?
Implemented in shell-interpreters it might be a factor to consider.

Janis

[toc] | [prev] | [next] | [standalone]

#21729

On 13.05.2021 10:34, Janis Papanagnou wrote:
> Kornshell introduced patterns (called extended patterns in bash) that
> allowed, for example, pattern expressions like @(A|B|C), matching any
> of the sub-patterns A, B, or C.  I was under the impression that I've
> somewhere - maybe in some intermediate or experimental ksh version -
> seen an extension of those patterns allowing expressions like @(A&B&C)
> with the semantics of matching all sub-patterns (in arbitrary order).
> 
> One can think of various work-arounds for that like the equivalents
> 
> [[ $1 == *@(A*B*C|A*C*B|B*A*C|B*C*A|C*A*B|C*B*A)* ]]
> 
> [[ $1 == *A* && $1 == *B* && $1 == *C* ]]
> 
> ((c=0))
> case $1 in
> (*A*) ((c++)) ;&
> (*B*) ((c++)) ;&
> (*C*) ((c++)) ;&
> (*) ((c==3))
> esac
> 
> none of them really satisfying (if compared to @(A&B&C) ).
> 
> [...]
> I'm quite sure I've seen it somewhere but don't recall where it was.

Doh! - I had obviously seen it described where it actually belongs, in
the ksh man page...

 "A pattern-list is a list of one or more patterns separated from each
  other with a & or ⎪.  A & signifies that all patterns must be matched
  whereas ⎪ requires that only one pattern be matched."

But I used the pattern incorrectly in my tests. This one actually works

  [[ $1 == @(*A*&*B*&*C*) ]]

(A bit more clumsy but still better than the workarounds.)

Janis

[toc] | [prev] | [next] | [standalone]

#21731

On Sun, 4 Jul 2021 00:56:13 +0200
Janis Papanagnou <janis_papanagnou@hotmail.com> wrote:
> On 13.05.2021 10:34, Janis Papanagnou wrote:
> > Kornshell introduced patterns (called extended patterns in bash) that
> > allowed, for example, pattern expressions like @(A|B|C), matching any
> > of the sub-patterns A, B, or C.  I was under the impression that I've
> > somewhere - maybe in some intermediate or experimental ksh version -
> > seen an extension of those patterns allowing expressions like @(A&B&C)
> > with the semantics of matching all sub-patterns (in arbitrary order).
> > 
> > One can think of various work-arounds for that like the equivalents
> > 
> > [[ $1 == *@(A*B*C|A*C*B|B*A*C|B*C*A|C*A*B|C*B*A)* ]]
> > 
> > [[ $1 == *A* && $1 == *B* && $1 == *C* ]]
> > 
> > ((c=0))
> > case $1 in
> > (*A*) ((c++)) ;&
> > (*B*) ((c++)) ;&
> > (*C*) ((c++)) ;&
> > (*) ((c==3))
> > esac
> > 
> > none of them really satisfying (if compared to @(A&B&C) ).
> > 
> > [...]
> > I'm quite sure I've seen it somewhere but don't recall where it was.
> 
> Doh! - I had obviously seen it described where it actually belongs, in
> the ksh man page...
> 
>  "A pattern-list is a list of one or more patterns separated from each
>   other with a & or ⎪.  A & signifies that all patterns must be matched
>   whereas ⎪ requires that only one pattern be matched."

I believe you want  |  rather than  ⎪ .Somehow you replaced the former
character with the latter which looks similar.

> But I used the pattern incorrectly in my tests. This one actually works
> 
>   [[ $1 == @(*A*&*B*&*C*) ]]
> 
> (A bit more clumsy but still better than the workarounds.)

[toc] | [prev] | [next] | [standalone]

#21732

On 04.07.2021 16:32, Spiros Bousbouras wrote:
> On Sun, 4 Jul 2021 00:56:13 +0200
> Janis Papanagnou <janis_papanagnou@hotmail.com> wrote:
>> the ksh man page...
>>
>>  "A pattern-list is a list of one or more patterns separated from each
>>   other with a & or ⎪.  A & signifies that all patterns must be matched
>>   whereas ⎪ requires that only one pattern be matched."
> 
> I believe you want  |  rather than  ⎪ .Somehow you replaced the former
> character with the latter which looks similar.

I haven't inspected the character's code number, I just copy/pasted it
from the ksh man page, I don't think I've replaced anything. If I copy
the text passages from the man page into 'od -c' I certainly don't get
the pipe symbol '|'. Does your ksh man page show the correct character?

Janis

[toc] | [prev] | [next] | [standalone]

#21733

On Sun, 4 Jul 2021 16:52:09 +0200
Janis Papanagnou <janis_papanagnou@hotmail.com> wrote:
> On 04.07.2021 16:32, Spiros Bousbouras wrote:
> > On Sun, 4 Jul 2021 00:56:13 +0200
> > Janis Papanagnou <janis_papanagnou@hotmail.com> wrote:
> >> the ksh man page...
> >>
> >>  "A pattern-list is a list of one or more patterns separated from each
> >>   other with a & or ⎪.  A & signifies that all patterns must be matched
> >>   whereas ⎪ requires that only one pattern be matched."
> > 
> > I believe you want  |  rather than  ⎪ .Somehow you replaced the former
> > character with the latter which looks similar.
> 
> I haven't inspected the character's code number, I just copy/pasted it
> from the ksh man page, I don't think I've replaced anything. If I copy
> the text passages from the man page into 'od -c' I certainly don't get
> the pipe symbol '|'. Does your ksh man page show the correct character?

It shows  |  i.e. ASCII 124. I also experimented with using  ⎪ (unicode
9130 decimal) and  ksh  takes it literally i.e. it matches itself. I think
it would be good if you examined the source code of the man page. If it
really has  ⎪  then it's a bug. If it doesn't then the bug is with one
of the tools of the man pipeline which translate the page.

-- 
If you want to appear cultured and hip at the same time you can say that Jar Jar
Binks is the Papageno of Star Wars.

[toc] | [prev] | [next] | [standalone]

#21735

On 04.07.2021 17:07, Spiros Bousbouras wrote:
> On Sun, 4 Jul 2021 16:52:09 +0200
> Janis Papanagnou <janis_papanagnou@hotmail.com> wrote:
>> On 04.07.2021 16:32, Spiros Bousbouras wrote:
>>> On Sun, 4 Jul 2021 00:56:13 +0200
>>> Janis Papanagnou <janis_papanagnou@hotmail.com> wrote:
>>>> the ksh man page...
>>>>
>>>>  "A pattern-list is a list of one or more patterns separated from each
>>>>   other with a & or ⎪.  A & signifies that all patterns must be matched
>>>>   whereas ⎪ requires that only one pattern be matched."
>>>
>>> I believe you want  |  rather than  ⎪ .Somehow you replaced the former
>>> character with the latter which looks similar.
>>
>> I haven't inspected the character's code number, I just copy/pasted it
>> from the ksh man page, I don't think I've replaced anything. If I copy
>> the text passages from the man page into 'od -c' I certainly don't get
>> the pipe symbol '|'. Does your ksh man page show the correct character?
> 
> It shows  |  i.e. ASCII 124. I also experimented with using  ⎪ (unicode
> 9130 decimal) and  ksh  takes it literally i.e. it matches itself. I think
> it would be good if you examined the source code of the man page. If it
> really has  ⎪  then it's a bug. If it doesn't then the bug is with one
> of the tools of the man pipeline which translate the page.

The man page is full of that "other" pipe symbol, there's only three
instances of the "real" one.

The files ksh.1.gz and ksh93.1.gz contain only the "real" pipe symbol.
It seems the *roff processor creates that symbol in specific contexts?

Janis

[toc] | [prev] | [next] | [standalone]

#21737

On 2021-07-04, Janis Papanagnou <janis_papanagnou@hotmail.com> wrote:
>
> The files ksh.1.gz and ksh93.1.gz contain only the "real" pipe symbol.
> It seems the *roff processor creates that symbol in specific contexts?

As I mentioned that is not true of the ksh.1.xz man page (Mageia 8).

ksh-2020.0.0.338.git8d91e8a-0.2.mga8

It contains the wrong (UTF) pipe symbol rather than the right ascii one. 

[toc] | [prev] | [next] | [standalone]

#21742

On Sun, 04 Jul 2021 13:03:14 -0400, William Unruh <unruh@invalid.ca> wrote:

> On 2021-07-04, Janis Papanagnou <janis_papanagnou@hotmail.com> wrote:
>>
>> The files ksh.1.gz and ksh93.1.gz contain only the "real" pipe symbol.
>> It seems the *roff processor creates that symbol in specific contexts?
>
> As I mentioned that is not true of the ksh.1.xz man page (Mageia 8).
>
> ksh-2020.0.0.338.git8d91e8a-0.2.mga8
>
> It contains the wrong (UTF) pipe symbol rather than the right ascii one.

The error is in the original source used to create the ksh source rpm for Mageia.

Line 109 of https://github.com/att/ast/blob/master/src/cmd/ksh93/sh.1 has the error.

Regards, Dave Hodgins

-- 
Change dwhodgins@nomail.afraid.org to davidwhodgins@teksavvy.com for
email replies.

[toc] | [prev] | [next] | [standalone]

#21743

On Sun, 04 Jul 2021 15:32:02 -0400, David W. Hodgins <dwhodgins@nomail.afraid.org> wrote:
> On Sun, 04 Jul 2021 13:03:14 -0400, William Unruh <unruh@invalid.ca> wrote:
>> On 2021-07-04, Janis Papanagnou <janis_papanagnou@hotmail.com> wrote:
>>> The files ksh.1.gz and ksh93.1.gz contain only the "real" pipe symbol.
>>> It seems the *roff processor creates that symbol in specific contexts?

>> As I mentioned that is not true of the ksh.1.xz man page (Mageia 8).
>> ksh-2020.0.0.338.git8d91e8a-0.2.mga8
>> It contains the wrong (UTF) pipe symbol rather than the right ascii one.

> The error is in the original source used to create the ksh source rpm for Mageia.
> Line 109 of https://github.com/att/ast/blob/master/src/cmd/ksh93/sh.1 has the error.

Now reported as https://github.com/att/ast/issues/1491 and
https://bugs.mageia.org/show_bug.cgi?id=29214

Regards, Dave Hodgins

-- 
Change dwhodgins@nomail.afraid.org to davidwhodgins@teksavvy.com for
email replies.

[toc] | [prev] | [next] | [standalone]

#21745

In article <sbspki$ovb$1@dont-email.me>,
William Unruh  <unruh@invalid.ca> wrote:
>On 2021-07-04, Janis Papanagnou <janis_papanagnou@hotmail.com> wrote:
>>
>> The files ksh.1.gz and ksh93.1.gz contain only the "real" pipe symbol.
>> It seems the *roff processor creates that symbol in specific contexts?
>
>As I mentioned that is not true of the ksh.1.xz man page (Mageia 8).
>
>ksh-2020.0.0.338.git8d91e8a-0.2.mga8

Be careful, the beast ksh2020 is the ksh destroyed bz RedHat...

-- 
EMail:joerg@schily.net			Jörg Schilling D-13353 Berlin
					Blog: http://schily.blogspot.com/
URL:  http://cdrecord.org/private/ http://sourceforge.net/projects/schilytools/files/

[toc] | [prev] | [next] | [standalone]

#21739

On Sun, 4 Jul 2021 17:36:46 +0200
Janis Papanagnou <janis_papanagnou@hotmail.com> wrote:
> On 04.07.2021 17:07, Spiros Bousbouras wrote:
> > On Sun, 4 Jul 2021 16:52:09 +0200
> > Janis Papanagnou <janis_papanagnou@hotmail.com> wrote:
> >> I haven't inspected the character's code number, I just copy/pasted it
> >> from the ksh man page, I don't think I've replaced anything. If I copy
> >> the text passages from the man page into 'od -c' I certainly don't get
> >> the pipe symbol '|'. Does your ksh man page show the correct character?
> > 
> > It shows  |  i.e. ASCII 124. I also experimented with using  ⎪ (unicode
> > 9130 decimal) and  ksh  takes it literally i.e. it matches itself. I think
> > it would be good if you examined the source code of the man page. If it
> > really has  ⎪  then it's a bug. If it doesn't then the bug is with one
> > of the tools of the man pipeline which translate the page.
> 
> The man page is full of that "other" pipe symbol, there's only three
> instances of the "real" one.
> 
> The files ksh.1.gz and ksh93.1.gz contain only the "real" pipe symbol.
> It seems the *roff processor creates that symbol in specific contexts?

It would seem so. I have encountered something analogous : doing
    groff -m mandoc -T html ./some-groff-source

produces
    &rsquo;    when there is   '   in the original
    &lsquo;    when there is   `   in the original
    &minus;    when there is   -   in the original

.According to
https://www.w3.org/wiki/Common_HTML_entities_used_for_typography

   &rsquo;  is unicode 8217
   &lsquo;  is unicode 8216
   &minus;  is unicode 8722

which of course was not what I wanted. In particular when the man page has
code examples , doing these replacements breaks the code. So I always use

    groff -m mandoc -T html ./some-groff-source | sed -e "s/&rsquo;/\&#39;/g" -e "s/&lsquo;/\&#96;/g" -e "s/&minus;/\&#45;/g" > man-page.html

-- 
About a car each minute gets blasted and 1000 liters of gasoline are burnt...
They sure did not have ecology in mind.
  www.imdb.com/review/rw2940189/

[toc] | [prev] | [next] | [standalone]

#21736

On 2021-07-04, Spiros Bousbouras <spibou@gmail.com> wrote:
> On Sun, 4 Jul 2021 16:52:09 +0200
> Janis Papanagnou <janis_papanagnou@hotmail.com> wrote:
>> On 04.07.2021 16:32, Spiros Bousbouras wrote:
>> > On Sun, 4 Jul 2021 00:56:13 +0200
>> > Janis Papanagnou <janis_papanagnou@hotmail.com> wrote:
>> >> the ksh man page...
>> >>
>> >>  "A pattern-list is a list of one or more patterns separated from each
>> >>   other with a & or ⎪.  A & signifies that all patterns must be matched
>> >>   whereas ⎪ requires that only one pattern be matched."
>> > 
>> > I believe you want  |  rather than  ⎪ .Somehow you replaced the former
>> > character with the latter which looks similar.
>> 
>> I haven't inspected the character's code number, I just copy/pasted it
>> from the ksh man page, I don't think I've replaced anything. If I copy
>> the text passages from the man page into 'od -c' I certainly don't get
>> the pipe symbol '|'. Does your ksh man page show the correct character?
>
> It shows  |  i.e. ASCII 124. I also experimented with using  ⎪ (unicode
> 9130 decimal) and  ksh  takes it literally i.e. it matches itself. I think
> it would be good if you examined the source code of the man page. If it
> really has  ⎪  then it's a bug. If it doesn't then the bug is with one
> of the tools of the man pipeline which translate the page.
>

Using hexedit the lines in the  unxz man page which are 

       ......................................................s:

              ;   &   (   )   ⎪   <   >   new-line   space   tab

       A

come out as 
00000744   73 3A 0A 2E  50 50 0A 2E  52 53 0A 5C  66 33 3B 20  20 20 26 20  s:..PP..RS.\f3;   &
00000758   20 20 28 20  20 20 29 20  20 20 5C 28  62 76 20 20  20 3C 20 20    (   )   \(bv   <
0000076C   20 3E 20 20  20 6E 65 77  2D 6C 69 6E  65 20 20 20  73 70 61 63   >   new-line   spac
00000780   65 20 20 20  74 61 62 5C  66 50 0A 2E  52 45 0A 2E  50 50 0A 41  e   tab\fP..RE..PP.A

Ie, it is the man page that is wrong-- consistantly using the Unicode
representation rather than that Ascii. 

And the symbol they use in the man page do not work as a substitute

Under ksh

echo a | cat

gives
a | cat

Not as it should

a

Ie, that symbol does NOT act as a pipe.

[toc] | [prev] | [next] | [standalone]

#21738

On Sun, 4 Jul 2021 15:57:26 -0000 (UTC)
William Unruh <unruh@invalid.ca> wrote:
> On 2021-07-04, Spiros Bousbouras <spibou@gmail.com> wrote:
> > On Sun, 4 Jul 2021 16:52:09 +0200
> > Janis Papanagnou <janis_papanagnou@hotmail.com> wrote:
> >> I haven't inspected the character's code number, I just copy/pasted it
> >> from the ksh man page, I don't think I've replaced anything. If I copy
> >> the text passages from the man page into 'od -c' I certainly don't get
> >> the pipe symbol '|'. Does your ksh man page show the correct character?
> >
> > It shows  |  i.e. ASCII 124. I also experimented with using  ⎪ (unicode
> > 9130 decimal) and  ksh  takes it literally i.e. it matches itself. I think
> > it would be good if you examined the source code of the man page. If it
> > really has  ⎪  then it's a bug. If it doesn't then the bug is with one
> > of the tools of the man pipeline which translate the page.
> >
> 
> Using hexedit the lines in the  unxz man page which are

This confused me. I'm guessing you mean the  ksh  page with  .xz  compression.

>        ......................................................s:
> 
>               ;   &   (   )   ⎪   <   >   new-line   space   tab
> 
>        A
> 
> come out as 
> 00000744   73 3A 0A 2E  50 50 0A 2E  52 53 0A 5C  66 33 3B 20  20 20 26 20  s:..PP..RS.\f3;   &
> 00000758   20 20 28 20  20 20 29 20  20 20 5C 28  62 76 20 20  20 3C 20 20    (   )   \(bv   <
> 0000076C   20 3E 20 20  20 6E 65 77  2D 6C 69 6E  65 20 20 20  73 70 61 63   >   new-line   spac
> 00000780   65 20 20 20  74 61 62 5C  66 50 0A 2E  52 45 0A 2E  50 50 0A 41  e   tab\fP..RE..PP.A
> 
> Ie, it is the man page that is wrong-- consistantly using the Unicode
> representation rather than that Ascii. 
> 
> And the symbol they use in the man page do not work as a substitute
> 
> Under ksh
> 
> echo a | cat
> 
> gives
> a | cat
> 
> Not as it should
> 
> a
> 
> Ie, that symbol does NOT act as a pipe.

But the symbol you posted is the usual pipe symbol namely ASCII 124.

[toc] | [prev] | [next] | [standalone]

#21744

On 2021-07-04, Spiros Bousbouras <spibou@gmail.com> wrote:
> On Sun, 4 Jul 2021 15:57:26 -0000 (UTC)
> William Unruh <unruh@invalid.ca> wrote:
>> On 2021-07-04, Spiros Bousbouras <spibou@gmail.com> wrote:
>> > On Sun, 4 Jul 2021 16:52:09 +0200
>> > Janis Papanagnou <janis_papanagnou@hotmail.com> wrote:
>> >> I haven't inspected the character's code number, I just copy/pasted it
>> >> from the ksh man page, I don't think I've replaced anything. If I copy
>> >> the text passages from the man page into 'od -c' I certainly don't get
>> >> the pipe symbol '|'. Does your ksh man page show the correct character?
>> >
>> > It shows  |  i.e. ASCII 124. I also experimented with using  ⎪ (unicode
>> > 9130 decimal) and  ksh  takes it literally i.e. it matches itself. I think
>> > it would be good if you examined the source code of the man page. If it
>> > really has  ⎪  then it's a bug. If it doesn't then the bug is with one
>> > of the tools of the man pipeline which translate the page.
>> >
>> 
>> Using hexedit the lines in the  unxz man page which are
>
> This confused me. I'm guessing you mean the  ksh  page with  .xz  compression.

Sorry, Yes, I meant the ksh.1.xz file  on which I ran the program unxz to
decompress it. 
>
>>        ......................................................s:
>> 
>>               ;   &   (   )   ⎪   <   >   new-line   space   tab
>> 
>>        A
>> 
>> come out as 
>> 00000744   73 3A 0A 2E  50 50 0A 2E  52 53 0A 5C  66 33 3B 20  20 20 26 20  s:..PP..RS.\f3;   &
>> 00000758   20 20 28 20  20 20 29 20  20 20 5C 28  62 76 20 20  20 3C 20 20    (   )   \(bv   <
>> 0000076C   20 3E 20 20  20 6E 65 77  2D 6C 69 6E  65 20 20 20  73 70 61 63   >   new-line   spac
>> 00000780   65 20 20 20  74 61 62 5C  66 50 0A 2E  52 45 0A 2E  50 50 0A 41  e   tab\fP..RE..PP.A
>> 
>> Ie, it is the man page that is wrong-- consistantly using the Unicode
>> representation rather than that Ascii. 
>> 
>> And the symbol they use in the man page do not work as a substitute
>> 
>> Under ksh
>> 
>> echo a | cat
>> 
>> gives
>> a | cat
>> 
>> Not as it should
>> 
>> a
>> 
>> Ie, that symbol does NOT act as a pipe.
>
> But the symbol you posted is the usual pipe symbol namely ASCII 124.

The symbol in the ksh.1 file derived from ksh.1.xz is  
5C 28 62 76
Which does not appear to be a unicode symbol, but some sort of command
to the troff interpreter which is coming out as the unicode ⎪, rather than
the ascii |

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | comp.unix.shell

csiph-web