Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.forth > #49175 > unrolled thread

Parse a sentence

Back to article view | Back to comp.lang.forth

Contents

  Parse a sentence "WJ" <w_a_x_man@yahoo.com> - 2016-07-15 13:47 +0000
    Re: Parse a sentence François <francois.ganza@wanadoo.fr> - 2016-07-15 07:10 -0700
    Re: Parse a sentence Richard Todd <richard.wesley.todd@gmail.com> - 2016-07-15 09:35 -0700
    Parse a sentence Mark Wills <markwills1970@gmail.com> - 2016-07-16 03:03 -0700
    Re: Parse a sentence "WJ" <w_a_x_man@yahoo.com> - 2016-07-17 10:24 +0000
    Re: Parse a sentence endlessboomcapitalism@gmail.com - 2016-07-17 21:19 -0700

#49175 — Parse a sentence

Ruby:

result = "  Alas poor Yorick!?  I knew him well  , Horatio .  ".
  scan(/\w+|[^\w\s]/)
 ===>
["Alas", "poor", "Yorick", "!", "?", "I", "knew", "him", "well",
 ",", "Horatio", "."]

result.join("|")
 ===>
"Alas|poor|Yorick|!|?|I|knew|him|well|,|Horatio|."


Ocaml:

#load "str.cma";;

let parse_sentence string =
  List.fold_right
    (fun el accum ->
      match el with
          Str.Text s -> s::accum
        | Str.Delim " " -> accum
        | Str.Delim s -> s::accum)
    (Str.full_split (Str.regexp "[^a-zA-Z]") string)
    [] ;;

let result = parse_sentence
  "  Alas poor Yorick!?  I knew him well  , Horatio .  " ;;
 ===>
["Alas"; "poor"; "Yorick"; "!"; "?"; "I"; "knew"; "him"; "well";
 ","; "Horatio"; "."]

String.concat "|" result ;;
 ===>
"Alas|poor|Yorick|!|?|I|knew|him|well|,|Horatio|."


ANS Forth?

-- 
Amazon bans book.  After nearly a month on the site, all traces of the book and
its 80 reviews have been removed.
http://jamesfetzer.blogspot.com/2015/11/debunking-sandy-hook-debunkers-5.html
https://www.youtube.com/watch?v=EEl_1HWFRfo

[toc] | [next] | [standalone]

#49177

Le vendredi 15 juillet 2016 15:47:03 UTC+2, WJ a écrit :
> Ruby:
> 
> result = "  Alas poor Yorick!?  I knew him well  , Horatio .  ".
>   scan(/\w+|[^\w\s]/)
>  ===>
> ["Alas", "poor", "Yorick", "!", "?", "I", "knew", "him", "well",
>  ",", "Horatio", "."]
> 
> result.join("|")
>  ===>
> "Alas|poor|Yorick|!|?|I|knew|him|well|,|Horatio|."
> 
> 
> Ocaml:
> 
> #load "str.cma";;
> 
> let parse_sentence string =
>   List.fold_right
>     (fun el accum ->
>       match el with
>           Str.Text s -> s::accum
>         | Str.Delim " " -> accum
>         | Str.Delim s -> s::accum)
>     (Str.full_split (Str.regexp "[^a-zA-Z]") string)
>     [] ;;
> 
> let result = parse_sentence
>   "  Alas poor Yorick!?  I knew him well  , Horatio .  " ;;
>  ===>
> ["Alas"; "poor"; "Yorick"; "!"; "?"; "I"; "knew"; "him"; "well";
>  ","; "Horatio"; "."]
> 
> String.concat "|" result ;;
>  ===>
> "Alas|poor|Yorick|!|?|I|knew|him|well|,|Horatio|."
> 
> 
> ANS Forth?
> 
> -- 
> Amazon bans book.  After nearly a month on the site, all traces of the book and
> its 80 reviews have been removed.
> http://jamesfetzer.blogspot.com/2015/11/debunking-sandy-hook-debunkers-5.html
> https://www.youtube.com/watch?v=EEl_1HWFRfo

mmmh.. Forth interpreter ?

[toc] | [prev] | [next] | [standalone]

#49179

On Friday, July 15, 2016 at 8:47:03 AM UTC-5, WJ wrote:
> ANS Forth?

\ Here's a state-machine implementation. No external libs 
\ for regex or anything.

0 constant stSPC  \ a space
1 constant stLTR  \ a letter
2 constant stNLS  \ a non-(letter|space)

: categorize ( c -- c cat )
  dup BL = IF stSPC ELSE
  dup [CHAR] a [CHAR] z 1+ within
  over [CHAR] A [CHAR] Z 1+ within or IF stLTR ELSE
    stNLS  THEN THEN  ;

: write ( addr c -- addr' ) over c! 1+ ;
: emit| ( addr c -- addr' ) drop [CHAR] | write ;
: |write ( addr c -- addr' ) swap [CHAR] | write swap write ;

CREATE (actions)
' drop ,  ' write ,  ' write ,
' emit| ,  ' write ,  ' |write ,
' emit| ,  ' |write ,  ' |write ,

: do-char ( addr st c  -- addr' st' )
   categorize dup >r rot 
   3 * + cells (actions) + @ EXECUTE r> ;

: parse-sentence ( addr u -- addr u )
   over dup >r swap stSPC -rot bounds
   ?DO I c@ do-char LOOP
   0= r@ - + r> swap ;


> result.join("|")
>  ===>
> "Alas|poor|Yorick|!|?|I|knew|him|well|,|Horatio|."

\ helper word to get a mutable string
: mutable tuck pad swap move pad swap ; 

s"   Alas poor Yorick!?  I knew him well  , Horatio .  " 
mutable parse-sentence type cr

Alas|poor|Yorick|!|?|I|knew|him|well|,|Horatio|.

[toc] | [prev] | [next] | [standalone]

#49186

create array 20 cells allot
S"  Alas poor Yorick!?  I knew him well  , Horatio ."

whitespace nonalpha + array splitString

Fills the array with the address of each sub string. Each sub string contains the string length as the first byte.

[toc] | [prev] | [next] | [standalone]

#49202

WJ wrote:

> Ruby:
> 
> result = "  Alas poor Yorick!?  I knew him well  , Horatio .  ".
>   scan(/\w+|[^\w\s]/)
>  ===>
> ["Alas", "poor", "Yorick", "!", "?", "I", "knew", "him", "well",
>  ",", "Horatio", "."]
> 
> result.join("|")
>  ===>
> "Alas|poor|Yorick|!|?|I|knew|him|well|,|Horatio|."
> 
> 
> Ocaml:
> 
> #load "str.cma";;
> 
> let parse_sentence string =
>   List.fold_right
>     (fun el accum ->
>       match el with
>           Str.Text s -> s::accum
>         | Str.Delim " " -> accum
>         | Str.Delim s -> s::accum)
>     (Str.full_split (Str.regexp "[^a-zA-Z]") string)
>     [] ;;
> 
> let result = parse_sentence
>   "  Alas poor Yorick!?  I knew him well  , Horatio .  " ;;
>  ===>
> ["Alas"; "poor"; "Yorick"; "!"; "?"; "I"; "knew"; "him"; "well";
>  ","; "Horatio"; "."]
> 
> String.concat "|" result ;;
>  ===>
> "Alas|poor|Yorick|!|?|I|knew|him|well|,|Horatio|."
> 
> 
> ANS Forth?

R:

str.scan = function(rx,str, ignore.case=FALSE, fixed=FALSE){
  m = gregexpr(rx, str, ignore.case=ignore.case, fixed=fixed, perl=TRUE)
  regmatches(str, m)[[1]]}

result = str.scan(
  "[a-z]+|[,.!?]",
  "  Alas poor Yorick!?  I knew him well  , Horatio .  ",
  ignore=TRUE)

result
 ===>
 [1] "Alas"    "poor"    "Yorick"  "!"       "?"       "I"       "knew"
 [8] "him"     "well"    ","       "Horatio" "."

paste(result, collapse="|")
 ===>
[1] "Alas|poor|Yorick|!|?|I|knew|him|well|,|Horatio|."

[toc] | [prev] | [next] | [standalone]

#49209

On Friday, July 15, 2016 at 9:47:03 AM UTC-4, WJ wrote:
> Ruby:
> 
> result = "  Alas poor Yorick!?  I knew him well  , Horatio .  ".
>   scan(/\w+|[^\w\s]/)
>  ===>
> ["Alas", "poor", "Yorick", "!", "?", "I", "knew", "him", "well",
>  ",", "Horatio", "."]
> 
> result.join("|")
>  ===>
> "Alas|poor|Yorick|!|?|I|knew|him|well|,|Horatio|."
> 
> 
> Ocaml:
> 
> #load "str.cma";;
> 
> let parse_sentence string =
>   List.fold_right
>     (fun el accum ->
>       match el with
>           Str.Text s -> s::accum
>         | Str.Delim " " -> accum
>         | Str.Delim s -> s::accum)
>     (Str.full_split (Str.regexp "[^a-zA-Z]") string)
>     [] ;;
> 
> let result = parse_sentence
>   "  Alas poor Yorick!?  I knew him well  , Horatio .  " ;;
>  ===>
> ["Alas"; "poor"; "Yorick"; "!"; "?"; "I"; "knew"; "him"; "well";
>  ","; "Horatio"; "."]
> 
> String.concat "|" result ;;
>  ===>
> "Alas|poor|Yorick|!|?|I|knew|him|well|,|Horatio|."
> 
> 
> ANS Forth?
> 
> -- 
> Amazon bans book.  After nearly a month on the site, all traces of the book and
> its 80 reviews have been removed.
> http://jamesfetzer.blogspot.com/2015/11/debunking-sandy-hook-debunkers-5.html
> https://www.youtube.com/watch?v=EEl_1HWFRfo

how about in gforth

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.forth

csiph-web