Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.forth > #21303

Re: The measure of a whale

Show all headers | View raw

WJ wrote:

> Consider all of the words in Project Gutenburg's "Moby Dick",
> where a word is simply a sequence of letters.
> Determine the number of distinct words.
> Display the most common word and the number of times it occurs.
> 
> Factor:
> 
> USING: hashtables regexp io.files locals io.encodings.ascii ;
> 
> : best-pair ( seq -- pair )
>   unclip
>   [ 2dup [ last ] bi@ < [ nip ] [ drop ] if ] reduce ;
> 
> :: doit ( -- )
>   H{ } clone :> table
>   "moby10b.txt" ascii file-contents
>   R/ [a-z]+/i all-matching-subseqs
>   [ 1 swap table at+ ] each
>   table count>> .
>   table >alist best-pair .
> ;
> 
> 
> [ doit ] time
> 19319
> { "the" 13784 }
> Running time: 0.247124425 seconds
> 

USING: formatting hashtables regexp io.files locals
       io.encodings.ascii ;

:: do-it ( -- )
  H{ } clone :> table
  "moby10b.txt" ascii file-contents
  R/ [a-z]+/i all-matching-subseqs
  [ 1 swap table at+ ] each
  table count>> "%d distinct words.\n" printf
  "The most common words:" print
  table >alist [ [ last ] bi@ >=< ] sort
  20 head [ 1 + swap first2 "%2d. %-4s %5d\n" printf   ] each-index
;

do-it

19319 distinct words.
The most common words:
 1. the  13784
 2. of    6597
 3. and   6065
 4. a     4605
 5. to    4595
 6. in    3933
 7. that  2992
 8. his   2459
 9. it    2227
10. I     2125
11. s     1746
12. is    1716
13. with  1666
14. he    1661
15. was   1635
16. as    1627
17. all   1470
18. for   1434
19. this  1311
20. at    1243

Back to comp.lang.forth | Previous | Next — Previous in thread | Find similar | Unroll thread

Thread

The measure of a whale "WJ" <w_a_x_man@yahoo.com> - 2013-03-14 08:56 +0000
  Re: The measure of a whale "WJ" <w_a_x_man@yahoo.com> - 2013-04-02 09:08 +0000

csiph-web