Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.lang.forth > #57272 > unrolled thread
| Started by | Julian Fondren <julian.fondren@gmail.com> |
|---|---|
| First post | 2017-04-13 22:46 -0700 |
| Last post | 2017-04-14 21:24 -0400 |
| Articles | 17 on this page of 57 — 11 participants |
Back to article view | Back to comp.lang.forth
[Random] Escape the trolls Julian Fondren <julian.fondren@gmail.com> - 2017-04-13 22:46 -0700
Re: [Random] Escape the trolls NN <november.nihal@gmail.com> - 2017-04-14 10:53 -0700
Re: [Random] Escape the trolls Julian Fondren <julian.fondren@gmail.com> - 2017-04-14 11:26 -0700
Re: [Random] Escape the trolls rickman <gnuarm@gmail.com> - 2017-04-14 15:30 -0400
Re: [Random] Escape the trolls Julian Fondren <julian.fondren@gmail.com> - 2017-04-14 12:45 -0700
Re: [Random] Escape the trolls rickman <gnuarm@gmail.com> - 2017-04-14 16:02 -0400
Re: [Random] Escape the trolls NN <november.nihal@gmail.com> - 2017-04-14 13:13 -0700
Re: [Random] Escape the trolls rickman <gnuarm@gmail.com> - 2017-04-14 16:18 -0400
Re: [Random] Escape the trolls NN <november.nihal@gmail.com> - 2017-04-14 13:25 -0700
Re: [Random] Escape the trolls rickman <gnuarm@gmail.com> - 2017-04-14 16:49 -0400
Re: [Random] Escape the trolls Alex <alex@rivadpm.com> - 2017-04-14 21:59 +0100
Re: [Random] Escape the trolls josv@planet.nl - 2017-04-15 04:09 -0700
Re: [Random] Escape the trolls rickman <gnuarm@gmail.com> - 2017-04-15 12:45 -0400
Re: [Random] Escape the trolls Coos Haak <htrofhc@gmail.com> - 2017-04-15 18:50 +0200
Re: [Random] Escape the trolls NN <november.nihal@gmail.com> - 2017-04-15 15:45 -0700
Re: [Random] Escape the trolls hughaguilar96@gmail.com - 2017-04-14 14:46 -0700
Re: [Random] Escape the trolls Julian Fondren <julian.fondren@gmail.com> - 2017-04-14 15:35 -0700
Re: [Random] Escape the trolls hughaguilar96@gmail.com - 2017-04-14 16:24 -0700
Re: [Random] Escape the trolls Alex <alex@rivadpm.com> - 2017-04-15 00:34 +0100
Re: [Random] Escape the trolls Alex <alex@rivadpm.com> - 2017-04-15 00:36 +0100
Re: [Random] Escape the trolls hughaguilar96@gmail.com - 2017-04-14 20:58 -0700
Re: [Random] Escape the trolls rickman <gnuarm@gmail.com> - 2017-04-15 00:24 -0400
Re: [Random] Escape the trolls hughaguilar96@gmail.com - 2017-04-14 21:40 -0700
Re: [Random] Escape the trolls rickman <gnuarm@gmail.com> - 2017-04-15 02:32 -0400
Re: [Random] Escape the trolls anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2017-04-15 07:50 +0000
Re: [Random] Escape the trolls anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2017-04-15 08:15 +0000
Re: [Random] Escape the trolls rickman <gnuarm@gmail.com> - 2017-04-15 13:10 -0400
Re: [Random] Escape the trolls Coos Haak <htrofhc@gmail.com> - 2017-04-15 19:27 +0200
Re: [Random] Escape the trolls rickman <gnuarm@gmail.com> - 2017-04-15 14:25 -0400
Re: [Random] Escape the trolls anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2017-04-17 14:05 +0000
Re: [Random] Escape the trolls "Rudy Velthuis" <newsgroups@rvelthuis.de> - 2017-04-16 01:42 +0000
Re: [Random] Escape the trolls rickman <gnuarm@gmail.com> - 2017-04-15 21:51 -0400
Re: [Random] Escape the trolls "Rudy Velthuis" <newsgroups@rvelthuis.de> - 2017-04-16 02:10 +0000
Re: [Random] Escape the trolls Coos Haak <htrofhc@gmail.com> - 2017-04-16 14:29 +0200
Re: [Random] Escape the trolls "Rudy Velthuis" <newsgroups@rvelthuis.de> - 2017-04-16 13:58 +0000
Re: [Random] Escape the trolls anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2017-04-16 16:11 +0000
Re: [Random] Escape the trolls "Rudy Velthuis" <newsgroups@rvelthuis.de> - 2017-04-16 22:18 +0000
Re: [Random] Escape the trolls Gerry Jackson <gerry@jackson9000.fsnet.co.uk> - 2017-04-17 07:54 +0100
Re: [Random] Escape the trolls "Rudy Velthuis" <newsgroups@rvelthuis.de> - 2017-04-17 23:32 +0000
Re: [Random] Escape the trolls "Rudy Velthuis" <newsgroups@rvelthuis.de> - 2017-04-17 23:33 +0000
Re: [Random] Escape the trolls Julian Fondren <julian.fondren@gmail.com> - 2017-04-14 22:33 -0700
Re: [Random] Escape the trolls Julian Fondren <julian.fondren@gmail.com> - 2017-04-14 22:52 -0700
Re: [Random] Escape the trolls anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2017-04-15 07:52 +0000
Re: [Random] Escape the trolls Alex <alex@rivadpm.com> - 2017-04-15 11:20 +0100
Re: [Random] Escape the trolls anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2017-04-15 11:53 +0000
Re: [Random] Escape the trolls Alex <alex@rivadpm.com> - 2017-04-15 19:10 +0100
Re: [Random] Escape the trolls anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2017-04-17 14:19 +0000
Re: [Random] Escape the trolls Bernd Linsel <bl1@gmx.com> - 2017-04-15 20:21 +0200
Re: [Random] Escape the trolls Alex <alex@rivadpm.com> - 2017-04-15 20:33 +0100
Re: [Random] Escape the trolls Bernd Linsel <bl1@gmx.com> - 2017-04-16 11:50 +0200
Re: [Random] Escape the trolls anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2017-04-16 14:22 +0000
Re: [Random] Escape the trolls anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2017-04-16 14:47 +0000
Re: [Random] Escape the trolls hughaguilar96@gmail.com - 2017-04-15 16:55 -0700
Re: [Random] Escape the trolls hughaguilar96@gmail.com - 2017-04-15 17:03 -0700
Re: [Random] Escape the trolls Alex <alex@rivadpm.com> - 2017-04-15 11:37 +0100
Re: [Random] Escape the trolls Julian Fondren <julian.fondren@gmail.com> - 2017-04-14 16:51 -0700
Re: [Random] Escape the trolls rickman <gnuarm@gmail.com> - 2017-04-14 21:24 -0400
Page 3 of 3 — ← Prev page 1 2 [3]
| From | Julian Fondren <julian.fondren@gmail.com> |
|---|---|
| Date | 2017-04-14 22:33 -0700 |
| Message-ID | <0c706cf8-2c6c-4f6d-9c63-bfd1cb91d27e@googlegroups.com> |
| In reply to | #57345 |
On Friday, April 14, 2017 at 10:58:50 PM UTC-5, hughag...@gmail.com wrote:
> -3
> enum last-\ \ the last char was \
> enum last-\x \ the last chars were \x
> enum last-done \ the last chars are irrelevent
> drop
>
...
>
> : <fix\$> { last char-adr | datum lc-datum -- last done? } \ DONE? is always FALSE
> char-adr c@ to datum
> datum char-lower to lc-datum
> last last-done = if
> [char] \ lc-datum = if last-\ false exit then \ backslash
> datum char+cstr last-done false exit \ raw char
> then
> last last-\ = if \ these should all be lower-case because we compare to LC-DATUM
> [char] a lc-datum = if 7 char+cstr last-done false exit then \ bell
> [char] b lc-datum = if 8 char+cstr last-done false exit then \ backspace
> [char] e lc-datum = if 27 char+cstr last-done false exit then \ escape
> [char] e lc-datum = if 27 char+cstr last-done false exit then \ escape
> [char] f lc-datum = if 12 char+cstr last-done false exit then \ FF (form feed)
> [char] l lc-datum = if 10 char+cstr last-done false exit then \ LF (line feed)
> [char] m lc-datum = if 13 char+cstr 10 char+cstr last-done false exit then \ CR/LF
> [char] n lc-datum = if 13 char+cstr 10 char+cstr last-done false exit then \ CR/LF
> [char] q lc-datum = if 34 char+cstr last-done false exit then \ double-quote
> [char] r lc-datum = if 13 char+cstr last-done false exit then \ CR (carriage return)
> [char] t lc-datum = if 9 char+cstr last-done false exit then \ HT (horizontal tab)
> [char] v lc-datum = if 11 char+cstr last-done false exit then \ VT (vertical tab)
> [char] z lc-datum = if 0 char+cstr last-done false exit then \ null
> [char] \ lc-datum = if 92 char+cstr last-done false exit then \ back-slash
> [char] x lc-datum = if last-\x false exit then \ start hex
> true abort" *** S\quote given \ with an illegal code after the \ char ***"
> then
> last last-\x = if datum hexchar>n false exit then
> \ LAST is the hex value of the high digit
> last 4 lshift datum hexchar>n + char+cstr last-done false exit ;
>
...
>
> : fix\$ ( -- ) \ string: str -- new-str
> <cstr last-done ['] <fix\$> forward$ 2drop cstr> count mut>$ ;
If someone came to me and said that they had an amazing new way to
program, with higher order functions and string stacks, and then they
showed me this code as an example of it, I would have to say that it's
hard to see any benefit. On the contrary, the main contribution to
FORWARD$ seems to be that <FIX\$> devotes incredible amounts of code
just towards telling FORWARD$ to keep going.
I'd like to see how much all of this costs... but not enough to load
novice.4th myself. You haven't published it recently anyway.
Instead, let's transform this code by removing the amazing
technologies it uses, so that we can see how much worse it is
afterwards.
-3 constant last-\
-2 constant last-\x
-1 constant last-letter
Removing ENUM makes this code so verbose and ugly. One immediately
misses how your code is framed by its -3 and DROP
: escape ( c-addr1 u1 -- c-addr1 u2 )
over last-letter 2over bounds ?do ( c-addr1 u a state )
...
loop 2drop ;
Removing FORWARD$ makes the structure of the code completely
incomprehensible. This is so procedural! It's an embarrassment to
Forth, really.
: escape ( c-addr1 u1 -- c-addr1 u2 )
over last-letter 2over bounds ?do ( c-addr1 u a state ) case
last-letter of
i c@ dup [char] \ = if
drop last-\
else
over c! 1+ last-letter
then
endof
last-\ of
i c@ bl or case
[char] a of 7 over c! 1+ -1 under+ last-letter endof
[char] b of 8 over c! 1+ -1 under+ last-letter endof
[char] e of 27 over c! 1+ -1 under+ last-letter endof
[char] f of 12 over c! 1+ -1 under+ last-letter endof
[char] l of 10 over c! 1+ -1 under+ last-letter endof
[char] m of 13 over c! 1+ 10 over c! 1+ last-letter endof
[char] n of 10 over c! 1+ -1 under+ last-letter endof
[char] q of [char] " over c! 1+ -1 under+ last-letter endof
[char] r of 13 over c! 1+ -1 under+ last-letter endof
[char] t of 9 over c! 1+ -1 under+ last-letter endof
[char] v of 11 over c! 1+ -1 under+ last-letter endof
[char] z of 0 over c! 1+ -1 under+ last-letter endof
[char] \ of [char] \ over c! 1+ -1 under+ last-letter endof
[char] x of last-\x endof
true abort" illegal \ escape"
endcase
endof
last-\x of
i c@ hexchar>n
endof
4 lshift i c@ hexchar>n or over c! 1+
-3 under+ last-letter
dup endcase loop 2drop ;
This is... well, unlike your code, it is extremely long and it is
extremely repetitive in parts. And I resort to CASE whereas your code
has a superior sequence of IFs. We can see already how worse the code
is when the purifying light of the string stack is withdrawn from it.
At this point I would be tempted to factor out the repetition from
that CASE structure, but ... honestly I am too depressed to continue.
This is a clear defeat for me. I will make sure to use local variables
and higher order functions in future string-iterating code so that I
can end at a result closer to your <FIX\$> . I don't think this code
can be possibly made more compact while it relies on the data stack as
heavily as it does.
I won't say I'll use the string stack but... please publish it. I am
now interesting in seeing just how much faster your code is, on top of
its aesthetic advantage.
[toc] | [prev] | [next] | [standalone]
| From | Julian Fondren <julian.fondren@gmail.com> |
|---|---|
| Date | 2017-04-14 22:52 -0700 |
| Message-ID | <e589e088-7be3-4da5-acda-d46ae07bc677@googlegroups.com> |
| In reply to | #57348 |
On Saturday, April 15, 2017 at 12:33:56 AM UTC-5, Julian Fondren wrote: > : escape ( c-addr1 u1 -- c-addr1 u2 ) ... > i c@ bl or case ... > [char] \ of [char] \ over c! 1+ -1 under+ last-letter endof The BL OR should be replaced with a legit LOWERCASE Alternatively you could switch to using | as the escape char :)
[toc] | [prev] | [next] | [standalone]
| From | anton@mips.complang.tuwien.ac.at (Anton Ertl) |
|---|---|
| Date | 2017-04-15 07:52 +0000 |
| Message-ID | <2017Apr15.095213@mips.complang.tuwien.ac.at> |
| In reply to | #57345 |
hughaguilar96@gmail.com writes:
>1.) I read that Forth-200x description of S\" and found this bizarre restri=
>ction:
>"A program shall not alter the returned string."
>I don't know why they have that restriction, and I certainly don't.
: foo s\" bla" ;
'x foo drop c!
foo type
If programs were allowed to alter the returned string, what should the
output of TYPE be? Is the returned string required to be in RAM?
Should it make a new copy every time it runs? How long is that copy
guaranteed to live?
>2.) They also said this:
>"An ambiguous condition exists if a \ is placed before any character, other=
> than those defined in here."
>They sure do love the term "ambiguous condition." I just abort with an erro=
>r message, because this is an error.
That's certainly a good approach in an implementation. However, if
Forth-2012 required it, and we wanted to add a new escape code in the
next standard, that would mean a system satisfying the next standard
would not be Forth-2012 compliant. OTOH, maybe that is preferable to
the present approach where each system can do it's own extension
(e.g., treat upper-case escapes as if they were lower case), and when
we want to add a new escape code, we find that we cannot do it because
they all are already taken in some system and entrenched in code
written for that system (what happened with "{").
- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2016: http://www.euroforth.org/ef16/
[toc] | [prev] | [next] | [standalone]
| From | Alex <alex@rivadpm.com> |
|---|---|
| Date | 2017-04-15 11:20 +0100 |
| Message-ID | <ocss07$6gd$1@dont-email.me> |
| In reply to | #57354 |
On 4/15/2017 08:52, Anton Ertl wrote:
>
> However, if Forth-2012 required it, and we wanted to add a new escape
> code in the next standard, that would mean a system satisfying the
> next standard would not be Forth-2012 compliant. OTOH, maybe that is
> preferable to the present approach where each system can do it's own
> extension (e.g., treat upper-case escapes as if they were lower
> case), and when we want to add a new escape code, we find that we
> cannot do it because they all are already taken in some system and
> entrenched in code written for that system (what happened with "{").
>
I asked for reservation of \u for up to $FFFF Unicode codepoints at the
time S\" was being proposed, and I can't remember why it was turned
down. It's supported in standard C, JavaScript, C# and probably a number
of other languages.
\U is often reserved for $10000 and above and doesn't match the \u case,
at least in C, so a case insensitive parse could be a future potential
issue too.
I have implemented \uxxxx in my escaped strings. I find it useful, as I
live in the UK but use a US keyboard. $ is readily typed, but I have no
key for the £ symbol \u00a3. And, perhaps less importantly since Brexit,
the Euro € \u20ac.
--
Alex
[toc] | [prev] | [next] | [standalone]
| From | anton@mips.complang.tuwien.ac.at (Anton Ertl) |
|---|---|
| Date | 2017-04-15 11:53 +0000 |
| Message-ID | <2017Apr15.135348@mips.complang.tuwien.ac.at> |
| In reply to | #57358 |
Alex <alex@rivadpm.com> writes:
>On 4/15/2017 08:52, Anton Ertl wrote:
>I asked for reservation of \u for up to $FFFF Unicode codepoints at the
>time S\" was being proposed, and I can't remember why it was turned
>down.
I think the argument was that you could instead encode it in UTF-8
with \x, e.g., \xc2\xa3 for Unicode Character 'POUND SIGN' (U+00A3)
and \xe2\x82\xac for Unicode Character 'EURO SIGN' (U+20AC). Of
course that means that the program is dependent on using the UTF-8
encoding, but that's probably not a big restriction; maybe some
word-addressed machine will use UTF-16 or UTF-32, and in such an
environment the \x method will not work as intended, but we have yet
to see such a system.
- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2016: http://www.euroforth.org/ef16/
[toc] | [prev] | [next] | [standalone]
| From | Alex <alex@rivadpm.com> |
|---|---|
| Date | 2017-04-15 19:10 +0100 |
| Message-ID | <octngc$tf$1@dont-email.me> |
| In reply to | #57369 |
On 4/15/2017 12:53, Anton Ertl wrote: > Alex <alex@rivadpm.com> writes: >> On 4/15/2017 08:52, Anton Ertl wrote: >> I asked for reservation of \u for up to $FFFF Unicode codepoints at the >> time S\" was being proposed, and I can't remember why it was turned >> down. > > I think the argument was that you could instead encode it in UTF-8 > with \x, e.g., \xc2\xa3 for Unicode Character 'POUND SIGN' (U+00A3) > and \xe2\x82\xac for Unicode Character 'EURO SIGN' (U+20AC). Of > course that means that the program is dependent on using the UTF-8 > encoding, but that's probably not a big restriction; maybe some > word-addressed machine will use UTF-16 or UTF-32, and in such an > environment the \x method will not work as intended, but we have yet > to see such a system. > > - anton > I may resurrect the request for the committee to consider. Is it a full RfD? -- Alex
[toc] | [prev] | [next] | [standalone]
| From | anton@mips.complang.tuwien.ac.at (Anton Ertl) |
|---|---|
| Date | 2017-04-17 14:19 +0000 |
| Message-ID | <2017Apr17.161924@mips.complang.tuwien.ac.at> |
| In reply to | #57382 |
Alex <alex@rivadpm.com> writes:
>On 4/15/2017 12:53, Anton Ertl wrote:
>> Alex <alex@rivadpm.com> writes:
>>> On 4/15/2017 08:52, Anton Ertl wrote:
>>> I asked for reservation of \u for up to $FFFF Unicode codepoints at the
>>> time S\" was being proposed, and I can't remember why it was turned
>>> down.
...
>I may resurrect the request for the committee to consider. Is it a full RfD?
You can make a full RfD for such a request.
Alternatively, you can make a proposal on that by making a
contribution at <http://forth-standard.org/standard/core/Seq>.
However, we have not yet decided how we handle such proposals, so it
might get treatment like an RfD, or it might get some other treatment,
maybe even be ignored (I hope not).
- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2016: http://www.euroforth.org/ef16/
[toc] | [prev] | [next] | [standalone]
| From | Bernd Linsel <bl1@gmx.com> |
|---|---|
| Date | 2017-04-15 20:21 +0200 |
| Message-ID | <octo9v$aja$1@newsreader4.netcologne.de> |
| In reply to | #57369 |
On 4/15/2017 1:53 PM, Anton Ertl wrote: > I think the argument was that you could instead encode it in UTF-8 > with \x, e.g., \xc2\xa3 for Unicode Character 'POUND SIGN' (U+00A3) > and \xe2\x82\xac for Unicode Character 'EURO SIGN' (U+20AC). Of > course that means that the program is dependent on using the UTF-8 > encoding, but that's probably not a big restriction; maybe some > word-addressed machine will use UTF-16 or UTF-32, and in such an > environment the \x method will not work as intended, but we have yet > to see such a system. UTF-8 is generally a bad idea when working with non-European languages, as most code points have to be encoded in three or more bytes. UCS-16, or, if you want full Unicode, UTF-16, scales much better in international applications. Regards -- Bernd
[toc] | [prev] | [next] | [standalone]
| From | Alex <alex@rivadpm.com> |
|---|---|
| Date | 2017-04-15 20:33 +0100 |
| Message-ID | <octsc8$hav$1@dont-email.me> |
| In reply to | #57383 |
On 4/15/2017 19:21, Bernd Linsel wrote: > On 4/15/2017 1:53 PM, Anton Ertl wrote: >> I think the argument was that you could instead encode it in UTF-8 >> with \x, e.g., \xc2\xa3 for Unicode Character 'POUND SIGN' (U+00A3) >> and \xe2\x82\xac for Unicode Character 'EURO SIGN' (U+20AC). Of >> course that means that the program is dependent on using the UTF-8 >> encoding, but that's probably not a big restriction; maybe some >> word-addressed machine will use UTF-16 or UTF-32, and in such an >> environment the \x method will not work as intended, but we have yet >> to see such a system. > > UTF-8 is generally a bad idea when working with non-European languages, > as most code points have to be encoded in three or more bytes. > UCS-16, or, if you want full Unicode, UTF-16, scales much better in > international applications. > > Regards > -- > Bernd > Space is a minor consideration, since bulk text is rarely stored or transmitted uncompressed. -- Alex
[toc] | [prev] | [next] | [standalone]
| From | Bernd Linsel <bl1@gmx.com> |
|---|---|
| Date | 2017-04-16 11:50 +0200 |
| Message-ID | <ocvena$c86$1@newsreader4.netcologne.de> |
| In reply to | #57386 |
On 4/15/2017 9:33 PM, Alex wrote: > > Space is a minor consideration, since bulk text is rarely stored or > transmitted uncompressed. > It's not just about space. Whe you are working with invidual characters in theses strings (even if you're just displaying them), the UTF-representation has to be unpacked every time. Regards -- Bernd
[toc] | [prev] | [next] | [standalone]
| From | anton@mips.complang.tuwien.ac.at (Anton Ertl) |
|---|---|
| Date | 2017-04-16 14:22 +0000 |
| Message-ID | <2017Apr16.162257@mips.complang.tuwien.ac.at> |
| In reply to | #57433 |
Bernd Linsel <bl1@gmx.com> writes:
>It's not just about space. Whe you are working with invidual characters
>in theses strings (even if you're just displaying them), the
>UTF-representation has to be unpacked every time.
No, you don't have to convert UTF-8 or UTF-16 into codepoint numbers
to deal with individual code points, you can also use single-codepoint
strings; and if you are dealing with characters, in particular in
decomposed form (and not every character exists in precomposed form),
you have to represent characters as strings anyway.
Moreover, you very rarely have to work with individual characters.
E.g., in the Forth systems I tested that don't know about UTF-8
(gforth 0.6, VFX, and SwiftForth), using UTF-8 tends to lead to
problems only in the command-line editor, and when indicating the
location of an error.
And if you write the code for displaying Unicode yourself, the
conversion from UTF-8 or UTF-16 to code points is the least of your
problems (and relatively easily solved). It also is the least costly
part of displaying the characters.
- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2016: http://www.euroforth.org/ef16/
[toc] | [prev] | [next] | [standalone]
| From | anton@mips.complang.tuwien.ac.at (Anton Ertl) |
|---|---|
| Date | 2017-04-16 14:47 +0000 |
| Message-ID | <2017Apr16.164730@mips.complang.tuwien.ac.at> |
| In reply to | #57383 |
Bernd Linsel <bl1@gmx.com> writes:
>UTF-8 is generally a bad idea when working with non-European languages,
>as most code points have to be encoded in three or more bytes.
>UCS-16, or, if you want full Unicode, UTF-16, scales much better in
>international applications.
I find it interesting that you mentioned only space savings here, and
when that was debunked shifted the goal posts.
Anyway, your claim is not new, and has been debunked more thoroughly
quite some time ago, in particular in
<http://utf8everywhere.org/#asian>. There they find that the article
about Japan in the Japanese Wikipedia takes 55% more space (1186KB
instead of 767KB, 419KB difference) in UTF-16 than in UTF-8. If you
convert it from HTML into dense plain text, UTF-16 saves 21% (176KB
instead of 222KB, 46KB). So if your computer deals with more than 9
times more dense plain text than HTML, UTF-16 starts being a (tiny)
win. Most people, though, deal with more HTML, and then UTF-8 is a
win.
- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2016: http://www.euroforth.org/ef16/
[toc] | [prev] | [next] | [standalone]
| From | hughaguilar96@gmail.com |
|---|---|
| Date | 2017-04-15 16:55 -0700 |
| Message-ID | <d4a7bcd3-f200-42c1-ae27-26607218aa12@googlegroups.com> |
| In reply to | #57354 |
On Saturday, April 15, 2017 at 1:08:54 AM UTC-7, Anton Ertl wrote: > hughaguilar96@gmail.com writes: > >1.) I read that Forth-200x description of S\" and found this bizarre restri= > >ction: > >"A program shall not alter the returned string." > >I don't know why they have that restriction, and I certainly don't. > > : foo s\" bla" ; > 'x foo drop c! > foo type > > If programs were allowed to alter the returned string, what should the > output of TYPE be? Is the returned string required to be in RAM? > Should it make a new copy every time it runs? How long is that copy > guaranteed to live? I responded this over here: https://groups.google.com/forum/#!topic/comp.lang.forth/CYR7FhjYMkg
[toc] | [prev] | [next] | [standalone]
| From | hughaguilar96@gmail.com |
|---|---|
| Date | 2017-04-15 17:03 -0700 |
| Message-ID | <f32c3cc3-09a9-4e17-94f0-0dae76d469bc@googlegroups.com> |
| In reply to | #57397 |
I have an updated version.
1.) I changed the name to S\| because I'm using the | as a delimiter so I can have \" in the strings (you need \! to get the vertical bar).
2.) I fixed a bug --- I needed NIP$ to get rid of the original string.
3.) I added some more error checking.
4.) I realized that the primary use of this is to support foreign languages with \x## to get at all the accented characters in the extended-ASCII. So I added a lot of shortcuts for getting at the Spanish characters, as well as some math symbols. You still need to use \x## for other languages.
5.) I switched from case-insensitive to case-sensitive. This is so the Spanish characters can have lower-case and upper-case --- for example: \n and \N for the 'n' or 'N' with a tilde over it.
Here is the upgrade:
----------------------------------------------------------------------------
\ ********
\ ******** My S\| is based on S\"
\ ******** described here: http://forth-standard.org/standard/core/Seq
\ ******** The Forth-200x version has this problem: "A program shall not alter the returned string."
\ ******** My version uses the string-stack, so there is no such problem.
\ ********
: hexchar>n ( char -- n )
s" 0123456789ABCDEF" >$
ifindc$ dup -1 = abort" *** HEXCHAR>N given a char that wasn't hex ***"
;
-3
enum last-done \ the last chars are irrelevent
enum last-\ \ the last char was \
enum last-\x \ the last chars were \x
drop
\ The LAST parameter given to <fix\$> is one of the above enums (all negative), or is the hexadecimal value of the last char.
: <fix\$> { last char-adr | datum -- last done? } \ DONE? is always FALSE
char-adr c@ to datum
last last-done = if
[char] \ datum = if last-\ false exit then \ backslash
datum char+cstr last-done false exit \ raw char
then
last last-\ = if
[char] a datum = if 7 char+cstr last-done false exit then \ bell
[char] b datum = if 8 char+cstr last-done false exit then \ backspace
[char] f datum = if 12 char+cstr last-done false exit then \ FF (form feed)
[char] l datum = if 10 char+cstr last-done false exit then \ LF (line feed)
[char] m datum = if 13 char+cstr 10 char+cstr last-done false exit then \ CR/LF
[char] " datum = if 34 char+cstr last-done false exit then \ double-quote
[char] r datum = if 13 char+cstr last-done false exit then \ CR (carriage return)
[char] t datum = if 9 char+cstr last-done false exit then \ HT (horizontal tab)
[char] v datum = if 11 char+cstr last-done false exit then \ VT (vertical tab)
[char] z datum = if 0 char+cstr last-done false exit then \ null
[char] \ datum = if 92 char+cstr last-done false exit then \ backslash
[char] ! datum = if 124 char+cstr last-done false exit then \ vertical bar
[char] t datum = if 153 char+cstr last-done false exit then \ trademark
[char] c datum = if 169 char+cstr last-done false exit then \ copyright
[char] d datum = if 176 char+cstr last-done false exit then \ degree
[char] + datum = if 177 char+cstr last-done false exit then \ +-
[char] 1 datum = if 188 char+cstr last-done false exit then \ 1/4
[char] 2 datum = if 189 char+cstr last-done false exit then \ 1/2
[char] 3 datum = if 190 char+cstr last-done false exit then \ 3/4
[char] ? datum = if 191 char+cstr last-done false exit then \ ? inverted
[char] n datum = if 241 char+cstr last-done false exit then \ n tilde
[char] N datum = if 209 char+cstr last-done false exit then \ N tilde
[char] a datum = if 225 char+cstr last-done false exit then \ a acute
[char] A datum = if 193 char+cstr last-done false exit then \ a acute
[char] e datum = if 233 char+cstr last-done false exit then \ e acute
[char] E datum = if 201 char+cstr last-done false exit then \ e acute
[char] i datum = if 237 char+cstr last-done false exit then \ i acute
[char] I datum = if 205 char+cstr last-done false exit then \ i acute
[char] o datum = if 243 char+cstr last-done false exit then \ o acute
[char] O datum = if 211 char+cstr last-done false exit then \ O acute
[char] u datum = if 250 char+cstr last-done false exit then \ u acute
[char] U datum = if 218 char+cstr last-done false exit then \ U acute
[char] d datum = if 252 char+cstr last-done false exit then \ u diaeresis
[char] D datum = if 220 char+cstr last-done false exit then \ U diaeresis
[char] x datum = if last-\x false exit then \ start hex
true abort" *** S\quote given \ with an illegal code after the \ char ***"
then
last last-\x = if datum hexchar>n false exit then
\ LAST is the hex value of the high digit
last 4 lshift datum hexchar>n + char+cstr last-done false exit ;
: fix\$ ( -- ) \ string: str -- new-str
<cstr last-done ['] <fix\$> forward$ drop last-done <> abort" *** FIX\$ had an incomplete \ code on the end ***"
cstr> count mut>$
nip$ ;
: s\| ( -- adr cnt ) \ stream: string \ this returns a string in the heap that will need DEALLOC eventually
[char] | parse mut>$
fix\$ $> ;
[toc] | [prev] | [next] | [standalone]
| From | Alex <alex@rivadpm.com> |
|---|---|
| Date | 2017-04-15 11:37 +0100 |
| Message-ID | <ocssuu$9ag$1@dont-email.me> |
| In reply to | #57345 |
On 4/15/2017 04:58, hughaguilar96@gmail.com wrote: > On Friday, April 14, 2017 at 4:34:40 PM UTC-7, Alex wrote: >> On 4/15/2017 00:24, hughaguilar96@gmail.com wrote: >>> On Friday, April 14, 2017 at 3:35:55 PM UTC-7, Julian Fondren >>> wrote: >>>> On Friday, April 14, 2017 at 4:46:18 PM UTC-5, >>>> hughag...@gmail.com wrote: >>>>> These ANS-Forth experts seem to know almost nothing about >>>>> ANS-Forth! Code that is trivial to write in ANS-Forth they >>>>> expect the compiler-writer to provide to them as a >>>>> non-standard extension. >>>> >>>> S\" is a very widely implemented standard extension that is >>>> also pretty easy to implement on the Forths I've found that >>>> lack it. >>> >>> It may be easy to implement, but you haven't done so --- you are >>> waiting for somebody smart to implement it for you. >>> >>> I don't know what S\" does --- if anybody knows what this does >>> and can describe it, I will try to write it --- if anybody has >>> already written it, and their code is ANS-Forth, I'll put it in >>> the novice-package. >>> >> >> Here's a challenge. Read the spec & write it without having others >> do your homework. >> >> http://forth-standard.org/standard/file/Sq >> >> See if you can do it without requiring 2500 lines of novice >> package to be included first. We'll wait. > > You'll wait forever for me to write an ANS-Forth program that > doesn't require NOVICE.4TH to be already loaded. You'll wait for me forever to run it as a consequence. This pebble of code really doesn't need a boulder as ballast. > 2.) They also said this: "An ambiguous condition exists if a \ is > placed before any character, other than those defined in here." They > sure do love the term "ambiguous condition." I just abort with an > error message, because this is an error. Except you then have point 5 below; you treat \q and \Q as the same. > > 4.) I don't support \" because I have \q for that. It seems > confusing to have \" inside of a " delimited string --- I suppose I > could write a smart PARSE that gets the whole string --- I didn't > bother with this. Which makes your code non standard. > > 5.) My tests are case-insensitive --- so \q and \Q are the same. > See point 2 above. Your code would benefit from a table and some serious factoring, rather than a huge if-then-else case-like statement. There are only 26 characters that require checking after '\' and most of them result in a 1 character substitution. -- Alex
[toc] | [prev] | [next] | [standalone]
| From | Julian Fondren <julian.fondren@gmail.com> |
|---|---|
| Date | 2017-04-14 16:51 -0700 |
| Message-ID | <0d2e3dce-7d8d-4d41-ac28-5003553a9d64@googlegroups.com> |
| In reply to | #57333 |
On Friday, April 14, 2017 at 6:24:29 PM UTC-5, hughag...@gmail.com wrote: > On Friday, April 14, 2017 at 3:35:55 PM UTC-7, Julian Fondren wrote: > > On Friday, April 14, 2017 at 4:46:18 PM UTC-5, hughag...@gmail.com wrote: > > > These ANS-Forth experts seem to know almost nothing about ANS-Forth! Code that is trivial to write in ANS-Forth they expect the compiler-writer to provide to them as a non-standard extension. > > > > S\" is a very widely implemented standard extension that is also > > pretty easy to implement on the Forths I've found that lack it. > > It may be easy to implement, but you haven't done so https://bitbucket.org/demonview/forth-httpd/src/3c02dfda761b7aa91c2eb7e1e32c657853fd8a34/ciforth/compat.fs Why would you say this so confidently? Please lay down on this comfortable chair and explain.
[toc] | [prev] | [next] | [standalone]
| From | rickman <gnuarm@gmail.com> |
|---|---|
| Date | 2017-04-14 21:24 -0400 |
| Message-ID | <ocrsil$upn$4@dont-email.me> |
| In reply to | #57333 |
On 4/14/2017 7:24 PM, hughaguilar96@gmail.com wrote: > On Friday, April 14, 2017 at 3:35:55 PM UTC-7, Julian Fondren wrote: >> On Friday, April 14, 2017 at 4:46:18 PM UTC-5, hughag...@gmail.com wrote: >>> These ANS-Forth experts seem to know almost nothing about ANS-Forth! Code that is trivial to write in ANS-Forth they expect the compiler-writer to provide to them as a non-standard extension. >> >> S\" is a very widely implemented standard extension that is also >> pretty easy to implement on the Forths I've found that lack it. > > It may be easy to implement, but you haven't done so --- you are waiting for somebody smart to implement it for you. > > I don't know what S\" does --- if anybody knows what this does and can describe it, I will try to write it --- if anybody has already written it, and their code is ANS-Forth, I'll put it in the novice-package. Here you go... from a non-Forth programmer... 6.2.2266 S\" “s-backslash-quote” CORE EXT Interpretation: Interpretation semantics for this word are undefined. Compilation: ( “ccchquotei” -- ) Parse ccc delimited by " (double-quote), using the translation rules below. Append the run-time semantics given below to the current definition. Translation rules: Characters are processed one at a time and appended to the compiled string. If the char- acter is a ‘\’ character it is processed by parsing and substituting one or more characters as follows, where the character after the backslash is case sensitive: \a BEL (alert, ASCII 7) \b BS (backspace, ASCII 8) \e ESC (escape, ASCII 27) \f FF (form feed, ASCII 12) \l LF (line feed, ASCII 10) \m CR/LF pair (ASCII 13, 10) \n newline (implementation dependent , e.g., CR/LF, CR, LF, LF/CR) \q double-quote (ASCII 34) \r CR (carriage return, ASCII 13) \t HT (horizontal tab, ASCII 9) \v VT (vertical tab, ASCII 11) \z NUL (no character, ASCII 0) \" double-quote (ASCII 34) \xhhexdigitihhexdigiti The resulting character is the conversion of these two hexadecimal digits. An ambiguous conditions exists if \x is not followed by two hexadecimal characters. \\ backslash itself (ASCII 92) An ambiguous condition exists if a \ is placed before any character, other than those defined in here. Run-time: ( -- c-addr u ) Return c-addr and u describing a string consisting of the translation of the characters ccc. A program shall not alter the returned string. See: 3.4.1 Parsing, 6.2.0855 C", 11.6.1.2165 S", A.6.1.2165 S". Sorry the formatting is lost. -- Rick C
[toc] | [prev] | [standalone]
Page 3 of 3 — ← Prev page 1 2 [3]
Back to top | Article view | comp.lang.forth
csiph-web