Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.lang.forth > #35894 > unrolled thread

Counted string

Started byMarek Brunda <marekb@phoenics.org>
First post2015-01-18 23:54 -0800
Last post2015-01-26 11:25 -1000
Articles 20 on this page of 31 — 16 participants

Back to article view | Back to comp.lang.forth


Contents

  Counted string Marek Brunda <marekb@phoenics.org> - 2015-01-18 23:54 -0800
    Re: Counted string "Elizabeth D. Rather" <erather@forth.com> - 2015-01-18 22:08 -1000
      Re: Counted string anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2015-01-19 09:36 +0000
        Re: Counted string "HAA" <someone@microsoft.com> - 2015-01-20 00:21 +1100
          Re: Counted string Coos Haak <chforth@hccnet.nl> - 2015-01-19 14:44 +0100
        Re: Counted string Raimond Dragomir <raimond.dragomir@gmail.com> - 2015-01-20 00:45 -0800
          Re: Counted string anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2015-01-20 13:12 +0000
            Re: Counted string Raimond Dragomir <raimond.dragomir@gmail.com> - 2015-01-21 01:05 -0800
              Re: Counted string albert@spenarnc.xs4all.nl (Albert van der Horst) - 2015-01-21 10:54 +0000
      Re: Counted string lehs <skydda.barnen@gmail.com> - 2015-01-30 11:18 -0800
    Re: Counted string Matthias Koch <matthias.koch@hot.uni-hannover.de> - 2015-01-21 11:58 +0100
      Re: Counted string "HAA" <someone@microsoft.com> - 2015-01-22 11:49 +1100
    Re: Counted string "Rod Pemberton" <buzz@notnotnontnooatno.cmm> - 2015-01-23 20:17 -0500
      Re: Counted string Marek Brunda <marekb@phoenics.org> - 2015-01-26 11:21 -0800
        Re: Counted string Mark Wills <markwills1970@gmail.com> - 2015-01-26 12:11 -0800
          Re: Counted string "Rod Pemberton" <buzz@notnotnontnooatno.cmm> - 2015-01-26 21:55 -0500
            Re: Counted string Julian Fondren <julian.fondren@gmail.com> - 2015-01-26 19:46 -0800
              Re: Counted string albert@spenarnc.xs4all.nl (Albert van der Horst) - 2015-01-27 12:07 +0000
                Re: Counted string "Rod Pemberton" <buzz@notnotnontnooatno.cmm> - 2015-01-28 20:13 -0500
              Re: Counted string anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2015-01-27 15:43 +0000
                Re: Counted string Marek Brunda <marekb@phoenics.org> - 2015-01-27 13:28 -0800
                  Re: Counted string Elizabeth D Rather <erather@forth.com> - 2015-01-27 13:06 -1000
                  Re: Counted string anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2015-01-30 17:33 +0000
              Re: Counted string "Rod Pemberton" <buzz@notnotnontnooatno.cmm> - 2015-01-28 20:07 -0500
                Re: Counted string "Rod Pemberton" <buzz@notnotnontnooatno.cmm> - 2015-01-28 20:16 -0500
                Re: Counted string Coos Haak <chforth@hccnet.nl> - 2015-01-29 03:14 +0100
        Re: Counted string Pablo Hugo Reda <pabloreda@gmail.com> - 2015-01-26 12:14 -0800
          Re: Counted string "Rod Pemberton" <buzz@notnotnontnooatno.cmm> - 2015-01-26 21:55 -0500
          Re: Counted string Ron Aaron <rambamist@gmail.com> - 2015-01-27 06:55 +0200
            Re: Counted string "Alex McDonald" <blog@rivadpm.com> - 2015-01-27 14:46 +0000
        Re: Counted string "Elizabeth D. Rather" <erather@forth.com> - 2015-01-26 11:25 -1000

Page 1 of 2  [1] 2  Next page →


#35894 — Counted string

FromMarek Brunda <marekb@phoenics.org>
Date2015-01-18 23:54 -0800
SubjectCounted string
Message-ID<174b22de-2454-4e17-ac18-6a1333ec408f@googlegroups.com>
Hi, can anyone explain me, what's wrong with counted strings? Maybe it's a stupid question, but I've never understood it. Thanks

[toc] | [next] | [standalone]


#35895

From"Elizabeth D. Rather" <erather@forth.com>
Date2015-01-18 22:08 -1000
Message-ID<pYydnUq72P_8JiHJnZ2dnUU7-S-dnZ2d@supernews.com>
In reply to#35894
On 1/18/15 9:54 PM, Marek Brunda wrote:
> Hi, can anyone explain me, what's wrong with counted strings? Maybe it's a stupid question, but I've never understood it. Thanks
>

Some people feel that allocating only one byte for the count, which has 
the effect of limiting strings kept in this format to 255 bytes, is 
inauspicious. My experience is that instances requiring longer strings 
are pretty rare, and easily dealt with by adding operators as necessary 
to handle longer strings, or declaring strings on this implementation to 
have longer count fields. The first approach adds words, the second 
costs bytes for every counted string in the system, which I personally 
feel is much worse, as it burdens systems with limited memory resources.

The current operating compromise is the convention that one passes 
strings using an addr/count pair on the stack, leaving the internal 
format of stings as an implementation dependency except for the few 
cases where it's specified.

Cheers,
Elizabeth

-- 
==================================================
Elizabeth D. Rather   (US & Canada)   800-55-FORTH
FORTH Inc.                         +1 310.999.6784
5959 West Century Blvd. Suite 700
Los Angeles, CA 90045
http://www.forth.com

"Forth-based products and Services for real-time
applications since 1973."
==================================================

[toc] | [prev] | [next] | [standalone]


#35896

Fromanton@mips.complang.tuwien.ac.at (Anton Ertl)
Date2015-01-19 09:36 +0000
Message-ID<2015Jan19.103605@mips.complang.tuwien.ac.at>
In reply to#35895
"Elizabeth D. Rather" <erather@forth.com> writes:
>On 1/18/15 9:54 PM, Marek Brunda wrote:
>> Hi, can anyone explain me, what's wrong with counted strings? Maybe it's a stupid question, but I've never understood it. Thanks
>>
>
>Some people feel that allocating only one byte for the count, which has 
>the effect of limiting strings kept in this format to 255 bytes, is 
>inauspicious.

That's just one of the drawbacks of standard counted strings.  That
could be fixed with a count cell instead of a count byte, but the main
other disadvantage would still remain:

If you need a substring of a counted string (returned as a counted
string), you have to put the substring in a new buffer.  One example
of this is when parsing the Forth input.  Word creates a counted
string, so it copies the substring to (usually) HERE; the next WORD
overwrites this.  In contrast, with the c-addr u representation (e.g.,
PARSE-NAME), you can refer to the substring in the input buffer; this
saves time, and allows having several parsed strings at the same time.

>My experience is that instances requiring longer strings 
>are pretty rare, and easily dealt with by adding operators as necessary 
>to handle longer strings, or declaring strings on this implementation to 
>have longer count fields.

All unnecessary with c-addr u strings.

>The current operating compromise is the convention that one passes 
>strings using an addr/count pair on the stack, leaving the internal 
>format of stings as an implementation dependency except for the few 
>cases where it's specified.

The internal representation of c-addr u strings is completely
specified.  It's just an array of chars.

- anton
-- 
M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
     New standard: http://www.forth200x.org/forth200x.html
   EuroForth 2014: http://www.euroforth.org/ef14/

[toc] | [prev] | [next] | [standalone]


#35899

From"HAA" <someone@microsoft.com>
Date2015-01-20 00:21 +1100
Message-ID<m9j09v$gld$1@speranza.aioe.org>
In reply to#35896
Anton Ertl wrote:
> "Elizabeth D. Rather" <erather@forth.com> writes:
> [...]
> >The current operating compromise is the convention that one passes
> >strings using an addr/count pair on the stack, leaving the internal
> >format of stings as an implementation dependency except for the few
> >cases where it's specified.
>
> The internal representation of c-addr u strings is completely
> specified. It's just an array of chars.

External representation perhaps.

"Counted strings remain useful as a way to store strings in memory.
This use is not discouraged, ..."


[toc] | [prev] | [next] | [standalone]


#35900

FromCoos Haak <chforth@hccnet.nl>
Date2015-01-19 14:44 +0100
Message-ID<358yrzephkoa.lxmsnvf52civ.dlg@40tude.net>
In reply to#35899
Op Tue, 20 Jan 2015 00:21:01 +1100 schreef HAA:

> Anton Ertl wrote:
>> "Elizabeth D. Rather" <erather@forth.com> writes:
>> [...]
>>>The current operating compromise is the convention that one passes
>>>strings using an addr/count pair on the stack, leaving the internal
>>>format of stings as an implementation dependency except for the few
>>>cases where it's specified.
>>
>> The internal representation of c-addr u strings is completely
>> specified. It's just an array of chars.
> 
> External representation perhaps.
> 
> "Counted strings remain useful as a way to store strings in memory.
> This use is not discouraged, ..."

Concatenating strings like:
"before" "after" concat type -> "beforeafter"
"after" "before" strcat type -> "beforeafter"
that have their lenghts on the stack is easier than with counted strings.

-- 
Coos

CHForth, 16 bit DOS applications
http://home.hccnet.nl/j.j.haak/forth.html 

[toc] | [prev] | [next] | [standalone]


#35914

FromRaimond Dragomir <raimond.dragomir@gmail.com>
Date2015-01-20 00:45 -0800
Message-ID<d7979ef9-2005-48cc-b41f-dcbfcf507423@googlegroups.com>
In reply to#35896
> That's just one of the drawbacks of standard counted strings.  That
> could be fixed with a count cell instead of a count byte, but the main
> other disadvantage would still remain:
> 
> If you need a substring of a counted string (returned as a counted
> string), you have to put the substring in a new buffer.  One example
> of this is when parsing the Forth input.  Word creates a counted
> string, so it copies the substring to (usually) HERE; the next WORD
> overwrites this.  In contrast, with the c-addr u representation (e.g.,
> PARSE-NAME), you can refer to the substring in the input buffer; this
> saves time, and allows having several parsed strings at the same time.

Those are just memory buffers, not "strings". String is a complex type, defined by two information: the start address, and the length. If you want strings you have to deal with this complex type. C has another type of "string", that one with 0 termination, with pros and cons.

You are basically saying that the "string" type is more complex that it should be for that particularly usage, so better use plain buffers instead.
But what if we want to use real strings? How can you store a string in memory for later usage? Following your logic, given the buffer content already somewhere in memory, you will need two further cells to store the addr and the length in memory. So the wonderfull "c-addr u" forth strings would end having two cells overhead instead of just one (the count) or just one char (the limited 255 max count).


Forth has here another problem: all the forth philosophy about the memory uasge is to deal with address of the object, then @ and ! the content. CREATE returns only the address, in the line of this philosophy. So, how would you create a "c-addr u" string at compile time and later use it as a "c-addr u" string? 

> 
> >My experience is that instances requiring longer strings 
> >are pretty rare, and easily dealt with by adding operators as necessary 
> >to handle longer strings, or declaring strings on this implementation to 
> >have longer count fields.
> 
> All unnecessary with c-addr u strings.
> 
> >The current operating compromise is the convention that one passes 
> >strings using an addr/count pair on the stack, leaving the internal 
> >format of stings as an implementation dependency except for the few 
> >cases where it's specified.
> 
> The internal representation of c-addr u strings is completely
> specified.  It's just an array of chars.
> 

Yup, and who is giving you the length when you really want to use it?
The internal representation is referring to how the count/length is stored in memory (or the string format in itself if a terminator is used).

The sad truth is that Forth has only one string type: the old counted string, with the obvious limitation of 255 chars. "c-addr u" is not a string, it's just a method of passing memory buffers at runtime. The blamed C has this too, memcpy(), memset(), etc. are exactly the same things, operating on "c-addr u" memory buffers.


> - anton
> -- 
> M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
> comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
>      New standard: http://www.forth200x.org/forth200x.html
>    EuroForth 2014: http://www.euroforth.org/ef14/

[toc] | [prev] | [next] | [standalone]


#35919

Fromanton@mips.complang.tuwien.ac.at (Anton Ertl)
Date2015-01-20 13:12 +0000
Message-ID<2015Jan20.141228@mips.complang.tuwien.ac.at>
In reply to#35914
Raimond Dragomir <raimond.dragomir@gmail.com> writes:
>> That's just one of the drawbacks of standard counted strings.  That
>> could be fixed with a count cell instead of a count byte, but the main
>> other disadvantage would still remain:
>>=20
>> If you need a substring of a counted string (returned as a counted
>> string), you have to put the substring in a new buffer.  One example
>> of this is when parsing the Forth input.  Word creates a counted
>> string, so it copies the substring to (usually) HERE; the next WORD
>> overwrites this.  In contrast, with the c-addr u representation (e.g.,
>> PARSE-NAME), you can refer to the substring in the input buffer; this
>> saves time, and allows having several parsed strings at the same time.
>
>Those are just memory buffers, not "strings".

Buffers are memory areas where you might store, e.g., a string.
Strings are memory where strings are actually stored.  Both are
described by "addr u" pairs, but that does not mean that "addr u"
describes a buffer.

 E.g.,

create buf 80 allot

produces a buffer that can contain 80 chars.  After

s" abc" buf swap move

you have a string "buf 3", and "buf 80" still describes the buffer.

>You are basically saying that the "string" type is more complex that it sho=
>uld be for that particularly usage, so better use plain buffers instead.
>But what if we want to use real strings? How can you store a string in memo=
>ry for later usage? Following your logic, given the buffer content already =
>somewhere in memory, you will need two further cells to store the addr and =
>the length in memory. So the wonderfull "c-addr u" forth strings would end =
>having two cells overhead instead of just one (the count) or just one char =
>(the limited 255 max count).

If you want to store an "addr u" descriptor somewhere, then yes, you
need two cells for that.  If you have a cell-counted string, you are
not any better of: You need the count cell at the start of the string,
and the address cell that you store where one stores two cells with
the addr u representation.

You might now argue that storing the descriptor for the same string in
n places requires n copies of the length, while the cell-counted
version stores the length only once.  True, that's the advantage you
get from not being able to share the characters for substrings.  I
consider the substring usage more important.

>Forth has here another problem: all the forth philosophy about the memory u=
>asge is to deal with address of the object, then @ and ! the content. CREAT=
>E returns only the address, in the line of this philosophy. So, how would y=
>ou create a "c-addr u" string at compile time and later use it as a "c-addr=
> u" string?=20

: bar s" abc" ;
bar type

Is that what you were asking for.  If not, please supply a clear
problem description.

>> The internal representation of c-addr u strings is completely
>> specified.  It's just an array of chars.
>>=20
>
>Yup, and who is giving you the length when you really want to use it?

As a programmer, it is my responsibility to provide both address and
length to the consumers.

>The internal representation is referring to how the count/length is stored =
>in memory (or the string format in itself if a terminator is used).

I use a cell pair to store a string descriptor, and use 2@ and 2! for
access.

>The blamed C has thi=
>s too, memcpy(), memset(), etc. are exactly the same things, operating on "=
>c-addr u" memory buffers.

But C string literals don't produce this representation, and C does
not allow to produce an addr length pair as result of a function that
could be directly used by functions like write() or fwrite().  In
Forth you have words that produce addr u and that can be directly
passed to consumers of addr u strings.  E.g.:

s" abcdefghi" s" cd" search . type

(Forth-2012 FILE S" assumed).

- anton
-- 
M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
     New standard: http://www.forth200x.org/forth200x.html
   EuroForth 2014: http://www.euroforth.org/ef14/

[toc] | [prev] | [next] | [standalone]


#35940

FromRaimond Dragomir <raimond.dragomir@gmail.com>
Date2015-01-21 01:05 -0800
Message-ID<135afdc7-81f0-4143-baa6-2601b1bd8b8a@googlegroups.com>
In reply to#35919
> : bar s" abc" ;
> bar type
> 
Yes, this simple code describes that the forth is not having support for a "c-addr u" string at all. You ultimately had to define a runtime word for a "c-addr u" string. This is the most inefficient way of dealing with the string. You have to call/execute "bar" to get the string information. Is there any chance in any forth compiler that the "bar" to be inlined? Better yet, being the "c-addr" and "u" both constants, is there any chance that those may be further involved in the literal optimization in the compiling process? I doubt it. Maybe VFX?

The only word forth has for "c-addr u" strings is S". This can hardly be called support for "c-addr u" strings.

I know that in old forths, even VARIABLEs were calls, not inline constants, but we are in 2015, not 70s or 80s ...

[toc] | [prev] | [next] | [standalone]


#35945

Fromalbert@spenarnc.xs4all.nl (Albert van der Horst)
Date2015-01-21 10:54 +0000
Message-ID<54bf8567$0$6926$e4fe514c@dreader36.news.xs4all.nl>
In reply to#35940
In article <135afdc7-81f0-4143-baa6-2601b1bd8b8a@googlegroups.com>,
Raimond Dragomir  <raimond.dragomir@gmail.com> wrote:
>> : bar s" abc" ;
>> bar type
>>
>Yes, this simple code describes that the forth is not having support for
>a "c-addr u" string at all. You ultimately had to define a runtime word
>for a "c-addr u" string. This is the most inefficient way of dealing
>with the string. You have to call/execute "bar" to get the string
>information. Is there any chance in any forth compiler that the "bar" to
>be inlined? Better yet, being the "c-addr" and "u" both constants, is
>there any chance that those may be further involved in the literal
>optimization in the compiling process? I doubt it. Maybe VFX?
>
>The only word forth has for "c-addr u" strings is S". This can hardly be
>called support for "c-addr u" strings.
>
>I know that in old forths, even VARIABLEs were calls, not inline
>constants, but we are in 2015, not 70s or 80s ...

The proof is in the pudding. In the end a compiler is a string manipulator.
One look at the implementation of yourforth should convince everybody
that using addr,len strings consequently (at the expense of some ISO
compatibility) leads to the simplest Forth compiler possible, which
means the simplest compiler, full stop.

So, adr,len strings are the best solution for the problem how to write
the simplest compiler possible. That means a lot (or most) of the
things you say can't be true.

Groetjes Albert
-- 
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

[toc] | [prev] | [next] | [standalone]


#36168

Fromlehs <skydda.barnen@gmail.com>
Date2015-01-30 11:18 -0800
Message-ID<13fef831-c733-45be-96ee-7f9c8f818a82@googlegroups.com>
In reply to#35895
Den måndag 19 januari 2015 kl. 09:08:02 UTC+1 skrev Elizabeth D. Rather:
> On 1/18/15 9:54 PM, Marek Brunda wrote:
> > Hi, can anyone explain me, what's wrong with counted strings? Maybe it's a stupid question, but I've never understood it. Thanks
> >
> 
> Some people feel that allocating only one byte for the count, which has 
> the effect of limiting strings kept in this format to 255 bytes, is 
> inauspicious. My experience is that instances requiring longer strings 
> are pretty rare, and easily dealt with by adding operators as necessary 
> to handle longer strings, or declaring strings on this implementation to 
> have longer count fields. The first approach adds words, the second 
> costs bytes for every counted string in the system, which I personally 
> feel is much worse, as it burdens systems with limited memory resources.
> 
> The current operating compromise is the convention that one passes 
> strings using an addr/count pair on the stack, leaving the internal 
> format of stings as an implementation dependency except for the few 
> cases where it's specified.
> 
> Cheers,
> Elizabeth
> 
> -- 
> ==================================================
> Elizabeth D. Rather   (US & Canada)   800-55-FORTH
> FORTH Inc.                         +1 310.999.6784
> 5959 West Century Blvd. Suite 700
> Los Angeles, CA 90045
> http://www.forth.com
> 
> "Forth-based products and Services for real-time
> applications since 1973."
> ==================================================

I have just about the same opinion. Most words dealing with strings has stack diagram of the type ( add n -- ) where n is a full length integer and only a few words has to be defined to handle strings of other types than the standard. And since Forth is Forth everything is so easy to customize. I sometimes use a string stack for dynamical strings and this works fine, even if I mostly use it to store dynamical big integer.

[toc] | [prev] | [next] | [standalone]


#35947

FromMatthias Koch <matthias.koch@hot.uni-hannover.de>
Date2015-01-21 11:58 +0100
Message-ID<m9o0sb$ber$1@newsserver.rrzn.uni-hannover.de>
In reply to#35894
> Hi, can anyone explain me, what's wrong with counted strings? Maybe it's a stupid question, but I've never understood it. Thanks

With addr-length strings it is possible to get a token or cut something else out without the need to change the underlying memory. Manipulation of counted strings always need a change in the buffer itself. If this is no concern, then both can be equally fine.

Matthias

[toc] | [prev] | [next] | [standalone]


#35954

From"HAA" <someone@microsoft.com>
Date2015-01-22 11:49 +1100
Message-ID<m9phcl$q7e$1@speranza.aioe.org>
In reply to#35947
Matthias Koch wrote:
> > Marek Brunda wrote:
> > Hi, can anyone explain me, what's wrong with counted strings? Maybe it's a stupid
> > question, but I've never understood it. Thanks
>
> With addr-length strings it is possible to get a token or cut something else out
> without the need to change the underlying memory. Manipulation of counted strings
> always need a change in the buffer itself. If this is no concern, then both can be
> equally fine.

There are no "address length strings" in ANS Forth - only "address length" operations.
Retrieving and storing strings is left to the programmer.


[toc] | [prev] | [next] | [standalone]


#35982

From"Rod Pemberton" <buzz@notnotnontnooatno.cmm>
Date2015-01-23 20:17 -0500
Message-ID<op.xsx0jbi0v6dtjm@localhost>
In reply to#35894
On Mon, 19 Jan 2015 02:54:49 -0500, Marek Brunda <marekb@phoenics.org>  
wrote:

> Hi, can anyone explain me, what's wrong with counted strings? Maybe it's  
> a stupid question, but I've never understood it. Thanks

This has been discussed many times here in the past.
You can search the Google Groups archive of this Usenet group.

Nul-terminated strings replaced counted strings for the most part
many decades ago in most programming languages, e.g., 1970's or 1980's.

The problem back then with counted strings was that the count could
be changed, but the length of the string wouldn't be.  I.e., there
is a mismatch in the size of the count and the actual length of the
string.  If the count is too short, then the string is truncated.
This isn't a problem.  If the count is too large, then you have a
very serious issue.  More space for a new larger string wasn't
allocated.  The string is the same size, but the count is larger.
So, the oversized count value allows you to access data PAST the
actual end-of-string.  I.e., you can read data you shouldn't have
access to, or write to variables you shouldn't have access to.
This is the same technique used by hackers for buffer overflow.

(FYI, to prevent flame war, there are those here who claim that
can't actually happen in Forth ...)

Languages like C which use nul-terminated strings frequently
only need to detect the end-of-string.  They generally don't
need to know the string's length.  So, not counting the length
of a string and just appending a nul-terminator to create a
string increased code speed.  Of course, not terminating
a string in C, which happens with some string functions under
certain conditions, can cause overruns or unintended access too.


Rod Pemberton

[toc] | [prev] | [next] | [standalone]


#36045

FromMarek Brunda <marekb@phoenics.org>
Date2015-01-26 11:21 -0800
Message-ID<d55e2379-92cb-454c-bb04-059c86ab8eb6@googlegroups.com>
In reply to#35982
Dne sobota 24. ledna 2015 2:17:25 UTC+1 Rod Pemberton napsal(a):
> On Mon, 19 Jan 2015 02:54:49 -0500, Marek Brunda <marekb@phoenics.org>  
> wrote:
> 
> > Hi, can anyone explain me, what's wrong with counted strings? Maybe it's  
> > a stupid question, but I've never understood it. Thanks
> 
> This has been discussed many times here in the past.
> You can search the Google Groups archive of this Usenet group.
> 
> Nul-terminated strings replaced counted strings for the most part
> many decades ago in most programming languages, e.g., 1970's or 1980's.
> 
> The problem back then with counted strings was that the count could
> be changed, but the length of the string wouldn't be.  I.e., there
> is a mismatch in the size of the count and the actual length of the
> string.  If the count is too short, then the string is truncated.
> This isn't a problem.  If the count is too large, then you have a
> very serious issue.  More space for a new larger string wasn't
> allocated.  The string is the same size, but the count is larger.
> So, the oversized count value allows you to access data PAST the
> actual end-of-string.  I.e., you can read data you shouldn't have
> access to, or write to variables you shouldn't have access to.
> This is the same technique used by hackers for buffer overflow.
> 
> (FYI, to prevent flame war, there are those here who claim that
> can't actually happen in Forth ...)
> 
> Languages like C which use nul-terminated strings frequently
> only need to detect the end-of-string.  They generally don't
> need to know the string's length.  So, not counting the length
> of a string and just appending a nul-terminator to create a
> string increased code speed.  Of course, not terminating
> a string in C, which happens with some string functions under
> certain conditions, can cause overruns or unintended access too.
> 
> 
> Rod Pemberton

I read all the posts and I understand the arguments. So why not use NULL terminated strings in Forth?

[toc] | [prev] | [next] | [standalone]


#36050

FromMark Wills <markwills1970@gmail.com>
Date2015-01-26 12:11 -0800
Message-ID<ec0e2ec8-fc53-41a1-9a16-0d5d151c527e@googlegroups.com>
In reply to#36045
A number of reasons:

1) you can't have ASCII code 0 in your string (not without escape codes)
2) you have to walk the entire length of a string just to get its length
3) copying a string from buffer a to buffer b would require the string to be walked twice: once to get its length (so that you can allocate buffer b) and a second time to do the actual copy.

How dumb is that? 

You know how we look back at flared trousers and wide collar shirts open to the navel, and we think to ourselves "how could they *do* that?!"? 

So it will be with null terminated strings.

[toc] | [prev] | [next] | [standalone]


#36065

From"Rod Pemberton" <buzz@notnotnontnooatno.cmm>
Date2015-01-26 21:55 -0500
Message-ID<op.xs3o14liv6dtjm@localhost>
In reply to#36050
On Mon, 26 Jan 2015 15:11:30 -0500, Mark Wills <markwills1970@gmail.com>  
wrote:

> A number of reasons:
>
> 1) you can't have ASCII code 0 in your string (not without escape codes)

If you're using ASCII, you would never need ASCII code 0 in you string.
If you're constructing Unicode using ASCII, then maybe ...

> 2) you have to walk the entire length of a string just to get its length

That must be done in Forth too to get the length in the first place.
It's true that you don't need to walk the entire length of the string,
but Forth is doing that for you.

Just how often do you need the length of a string in Forth?
I.e., once it's stored, you'd only need the length when copied.
You only need to copy if you intend to modify.  For any read only
or display ccess, you just need it's address.

> 3) copying a string from buffer a to buffer b would require the string  
> to be walked twice: once to get its length (so that you can allocate  
> buffer b) and a second time to do the actual copy.

In Forth, you could save the size of "buffer a" as a variable or constant.
That would eliminate this problem.  ISTM, this is a rather low occurance
anyway.


Rod Pemberton

[toc] | [prev] | [next] | [standalone]


#36066

FromJulian Fondren <julian.fondren@gmail.com>
Date2015-01-26 19:46 -0800
Message-ID<6a3a8959-4e74-42f8-a050-069e924fdfe2@googlegroups.com>
In reply to#36065
On Monday, January 26, 2015 at 8:55:05 PM UTC-6, Rod Pemberton wrote:
> On Mon, 26 Jan 2015 15:11:30 -0500, Mark Wills <markwills1970@gmail.com>  
> wrote:
> > 2) you have to walk the entire length of a string just to get its length
> 
> That must be done in Forth too to get the length in the first place.
> It's true that you don't need to walk the entire length of the string,
> but Forth is doing that for you.

Huh?  No.  You seem to think that C's ways of doings things are how
things are actually done.  In unix-speak, You should read section 2 of
the manpages, instead of just section 3.  The fundamental I/O functions
all work with ANS Forth strings, basically.  They demand buffers with
lengths.  They write arbitrary, '\0'-including data into buffers and
then return a count of data written.

http://linux.die.net/man/2/write
http://linux.die.net/man/2/read
http://linux.die.net/man/2/mmap

C's fussing about with finding and setting the length is extra,
unnecessary effort; there's no "in the first place" about it.


-- Julian

[toc] | [prev] | [next] | [standalone]


#36073

Fromalbert@spenarnc.xs4all.nl (Albert van der Horst)
Date2015-01-27 12:07 +0000
Message-ID<54c77f9e$0$6894$e4fe514c@dreader36.news.xs4all.nl>
In reply to#36066
In article <6a3a8959-4e74-42f8-a050-069e924fdfe2@googlegroups.com>,
Julian Fondren  <julian.fondren@gmail.com> wrote:
>On Monday, January 26, 2015 at 8:55:05 PM UTC-6, Rod Pemberton wrote:
>> On Mon, 26 Jan 2015 15:11:30 -0500, Mark Wills <markwills1970@gmail.com>
>> wrote:
>> > 2) you have to walk the entire length of a string just to get its length
>>
>> That must be done in Forth too to get the length in the first place.
>> It's true that you don't need to walk the entire length of the string,
>> but Forth is doing that for you.
>
>Huh?  No.  You seem to think that C's ways of doings things are how
>things are actually done.  In unix-speak, You should read section 2 of
>the manpages, instead of just section 3.  The fundamental I/O functions
>all work with ANS Forth strings, basically.  They demand buffers with
>lengths.  They write arbitrary, '\0'-including data into buffers and
>then return a count of data written.
>
>http://linux.die.net/man/2/write
>http://linux.die.net/man/2/read
>http://linux.die.net/man/2/mmap
>
>C's fussing about with finding and setting the length is extra,
>unnecessary effort; there's no "in the first place" about it.

Or Rod could study yourforth. It is simple and unencumbered with
C-strings. The only places where c-strings are used is for
passing to linux.
Strings that are passed to linux use the word ZEN ( "zrrren")
an abbreviation of ZERO-END
"
ZEN      Stack effect: sc -- addr
Leaves an address that contains a zero-ended (c-type) equivalent of
sc. The same buffer is reused, such that this word is not reentrant.
Use the word immediately, e.g. its intended used is passing parameters
to the operating system. In fact this is RW-BUFFER .
"
(A sc stringconstant is of course an address len pair).

It is used like so:

( sc flags -- fd errorcode )
: OPEN-FILE >R ZEN R> _ 5 XOS DUP 0 MIN ;

Not that I hope to educate Rod, but others may find it illuminating
that c-strings are only present at the outer borders of the Forth.
Linux doesnot use zero ended strings all the time:
"
: DSSWAP ROT ROT ;
: TYPE   1 DSSWAP 4 XOS 0 MIN THROW ;
"
The write operating system call (4) accepts a regular Forth
string constant.

Groetjes Albert

>
>
>-- Julian
-- 
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

[toc] | [prev] | [next] | [standalone]


#36101

From"Rod Pemberton" <buzz@notnotnontnooatno.cmm>
Date2015-01-28 20:13 -0500
Message-ID<op.xs69otcbv6dtjm@localhost>
In reply to#36073
On Tue, 27 Jan 2015 07:07:58 -0500, Albert van der Horst  
<albert@spenarnc.xs4all.nl> wrote:

> Or Rod could study yourforth. It is simple and unencumbered with
> C-strings. The only places where c-strings are used is for passing
> to linux. Strings that are passed to linux use the word ZEN ( "zrrren")
> an abbreviation of ZERO-END

As mentioned elsewhere, I already use C style strings in my Forth,
everywhere.  It's built right into the inner interpreter.  Why would
I want to limit their use to only FFIs or host-specific interfaces?
Most of Hayes core and core ext is implemented.  Also, it only
takes one byte and a change to one or two Forth words to convert
to counted strings.  BTW, it's coded in ANSI C, using GCC for DOS,
i.e., DJGPP, but should compile cleanly for Linux, except for maybe
one C function.  Currently, most higher level Forth words are coded
in Forth, i.e., only primitives, stacks, and system variables are in C.


Rod Pemberton

[toc] | [prev] | [next] | [standalone]


#36079

Fromanton@mips.complang.tuwien.ac.at (Anton Ertl)
Date2015-01-27 15:43 +0000
Message-ID<2015Jan27.164302@mips.complang.tuwien.ac.at>
In reply to#36066
Julian Fondren <julian.fondren@gmail.com> writes:
>Huh?  No.  You seem to think that C's ways of doings things are how
>things are actually done.  In unix-speak, You should read section 2 of
>the manpages, instead of just section 3.  The fundamental I/O functions
>all work with ANS Forth strings, basically.

write() and read() do, but, e.g., open() takes a zero-terminated
string.  That's not surprising, after C was developed to implement
Unix, and it's no wonder the same string formats are used for both.

- anton
-- 
M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
     New standard: http://www.forth200x.org/forth200x.html
   EuroForth 2014: http://www.euroforth.org/ef14/

[toc] | [prev] | [next] | [standalone]


Page 1 of 2  [1] 2  Next page →

Back to top | Article view | comp.lang.forth


csiph-web