Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.forth > #35894 > unrolled thread

Counted string

Back to article view | Back to comp.lang.forth

Contents

  Counted string Marek Brunda <marekb@phoenics.org> - 2015-01-18 23:54 -0800
    Re: Counted string "Elizabeth D. Rather" <erather@forth.com> - 2015-01-18 22:08 -1000
      Re: Counted string anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2015-01-19 09:36 +0000
        Re: Counted string "HAA" <someone@microsoft.com> - 2015-01-20 00:21 +1100
          Re: Counted string Coos Haak <chforth@hccnet.nl> - 2015-01-19 14:44 +0100
        Re: Counted string Raimond Dragomir <raimond.dragomir@gmail.com> - 2015-01-20 00:45 -0800
          Re: Counted string anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2015-01-20 13:12 +0000
            Re: Counted string Raimond Dragomir <raimond.dragomir@gmail.com> - 2015-01-21 01:05 -0800
              Re: Counted string albert@spenarnc.xs4all.nl (Albert van der Horst) - 2015-01-21 10:54 +0000
      Re: Counted string lehs <skydda.barnen@gmail.com> - 2015-01-30 11:18 -0800
    Re: Counted string Matthias Koch <matthias.koch@hot.uni-hannover.de> - 2015-01-21 11:58 +0100
      Re: Counted string "HAA" <someone@microsoft.com> - 2015-01-22 11:49 +1100
    Re: Counted string "Rod Pemberton" <buzz@notnotnontnooatno.cmm> - 2015-01-23 20:17 -0500
      Re: Counted string Marek Brunda <marekb@phoenics.org> - 2015-01-26 11:21 -0800
        Re: Counted string Mark Wills <markwills1970@gmail.com> - 2015-01-26 12:11 -0800
          Re: Counted string "Rod Pemberton" <buzz@notnotnontnooatno.cmm> - 2015-01-26 21:55 -0500
            Re: Counted string Julian Fondren <julian.fondren@gmail.com> - 2015-01-26 19:46 -0800
              Re: Counted string albert@spenarnc.xs4all.nl (Albert van der Horst) - 2015-01-27 12:07 +0000
                Re: Counted string "Rod Pemberton" <buzz@notnotnontnooatno.cmm> - 2015-01-28 20:13 -0500
              Re: Counted string anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2015-01-27 15:43 +0000
                Re: Counted string Marek Brunda <marekb@phoenics.org> - 2015-01-27 13:28 -0800
                  Re: Counted string Elizabeth D Rather <erather@forth.com> - 2015-01-27 13:06 -1000
                  Re: Counted string anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2015-01-30 17:33 +0000
              Re: Counted string "Rod Pemberton" <buzz@notnotnontnooatno.cmm> - 2015-01-28 20:07 -0500
                Re: Counted string "Rod Pemberton" <buzz@notnotnontnooatno.cmm> - 2015-01-28 20:16 -0500
                Re: Counted string Coos Haak <chforth@hccnet.nl> - 2015-01-29 03:14 +0100
        Re: Counted string Pablo Hugo Reda <pabloreda@gmail.com> - 2015-01-26 12:14 -0800
          Re: Counted string "Rod Pemberton" <buzz@notnotnontnooatno.cmm> - 2015-01-26 21:55 -0500
          Re: Counted string Ron Aaron <rambamist@gmail.com> - 2015-01-27 06:55 +0200
            Re: Counted string "Alex McDonald" <blog@rivadpm.com> - 2015-01-27 14:46 +0000
        Re: Counted string "Elizabeth D. Rather" <erather@forth.com> - 2015-01-26 11:25 -1000

Page 2 of 2 — ← Prev page 1 [2]

#36080

Dne úterý 27. ledna 2015 16:47:27 UTC+1 Anton Ertl napsal(a):
> Julian Fondren <julian.fondren@gmail.com> writes:
> >Huh?  No.  You seem to think that C's ways of doings things are how
> >things are actually done.  In unix-speak, You should read section 2 of
> >the manpages, instead of just section 3.  The fundamental I/O functions
> >all work with ANS Forth strings, basically.
> 
> write() and read() do, but, e.g., open() takes a zero-terminated
> string.  That's not surprising, after C was developed to implement
> Unix, and it's no wonder the same string formats are used for both.
> 
> - anton
> -- 
> M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
> comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
>      New standard: http://www.forth200x.org/forth200x.html
>    EuroForth 2014: http://www.euroforth.org/ef14/


I think you freely interchange a "string" and a "buffer" which holds binary data. write() and read() works with binary data. open() takes a null terminated character string as filename. It's not the same thing. 

If I understand it, ANS Forth dosesn't specify a format of character string. It only specifies, how to handle a string on stack: "A string IS SPECIFIED by a cell pair (c-addr u) representing its starting address and length in characters".  It hides how strings are implemented at cost of two stack items instead of one.

[toc] | [prev] | [next] | [standalone]

#36081

On 1/27/2015 11:28 AM, Marek Brunda wrote:
> Dne úterý 27. ledna 2015 16:47:27 UTC+1 Anton Ertl napsal(a):
>> Julian Fondren <julian.fondren@gmail.com> writes:
>>> Huh?  No.  You seem to think that C's ways of doings things are how
>>> things are actually done.  In unix-speak, You should read section 2 of
>>> the manpages, instead of just section 3.  The fundamental I/O functions
>>> all work with ANS Forth strings, basically.
>>
>> write() and read() do, but, e.g., open() takes a zero-terminated
>> string.  That's not surprising, after C was developed to implement
>> Unix, and it's no wonder the same string formats are used for both.
>>
>> - anton
>> --
>> M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
>> comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
>>       New standard: http://www.forth200x.org/forth200x.html
>>     EuroForth 2014: http://www.euroforth.org/ef14/
>
>
> I think you freely interchange a "string" and a "buffer" which holds binary data. write() and read() works with binary data. open() takes a null terminated character string as filename. It's not the same thing.
>
> If I understand it, ANS Forth dosesn't specify a format of character string. It only specifies, how to handle a string on stack: "A string IS SPECIFIED by a cell pair (c-addr u) representing its starting address and length in characters".  It hides how strings are implemented at cost of two stack items instead of one.

Correct. Many systems still store strings (e.g. word names, ." strings, 
strings for other purposes) as counted strings. They are accessed by
addr COUNT (which leaves the appropriate addr/len pair).

Cheers,
Elizabeth

-- 
==================================================
Elizabeth D. Rather   (US & Canada)   800-55-FORTH
FORTH Inc.                         +1 310.999.6784
5959 West Century Blvd. Suite 700
Los Angeles, CA 90045
http://www.forth.com

"Forth-based products and Services for real-time
applications since 1973."
==================================================

[toc] | [prev] | [next] | [standalone]

#36167

Marek Brunda <marekb@phoenics.org> writes:
>I think you freely interchange a "string" and a "buffer" which holds binary=
> data.

Buffer in Forth and in other common usage denotes a memory area where
some data (whether string or binary) can be stored.

Concerning string data and a piece of binary data in memory, yes, for
many Forth words (e.g., MOVE) there is no difference between them, and
that's one of the strengths of the addr length representation.

> write() and read() works with binary data

It also works with character data.  Like Forth, in Unix the addr
length representation leads to flexible interfaces, unlike the
zero-terminated string.

>If I understand it, ANS Forth dosesn't specify a format of character string=
>. It only specifies, how to handle a string on stack: "A string IS SPECIFIE=
>D by a cell pair (c-addr u) representing its starting address and length in=
> characters".  It hides how strings are implemented at cost of two stack it=
>ems instead of one.

That might be a short-coming of Forth-94 (and IIRC we have not fixed
that in Forth-2012), but everybody knows that the char with the lowest
address is the first one in the string, the one at addr+1 is the
second char, etc.  So nobody has implemented a TYPE that expects the
first char at the last address or somesuch.  And all other string
words also work in that way.

However, the way that strings are used is mostly opaque, though, which
has allowed us to add the xchar wordset that treat strings as
containing sequences of chars that may form a multi-char character.

- anton
-- 
M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
     New standard: http://www.forth200x.org/forth200x.html
   EuroForth 2014: http://www.euroforth.org/ef14/

[toc] | [prev] | [next] | [standalone]

#36100

On Mon, 26 Jan 2015 22:46:05 -0500, Julian Fondren  
<julian.fondren@gmail.com> wrote:
> On Monday, January 26, 2015 at 8:55:05 PM UTC-6, Rod Pemberton wrote:
>> On Mon, 26 Jan 2015 15:11:30 -0500, Mark Wills <markwills1970@gmail.com>
>> wrote:

>> > 2) you have to walk the entire length of a string just to get its  
>> length
>>
>> That must be done in Forth too to get the length in the first place.
>> It's true that you don't need to walk the entire length of the string,
>> but Forth is doing that for you.
>
> Huh?  No.

Huh? Yes.

To repeat, I said the length byte of a counted string in
Forth isn't set by the user or programmer.

How can you disagree with that? ...

I have idea what you C answer or explanation below has
to do with Forth's counted strings or your disagreement
with my statement on Forth.

C wasn't even mentioned here ...

> You seem to think that C's ways of doings things are how
> things are actually done.  In unix-speak, You should read section 2 of
> the manpages, instead of just section 3.  The fundamental I/O functions
> all work with ANS Forth strings, basically.  They demand buffers with
> lengths.  They write arbitrary, '\0'-including data into buffers and
> then return a count of data written.
>
> http://linux.die.net/man/2/write
> http://linux.die.net/man/2/read
> http://linux.die.net/man/2/mmap
>
> C's fussing about with finding and setting the length is extra,
> unnecessary effort; there's no "in the first place" about it.
>

I really don't understand why you posted a C explanation to a Forth
statement that you disagreed with.  I almost think this was in regards
to another post on using C style strings for my Forth.


Rod Pemberton

[toc] | [prev] | [next] | [standalone]

#36102

On Wed, 28 Jan 2015 20:07:40 -0500, Rod Pemberton  
<buzz@notnotnontnooatno.cmm> wrote:

> How can you disagree with that? ...
>
> I have idea what you C answer or

I have no idea what your C answer or ...


RP

[toc] | [prev] | [next] | [standalone]

#36104

Op Wed, 28 Jan 2015 20:07:40 -0500 schreef Rod Pemberton:

> On Mon, 26 Jan 2015 22:46:05 -0500, Julian Fondren  
> <julian.fondren@gmail.com> wrote:
>> On Monday, January 26, 2015 at 8:55:05 PM UTC-6, Rod Pemberton wrote:
>>> On Mon, 26 Jan 2015 15:11:30 -0500, Mark Wills <markwills1970@gmail.com>
>>> wrote:
> 
>>> > 2) you have to walk the entire length of a string just to get its  
>>> length
>>>
>>> That must be done in Forth too to get the length in the first place.
>>> It's true that you don't need to walk the entire length of the string,
>>> but Forth is doing that for you.
>>
>> Huh?  No.
> 
> Huh? Yes.
> 
> To repeat, I said the length byte of a counted string in
> Forth isn't set by the user or programmer.
It's set by the compiler. Look up what e.g. WORD does(*). Getting the count
is simply C@ or using the word COUNT. Forths that use cells for the count
use @ or @+ (DUP CELL+ SWAP @)
No scanning for NUL needed.

(*) IMHO WORD is a remnant of the use of counted strings, PARSE-NAME is
more versatile, it leaves the address AND the count of the string, still in
the input stream, so more instances of PARSE-NAME can be in action,
contrary to WORD.

-- 
Coos

CHForth, 16 bit DOS applications
http://home.hccnet.nl/j.j.haak/forth.html 

[toc] | [prev] | [next] | [standalone]

#36052

El lunes, 26 de enero de 2015, 16:21:52 (UTC-3), Marek Brunda  escribió:
> Dne sobota 24. ledna 2015 2:17:25 UTC+1 Rod Pemberton napsal(a):
> > On Mon, 19 Jan 2015 02:54:49 -0500, Marek Brunda <marekb@phoenics.org>  
> > wrote:
> > 
> > > Hi, can anyone explain me, what's wrong with counted strings? Maybe it's  
> > > a stupid question, but I've never understood it. Thanks
> > 
> > This has been discussed many times here in the past.
> > You can search the Google Groups archive of this Usenet group.
> > 
> > Nul-terminated strings replaced counted strings for the most part
> > many decades ago in most programming languages, e.g., 1970's or 1980's.
> > 
> > The problem back then with counted strings was that the count could
> > be changed, but the length of the string wouldn't be.  I.e., there
> > is a mismatch in the size of the count and the actual length of the
> > string.  If the count is too short, then the string is truncated.
> > This isn't a problem.  If the count is too large, then you have a
> > very serious issue.  More space for a new larger string wasn't
> > allocated.  The string is the same size, but the count is larger.
> > So, the oversized count value allows you to access data PAST the
> > actual end-of-string.  I.e., you can read data you shouldn't have
> > access to, or write to variables you shouldn't have access to.
> > This is the same technique used by hackers for buffer overflow.
> > 
> > (FYI, to prevent flame war, there are those here who claim that
> > can't actually happen in Forth ...)
> > 
> > Languages like C which use nul-terminated strings frequently
> > only need to detect the end-of-string.  They generally don't
> > need to know the string's length.  So, not counting the length
> > of a string and just appending a nul-terminator to create a
> > string increased code speed.  Of course, not terminating
> > a string in C, which happens with some string functions under
> > certain conditions, can cause overruns or unintended access too.
> > 
> > 
> > Rod Pemberton
> 
> I read all the posts and I understand the arguments. So why not use NULL terminated strings in Forth?

some forth use null terminate strings !! mine is one

[toc] | [prev] | [next] | [standalone]

#36064

On Mon, 26 Jan 2015 15:14:09 -0500, Pablo Hugo Reda <pabloreda@gmail.com>  
wrote:
> El lunes, 26 de enero de 2015, 16:21:52 (UTC-3), Marek Brunda  escribió:
>> Dne sobota 24. ledna 2015 2:17:25 UTC+1 Rod Pemberton napsal(a):
>> > On Mon, 19 Jan 2015 02:54:49 -0500, Marek Brunda <marekb@phoenics.org>
>> > wrote:

>> > > Hi, can anyone explain me, what's wrong with counted strings? Maybe  
>> it's
>> > > a stupid question, but I've never understood it. Thanks
>> >
>> > This has been discussed many times here in the past.
>> > You can search the Google Groups archive of this Usenet group.
>> >
>> > Nul-terminated strings replaced counted strings for the most part
>> > many decades ago in most programming languages, e.g., 1970's or  
>> 1980's.
>> >
>> > The problem back then with counted strings was that the count could
>> > be changed, but the length of the string wouldn't be.  I.e., there
>> > is a mismatch in the size of the count and the actual length of the
>> > string.  If the count is too short, then the string is truncated.
>> > This isn't a problem.  If the count is too large, then you have a
>> > very serious issue.  More space for a new larger string wasn't
>> > allocated.  The string is the same size, but the count is larger.
>> > So, the oversized count value allows you to access data PAST the
>> > actual end-of-string.  I.e., you can read data you shouldn't have
>> > access to, or write to variables you shouldn't have access to.
>> > This is the same technique used by hackers for buffer overflow.
>> >
>> > (FYI, to prevent flame war, there are those here who claim that
>> > can't actually happen in Forth ...)
>> >
>> > Languages like C which use nul-terminated strings frequently
>> > only need to detect the end-of-string.  They generally don't
>> > need to know the string's length.  So, not counting the length
>> > of a string and just appending a nul-terminator to create a
>> > string increased code speed.  Of course, not terminating
>> > a string in C, which happens with some string functions under
>> > certain conditions, can cause overruns or unintended access too.
>> >
>>
>> I read all the posts and I understand the arguments. So why not use  
>> NULL terminated strings in Forth?
>
> some forth use null terminate strings !! mine is one

I use nul-terminated, C-compatible strings in my Forth too.
2/3 of it passes Hayes Core.  Mostly, the floating point and
a few odd words remain from ANS "core" and "core ext".  It's
coded in C.  It fails one Hayes Core test for strings.  This
is due to the way the test constructs strings.  It makes an
assumption about the implementation.  This can be fixed by
providing an extra, unused byte at the beginning for the length
field that the tests expect.  Although, I haven't done so yet.
So, there isn't much of a reason to not use them, AFAICT so far.
Most of the mechanics of forth strings are hidden behind a few
words and don't affect the user or programmer.  I.e., they can
be changed to use other string representations.


Rod Pemberton

[toc] | [prev] | [next] | [standalone]

#36068

On 1/26/15 22:14, Pablo Hugo Reda wrote:

> some forth use null terminate strings !! mine is one

Mine is two (or is, too).  8th strings are single items on the stack,
but hold the string's length as well as making sure the string is NUL
terminated.

IMO it costs almost nothing to do that, and makes passing strings to OS
or library routines easier and less error-prone.

[toc] | [prev] | [next] | [standalone]

#36077

on 27/01/2015 04:55:03, Ron Aaron wrote:
> 
> 
> On 1/26/15 22:14, Pablo Hugo Reda wrote:
> 
>> some forth use null terminate strings !! mine is one
> 
> Mine is two (or is, too).  8th strings are single items on the stack,
> but hold the string's length as well as making sure the string is NUL
> terminated.
> 
> IMO it costs almost nothing to do that, and makes passing strings to
> OS or library routines easier and less error-prone.
> 

Mine is number 3 (it's one too), and it passes all the Hayes tests.

[toc] | [prev] | [next] | [standalone]

#36058

On 1/26/15 9:21 AM, Marek Brunda wrote:
> Dne sobota 24. ledna 2015 2:17:25 UTC+1 Rod Pemberton napsal(a):
>> On Mon, 19 Jan 2015 02:54:49 -0500, Marek Brunda <marekb@phoenics.org>
>> wrote:
>>
>>> Hi, can anyone explain me, what's wrong with counted strings? Maybe it's
>>> a stupid question, but I've never understood it. Thanks
>>
>> This has been discussed many times here in the past.
>> You can search the Google Groups archive of this Usenet group.
>>
>> Nul-terminated strings replaced counted strings for the most part
>> many decades ago in most programming languages, e.g., 1970's or 1980's.
>>
>> The problem back then with counted strings was that the count could
>> be changed, but the length of the string wouldn't be.  I.e., there
>> is a mismatch in the size of the count and the actual length of the
>> string.  If the count is too short, then the string is truncated.
>> This isn't a problem.  If the count is too large, then you have a
>> very serious issue.  More space for a new larger string wasn't
>> allocated.  The string is the same size, but the count is larger.
>> So, the oversized count value allows you to access data PAST the
>> actual end-of-string.  I.e., you can read data you shouldn't have
>> access to, or write to variables you shouldn't have access to.
>> This is the same technique used by hackers for buffer overflow.
>>
>> (FYI, to prevent flame war, there are those here who claim that
>> can't actually happen in Forth ...)
>>
>> Languages like C which use nul-terminated strings frequently
>> only need to detect the end-of-string.  They generally don't
>> need to know the string's length.  So, not counting the length
>> of a string and just appending a nul-terminator to create a
>> string increased code speed.  Of course, not terminating
>> a string in C, which happens with some string functions under
>> certain conditions, can cause overruns or unintended access too.
>
>
> I read all the posts and I understand the arguments. So why not use NULL terminated strings in Forth?

We do, sometimes, when we have to interact with an OS or C program that 
requires them. But Mark gave an excellent summary of why many of us 
dislike them. None of the horrors that Rod cites actually plague 
well-designed Forth programs, and badly-designed programs suffer from 
problems that NULL-terminated strings can't solve. You must realize that 
Rod has very little Forth application experience.

But the direction the standards are taking of referencing all strings by 
addr/len pairs on the stack makes the language agnostic as to internal 
storage format.

Cheers,
Elizabeth

-- 
==================================================
Elizabeth D. Rather   (US & Canada)   800-55-FORTH
FORTH Inc.                         +1 310.999.6784
5959 West Century Blvd. Suite 700
Los Angeles, CA 90045
http://www.forth.com

"Forth-based products and Services for real-time
applications since 1973."
==================================================

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | comp.lang.forth

csiph-web