Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.forth > #59642 > unrolled thread

allot which initialize the alloted space ?

Back to article view | Back to comp.lang.forth

Contents

  allot which initialize the alloted space ? Andreas Klimas <klimas@w3group.de> - 2017-05-21 13:50 +0200
    Re: allot which initialize the alloted space ? Coos Haak <htrofhc@gmail.com> - 2017-05-21 14:13 +0200
    Re: allot which initialize the alloted space ? hughaguilar96@gmail.com - 2017-05-21 08:01 -0700
      Re: allot which initialize the alloted space ? Ron Aaron <rambamist@gmail.com> - 2017-05-21 18:16 +0300
        Re: allot which initialize the alloted space ? hughaguilar96@gmail.com - 2017-05-21 09:02 -0700
          Re: allot which initialize the alloted space ? Ron Aaron <rambamist@gmail.com> - 2017-05-21 19:17 +0300
            Re: allot which initialize the alloted space ? anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2017-05-22 11:22 +0000
            Re: allot which initialize the alloted space ? hughaguilar96@gmail.com - 2017-05-24 08:03 -0700
              Re: allot which initialize the alloted space ? Alex <alex@rivadpm.com> - 2017-05-24 16:58 +0100
          Re: allot which initialize the alloted space ? anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2017-05-22 07:40 +0000
            Re: allot which initialize the alloted space ? Mark Wills <markwills1970@gmail.com> - 2017-05-22 08:02 -0700
              Re: allot which initialize the alloted space ? "HAA" <someone@microsoft.com> - 2017-05-24 16:40 +1000
            Re: allot which initialize the alloted space ? hughaguilar96@gmail.com - 2017-05-22 13:27 -0700
        Re: allot which initialize the alloted space ? Alex <alex@rivadpm.com> - 2017-05-21 17:16 +0100
          Re: allot which initialize the alloted space ? Ron Aaron <rambamist@gmail.com> - 2017-05-21 19:22 +0300
            Re: allot which initialize the alloted space ? Alex <alex@rivadpm.com> - 2017-05-21 18:03 +0100
            Re: allot which initialize the alloted space ? albert@cherry.spenarnc.xs4all.nl (Albert van der Horst) - 2017-05-21 20:46 +0200
              Re: allot which initialize the alloted space ? Alex <alex@rivadpm.com> - 2017-05-21 20:34 +0100
        Re: allot which initialize the alloted space ? "HAA" <someone@microsoft.com> - 2017-05-24 16:41 +1000
    Re: allot which initialize the alloted space ? Julian Fondren <julian.fondren@gmail.com> - 2017-05-21 08:36 -0700

#59642 — allot which initialize the alloted space ?

is there a common known word which performs allot but initialize the 
memory with all bits set to 0 ?

thank you

- Andreas

[toc] | [next] | [standalone]

#59645

Op Sun, 21 May 2017 13:50:12 +0200 schreef Andreas Klimas:

> is there a common known word which performs allot but initialize the 
> memory with all bits set to 0 ?
> 
> thank you
> 
> - Andreas

The only one I'm aware of is /ALLOT in SwiftForth

groet Coos

[toc] | [prev] | [next] | [standalone]

#59656

On Sunday, May 21, 2017 at 4:50:10 AM UTC-7, Andreas Klimas wrote:
> is there a common known word which performs allot but initialize the 
> memory with all bits set to 0 ?
> 
> thank you
> 
> - Andreas

: allot ( size -- )                                             \ zeros out the memory
    dup 0> if                                                   \ only zero out the memory for a positive allotment
        here >r  allot
        r@  here r> -  erase
    else
        allot  then ;

I used to have ZALLOT in the novice-package but I realized that it is redundant --- there is no case in which I don't want to zero out the memory --- so I just renamed it ALLOT ash shown above.

[toc] | [prev] | [next] | [standalone]

#59659

On 21/05/2017 18:01, hughaguilar96@gmail.com wrote:

> I used to have ZALLOT in the novice-package but I realized that it is redundant --- there is no case in which I don't want to zero out the memory --- so I just renamed it ALLOT ash shown above.

Really?  Why would you zero out the memory if, for instance, you were
going to read from a file into that memory area?  Or if you were going
to copy some other data to it?  The zeroing takes time.

[toc] | [prev] | [next] | [standalone]

#59665

On Sunday, May 21, 2017 at 8:16:10 AM UTC-7, Ron Aaron wrote:
> On 21/05/2017 18:01, hughaguilar96@gmail.com wrote:
> 
> > I used to have ZALLOT in the novice-package but I realized that it is redundant --- there is no case in which I don't want to zero out the memory --- so I just renamed it ALLOT ash shown above.
> 
> Really?  Why would you zero out the memory if, for instance, you were
> going to read from a file into that memory area?  Or if you were going
> to copy some other data to it?  The zeroing takes time.

This is the same argument that Anton Ertl made in regard to his {: definer. He wasn't zeroing out the uninitialized locals (the locals to the right of the | in the list of locals). He wanted to make it legal to initialize them to whatever data happened to be in memory already.

This is a very bad idea! Uninitialized data needs to be explicitly initialized by the programmer, but a common bug is that the programmer forgets to initialize it. If it is randomly initialized, the program might happen to work due to luck, and so the bug doesn't get noticed. Then the program is recompiled and suddenly the bug causes the program to not work, because the values in there happen to be different. Sporadic bugs like this are very difficult to find! Also, sporadic bugs have a tendency to appear at awkward moments --- for example, compiling the production version for the customer --- it looks very bad when a customer finds a bug in 5 minutes, that you failed to notice in several weeks of testing.

Initializing to zero is always a good idea! This isn't because zero is what you want. This is because, if you forget to explicitly initialize it to what you do want, then you will discover the bug immediately. The program will do the same thing every time that it is compiled --- it won't necessarily do what you want it to do (bugs happen!) --- but it will do the same thing every time.

Zero is a good value for uninitialized locals. In my { I initialize the locals to the right of the | to zero. Because this is defined behavior, not just a random happenstance, I can assume that this is always what happens. A lot of times (initializing a flag or a counter to zero), this is what I want to happen, so I don't bother with explicitly doing it. My HOTTIE.4TH program is an example of this, as all of the counters are initialized to zero which is what I want.

As for the time that it takes to initialize data to zero, I don't worry about that. Modern processors are pretty fast! Even on the 65c02 though, I wouldn't have worried about it --- defensive programming is always a good idea --- premature optimization usually results in a bug-ridden program, in which case speed is not your #1 concern.

[toc] | [prev] | [next] | [standalone]

#59667

On 21/05/2017 19:02, hughaguilar96@gmail.com wrote:

> This is a very bad idea! Uninitialized data needs to be explicitly initialized by the programmer

No, it's not a very bad idea.  It's a smart idea if you know that the
area will be initialized before use.  Yes, it's a bad idea if you don't
know that, and depend on the data being initialized.


> As for the time that it takes to initialize data to zero, I don't worry about that. Modern processors are pretty fast! 

Memory speed is not insignificant, and is considerably slower than
processor speed.  If the data being initialized is (for example) a 10000
byte long buffer, then the memory initialization time is significant,
and if it's happening in a time-critical loop, you won't want to do the
extra work.  That is, if efficiency is important.

[toc] | [prev] | [next] | [standalone]

#59715

Ron Aaron <rambamist@gmail.com> writes:
>> As for the time that it takes to initialize data to zero, I don't worry about that. Modern processors are pretty fast! 
>
>Memory speed is not insignificant, and is considerably slower than
>processor speed.  If the data being initialized is (for example) a 10000
>byte long buffer, then the memory initialization time is significant,
>and if it's happening in a time-critical loop, you won't want to do the
>extra work.

Because memory speed is significant, it can actually be more efficient
to zero out the memory before use than to load it from main memory
(which typically happens when you write only a part of a cache line):
Zeroing out can be performed as a pure cache operation.  Of course, a
lot of circumstances play a role: Is the memory area under
consideration currently in the cache or not, how bit is it, will the
stores to it actually happen before the line is evicted from the
cache, etc.  Anyway, various architectures have instructions for
zeroing: E.g., Alpha has WH64, PowerPC has dcbz, there is CLZERO in
recent AMD CPUs.

- anton
-- 
M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
     New standard: http://www.forth200x.org/forth200x.html
   EuroForth 2017: http://www.euroforth.org/ef17/

[toc] | [prev] | [next] | [standalone]

#59776

On Sunday, May 21, 2017 at 9:17:33 AM UTC-7, Ron Aaron wrote:
> On 21/05/2017 19:02, hughaguilar96@gmail.com wrote:
> > As for the time that it takes to initialize data to zero, I don't worry about that. Modern processors are pretty fast! 
> 
> Memory speed is not insignificant, and is considerably slower than
> processor speed.  If the data being initialized is (for example) a 10000
> byte long buffer, then the memory initialization time is significant,
> and if it's happening in a time-critical loop, you won't want to do the
> extra work.  That is, if efficiency is important.

ALLOT is almost always done at compile-time, not run-time. 
A 10000 ALLOT in a time-critical loop is nonsense.

I also zero out ALLOCATE and speed is an issue there. ALLOCATE is already horribly slow though, but that is the OS --- that has nothing to do with my zeroing out the memory, which takes a few microseconds, even with a 10000 byte block --- I typically use ALLOCATE for structs, and they are typically less than 100 bytes in size.
A 10000 ALLOCATE in a time-critical loop is nonsense.

A lot of these comp.lang.forth threads seem to quickly devolve into nonsense, and this is one of them --- just zero out the memory as typical defensive-programming to help with debugging --- don't worry, be happy!

[toc] | [prev] | [next] | [standalone]

#59777

On 5/24/2017 16:03, hughaguilar96@gmail.com wrote:
> On Sunday, May 21, 2017 at 9:17:33 AM UTC-7, Ron Aaron wrote:
>> On 21/05/2017 19:02, hughaguilar96@gmail.com wrote:
>>> As for the time that it takes to initialize data to zero, I don't
>>> worry about that. Modern processors are pretty fast!
>> 
>> Memory speed is not insignificant, and is considerably slower than 
>> processor speed.  If the data being initialized is (for example) a
>> 10000 byte long buffer, then the memory initialization time is
>> significant, and if it's happening in a time-critical loop, you
>> won't want to do the extra work.  That is, if efficiency is
>> important.
> 
> ALLOT is almost always done at compile-time, not run-time. A 10000
> ALLOT in a time-critical loop is nonsense.
> 
> I also zero out ALLOCATE and speed is an issue there. ALLOCATE is
> already horribly slow though, but that is the OS --- that has nothing
> to do with my zeroing out the memory, which takes a few microseconds,
> even with a 10000 byte block --- I typically use ALLOCATE for
> structs, and they are typically less than 100 bytes in size. A 10000
> ALLOCATE in a time-critical loop is nonsense.
> 
> A lot of these comp.lang.forth threads seem to quickly devolve into
> nonsense, and this is one of them --- just zero out the memory as
> typical defensive-programming to help with debugging --- don't worry,
> be happy!
> 

I agree that there are few times you will be ALLOTing, and ALLOCATE is 
overhead laden in many systems; so zeroing is a marginal loss.

However buffer management is one area where large allocations, 
frequently performed, are an issue. Zeroing storage at every allocation 
out of a pool may have significant overheads. A few microseconds is more 
than hugely significant when you're shifting data across a network at 
the rate of Tb/sec. Don't do it unless you have hardware assistance 
would be my advice.

I also disagree that zeroing is good for defensive-programming to help 
with debugging. In the 40+ years or so that I've been do this, I've 
found oddball bit patterns more useful. There's nothing quite like 
seeing DEADBEEF (obvious) FA17FA17 (FAILFAIL) 71077345 (Shell Oil upside 
down) or your chosen fill a few digits or bits off, in a register or key 
memory location, to know that there's something quite wrong. Zero is too 
ubiquitous to be of much use.

-- 
Alex

[toc] | [prev] | [next] | [standalone]

#59711

hughaguilar96@gmail.com writes:
>This is the same argument that Anton Ertl made in regard to his {: definer.=
> He wasn't zeroing out the uninitialized locals (the locals to the right of=
> the | in the list of locals). He wanted to make it legal to initialize the=
>m to whatever data happened to be in memory already.

Reality check: The "enhanced local variable syntax" proposal is by
Stephen Pelc, and he included the "|" feature.

I argued that "|" is unnecessary, because you can write

0 0 {: a b c d :} 

instead of

{: a b | c d :}

and voila, you get initialized c and d.  And the better approach is to
define c and d where you get the actual data instead of at the start.
So the "|" feature is mainly a workaround for the inability of most
Forth systems to support multiple locals definitions and locals
definitions inside control structures.

Stephen Pelc included a "reference implementation" based on a word
BUILDLV that is not implemented in a working way in the reference
implementation, nor implemented in any Forth system.

I then implemented {: twice in standard Forth, based on (LOCAL):

http://www.forth200x.org/reference-implementations/locals.fs
http://www.forth200x.org/reference-implementations/extended-locals.fs

Both initialize the locals after |, because this is a good idea, and
because (LOCAL) requires an initialization value.  locals.fs
initializes the locals after | to 0, extended-locals.fs to 12345.

- anton
-- 
M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
     New standard: http://www.forth200x.org/forth200x.html
   EuroForth 2017: http://www.euroforth.org/ef17/

[toc] | [prev] | [next] | [standalone]

#59726

On Monday, 22 May 2017 09:05:55 UTC+1, Anton Ertl  wrote:
> I argued that "|" is unnecessary, because you can write
> 
> 0 0 {: a b c d :} 
> 
> instead of
> 
> {: a b | c d :}
> 

Yes you can do that, but it's vile.

[toc] | [prev] | [next] | [standalone]

#59766

Mark Wills wrote:
> On Monday, 22 May 2017 09:05:55 UTC+1, Anton Ertl  wrote:
> > I argued that "|" is unnecessary, because you can write
> >
> > 0 0 {: a b c d :}
> >
> > instead of
> >
> > {: a b | c d :}
> >
>
> Yes you can do that, but it's vile.

It wouldn't be vile under ANS syntax :)

[toc] | [prev] | [next] | [standalone]

#59736

On Monday, May 22, 2017 at 1:05:55 AM UTC-7, Anton Ertl wrote:
> hughaguilar96@gmail.com writes:
> >This is the same argument that Anton Ertl made in regard to his {: definer.=
> > He wasn't zeroing out the uninitialized locals (the locals to the right of=
> > the | in the list of locals). He wanted to make it legal to initialize the=
> >m to whatever data happened to be in memory already.
> 
> Reality check: The "enhanced local variable syntax" proposal is by
> Stephen Pelc, and he included the "|" feature.

This is a straw-man argument --- I never said that the specification didn't include the | feature --- I am aware that it does.

What I said is that the specification failed to require the locals to the right of the | to be initialized to a specific value (zero would be the best choice).

> I then implemented {: twice in standard Forth, based on (LOCAL):
> 
> http://www.forth200x.org/reference-implementations/locals.fs
> http://www.forth200x.org/reference-implementations/extended-locals.fs
> 
> Both initialize the locals after |, because this is a good idea, and
> because (LOCAL) requires an initialization value.  locals.fs
> initializes the locals after | to 0, extended-locals.fs to 12345.

Initializing the locals after the | is a "good idea," but this is not what the specification says. If Stephen Pelc wrote the specification, then he screwed it up. You shouldn't have implemented a screwed-up specification --- you are the Forth-200x chair-person --- you should have fixed the specification rather than just blindly implement it.

This is what you said before:

On Wednesday, October 27, 2010 at 8:24:14 AM UTC-7, Anton Ertl wrote:
> Hugh Aguilar <hughaguilar96@yahoo.com> writes:
> >P.S.: Why did you set your undefined value to 12345?
> 
> Why not?  It's not defined what the value is, so it can be anything.

You are admitting that the specification doesn't define a value that the locals to the right of the | must be initialized to. This results in ambiguity! The locals to the right of the | can be initialized to different values in different standard-compliant compilers, or could even be initialized to whatever data happened to be in memory at the time that the locals are initialized, which will vary from from execution of the function to the next. Your specification essentially says: 
"Well, um, initialize the locals to the right of the | to anything that you feel like, including random numbers  --- every Forth-200x compiler will have a different behavior from every other Forth-200x compiler --- ambiguity was the hallmark of ANS-Forth, and ambiguity will be the hallmark of Forth-200x!"

What I said before (that you provided a "reality check" on) was: 
"[Anton Ertl] wanted to make it legal to initialize [locals after the | symbol] to whatever data happened to be in memory already."
What I said was true --- you were totally okay with Stephen Pelc's specification that allowed the locals to the right of the | to be initialized to whatever data happened to be in memory already --- this continues to be the specification, so apparently you continue to be okay with this blunder.

The Forth-200x committee are similar to the ANS-Forth committee --- you are obsessed with making everything ambiguous and undefined --- these are standards that don't standardize anything!

Gratuitous ambiguity is the major problem in ANS-Forth, and this continues to be the major problem in Forth-200x --- this is why both ANS-Forth and Forth-200x deserve to be ignored completely!

[toc] | [prev] | [next] | [standalone]

#59666

On 5/21/2017 16:16, Ron Aaron wrote:
> 
> 
> On 21/05/2017 18:01, hughaguilar96@gmail.com wrote:
> 
>> I used to have ZALLOT in the novice-package but I realized that it
>> is redundant --- there is no case in which I don't want to zero out
>> the memory --- so I just renamed it ALLOT ash shown above.
> 
> Really?  Why would you zero out the memory if, for instance, you
> were going to read from a file into that memory area?  Or if you were
> going to copy some other data to it?  The zeroing takes time.
> 

The only time an OS like Windows gives you memory that's not initialised 
to zero is memory you've allocated, used and freed yourself (where 
yourself is anything running in the process) on the heap. It can be 
requested zeroed by passing HEAP_ZERO_MEMORY to HeapAlloc.

VirtualAlloc (the basis of all memory reservation in Windows, including 
the provision of heaps) always zeroes memory.

About the only time you'd need to zero memory is when you're dealing 
with secure info like passwords, and the app should do that before 
passing the memory around or freeing it. Not doing so is a security hole.

In the future, zeroing persistent memory (persistent stuff faster than 
flash SSD but slower than DRAM) will be very expensive indeed, and will 
normally be done by requesting the device to do it, and not 
programmatically.

-- 
Alex

[toc] | [prev] | [next] | [standalone]

#59669

On 21/05/2017 19:16, Alex wrote:

> The only time an OS like Windows gives you memory that's not initialised
> to zero is memory you've allocated, used and freed yourself (where
> yourself is anything running in the process) on the heap. It can be
> requested zeroed by passing HEAP_ZERO_MEMORY to HeapAlloc.

Not everyone uses Windows, and if you use "malloc()" you are not
guaranteed that the memory is cleared.  That's what "calloc()" is for.

> 
> VirtualAlloc (the basis of all memory reservation in Windows, including
> the provision of heaps) always zeroes memory.

But that is not the case when a suballocator (e.g. a malloc() library)
is parcelling out the VirtualAlloced chunk.  And when you malloc/free
blocks, you are definitely not guaranteed any zeroed-out memory.

> About the only time you'd need to zero memory is when you're dealing
> with secure info like passwords, and the app should do that before
> passing the memory around or freeing it. Not doing so is a security hole.

Yes indeed, clearing a password buffer is a Good Idea.  You might also
want to do that if you have a struct that should all be zeroes, saving
the time to manually set each field.

> In the future, zeroing persistent memory (persistent stuff faster than
> flash SSD but slower than DRAM) will be very expensive indeed, and will
> normally be done by requesting the device to do it, and not
> programmatically.

Well, we'll see when we get there...

[toc] | [prev] | [next] | [standalone]

#59672

On 5/21/2017 17:22, Ron Aaron wrote:
>> In the future, zeroing persistent memory (persistent stuff faster
>> than flash SSD but slower than DRAM) will be very expensive indeed,
>> and will normally be done by requesting the device to do it, and
>> not programmatically.
> Well, we'll see when we get there...

It's here already. Eye wateringly expensive but damn, it's fast.

https://arstechnica.co.uk/information-technology/2017/03/intel-optane-3d-xpoint-details-price/

-- 
Alex

[toc] | [prev] | [next] | [standalone]

#59685

In article <ofsemm$2tb$1@dont-email.me>,
Ron Aaron  <rambamist@gmail.com> wrote:
>
>
>On 21/05/2017 19:16, Alex wrote:
<SNP>
>> About the only time you'd need to zero memory is when you're dealing
>> with secure info like passwords, and the app should do that before
>> passing the memory around or freeing it. Not doing so is a security hole.
>
>Yes indeed, clearing a password buffer is a Good Idea.  You might also
>want to do that if you have a struct that should all be zeroes, saving
>the time to manually set each field.

Unless the compiler is smart enough to detect that you don't use
the zeroed buffer anyway, and optimises out the zeroing.

>Well, we'll see when we get there...

Groetjes Albert
-- 
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

[toc] | [prev] | [next] | [standalone]

#59692

On 5/21/2017 19:46, Albert van der Horst wrote:

> 
> Unless the compiler is smart enough to detect that you don't use
> the zeroed buffer anyway, and optimises out the zeroing.

Which was the case with the Windows macro ZeroMemory. SecureZeroMemory 
is a function to replace it. "To avoid any undesired effects of 
optimizing compilers, use the SecureZeroMemory function."

-- 
Alex

[toc] | [prev] | [next] | [standalone]

#59767

Ron Aaron wrote:
> On 21/05/2017 18:01, hughaguilar96@gmail.com wrote:
>
> > I used to have ZALLOT in the novice-package but I realized that it is redundant ---
> > there is no case in which I don't want to zero out the memory --- so I just renamed
> > it ALLOT ash shown above.
>
> Really?  Why would you zero out the memory if, for instance, you were
> going to read from a file into that memory area?  Or if you were going
> to copy some other data to it?  The zeroing takes time.

It will take time.  Whether it matters in practice depends on how large the
memory is, how frequently you ALLOT, whether it occurs at run-time or
compile-time etc.  Some reasons why you might want it zeroed:
- you expect it
- you won't see garbage when DUMPing
- turnkeyed executables compress better

[toc] | [prev] | [next] | [standalone]

#59662

On Sunday, May 21, 2017 at 6:50:10 AM UTC-5, Andreas Klimas wrote:
> is there a common known word which performs allot but initialize the 
> memory with all bits set to 0 ?

When I've been tempted to write one, usually there was an INIT-BLAH
that should do the work instead, or I just used this idiom:

  create blah here 100 dup allot erase


On Sunday, May 21, 2017 at 10:01:47 AM UTC-5, hughag...@gmail.com wrote:
>     dup 0> if                                                   \ only zero out the memory for a positive allotment
...
> there is no case in which I don't want to zero out the memory

I found a case in your code where you don't want that.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.forth

csiph-web