Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.forth > #25699 > unrolled thread

gForth and unix/pthread.fs

Back to article view | Back to comp.lang.forth

Contents

  gForth and unix/pthread.fs voidvolker@gmail.com - 2013-09-16 00:43 -0700
    Re: gForth and unix/pthread.fs anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2013-09-16 09:05 +0000
      Re: gForth and unix/pthread.fs voidvolker@gmail.com - 2013-09-16 05:43 -0700
    Re: gForth and unix/pthread.fs VoidVolker <voidvolker@gmail.com> - 2013-09-19 10:14 -0700
      Re: gForth and unix/pthread.fs Bernd Paysan <bernd.paysan@gmx.de> - 2013-09-20 01:26 +0200
        Re: gForth and unix/pthread.fs VoidVolker <voidvolker@gmail.com> - 2013-09-19 19:17 -0700
          Re: gForth and unix/pthread.fs Bernd Paysan <bernd.paysan@gmx.de> - 2013-09-20 16:20 +0200
            Re: gForth and unix/pthread.fs VoidVolker <voidvolker@gmail.com> - 2013-09-20 08:43 -0700
              Re: gForth and unix/pthread.fs Bernd Paysan <bernd.paysan@gmx.de> - 2013-09-20 19:07 +0200
                Re: gForth and unix/pthread.fs Bernd Paysan <bernd.paysan@gmx.de> - 2013-09-20 19:14 +0200
                  Re: gForth and unix/pthread.fs Andrew Haley <andrew29@littlepinkcloud.invalid> - 2013-09-21 05:02 -0500
                    Re: gForth and unix/pthread.fs Bernd Paysan <bernd.paysan@gmx.de> - 2013-09-21 15:55 +0200
                      Re: gForth and unix/pthread.fs Andrew Haley <andrew29@littlepinkcloud.invalid> - 2013-09-21 11:32 -0500
                        Re: gForth and unix/pthread.fs Bernd Paysan <bernd.paysan@gmx.de> - 2013-09-21 20:44 +0200
                          Re: gForth and unix/pthread.fs Andrew Haley <andrew29@littlepinkcloud.invalid> - 2013-09-22 05:03 -0500
                            Re: gForth and unix/pthread.fs Bernd Paysan <bernd.paysan@gmx.de> - 2013-09-22 21:18 +0200
                  Re: gForth and unix/pthread.fs VoidVolker <voidvolker@gmail.com> - 2013-09-21 11:41 -0700
                    Re: gForth and unix/pthread.fs Bernd Paysan <bernd.paysan@gmx.de> - 2013-09-21 21:18 +0200
                      Re: gForth and unix/pthread.fs VoidVolker <voidvolker@gmail.com> - 2013-09-21 13:55 -0700
                        Re: gForth and unix/pthread.fs Bernd Paysan <bernd.paysan@gmx.de> - 2013-09-21 23:40 +0200
                Re: gForth and unix/pthread.fs albert@spenarnc.xs4all.nl (Albert van der Horst) - 2013-09-21 12:59 +0000

Page 1 of 2  [1] 2  Next page →

#25699 — gForth and unix/pthread.fs

Hi!
I found library on git: https://github.com/earl/gforth-mirror/blob/master/unix/pthread.fs
But it seems it's for older version of gForth or not for linux and it doesn't work in gForth 0.7.2.
How make it work in new gForth?

OS: Debian 7 x64 / Ubuntu x64

Write now I use https://github.com/ayrnieu/forth-garden/blob/master/gforth/fork.fs
But it's very simple and there are no USER variables.

Maybe somewhere exist normal library for treading for gforth?

[toc] | [next] | [standalone]

#25701

voidvolker@gmail.com writes:
>Hi!
>I found library on git: https://github.com/earl/gforth-mirror/blob/master/unix/pthread.fs
>But it seems it's for older version of gForth or not for linux and it doesn't work in gForth 0.7.2.
>How make it work in new gForth?

Bernd Paysan has been doing a lot of work on multi-threading in the
last years, and it is newer than 0.7.x (0.7.2 just fixes some build
problems).  So if you want to use it, build Gforth from the git
sources at <http://git.savannah.gnu.org/cgit/gforth.git/tree/>, which
also includes an up-to-date version of pthread.fs.  Note however, that
this stuff is still under development and may change, and it is most
likely undocumented.

- anton
-- 
M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
     New standard: http://www.forth200x.org/forth200x.html
   EuroForth 2013: http://www.euroforth.org/ef13/

[toc] | [prev] | [next] | [standalone]

#25703

16/09/2013, 13:05:15 UTC+4 Anton Ertl wrote:
> Bernd Paysan has been doing a lot of work on multi-threading in the
> last years, and it is newer than 0.7.x (0.7.2 just fixes some build
> problems).  So if you want to use it, build Gforth from the git
> sources at <http://git.savannah.gnu.org/cgit/gforth.git/tree/>, which
> also includes an up-to-date version of pthread.fs.  Note however, that
> this stuff is still under development and may change, and it is most
> likely undocumented.

Thanks for answer! I builded gforth and it works.

[toc] | [prev] | [next] | [standalone]

#25821

Sometimes, when i try to start new thread, the gForth show error:
   uncaught thread exception: Return stack underflow
The new thread is socket-server or client and it doesn't work with return stack. It happens before user-code.
How to find error?

Here the code:

: init
  stacksize4 NewTask4 activate
  main-word
;

[toc] | [prev] | [next] | [standalone]

#25824

VoidVolker wrote:

> Sometimes, when i try to start new thread, the gForth show error:
>    uncaught thread exception: Return stack underflow
> The new thread is socket-server or client and it doesn't work with return
> stack. It happens before user-code. How to find error?

Then it probably happens in the C code, and needs to be debugged in C.

> Here the code:
> 
> : init
>   stacksize4 NewTask4 activate
>   main-word
> ;

How often does this happen? Platform? I did run

forth-fast unix/pthread.fs -e ": init stacksize4 newtask4 activate cr ; init 
10 ms bye"

a 1000 times, and there was no problem (and each of the 1000 crs did show up 
in the output).

-- 
Bernd Paysan
"If you want it done right, you have to do it yourself"
http://bernd-paysan.de/

[toc] | [prev] | [next] | [standalone]

#25827

пятница, 20 сентября 2013 г., 3:26:35 UTC+4 пользователь Bernd Paysan написал:
> How often does this happen? Platform? I did run
> 
> forth-fast unix/pthread.fs -e ": init stacksize4 newtask4 activate cr ; init 
> 10 ms bye"
> 
> a 1000 times, and there was no problem (and each of the 1000 crs did show up 
> in the output).

It can be 20 times in sequence, even after restart of gforth, and often another copy of gforth can't start no one tread within in few minutes. Few minutes later it works normal.
I use 'htop' for thread control - it to sometimes closing without error message.

Platform: Debian 7 x64.

[toc] | [prev] | [next] | [standalone]

#25844

VoidVolker wrote:

> пятница, 20 сентября 2013 г., 3:26:35 UTC+4 пользователь Bernd Paysan
> написал:
>> How often does this happen? Platform? I did run
>> 
>> forth-fast unix/pthread.fs -e ": init stacksize4 newtask4 activate cr ;
>> init 10 ms bye"
>> 
>> a 1000 times, and there was no problem (and each of the 1000 crs did show
>> up in the output).
> 
> It can be 20 times in sequence, even after restart of gforth, and often
> another copy of gforth can't start no one tread within in few minutes. Few
> minutes later it works normal. I use 'htop' for thread control - it to
> sometimes closing without error message.
> 
> Platform: Debian 7 x64.

Hm, I can somehow reproduce this, but it certainly doesn't happen that 
often.  Apparently there is some problem with cleaning up threads after use.

BTW: The right way to use threads is *not* to create/destroy them 
frequently, use worker threads which wait for commands.  Ok, that's what I'm 
doing, and that's what doesn't trigger this problem.

-- 
Bernd Paysan
"If you want it done right, you have to do it yourself"
http://bernd-paysan.de/

[toc] | [prev] | [next] | [standalone]

#25845

> BTW: The right way to use threads is *not* to create/destroy them 
> frequently, use worker threads which wait for commands.  Ok, that's what I'm 
> doing, and that's what doesn't trigger this problem.
> 

Yes, this is what I do. At program start create threads and mutex, then the threads just wait mutex unlock, work, and wait again mutex. Simple and fast.

[toc] | [prev] | [next] | [standalone]

#25846

VoidVolker wrote:

>> BTW: The right way to use threads is *not* to create/destroy them
>> frequently, use worker threads which wait for commands.  Ok, that's what
>> I'm doing, and that's what doesn't trigger this problem.
>> 
> 
> Yes, this is what I do. At program start create threads and mutex, then
> the threads just wait mutex unlock, work, and wait again mutex. Simple and
> fast.

How many threads are you creating?  I've now changed the code so that it 
works better through unexpected failures, and is more robust in general, but 
the whole problem is still a bit strange.

My impression is that glibc's memory management is not really thread-safe, 
and that this causes severe problems.  I've enabled malloc checks to get a 
chance to print meaningful error messages in Forth, but that seems to rather 
increase the number of potential problems...

-- 
Bernd Paysan
"If you want it done right, you have to do it yourself"
http://bernd-paysan.de/

[toc] | [prev] | [next] | [standalone]

#25847

Bernd Paysan wrote:

> My impression is that glibc's memory management is not really thread-safe,
> and that this causes severe problems.  I've enabled malloc checks to get a
> chance to print meaningful error messages in Forth, but that seems to
> rather increase the number of potential problems...

I've now disabled the use of mcheck(), and this seems to cure the problem. 
Please try.  I've added mcheck() to make it possible to debug memory errors 
from within Forth (and getting a meaningful backtrace), but Ulrich Drepper 
defeats that idea...

Ulrich Drepper's comment on this is:

"mcheck does not work for multi-threaded code. It cannot possibly do. There 
is no way to fix this with the technology underlying mcheck."

m(

This is a typical Ulrich Drepper: implement something in a way that can't 
possibly work, and then claim it is impossible to fix...

-- 
Bernd Paysan
"If you want it done right, you have to do it yourself"
http://bernd-paysan.de/

[toc] | [prev] | [next] | [standalone]

#25880

Bernd Paysan <bernd.paysan@gmx.de> wrote:
> Bernd Paysan wrote:
> 
>> My impression is that glibc's memory management is not really
>> thread-safe, and that this causes severe problems.  I've enabled
>> malloc checks to get a chance to print meaningful error messages in
>> Forth, but that seems to rather increase the number of potential
>> problems...
> 
> I've now disabled the use of mcheck(), and this seems to cure the
> problem.  Please try.  I've added mcheck() to make it possible to
> debug memory errors from within Forth (and getting a meaningful
> backtrace), but Ulrich Drepper defeats that idea...
> 
> Ulrich Drepper's comment on this is:
> 
> "mcheck does not work for multi-threaded code. It cannot possibly
> do. There is no way to fix this with the technology underlying
> mcheck."
> 
> m(
> 
> This is a typical Ulrich Drepper: implement something in a way that can't 
> possibly work, and then claim it is impossible to fix...

C'mon Bernd, your English comprehension is better than that.  He
didn't say mcheck was impossible to fix, he said it's impossible to
fix the technology underlying mcheck.

Andrew.

[toc] | [prev] | [next] | [standalone]

#25887

Andrew Haley wrote:

> Bernd Paysan <bernd.paysan@gmx.de> wrote:
>> This is a typical Ulrich Drepper: implement something in a way that can't
>> possibly work, and then claim it is impossible to fix...
> 
> C'mon Bernd, your English comprehension is better than that.  He
> didn't say mcheck was impossible to fix, he said it's impossible to
> fix the technology underlying mcheck.

And he did the underlying technology (which are the malloc hooks).  So he 
did implement something that isn't possible to work in a multithreaded 
environmnent, and then is incapable of fixing it.  Why the heck is it 
impossible to have thread-safe hooks for exception handling or other things 
like sanity checking?

The whole exception stuff in malloc() is impossible to use reasonable.  If 
something goes wrong in malloc() or free(), the default (thread-safe) 
behavior is to print a C backtrace and a memory map and terminate the 
program.  For a VM, this is inadequate, because the C backtrace is 
meaningless, and terminating the program is a bad idea, too.  We have our 
own exception handling.

What I actually want is a hook for the exception reporting, which mcheck() 
doesn't really provide - it *also* turns on some more expensive sanity 
checking.  I don't need or want that, the only thing I want is to take over 
when the normal code detects that there is something wrong.  I would expect 
that only mcheck_pedantic() would install those sanity checks.

And first of all, I would expect that a non-thread-safe function for 
otherwise thread-safe stuff (malloc is very explicitely stated to be thread-
safe) would have a big warning label in the manpage.

-- 
Bernd Paysan
"If you want it done right, you have to do it yourself"
http://bernd-paysan.de/

[toc] | [prev] | [next] | [standalone]

#25892

Bernd Paysan <bernd.paysan@gmx.de> wrote:
> Andrew Haley wrote:
> 
>> Bernd Paysan <bernd.paysan@gmx.de> wrote:
>>> This is a typical Ulrich Drepper: implement something in a way that can't
>>> possibly work, and then claim it is impossible to fix...
>> 
>> C'mon Bernd, your English comprehension is better than that.  He
>> didn't say mcheck was impossible to fix, he said it's impossible to
>> fix the technology underlying mcheck.
> 
> And he did the underlying technology (which are the malloc hooks).
> So he did implement something that isn't possible to work in a
> multithreaded environmnent, and then is incapable of fixing it. 

So it seems.

> Why the heck is it impossible to have thread-safe hooks for
> exception handling or other things like sanity checking?

It's not, and he didn't say it was.  God knows, it's strange for me to
be defending him: Uli can annoy me just as much as he annoys everyone
else!

> The whole exception stuff in malloc() is impossible to use
> reasonable.  If something goes wrong in malloc() or free(), the
> default (thread-safe) behavior is to print a C backtrace and a
> memory map and terminate the program.  For a VM, this is inadequate,
> because the C backtrace is meaningless, and terminating the program
> is a bad idea, too.  We have our own exception handling.

The trouble with memory rashing bugs is that, in general, it's
extremely hard to recover from them in any reasonable way.  So, in
many cases terminating the programs makes very good sense.

> And first of all, I would expect that a non-thread-safe function for
> otherwise thread-safe stuff (malloc is very explicitely stated to be
> thread- safe) would have a big warning label in the manpage.

So would I.

Andrew.

[toc] | [prev] | [next] | [standalone]

#25903

Andrew Haley wrote:

> Bernd Paysan <bernd.paysan@gmx.de> wrote:
>> The whole exception stuff in malloc() is impossible to use
>> reasonable.  If something goes wrong in malloc() or free(), the
>> default (thread-safe) behavior is to print a C backtrace and a
>> memory map and terminate the program.  For a VM, this is inadequate,
>> because the C backtrace is meaningless, and terminating the program
>> is a bad idea, too.  We have our own exception handling.
> 
> The trouble with memory rashing bugs is that, in general, it's
> extremely hard to recover from them in any reasonable way.  So, in
> many cases terminating the programs makes very good sense.

This assumption is pretty wrong in Forth.  Well, I certainly can't continue 
to run the program which uses malloc() and then has a buffer overflow.  But: 
I can certainly keep the normal Forth system running, as this works pretty 
well without using malloc (unless you enter too long lines or so).  That 
allows to examine the corrupted memory, and possibly determine what went 
wrong.

bigForth has its own memory management system, and it adds enough stuff 
around the memory blocks to do sanity checking.  There's a background task 
which regularly sweeps the memory blocks, does compacting of movable blocks 
and also reports corruped blocks; it of course stops when it finds such a 
block.

This was very helpful to eradicate problems.

I'll now check if setting mallopt(M_CHECK_ACTION, 1) is a better way to 
avoid program termination.  It will not throw an exception, it will only 
print that there is a problem, and where it is.  This is of *much* more 
dubious quality than throwing an exception, because you really can't let a 
program continue to run in normal mode after it had corrupted its memory.

>> And first of all, I would expect that a non-thread-safe function for
>> otherwise thread-safe stuff (malloc is very explicitely stated to be
>> thread- safe) would have a big warning label in the manpage.
> 
> So would I.

I think that's the most annoying part.

-- 
Bernd Paysan
"If you want it done right, you have to do it yourself"
http://bernd-paysan.de/

[toc] | [prev] | [next] | [standalone]

#25931

Bernd Paysan <bernd.paysan@gmx.de> wrote:
> Andrew Haley wrote:
> 
>> Bernd Paysan <bernd.paysan@gmx.de> wrote:
>>> The whole exception stuff in malloc() is impossible to use
>>> reasonable.  If something goes wrong in malloc() or free(), the
>>> default (thread-safe) behavior is to print a C backtrace and a
>>> memory map and terminate the program.  For a VM, this is
>>> inadequate, because the C backtrace is meaningless, and
>>> terminating the program is a bad idea, too.  We have our own
>>> exception handling.
>> 
>> The trouble with memory [t]rashing bugs is that, in general, it's
>> extremely hard to recover from them in any reasonable way.  So, in
>> many cases terminating the programs makes very good sense.
> 
> This assumption is pretty wrong in Forth.

It's a statement about the C library, and more particularly malloc().
It's true because memory trashing often corrupts malloc()'s own data
structures.  It's also true of Forth if you damage the dictionary.  In
some cases you might be able to recover, of course; but there's no
general way to do so.

>>> And first of all, I would expect that a non-thread-safe function for
>>> otherwise thread-safe stuff (malloc is very explicitely stated to be
>>> thread- safe) would have a big warning label in the manpage.
>> 
>> So would I.
> 
> I think that's the most annoying part.

Mmm.  Patches welcome.  :-)

Andrew.

[toc] | [prev] | [next] | [standalone]

#25946

Andrew Haley wrote:

> Bernd Paysan <bernd.paysan@gmx.de> wrote:
>> Andrew Haley wrote:
>>> The trouble with memory [t]rashing bugs is that, in general, it's
>>> extremely hard to recover from them in any reasonable way.  So, in
>>> many cases terminating the programs makes very good sense.
>> 
>> This assumption is pretty wrong in Forth.
> 
> It's a statement about the C library, and more particularly malloc().
> It's true because memory trashing often corrupts malloc()'s own data
> structures.  It's also true of Forth if you damage the dictionary.  In
> some cases you might be able to recover, of course; but there's no
> general way to do so.

Usually you don't recover from errors in Forth, but you try to debug the 
situation.  That's the intention of the default debug handling code in 
glibc, but this default code only works for C programs - at least partially.  
The usual cause for this kind of error are buffer overflows, dumping the 
memory in the proximity of the problem also is helpful.

>>>> And first of all, I would expect that a non-thread-safe function for
>>>> otherwise thread-safe stuff (malloc is very explicitely stated to be
>>>> thread- safe) would have a big warning label in the manpage.
>>> 
>>> So would I.
>> 
>> I think that's the most annoying part.
> 
> Mmm.  Patches welcome.  :-)

To the manpage?  That's doable ;-).  To Uli's code?  :-/

-- 
Bernd Paysan
"If you want it done right, you have to do it yourself"
http://bernd-paysan.de/

[toc] | [prev] | [next] | [standalone]

#25902

> Bernd Paysan wrote:
> I've now disabled the use of mcheck(), and this seems to cure the problem. 
> Please try.  I've added mcheck() to make it possible to debug memory errors 
> from within Forth (and getting a meaningful backtrace), but Ulrich Drepper 
> defeats that idea...

Thx! Where/how? I didn't see updates at cvs or github.

[toc] | [prev] | [next] | [standalone]

#25905

VoidVolker wrote:

>> Bernd Paysan wrote:
>> I've now disabled the use of mcheck(), and this seems to cure the
>> problem.
>> Please try.  I've added mcheck() to make it possible to debug memory
>> errors from within Forth (and getting a meaningful backtrace), but Ulrich
>> Drepper defeats that idea...
> 
> Thx! Where/how? I didn't see updates at cvs or github.

In the official git repository of Gforth at Savannah.

git clone git://git.savannah.gnu.org/gforth.git

The github clone is done by Andreas Bolka; I don't know how to contact him.  
I've set up another git clone on GitHub, if people search for that stuff 
there.  Fortunately, it is fairly easy to keep multiple git repos in sync... 
so now you also can get it via

https://github.com/forthy42/gforth.git

or

git@github.com:forthy42/gforth.git

(ssh)

-- 
Bernd Paysan
"If you want it done right, you have to do it yourself"
http://bernd-paysan.de/

[toc] | [prev] | [next] | [standalone]

#25911

> I've set up another git clone on GitHub, if people search for that stuff 
> there.  Fortunately, it is fairly easy to keep multiple git repos in sync... 
> so now you also can get it via
> 
> https://github.com/forthy42/gforth.git

Nice! Thx!
Hmm... There are to some problem with date in arch/power/test /d/north.* and /x/north.* - this files have wrong date property. Plz fix it - git under win can't work with them correct.

[toc] | [prev] | [next] | [standalone]

#25913

VoidVolker wrote:

>> I've set up another git clone on GitHub, if people search for that stuff
>> there.  Fortunately, it is fairly easy to keep multiple git repos in
>> sync... so now you also can get it via
>> 
>> https://github.com/forthy42/gforth.git
> 
> Nice! Thx!
> Hmm... There are to some problem with date in arch/power/test /d/north.*
> and /x/north.* - this files have wrong date property. Plz fix it - git
> under win can't work with them correct.

Can you please send a detailed log of the problem.  I'm going to check 
myself on Windows, but that means first installing updates to Cygwin... my 
only Windows PC is a Eee PC, not very fast.

-- 
Bernd Paysan
"If you want it done right, you have to do it yourself"
http://bernd-paysan.de/

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | comp.lang.forth

csiph-web