Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #17475 > unrolled thread

malloc

Back to article view | Back to comp.lang.c

Contents

  malloc jgk@panix.com (Joe keane) - 2012-02-16 21:04 +0000
    Re: malloc Devil with the China Blue Dress <chine.bleu@yahoo.com> - 2012-02-16 13:21 -0800
      Re: malloc BGB <cr88192@hotmail.com> - 2012-02-16 16:16 -0700
        Re: malloc Paavo Helde <myfirstname@osa.pri.ee> - 2012-02-17 14:46 -0600
          Re: malloc Joshua Maurice <joshuamaurice@gmail.com> - 2012-02-17 14:51 -0800
            Re: malloc BGB <cr88192@hotmail.com> - 2012-02-17 19:58 -0700
              Re: malloc Paavo Helde <myfirstname@osa.pri.ee> - 2012-02-18 02:18 -0600
                Re: malloc BGB <cr88192@hotmail.com> - 2012-02-18 01:58 -0700
                Re: malloc glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2012-02-18 15:30 +0000
              Re: malloc Kaz Kylheku <kaz@kylheku.com> - 2012-02-18 10:36 +0000
                Re: malloc BGB <cr88192@hotmail.com> - 2012-02-18 10:53 -0700
              Re: malloc William Ahern <william@wilbur.25thandClement.com> - 2012-02-18 16:43 -0800
                Re: malloc BGB <cr88192@hotmail.com> - 2012-02-18 20:03 -0700
                  Re: malloc Miles Bader <miles@gnu.org> - 2012-02-19 12:49 +0900
                    Re: malloc William Ahern <william@wilbur.25thandClement.com> - 2012-02-18 21:08 -0800
                    Re: malloc "io_x" <a@b.c.invalid> - 2012-02-19 18:23 +0100
                  Re: malloc William Ahern <william@wilbur.25thandClement.com> - 2012-02-18 21:06 -0800
                    Re: malloc BGB <cr88192@hotmail.com> - 2012-02-18 23:06 -0700
        Re: malloc "Asger Joergensen" <Junk@Asger-P.dk> - 2012-02-19 15:54 +0000
          Re: malloc BGB <cr88192@hotmail.com> - 2012-02-19 09:37 -0700
      Re: malloc Eric Sosman <esosman@ieee-dot-org.invalid> - 2012-02-16 20:27 -0500
        Re: malloc Goran <goran.pusic@gmail.com> - 2012-02-16 23:40 -0800
          Re: malloc Keith Thompson <kst-u@mib.org> - 2012-02-17 02:26 -0800
            Re: malloc Goran <goran.pusic@gmail.com> - 2012-02-17 04:01 -0800
              Re: malloc Keith Thompson <kst-u@mib.org> - 2012-02-17 09:25 -0800
                Re: malloc Joshua Maurice <joshuamaurice@gmail.com> - 2012-02-17 14:49 -0800
                  Re: malloc glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2012-02-18 01:43 +0000
                    Re: malloc Kaz Kylheku <kaz@kylheku.com> - 2012-02-18 02:04 +0000
                  Re: malloc scott@slp53.sl.home (Scott Lurndal) - 2012-02-18 03:04 +0000
                    Re: malloc Robert Wessel <robertwessel2@yahoo.com> - 2012-02-17 22:15 -0600
                    Re: malloc Joshua Maurice <joshuamaurice@gmail.com> - 2012-02-17 23:48 -0800
                Re: malloc Kaz Kylheku <kaz@kylheku.com> - 2012-02-17 23:29 +0000
                  Re: malloc William Ahern <william@wilbur.25thandClement.com> - 2012-02-17 15:42 -0800
                    Re: malloc "io_x" <a@b.c.invalid> - 2012-02-19 18:24 +0100
                      Re: malloc Kleuske <kleuske@nowhere.net> - 2012-02-19 20:41 +0000
                        Re: malloc James Kuyper <jameskuyper@verizon.net> - 2012-02-19 18:05 -0500
                          Re: malloc "io_x" <a@b.c.invalid> - 2012-02-20 12:24 +0100
                          Re: malloc Kleuske <kleuske@nowhere.net> - 2012-02-20 13:56 +0000
                            Re: malloc "io_x" <a@b.c.invalid> - 2012-02-20 15:14 +0100
                      Re: malloc William Ahern <william@wilbur.25thandClement.com> - 2012-02-19 15:19 -0800
          Re: malloc Eric Sosman <esosman@ieee-dot-org.invalid> - 2012-02-17 08:06 -0500
            Re: malloc Goran <goran.pusic@gmail.com> - 2012-02-18 07:57 -0800
              Re: malloc Keith Thompson <kst-u@mib.org> - 2012-02-18 10:57 -0800
              Re: malloc Eric Sosman <esosman@ieee-dot-org.invalid> - 2012-02-19 08:51 -0500
        Re: malloc Noob <root@127.0.0.1> - 2012-02-17 11:44 +0100
          Re: malloc "Joachim Schmitz" <nospam.jojo@schmitz-digital.de> - 2012-02-17 17:05 +0100
            Re: malloc pete <pfiland@mindspring.com> - 2012-02-17 11:19 -0500
              Re: malloc Noob <root@127.0.0.1> - 2012-02-17 17:30 +0100
            Re: malloc James Kuyper <jameskuyper@verizon.net> - 2012-02-17 11:31 -0500
        Re: malloc scott@slp53.sl.home (Scott Lurndal) - 2012-02-17 20:31 +0000
          Re: malloc Joshua Maurice <joshuamaurice@gmail.com> - 2012-02-18 00:01 -0800
            Re: malloc scott@slp53.sl.home (Scott Lurndal) - 2012-02-18 17:07 +0000
              Re: malloc Joshua Maurice <joshuamaurice@gmail.com> - 2012-02-18 18:14 -0800
                Re: malloc scott@slp53.sl.home (Scott Lurndal) - 2012-02-19 17:14 +0000
    Re: malloc Goran <goran.pusic@gmail.com> - 2012-02-16 23:43 -0800
    Re: malloc "io_x" <a@b.c.invalid> - 2012-02-19 16:44 +0100

Page 2 of 3 — ← Prev page 1 [2] 3  Next page →

#17496

On 2/16/2012 4:21 PM, Devil with the China Blue Dress wrote:
> In article<jhjr14$rjk$1@reader1.panix.com>, jgk@panix.com (Joe keane) wrote:
>
>> Bugs of memory allocation will make you mad.
>>
>> Bugs *in* memory allocation will put you in the cuckoo people place.
>
> Which is why they are exceedingly rare. Nearly all allocation problems are due
> to the program storing outside array bounds.

     The only allocator bug I have personally encountered was with
a malloc() implementation that never, never returned NULL.  When it
ought to have returned NULL, it crashed the program instead ...

-- 
Eric Sosman
esosman@ieee-dot-org.invalid

[toc] | [prev] | [next] | [standalone]

#17499

On Feb 17, 2:27 am, Eric Sosman <esos...@ieee-dot-org.invalid> wrote:
> On 2/16/2012 4:21 PM, Devil with the China Blue Dress wrote:
>
> > In article<jhjr14$rj...@reader1.panix.com>, j...@panix.com (Joe keane) wrote:
>
> >> Bugs of memory allocation will make you mad.
>
> >> Bugs *in* memory allocation will put you in the cuckoo people place.
>
> > Which is why they are exceedingly rare. Nearly all allocation problems are due
> > to the program storing outside array bounds.
>
>      The only allocator bug I have personally encountered was with
> a malloc() implementation that never, never returned NULL.  When it
> ought to have returned NULL, it crashed the program instead ...

If you don't provide a way to verify that, it didn't happen ;-), and
there was a bug in your code.

Goran.

[toc] | [prev] | [next] | [standalone]

#17505

Goran <goran.pusic@gmail.com> writes:
> On Feb 17, 2:27 am, Eric Sosman <esos...@ieee-dot-org.invalid> wrote:
>> On 2/16/2012 4:21 PM, Devil with the China Blue Dress wrote:
>>
>> > In article<jhjr14$rj...@reader1.panix.com>, j...@panix.com (Joe keane) wrote:
>>
>> >> Bugs of memory allocation will make you mad.
>>
>> >> Bugs *in* memory allocation will put you in the cuckoo people place.
>>
>> > Which is why they are exceedingly rare. Nearly all allocation problems are due
>> > to the program storing outside array bounds.
>>
>>      The only allocator bug I have personally encountered was with
>> a malloc() implementation that never, never returned NULL.  When it
>> ought to have returned NULL, it crashed the program instead ...
>
> If you don't provide a way to verify that, it didn't happen ;-), and
> there was a bug in your code.

I suspect he's referring to the malloc() implementation
on typical Linux systems, which overcommits memory by default.
It can allocate a large chunk of address space for which no actual
memory is available.  The memory isn't actually allocated until
the process attempts to access it.  If there isn't enough memory
available for the allocation, the "OOM killer" kills some process
(not necessarily the one that did the allocation).

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
    Will write code for food.
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

[toc] | [prev] | [next] | [standalone]

#17509

On Feb 17, 11:26 am, Keith Thompson <ks...@mib.org> wrote:
> Goran <goran.pu...@gmail.com> writes:
> > On Feb 17, 2:27 am, Eric Sosman <esos...@ieee-dot-org.invalid> wrote:
> >> On 2/16/2012 4:21 PM, Devil with the China Blue Dress wrote:
>
> >> > In article<jhjr14$rj...@reader1.panix.com>, j...@panix.com (Joe keane) wrote:
>
> >> >> Bugs of memory allocation will make you mad.
>
> >> >> Bugs *in* memory allocation will put you in the cuckoo people place.
>
> >> > Which is why they are exceedingly rare. Nearly all allocation problems are due
> >> > to the program storing outside array bounds.
>
> >>      The only allocator bug I have personally encountered was with
> >> a malloc() implementation that never, never returned NULL.  When it
> >> ought to have returned NULL, it crashed the program instead ...
>
> > If you don't provide a way to verify that, it didn't happen ;-), and
> > there was a bug in your code.
>
> I suspect he's referring to the malloc() implementation
> on typical Linux systems, which overcommits memory by default.
> It can allocate a large chunk of address space for which no actual
> memory is available.  The memory isn't actually allocated until
> the process attempts to access it.  If there isn't enough memory
> available for the allocation, the "OOM killer" kills some process
> (not necessarily the one that did the allocation).

That crossed my mind, but what he said doesn't correspond with what
happens: malloc does return something and __doesn't__ crash the
program. OOM killer kills the code upon an attempt to access that
memory.

But given the way he explained it, it's possible that he's affected by
OOM killer, and he forgot, or never knew, what really happened.

Goran.

[toc] | [prev] | [next] | [standalone]

#17523

Goran <goran.pusic@gmail.com> writes:
> On Feb 17, 11:26 am, Keith Thompson <ks...@mib.org> wrote:
[...]
>> I suspect he's referring to the malloc() implementation
>> on typical Linux systems, which overcommits memory by default.
>> It can allocate a large chunk of address space for which no actual
>> memory is available.  The memory isn't actually allocated until
>> the process attempts to access it.  If there isn't enough memory
>> available for the allocation, the "OOM killer" kills some process
>> (not necessarily the one that did the allocation).
>
> That crossed my mind, but what he said doesn't correspond with what
> happens: malloc does return something and __doesn't__ crash the
> program. OOM killer kills the code upon an attempt to access that
> memory.

If you call malloc() and it overcommits, it won't crash the
program until you access the allocated memory.  (The rationale for
overcommitting is that most programs don't actually use most of
the memory the memory the allocate.  I find that odd)

> But given the way he explained it, it's possible that he's affected by
> OOM killer, and he forgot, or never knew, what really happened.

Given the post you're responding to, I would find it more likely that
he knows exactly what happened and didn't mention it.  Reading his
more recent followup, apparently it was on VMS, not Linux, and was
likely a bug in DEC's C library.

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
    Will write code for food.
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

[toc] | [prev] | [next] | [standalone]

#17542

On Feb 17, 9:25 am, Keith Thompson <ks...@mib.org> wrote:
> If you call malloc() and it overcommits, it won't crash the
> program until you access the allocated memory.  (The rationale for
> overcommitting is that most programs don't actually use most of
> the memory the memory the allocate.  I find that odd)

Really? The explanation that I'm most familiar with is that most fork
calls are immediately followed by exec, and thus if you're low on
memory, then a large process cannot spawn a new process without
overcommit because the only process create primitive is fork, which
"copies" the memory space of the parent process.

I of course think this is a broken state of affairs for several
reasons. 1- Just introduce a new create process primitive that creates
a process from an executable file with a copy options for specifying
the env vars, the command line, the working dir, etc. Sure it lacks
the full "power" of fork + exec, but it's a lot easier to use, and for
most uses of fork, I expect this would suffice, and then we could
avoid this nasty overcommit problem. 2- I'm annoyed at the ease at
which resources can be leaked to child processes and the near
impossibility to do anything portably about it.

However, due to discussions on this board, I've come to learn that OOM
situations are very hard to program for, and OOM in a common desktop
just can't be handled gracefully.

[toc] | [prev] | [next] | [standalone]

#17551

In comp.lang.c++ Joshua Maurice <joshuamaurice@gmail.com> wrote:

(snip)

> Really? The explanation that I'm most familiar with is that most fork
> calls are immediately followed by exec, and thus if you're low on
> memory, then a large process cannot spawn a new process without
> overcommit because the only process create primitive is fork, which
> "copies" the memory space of the parent process.

That was fixed about 20 years ago. Among others, there is vfork()

  "vfork - spawn new process in a virtual memory efficient way"

A simple explanation is that vfork() tells the system that you 
expect to call exec() next, and it can optimize for that case.

> I of course think this is a broken state of affairs for several
> reasons. 1- Just introduce a new create process primitive that creates
> a process from an executable file with a copy options for specifying
> the env vars, the command line, the working dir, etc. 

(snip)

-- glen

[toc] | [prev] | [next] | [standalone]

#17552

On 2012-02-18, glen herrmannsfeldt <gah@ugcs.caltech.edu> wrote:
> In comp.lang.c++ Joshua Maurice <joshuamaurice@gmail.com> wrote:
>
> (snip)
>
>> Really? The explanation that I'm most familiar with is that most fork
>> calls are immediately followed by exec, and thus if you're low on
>> memory, then a large process cannot spawn a new process without
>> overcommit because the only process create primitive is fork, which
>> "copies" the memory space of the parent process.
>
> That was fixed about 20 years ago. Among others, there is vfork()
>
>   "vfork - spawn new process in a virtual memory efficient way"
>
> A simple explanation is that vfork() tells the system that you 
> expect to call exec() next, and it can optimize for that case.

vfork is a dangerous hack which exposes the semantics of the optimization
of fork to the program.

A modern copy-on-write fork hides the semantics: the parent and child
spaces are shared, but appear duplicated.

A copy-on-write fork does have to account for the virtual space required
to duplicate the private mappings of the parent process, because those
will be copied physically if they are touched.

If you have a 500 megabyte process, of which 400 megabytes are private
mappings, and that process forks, the virtual layout of the system
increases by 400 megabytes. If overcommit is not allowed, that means
that the 400 megabytes has to be counted as physical memory.

Joshua is completely right here: forking is one of the use cases for
overcommit for this reason.

[toc] | [prev] | [next] | [standalone]

#17554

Joshua Maurice <joshuamaurice@gmail.com> writes:
>On Feb 17, 9:25=A0am, Keith Thompson <ks...@mib.org> wrote:
>> If you call malloc() and it overcommits, it won't crash the
>> program until you access the allocated memory. =A0(The rationale for
>> overcommitting is that most programs don't actually use most of
>> the memory the memory the allocate. =A0I find that odd)
>
>Really? The explanation that I'm most familiar with is that most fork
>calls are immediately followed by exec, and thus if you're low on
>memory, then a large process cannot spawn a new process without
>overcommit because the only process create primitive is fork, which
>"copies" the memory space of the parent process.

You're confusing overcommit with copy-on-write.  Fork uses COW[*] in
which the parent and child share the physical pages until the child
writes to one - at that point, they child gets a copy (and an allocation
occurs which may fail at that point if memory and swap are exhausted).

Overcommit was allowed to support sparse arrays which are common
with some workloads.

scott

(p.s.  see 'posix_spawn').

[*] COW came into general use in the SVR3.2/SVR4 timeframe.  Linux has always
    used COW on fork.   The only cost for the child is the page table
    (which actually can be quite a bit for a 1TB virtual address space using
     4k pages - IIRC about 2GB just for page tables to map that much VA; makes
     1G pages much more attractive (drops the page table size to 8k)).

[toc] | [prev] | [next] | [standalone]

#17555

On 18 Feb 2012 03:04:14 GMT, scott@slp53.sl.home (Scott Lurndal)
wrote:

>Joshua Maurice <joshuamaurice@gmail.com> writes:
>>On Feb 17, 9:25=A0am, Keith Thompson <ks...@mib.org> wrote:
>>> If you call malloc() and it overcommits, it won't crash the
>>> program until you access the allocated memory. =A0(The rationale for
>>> overcommitting is that most programs don't actually use most of
>>> the memory the memory the allocate. =A0I find that odd)
>>
>>Really? The explanation that I'm most familiar with is that most fork
>>calls are immediately followed by exec, and thus if you're low on
>>memory, then a large process cannot spawn a new process without
>>overcommit because the only process create primitive is fork, which
>>"copies" the memory space of the parent process.
>
>You're confusing overcommit with copy-on-write.  Fork uses COW[*] in
>which the parent and child share the physical pages until the child
>writes to one - at that point, they child gets a copy (and an allocation
>occurs which may fail at that point if memory and swap are exhausted).
>
>Overcommit was allowed to support sparse arrays which are common
>with some workloads.


I don't think there's much daylight between the two.  In both cases a
process has been implicitly allocated virtual pages, without
(necessarily) the presence of enough actual pages (real and/or disk)
to make good on that allocation, and the system can fail in the future
when the application does something apparently innocuous, like try to
write to a page it appears to have been allocated.

And while I'm not familiar with the implementation of all over-commit
schemes, at least a couple I've seen actually map all allocated pages
to a read-only pages full of zeros, and then use the COW mechanism to
allocate actual pages when they're written to.  This has the advantage
of not requiring a second mechanism, and possibly further delaying the
actual allocation (as an ordinary read does not trigger it).

[toc] | [prev] | [next] | [standalone]

#17556

On Feb 17, 7:04 pm, sc...@slp53.sl.home (Scott Lurndal) wrote:
> Joshua Maurice <joshuamaur...@gmail.com> writes:
> >Really? The explanation that I'm most familiar with is that most fork
> >calls are immediately followed by exec, and thus if you're low on
> >memory, then a large process cannot spawn a new process without
> >overcommit because the only process create primitive is fork, which
> >"copies" the memory space of the parent process.
>
> You're confusing overcommit with copy-on-write.  Fork uses COW[*] in
> which the parent and child share the physical pages until the child
> writes to one - at that point, they child gets a copy (and an allocation
> occurs which may fail at that point if memory and swap are exhausted).
>
> Overcommit was allowed to support sparse arrays which are common
> with some workloads.
[...]
> [*] COW came into general use in the SVR3.2/SVR4 timeframe.  Linux has always
>     used COW on fork.   The only cost for the child is the page table
>     (which actually can be quite a bit for a 1TB virtual address space using
>      4k pages - IIRC about 2GB just for page tables to map that much VA; makes
>      1G pages much more attractive (drops the page table size to 8k)).

So, I thought that if I turned overcommit off in Linux that if I tried
to fork with a large process and low commit, then the fork would fail.
(We're getting a little off topic, but I do not care.)

> (p.s.  see 'posix_spawn').

posix_spawn solves the "COW" and fork memory problem, but posix_spawn
still has all of the same process w.r.t. leaking resources to child
processes because its defined semantics are "as if fork followed by
exec".

[toc] | [prev] | [next] | [standalone]

#17547

["Followup-To:" header set to comp.lang.c.]
On 2012-02-17, Keith Thompson <kst-u@mib.org> wrote:
> Goran <goran.pusic@gmail.com> writes:
>> On Feb 17, 11:26 am, Keith Thompson <ks...@mib.org> wrote:
> [...]
>>> I suspect he's referring to the malloc() implementation
>>> on typical Linux systems, which overcommits memory by default.
>>> It can allocate a large chunk of address space for which no actual
>>> memory is available.  The memory isn't actually allocated until
>>> the process attempts to access it.  If there isn't enough memory
>>> available for the allocation, the "OOM killer" kills some process
>>> (not necessarily the one that did the allocation).
>>
>> That crossed my mind, but what he said doesn't correspond with what
>> happens: malloc does return something and __doesn't__ crash the
>> program. OOM killer kills the code upon an attempt to access that
>> memory.
>
> If you call malloc() and it overcommits, it won't crash the
> program until you access the allocated memory.  (The rationale for
> overcommitting is that most programs don't actually use most of
> the memory the memory the allocate.  I find that odd)

Odd or not, it is borne out empirically. Applications are physically
smaller than their virtual footprints.

It may be the case that C programs that malloc something usually use
the whole block.

But overcommitting is not implemented at the level of malloc, but
at the level of a lower level allocator like mmap.

If the system maps a large block to give you a smaller one, that large
block will not be all used immediately.

Another example is thread stacks. If you give each thread a one megabyte
stack and make 100 threads, that's 100 megs of virtual space. But that one
megabyte is a worst case that few, if any, of the threads will hit.

Programs with lots of threads on GNU/Linux have inflated virtual footprints
due to the generous default stack size.

[toc] | [prev] | [next] | [standalone]

#17548

In comp.lang.c Kaz Kylheku <kaz@kylheku.com> wrote:
> ["Followup-To:" header set to comp.lang.c.]
> On 2012-02-17, Keith Thompson <kst-u@mib.org> wrote:
<snip>
> > If you call malloc() and it overcommits, it won't crash the
> > program until you access the allocated memory.  (The rationale for
> > overcommitting is that most programs don't actually use most of
> > the memory the memory the allocate.  I find that odd)

> Odd or not, it is borne out empirically. Applications are physically
> smaller than their virtual footprints.

Supposedly, and years ago when application developers where chummy with
the kernel folks.

> It may be the case that C programs that malloc something usually use
> the whole block.
>
> But overcommitting is not implemented at the level of malloc, but
> at the level of a lower level allocator like mmap.
>
> If the system maps a large block to give you a smaller one, that large
> block will not be all used immediately.

No, but it's not like allocators are mmap'ing large fractions of available
memory. The trend is toward more and smaller mmap'd backing blocks to, e.g.,
improve address randomization.

> Another example is thread stacks. If you give each thread a one megabyte
> stack and make 100 threads, that's 100 megs of virtual space. But that one
> megabyte is a worst case that few, if any, of the threads will hit.

Fortunately GCC already supports segmented stacks. Such support would have
been there sooner were it not for overcommit reducing the apparent demand
for such a feature. Although, glibc doesn't seem to be too eager to jump
aboard. Perhaps they feel that a 5% performance hit is too high a cost to
get rid of applications randomly crashing under high loads.

[toc] | [prev] | [next] | [standalone]

#17613

"William Ahern" <william@wilbur.25thandClement.com> ha scritto nel messaggio
news:s85319-dhd.ln1@wilbur.25thandClement.com...
> In comp.lang.c Kaz Kylheku <kaz@kylheku.com> wrote:
>> ["Followup-To:" header set to comp.lang.c.]
>> On 2012-02-17, Keith Thompson <kst-u@mib.org> wrote:
> <snip>
>> > If you call malloc() and it overcommits, it won't crash the
>> > program until you access the allocated memory.  (The rationale for
>> > overcommitting is that most programs don't actually use most of
>> > the memory the memory the allocate.  I find that odd)
>
>> Odd or not, it is borne out empirically. Applications are physically
>> smaller than their virtual footprints.
>
> Supposedly, and years ago when application developers where chummy with
> the kernel folks.
>
>> It may be the case that C programs that malloc something usually use
>> the whole block.
>>
>> But overcommitting is not implemented at the level of malloc, but
>> at the level of a lower level allocator like mmap.
>>
>> If the system maps a large block to give you a smaller one, that large
>> block will not be all used immediately.
>
> No, but it's not like allocators are mmap'ing large fractions of available
> memory. The trend is toward more and smaller mmap'd backing blocks to, e.g.,
> improve address randomization.
>
>> Another example is thread stacks. If you give each thread a one megabyte
>> stack and make 100 threads, that's 100 megs of virtual space. But that one
>> megabyte is a worst case that few, if any, of the threads will hit.
>
> Fortunately GCC already supports segmented stacks. Such support would have
> been there sooner were it not for overcommit reducing the apparent demand
> for such a feature. Although, glibc doesn't seem to be too eager to jump
> aboard. Perhaps they feel that a 5% performance hit is too high a cost to
> get rid of applications randomly crashing under high loads.

the problem is not the number of stacks
the problem is not the few memory....
now we have enougjht memory for doing all even 100000 threads etc
[here there are pc with 1GB, 10GB etc cpu are always better with
always more cores]
if people not know how to do it, it is because use their fat languages
and their not like of say few words to cpu for doing operations

so for one not problem, they use the wrong solution
2 wrongs
Saluti

[toc] | [prev] | [next] | [standalone]

#17625

On Sun, 19 Feb 2012 18:24:28 +0100, io_x saw fit to publish the following:

<snip sensible part>

> the problem is not the number of stacks the problem is not the few
> memory.... now we have enougjht memory for doing all even 100000 threads
> etc [here there are pc with 1GB, 10GB etc cpu are always better with
> always more cores]
> if people not know how to do it, it is because use their fat languages
> and their not like of say few words to cpu for doing operations
> 
> so for one not problem, they use the wrong solution 2 wrongs
> Saluti

Is the above intended to be coherent in any way, shape or form? If so,
the coherent bits managed to elude me.
-- 
He's dead, Jim.
		-- McCoy, "The Devil in the Dark", stardate 3196.1

[toc] | [prev] | [next] | [standalone]

#17630

On 02/19/2012 03:41 PM, Kleuske wrote:
> On Sun, 19 Feb 2012 18:24:28 +0100, io_x saw fit to publish the following:
...
> Is the above intended to be coherent in any way, shape or form?

Consider who you're responding to - that should answer your question.
-- 
James Kuyper

[toc] | [prev] | [next] | [standalone]

#17647

"James Kuyper" <jameskuyper@verizon.net> ha scritto nel messaggio 
news:jhrv75$nl4$1@dont-email.me...
> On 02/19/2012 03:41 PM, Kleuske wrote:
>> On Sun, 19 Feb 2012 18:24:28 +0100, io_x saw fit to publish the following:
> ...
>> Is the above intended to be coherent in any way, shape or form?
>
> Consider who you're responding to - that should answer your question.

who is speaking.... the champion of longer post

> -- 
> James Kuyper 

[toc] | [prev] | [next] | [standalone]

#17660

On Sun, 19 Feb 2012 18:05:09 -0500, James Kuyper saw fit to publish the
following:

> On 02/19/2012 03:41 PM, Kleuske wrote:
>> On Sun, 19 Feb 2012 18:24:28 +0100, io_x saw fit to publish the
>> following:
> ...
>> Is the above intended to be coherent in any way, shape or form?
> 
> Consider who you're responding to - that should answer your question.

It was rather rhetorical, but periodically, someone should point it out.
-- 
It is necessary to have purpose.
		-- Alice #1, "I, Mudd", stardate 4513.3

[toc] | [prev] | [next] | [standalone]

#17661

"Kleuske" <kleuske@nowhere.net> ha scritto nel messaggio 
news:jhtjdq$ato$1@dont-email.me...
> On Sun, 19 Feb 2012 18:05:09 -0500, James Kuyper saw fit to publish the
> following:
>
>> On 02/19/2012 03:41 PM, Kleuske wrote:
>>> On Sun, 19 Feb 2012 18:24:28 +0100, io_x saw fit to publish the
>>> following:
>> ...
>>> Is the above intended to be coherent in any way, shape or form?
>>
>> Consider who you're responding to - that should answer your question.
>
> It was rather rhetorical, but periodically, someone should point it out.

yes periodically...
why you not plonk me so i not have to read your think about??
thanks

> -- 
> It is necessary to have purpose.

it is necessary to have the clue

> -- Alice #1, "I, Mudd", stardate 4513.3 

[toc] | [prev] | [next] | [standalone]

#17632

io_x <a@b.c.invalid> wrote:
> "William Ahern" <william@wilbur.25thandClement.com> ha scritto nel messaggio
> news:s85319-dhd.ln1@wilbur.25thandClement.com...
<snip>
> the problem is not the number of stacks
> the problem is not the few memory....
> now we have enougjht memory for doing all even 100000 threads etc
> [here there are pc with 1GB, 10GB etc cpu are always better with
> always more cores]

Let's add that up:

	100000 * (8 * 2^20) == 838860800000

That's 800GB!

The most recent server I purchased is a single-core 1Ghz Atom with 1GB of
memory. Specifically, a Soekris net6501. And I disagree that more cores is
better. I prefer scaling out rather than up, and when you have to design for
message passing anyhow the conveniences of threading can turn into
liabilities.

> if people not know how to do it, it is because use their fat languages
> and their not like of say few words to cpu for doing operations

> so for one not problem, they use the wrong solution
> 2 wrongs

Not sure what you meant here. I agree selecting the right solution (e.g.
data structure) is important. But the type of language one uses usually
doesn't have much to do with anything except personal and organizational
preference. It's rare that low-level hardware access supports a significant
algorithmic improvement, and when such possibilities do exist they can
usually be implemented in a language agnostic manner.

[toc] | [prev] | [next] | [standalone]

Page 2 of 3 — ← Prev page 1 [2] 3  Next page →

Back to top | Article view | comp.lang.c

csiph-web