Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c++ > #3466 > unrolled thread

To standardize Boost.Pool

Back to article view | Back to comp.lang.c++

Contents

  To standardize Boost.Pool  Phil Bouchard <phil@fornux.com> - 2011-04-12 07:44 -0600
    Re: To standardize Boost.Pool Jorgen Grahn <grahn+nntp@snipabacken.se> - 2011-04-12 21:56 +0000
      Re: To standardize Boost.Pool Phil Bouchard <phil@fornux.com> - 2011-04-12 17:31 -0700
        Re: To standardize Boost.Pool Phil Bouchard <phil@fornux.com> - 2011-04-13 00:08 -0700
          Re: To standardize Boost.Pool Öö Tiib <ootiib@hot.ee> - 2011-04-13 08:28 -0700
            Re: To standardize Boost.Pool Phil Bouchard <phil@fornux.com> - 2011-04-13 09:41 -0700
              Re: To standardize Boost.Pool Phil Bouchard <phil@fornux.com> - 2011-04-13 14:49 -0700
                Re: To standardize Boost.Pool Phil Bouchard <phil@fornux.com> - 2011-04-13 15:57 -0700
                  Re: To standardize Boost.Pool Paavo Helde <myfirstname@osa.pri.ee> - 2011-04-13 23:39 -0500
                    Re: To standardize Boost.Pool Phil Bouchard <phil@fornux.com> - 2011-04-13 22:43 -0700
                      Re: To standardize Boost.Pool Phil Bouchard <phil@fornux.com> - 2011-04-13 22:53 -0700
                        Re: To standardize Boost.Pool Miles Bader <miles@gnu.org> - 2011-04-14 15:55 +0900
                          Re: To standardize Boost.Pool Phil Bouchard <phil@fornux.com> - 2011-04-14 00:15 -0700

#3466 — To standardize Boost.Pool

Boost.Pool provides an is_from() member function that does a simple
range checks of the heap memory pages and returns whether a pointer is
part of the pool or not.  This is a very useful function which I think
should be part of the standards because it could be used for the global
pool as well; i.e. the one used by operator ::new and ::delete.  The
benefits for memory management of knowing whether an object resides on
the heap or stack outweigh the costs of having to pollute the global
namespace.


Thanks,
-Phil


-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

[toc] | [next] | [standalone]

#3527

[I skip the c.l.c++.moderated crosspost]

On Tue, 2011-04-12, Phil Bouchard wrote:
>
> Boost.Pool provides an is_from() member function that does a simple
> range checks of the heap memory pages and returns whether a pointer is
> part of the pool or not.  This is a very useful function which I think
> should be part of the standards because it could be used for the global
> pool as well; i.e. the one used by operator ::new and ::delete.  The
> benefits for memory management of knowing whether an object resides on
> the heap or stack outweigh the costs of having to pollute the global
> namespace.

What would those benefits be, according to you? Detecting bugs (e.g.
assert_on_heap(ptr)), or something else?

/Jorgen

-- 
  // Jorgen Grahn <grahn@  Oo  o.   .  .
\X/     snipabacken.se>   O  o   .

[toc] | [prev] | [next] | [standalone]

#3531

On 4/12/2011 2:56 PM, Jorgen Grahn wrote:
>
> What would those benefits be, according to you? Detecting bugs (e.g.
> assert_on_heap(ptr)), or something else?

Garbage collectors make use of such a function but also another memory 
manager I wrote called: Shifted Pointer where we can see a repetitive 
use of the function is_from() here:
http://www.fornux.com/personal/philippe/devel/shifted_ptr/boost/shifted_ptr.hpp


Thanks,
-Phil

[toc] | [prev] | [next] | [standalone]

#3539

On 4/12/2011 5:31 PM, Phil Bouchard wrote:
> On 4/12/2011 2:56 PM, Jorgen Grahn wrote:
>>
>> What would those benefits be, according to you? Detecting bugs (e.g.
>> assert_on_heap(ptr)), or something else?
>
> Garbage collectors make use of such a function but also another memory
> manager I wrote called: Shifted Pointer where we can see a repetitive
> use of the function is_from() here:
> http://www.fornux.com/personal/philippe/devel/shifted_ptr/boost/shifted_ptr.hpp

After some discussion with Boost, contiguous memory pool objects could 
represent the process' stack, data and heap segments legally.  For example:

struct process
{
	static pool data;
	static pool heap;
	static pool stack;
};

The default implementation of the global operator ::new could be:

void * operator new(size_t n) { return process::heap.malloc(n); }


A call to is_from() with a pointer guaranteed to be within one of the 3 
pools will then be valid:

int * h = new int;
process::heap.is_from(h) // true
int s;
process::heap.is_from(& s) // false
process::stack.is_from(& s) // true
static int d;
process::heap.is_from(& d) // false
process::stack.is_from(& d) // false
process::data.is_from(& d) // true


Thanks,
-Phil

[toc] | [prev] | [next] | [standalone]

#3577

On Apr 13, 10:08 am, Phil Bouchard <p...@fornux.com> wrote:
> On 4/12/2011 5:31 PM, Phil Bouchard wrote:
>
> > On 4/12/2011 2:56 PM, Jorgen Grahn wrote:
>
> >> What would those benefits be, according to you? Detecting bugs (e.g.
> >> assert_on_heap(ptr)), or something else?
>
> > Garbage collectors make use of such a function but also another memory
> > manager I wrote called: Shifted Pointer where we can see a repetitive
> > use of the function is_from() here:
> >http://www.fornux.com/personal/philippe/devel/shifted_ptr/boost/shift...
>
> After some discussion with Boost, contiguous memory pool objects could
> represent the process' stack, data and heap segments legally.  For example:
>
> struct process
> {
>         static pool data;
>         static pool heap;
>         static pool stack;
>
> };
>
> The default implementation of the global operator ::new could be:
>
> void * operator new(size_t n) { return process::heap.malloc(n); }
>
> A call to is_from() with a pointer guaranteed to be within one of the 3
> pools will then be valid:
>
> int * h = new int;
> process::heap.is_from(h) // true
> int s;
> process::heap.is_from(& s) // false
> process::stack.is_from(& s) // true
> static int d;
> process::heap.is_from(& d) // false
> process::stack.is_from(& d) // false
> process::data.is_from(& d) // true

Please elaborate what is process::stack? On most platforms a process
may have several threads, number of these threads often changes
dynamically during processes lifetime and each of these threads does
have its own stack.

[toc] | [prev] | [next] | [standalone]

#3585

On 4/13/2011 8:28 AM, Öö Tiib wrote:
>
> Please elaborate what is process::stack? On most platforms a process
> may have several threads, number of these threads often changes
> dynamically during processes lifetime and each of these threads does
> have its own stack.

process::stack represents the memory pages reserved for the stack of the 
current thread quite simply.  The details of the implementation would 
have to be hidden from the programmer so that all that he is exposed to 
are member functions such as malloc(size_t), free(void *), if_from(void 
*), etc.  Note that the malloc(size_t) call would be available only for 
the heap memory pool and so on.

I fail to see any other alternatives than that if a memory manager 
becomes eventually part of the standards.


Thanks,
-Phil

[toc] | [prev] | [next] | [standalone]

#3606

On 4/13/2011 9:41 AM, Phil Bouchard wrote:
> On 4/13/2011 8:28 AM, Öö Tiib wrote:
>>
>> Please elaborate what is process::stack? On most platforms a process
>> may have several threads, number of these threads often changes
>> dynamically during processes lifetime and each of these threads does
>> have its own stack.
>
> process::stack represents the memory pages reserved for the stack of the
> current thread quite simply. The details of the implementation would
> have to be hidden from the programmer so that all that he is exposed to
> are member functions such as malloc(size_t), free(void *), if_from(void
> *), etc. Note that the malloc(size_t) call would be available only for
> the heap memory pool and so on.

There was a discussion on the validity of a 'false' statement returned 
by is_from() because out-of-bounds pointer comparison would invoke 
undefined behavior according to 5.7.6.  This was bringing the idea of 
having adjacent pools stack & data to the heap pool.

But it looks like heap.is_from() could work without adjacent stack and 
data pools in all cases, even if the pointers are arbitrary.  This is 
because they can be casted to an integral type before they are compared 
together as stated by 5.2.10/4.  The following example is thus valid for 
arbitrary pointers:

bool is_from(const char* p)
{
     return reinterpret_cast<long>(buffer)<= reinterpret_cast<long>(p)&& 
  reinterpret_cast<long>(p)<  reinterpret_cast<long>(buffer + 
sizeof(buffer));
}


Thanks,
-Phil

[toc] | [prev] | [next] | [standalone]

#3623

On 4/13/2011 2:49 PM, Phil Bouchard wrote:
>
> There was a discussion on the validity of a 'false' statement returned
> by is_from() because out-of-bounds pointer comparison would invoke
> undefined behavior according to 5.7.6. This was bringing the idea of
> having adjacent pools stack & data to the heap pool.
>
> But it looks like heap.is_from() could work without adjacent stack and
> data pools in all cases, even if the pointers are arbitrary. This is
> because they can be casted to an integral type before they are compared
> together as stated by 5.2.10/4. The following example is thus valid for
> arbitrary pointers:
>
> bool is_from(const char* p)
> {
> return reinterpret_cast<long>(buffer)<= reinterpret_cast<long>(p)&&
> reinterpret_cast<long>(p)< reinterpret_cast<long>(buffer + sizeof(buffer));
> }

Sorry, it's actually "implementation defined" but perhaps a 
reinterpret_cast of a pointer to a long integer could be guaranteed to 
be valid by the standards, long being the greatest integer available on 
a system.


-Phil

[toc] | [prev] | [next] | [standalone]

#3640

Phil Bouchard <phil@fornux.com> wrote in
news:4da62a76@news.x-privat.org: 

> On 4/13/2011 2:49 PM, Phil Bouchard wrote:
>>
>> There was a discussion on the validity of a 'false' statement
>> returned by is_from() because out-of-bounds pointer comparison would
>> invoke undefined behavior according to 5.7.6. This was bringing the
>> idea of having adjacent pools stack & data to the heap pool.
>>
>> But it looks like heap.is_from() could work without adjacent stack
>> and data pools in all cases, even if the pointers are arbitrary. This
>> is because they can be casted to an integral type before they are
>> compared together as stated by 5.2.10/4. The following example is
>> thus valid for arbitrary pointers:
>>
>> bool is_from(const char* p)
>> {
>> return reinterpret_cast<long>(buffer)<= reinterpret_cast<long>(p)&&
>> reinterpret_cast<long>(p)< reinterpret_cast<long>(buffer +
>> sizeof(buffer)); }

5.2.10/4 only says that you can cast a pointer to and back to an integral 
type large enough (if such exists). You can of course compare integers, 
but there is no guarantee that these comparisons make any sense for the 
original pointers (hint: think about x86 segmented memory model). For 
comparing pointers they must point to elements inside the same array (or 
immediately past the array) and there is no such easy way to work around 
this limitation.

> Sorry, it's actually "implementation defined" but perhaps a 
> reinterpret_cast of a pointer to a long integer could be guaranteed to
> be valid by the standards, long being the greatest integer available
> on a system.

In systems with flat address spaces the implementations indeed guarantee 
a total ordering of pointers as they were integers of a sufficiently 
large type. However, no hope to get 'long' working here, in Win64 long is 
only 32 bit and this is not going to change. Besides, it has wrong 
signedness. 

If Boost.Pool was standardized for process' memory areas its 
implementation would be quite implementation-specific anyway so there is 
no need to worry about such minor details like the suitable integer type, 
if such is needed at all. For example, in some x86 segmented memory 
models one could tell the type of the pointer just by looking at its 
segment selector, no address range comparisons needed.

Cheers
Paavo

[toc] | [prev] | [next] | [standalone]

#3642

On 4/13/2011 9:39 PM, Paavo Helde wrote:
>
> 5.2.10/4 only says that you can cast a pointer to and back to an integral
> type large enough (if such exists). You can of course compare integers,
> but there is no guarantee that these comparisons make any sense for the
> original pointers (hint: think about x86 segmented memory model). For
> comparing pointers they must point to elements inside the same array (or
> immediately past the array) and there is no such easy way to work around
> this limitation.

The C standards actually define integer types called "intptr_t" in 
"cstdint" capable of holding object pointers (7.18.1.4).  The following 
version could then be perfectly portable:

bool is_from(const char* p)
{
     return reinterpret_cast<intptr_t>(buffer)<= 
reinterpret_cast<intptr_t>(p)&& reinterpret_cast<intptr_t>(p)< 
reinterpret_cast<intptr_t>(buffer + sizeof(buffer));
}

> In systems with flat address spaces the implementations indeed guarantee
> a total ordering of pointers as they were integers of a sufficiently
> large type. However, no hope to get 'long' working here, in Win64 long is
> only 32 bit and this is not going to change. Besides, it has wrong
> signedness.
>
> If Boost.Pool was standardized for process' memory areas its
> implementation would be quite implementation-specific anyway so there is
> no need to worry about such minor details like the suitable integer type,
> if such is needed at all. For example, in some x86 segmented memory
> models one could tell the type of the pointer just by looking at its
> segment selector, no address range comparisons needed.

Currently is_from() invokes undefined behavior with a false statement. 
This makes the utility useless unless it is reimplemented or 5.7/6 
becomes less restrictive on the comparison of pointers not part of the 
same array.


Thanks,
-Phil

[toc] | [prev] | [next] | [standalone]

#3643

On 4/13/2011 10:43 PM, Phil Bouchard wrote:
>>
>> If Boost.Pool was standardized for process' memory areas its
>> implementation would be quite implementation-specific anyway so there is
>> no need to worry about such minor details like the suitable integer type,
>> if such is needed at all. For example, in some x86 segmented memory
>> models one could tell the type of the pointer just by looking at its
>> segment selector, no address range comparisons needed.

My apologies but I forgot mentioning that we wouldn't want if_from to 
get discredited before it upgrades to the standards.  We are looking for 
both a standard compliant version of is_from and a system specific 
version of it.


Thanks,
-Phil

[toc] | [prev] | [next] | [standalone]

#3645

Phil Bouchard <phil@fornux.com> writes:
> My apologies but I forgot mentioning that we wouldn't want if_from to
> get discredited before it upgrades to the standards.  We are looking for
> both a standard compliant version of is_from and a system specific
> version of it.

BTW, the name "is_from" seems very odd -- "foo->is_from (ptr)" _looks_
like it should ask the question "foo is from ptr?", but really it means
the opposite ("ptr is from foo?").

A better name might be "contains":   foo->contains (ptr)

-miles

-- 
Twice, adv. Once too often.

[toc] | [prev] | [next] | [standalone]

#3646

On 4/13/2011 11:55 PM, Miles Bader wrote:
>
> BTW, the name "is_from" seems very odd -- "foo->is_from (ptr)" _looks_
> like it should ask the question "foo is from ptr?", but really it means
> the opposite ("ptr is from foo?").
>
> A better name might be "contains":   foo->contains (ptr)

contains sounds good to me.


-Phil

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.c++

csiph-web