Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c++ > #83984 > unrolled thread

nullptr is not 0

Back to article view | Back to comp.lang.c++

Contents

  nullptr is not 0 Sam <sam@email-scan.com> - 2022-05-08 09:45 -0400
    Re: nullptr is not 0 "Alf P. Steinbach" <alf.p.steinbach@gmail.com> - 2022-05-08 16:21 +0200
    Re: nullptr is not 0 Andrey Tarasevich <andreytarasevich@hotmail.com> - 2022-05-08 10:00 -0700
    Re: nullptr is not 0 Bonita Montero <Bonita.Montero@gmail.com> - 2022-05-09 06:38 +0200
    Re: nullptr is not 0 Juha Nieminen <nospam@thanks.invalid> - 2022-05-09 05:06 +0000
      Re: nullptr is not 0 Vir Campestris <vir.campestris@invalid.invalid> - 2022-05-09 21:46 +0100
        Re: nullptr is not 0 Andrey Tarasevich <andreytarasevich@hotmail.com> - 2022-05-09 13:57 -0700
          Re: nullptr is not 0 Juha Nieminen <nospam@thanks.invalid> - 2022-05-10 05:04 +0000
            Re: nullptr is not 0 Andrey Tarasevich <andreytarasevich@hotmail.com> - 2022-05-09 22:33 -0700
        Re: nullptr is not 0 scott@slp53.sl.home (Scott Lurndal) - 2022-05-09 21:56 +0000
          Function pointer size (was Re: nullptr is not 0) Vir Campestris <vir.campestris@invalid.invalid> - 2022-05-17 21:30 +0100
            Re: Function pointer size (was Re: nullptr is not 0) scott@slp53.sl.home (Scott Lurndal) - 2022-05-17 21:00 +0000
              Re: Function pointer size (was Re: nullptr is not 0) Vir Campestris <vir.campestris@invalid.invalid> - 2022-05-19 21:52 +0100
              Re: Function pointer size (was Re: nullptr is not 0) Andrey Tarasevich <andreytarasevich@hotmail.com> - 2022-05-19 14:08 -0700
            Re: Function pointer size (was Re: nullptr is not 0) Andrey Tarasevich <andreytarasevich@hotmail.com> - 2022-05-17 17:05 -0700
            Re: Function pointer size (was Re: nullptr is not 0) Juha Nieminen <nospam@thanks.invalid> - 2022-05-18 04:46 +0000
              Re: Function pointer size (was Re: nullptr is not 0) Andrey Tarasevich <andreytarasevich@hotmail.com> - 2022-05-18 02:14 -0700
                Re: Function pointer size (was Re: nullptr is not 0) Juha Nieminen <nospam@thanks.invalid> - 2022-05-19 14:53 +0000
                  Re: Function pointer size (was Re: nullptr is not 0) "Alf P. Steinbach" <alf.p.steinbach@gmail.com> - 2022-05-19 17:19 +0200
                    Re: Function pointer size (was Re: nullptr is not 0) Andrey Tarasevich <andreytarasevich@hotmail.com> - 2022-05-19 10:14 -0700
                      Re: Function pointer size (was Re: nullptr is not 0) Richard Damon <Richard@Damon-Family.org> - 2022-05-19 13:45 -0400
                        Re: Function pointer size (was Re: nullptr is not 0) Andrey Tarasevich <andreytarasevich@hotmail.com> - 2022-05-19 11:31 -0700
                      Re: Function pointer size (was Re: nullptr is not 0) Juha Nieminen <nospam@thanks.invalid> - 2022-05-20 06:58 +0000
                        Re: Function pointer size (was Re: nullptr is not 0) Andrey Tarasevich <andreytarasevich@hotmail.com> - 2022-05-20 00:11 -0700
                  Re: Function pointer size (was Re: nullptr is not 0) Richard Damon <Richard@Damon-Family.org> - 2022-05-19 12:27 -0400
                    Re: Function pointer size (was Re: nullptr is not 0) Andrey Tarasevich <andreytarasevich@hotmail.com> - 2022-05-19 09:47 -0700
                      Re: Function pointer size (was Re: nullptr is not 0) Richard Damon <Richard@Damon-Family.org> - 2022-05-19 13:36 -0400
                        Re: Function pointer size (was Re: nullptr is not 0) Andrey Tarasevich <andreytarasevich@hotmail.com> - 2022-05-19 11:24 -0700
                          Re: Function pointer size (was Re: nullptr is not 0) Richard Damon <Richard@Damon-Family.org> - 2022-05-19 21:32 -0400
                            Re: Function pointer size (was Re: nullptr is not 0) Andrey Tarasevich <andreytarasevich@hotmail.com> - 2022-05-19 20:28 -0700
                              Re: Function pointer size (was Re: nullptr is not 0) Richard Damon <Richard@Damon-Family.org> - 2022-05-20 10:57 -0400
                  Re: Function pointer size (was Re: nullptr is not 0) Andrey Tarasevich <andreytarasevich@hotmail.com> - 2022-05-19 09:35 -0700
                    Re: Function pointer size (was Re: nullptr is not 0) Andrey Tarasevich <andreytarasevich@hotmail.com> - 2022-05-19 10:10 -0700
                    Re: Function pointer size (was Re: nullptr is not 0) Juha Nieminen <nospam@thanks.invalid> - 2022-05-20 07:15 +0000
            Re: Function pointer size (was Re: nullptr is not 0) "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2022-05-18 11:50 -0700
              Re: Function pointer size (was Re: nullptr is not 0) Andrey Tarasevich <andreytarasevich@hotmail.com> - 2022-05-18 15:46 -0700
                Re: Function pointer size (was Re: nullptr is not 0) "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2022-05-19 20:52 -0700

Page 2 of 2 — ← Prev page 1 [2]

#84195 — Re: Function pointer size (was Re: nullptr is not 0)

On 5/19/22 1:14 PM, Andrey Tarasevich wrote:
> On 5/19/2022 8:19 AM, Alf P. Steinbach wrote:
>> On 19 May 2022 16:53, Juha Nieminen wrote:
>>> Andrey Tarasevich <andreytarasevich@hotmail.com> wrote:
>>>>> If you have a reference or pointer to an object of type class A, and a
>>>>> member pointer to a member function of class A, which is virtual, if
>>>>> the actual object pointed to is of some derived type B which has its
>>>>> own specialization of that virtual function, calling the function 
>>>>> using
>>>>> the pointer will call the derived implementation not, the one in 
>>>>> class A.
>>>>>
>>>>> For this to be possible the pointer to member function needs 
>>>>> additional
>>>>> data.
>>>>
>>>> No, that's incorrect.
>>>>
>>>> This has nothing to do with virtual functions. On the contrary,
>>>> pointers-to-virtual-functions are "easy": regular virtual call 
>>>> mechanism
>>>> itself is already required to incorporate all necessary mechanics to
>>>> properly invoke virtual functions across the entire hierarchy of
>>>> classes. This mechanics is already present in "regular" virtual calls
>>>> (without involving any pointers-to-members).
>>>
>>> I'm not sure that's correct.
>>>
>>> When you have a pointer-to-member, in the location of the call the 
>>> compiler
>>> doesn't know *which* member function it's pointing to. It only knows its
>>> signature, not its name. The class may have several differently-named
>>> member fuctions with the same signature. I don't think the compiler can
>>> even know (from the source code alone) if the pointed-to function is
>>> virtual or not.
>>>
>>> Thus, the compiler needs to create code that somehow figures out if
>>> that member function is virtual, and which function it is, and then
>>> use the vtable as normal to jump to the actual most-derived 
>>> implementation.
>>> I don't think it can do this without the extra data in the pointer.
>>> (Well, I suppose it theoretically could, but that might require some
>>> searching).
>>>
>>> Please correct me if I'm wrong. (Honestly. This isn't sarcasm.)
>>
>> Just an example of what you're saying:
>>
>>
>> #include <stdio.h>
>>
>> struct A
>> {
>>      void foo() const { printf( "A::foo\n" ); }
>>      virtual void bar() const { printf( "A::bar\n" ); }
>> };
>>
>> struct B: A
>> {
>>      void bar() const override { printf( "B::bar\n" ); }
>> };
>>
>> auto main() -> int
>> {
>>      auto        f = &A::foo;
>>      const A&    o = B();
>>
>>      f = &A::bar;
>>      (o.*f)();    // "B::bar"
>> }
>>
> 
> Not sure what this is supposed to illustrate. That 
> pointers-to-member-functions implement "late binding" at the point of 
> the call? Yes, that's true and that's banal.
> 
> However, this still does not in any way mean that 
> pointers-to-member-function has to distinguish between pointing to a 
> "regular" function and pointing to a virtual function.
> 

Look at an actual layout of a class

Since struct A has a virtual function in it, the class needs a vtable, 
so the layout of A will be something like:


struct A:
     VTable* vtbl = vtable_A;  // this gets filled in by the constructor.
...rest of the data in A

vtable_a:
     void (*vtbl_A_bar)(A*);		// filled in by loader to point to the 
implementation of A::bar


struct B:
     VTable* vtbl = vtable_B;	// this gets filled in by the constructor

...rest of data in A
...rest of data in B


vtable_b:
     void (*vtbl_B_bar)(A*);		// filled in by loader to point to the 
implementation of B::bar


&A::foo results in a pointer that says it is a "normal" pointer, and the 
address of A::foo;

&A::bar resutls in a pointer that says it is a "Virtual" pointer to the 
first element of the vtable (which will point to A::bar in an A object 
and B::bar in a B object)

There needs to be a bit when we evalute o.*f so it knows if f is storing 
the actual address of the function or the offset into the vtable.

[toc] | [prev] | [next] | [standalone]

#84197 — Re: Function pointer size (was Re: nullptr is not 0)

On 5/19/2022 10:45 AM, Richard Damon wrote:
> 
> Look at an actual layout of a class
> 
> Since struct A has a virtual function in it, the class needs a vtable, 
> so the layout of A will be something like:
> 
> 
> struct A:
>      VTable* vtbl = vtable_A;  // this gets filled in by the constructor.
> ...rest of the data in A
> 
> vtable_a:
>      void (*vtbl_A_bar)(A*);        // filled in by loader to point to 
> the implementation of A::bar
> 
> 
> struct B:
>      VTable* vtbl = vtable_B;    // this gets filled in by the constructor
> 
> ...rest of data in A
> ...rest of data in B
> 
> 
> vtable_b:
>      void (*vtbl_B_bar)(A*);        // filled in by loader to point to 
> the implementation of B::bar
> 
> 
> &A::foo results in a pointer that says it is a "normal" pointer, and the 
> address of A::foo;
> 
> &A::bar resutls in a pointer that says it is a "Virtual" pointer to the 
> first element of the vtable (which will point to A::bar in an A object 
> and B::bar in a B object)

You keep describing GCC's implementation. I've already described this in 
great detail previously. You are just repeating my previous explanations.

For what reason?

> There needs to be a bit when we evalute o.*f so it knows if f is storing 
> the actual address of the function or the offset into the vtable.

Well, of course, if you follow GCC's approach, then yes, you need a bit, 
since GCC's approach is based on that extra bit.

But you don't have to use GCC's approach. Take MSVC's approach, which 
does not need any extra bits. In MSVC's approach pointers never store 
"offset into the vtable".

So, what point are you trying to make? So far I see only a tautology: 
"if you use a approach that need an extra bit, then you need an extra 
bit". Well, duh... And?

-- 
Best regards,
Andrey

[toc] | [prev] | [next] | [standalone]

#84204 — Re: Function pointer size (was Re: nullptr is not 0)

Andrey Tarasevich <andreytarasevich@hotmail.com> wrote:
> Not sure what this is supposed to illustrate. That 
> pointers-to-member-functions implement "late binding" at the point of 
> the call? Yes, that's true and that's banal.
> 
> However, this still does not in any way mean that 
> pointers-to-member-function has to distinguish between pointing to a 
> "regular" function and pointing to a virtual function.

I suppose the compiler could make *all* function pointers be of the same
size (ie. eg. the size of two pointers), but with regular (free-floating)
functions that would be rather useless. A single pointer is enough for
those.

If by "regular" function you meant "a non-virtual member function", then
my point wasn't that those have to be of a different size compared to a
pointer-to-virtual-function. My point was that they have to be larger than
a single pointer because of how they have to behave (ie. late binding,
as you say).

I suppose theoretically they wouldn't have to be larger, but in that case
the code that calls a member function via a pointer would probably need to
be a lot more complicated (and somewhat more inefficient). I haven't thought
about the technical details in enough depth to tell for sure.

[toc] | [prev] | [next] | [standalone]

#84205 — Re: Function pointer size (was Re: nullptr is not 0)

On 5/19/2022 11:58 PM, Juha Nieminen wrote:
> Andrey Tarasevich <andreytarasevich@hotmail.com> wrote:
>> Not sure what this is supposed to illustrate. That
>> pointers-to-member-functions implement "late binding" at the point of
>> the call? Yes, that's true and that's banal.
>>
>> However, this still does not in any way mean that
>> pointers-to-member-function has to distinguish between pointing to a
>> "regular" function and pointing to a virtual function.
> 
> I suppose the compiler could make *all* function pointers be of the same
> size (ie. eg. the size of two pointers), but with regular (free-floating)
> functions that would be rather useless. A single pointer is enough for
> those.
> 
> If by "regular" function you meant "a non-virtual member function", 

Yes. In the context of this topic I'm (we are) talking about non-static 
member functions only, and pointers to such member functions. Virtual 
ones and non-virtual ones. By "regular" I simply mean the latter.

> then
> my point wasn't that those have to be of a different size compared to a
> pointer-to-virtual-function. My point was that they have to be larger than
> a single pointer because of how they have to behave (ie. late binding,
> as you say).

But that is incorrect.

Late binding does not _require_ pointers-to-member-functions to be 
larger. Once again, see the MSVC's implementation of 
pointers-to-member-functions. In that implementation all pointers are 
represented identically, regardless of whether they point to virtual or 
non-virtual member functions.

Also, see GCC/Clang implementation for x86/x64. They use different 
representations for virtual/non-virtual, but they still do not (!) 
require extra bytes for that. They manage to stuff everything into the 
lower word.

In all of the above implementations the upper word is needed for a 
different purpose. Not related to late binding at all.

> I suppose theoretically they wouldn't have to be larger, but in that case
> the code that calls a member function via a pointer would probably need to
> be a lot more complicated (and somewhat more inefficient). I haven't thought
> about the technical details in enough depth to tell for sure.

In the available mainstream implementations pointers-to-member-functions 
are larger for a good reason. But that reason, as I stated repeatedly, 
has absolutely nothing to do with pointing to virtual member functions 
and late binding.

This is point I'm trying to convey to you for the last ten or so 
messages. I don't understand why it is so difficult to get this point 
across...

-- 
Best regards,
Andrey

[toc] | [prev] | [next] | [standalone]

#84187 — Re: Function pointer size (was Re: nullptr is not 0)

On 5/19/22 10:53 AM, Juha Nieminen wrote:
> Andrey Tarasevich <andreytarasevich@hotmail.com> wrote:
>>> If you have a reference or pointer to an object of type class A, and a
>>> member pointer to a member function of class A, which is virtual, if
>>> the actual object pointed to is of some derived type B which has its
>>> own specialization of that virtual function, calling the function using
>>> the pointer will call the derived implementation not, the one in class A.
>>>
>>> For this to be possible the pointer to member function needs additional
>>> data.
>>
>> No, that's incorrect.
>>
>> This has nothing to do with virtual functions. On the contrary,
>> pointers-to-virtual-functions are "easy": regular virtual call mechanism
>> itself is already required to incorporate all necessary mechanics to
>> properly invoke virtual functions across the entire hierarchy of
>> classes. This mechanics is already present in "regular" virtual calls
>> (without involving any pointers-to-members).
> 
> I'm not sure that's correct.
> 
> When you have a pointer-to-member, in the location of the call the compiler
> doesn't know *which* member function it's pointing to. It only knows its
> signature, not its name. The class may have several differently-named
> member fuctions with the same signature. I don't think the compiler can
> even know (from the source code alone) if the pointed-to function is
> virtual or not.
> 
> Thus, the compiler needs to create code that somehow figures out if
> that member function is virtual, and which function it is, and then
> use the vtable as normal to jump to the actual most-derived implementation.
> I don't think it can do this without the extra data in the pointer.
> (Well, I suppose it theoretically could, but that might require some
> searching).
> 
> Please correct me if I'm wrong. (Honestly. This isn't sarcasm.)

Yes that sort of describes the problem. The pointer to member needs to 
be able to refer to a virtual function, or even a function that is a 
member of a virtual base class, and handling that sort of stuff is what 
forces the extra memory in the pointer-to-member.

I remember seeing documentation in one compiler that gave options for 
"optimizations" where you could limit what sort of member functions you 
could take the address of to make the pointers smaller. Omitting the 
handling of Virtual Base classes, or those and virtual functions (I 
forget if there was some other level).

[toc] | [prev] | [next] | [standalone]

#84190 — Re: Function pointer size (was Re: nullptr is not 0)

On 5/19/2022 9:27 AM, Richard Damon wrote:
>>
>> Thus, the compiler needs to create code that somehow figures out if
>> that member function is virtual, and which function it is, and then
>> use the vtable as normal to jump to the actual most-derived 
>> implementation.
>> I don't think it can do this without the extra data in the pointer.
>> (Well, I suppose it theoretically could, but that might require some
>> searching).
>>
>> Please correct me if I'm wrong. (Honestly. This isn't sarcasm.)
> 
> Yes that sort of describes the problem. The pointer to member needs to 
> be able to refer to a virtual function, or even a function that is a 
> member of a virtual base class, and handling that sort of stuff is what 
> forces the extra memory in the pointer-to-member.

That's incorrect in general. Again, using x86 as a counterexample:

Implementations that use different representations for virtual vs. 
non-virtual member pointers (GCC, Clang) manage to stuff that extra 
information into the "original" memory - they use the lowest bit of the 
pointer as a flag.

Implementations that use identical representations for virtual vs. 
non-virtual member pointers (MSVC) naturally don't have this issue at all.

So, no, the need to handle virtual functions does not require extra 
memory in pointer-to-member function.

> I remember seeing documentation in one compiler that gave options for 
> "optimizations" where you could limit what sort of member functions you 
> could take the address of to make the pointers smaller. Omitting the 
> handling of Virtual Base classes, or those and virtual functions (I 
> forget if there was some other level).

Well, you forget.

These optimizations have absolutely nothing to do with virtual 
functions. They have everything to do with the hierarchy structure. They 
depend on the answer to one question: is it necessary to correct the 
`this` pointer value when making a call?

Correction to `this` might be necessary in case of multiple inheritance, 
in case of virtual inheritance and in some other niche cases. This 
correction is what the extra memory in pointer-to-member function 
stores. This is why pointer-to-member-function is larger than an 
ordinary pointer.

But it has nothing to do with calls to virtual functions.

-- 
Best regards,
Andrey

[toc] | [prev] | [next] | [standalone]

#84194 — Re: Function pointer size (was Re: nullptr is not 0)

On 5/19/22 12:47 PM, Andrey Tarasevich wrote:
> On 5/19/2022 9:27 AM, Richard Damon wrote:
>>>
>>> Thus, the compiler needs to create code that somehow figures out if
>>> that member function is virtual, and which function it is, and then
>>> use the vtable as normal to jump to the actual most-derived 
>>> implementation.
>>> I don't think it can do this without the extra data in the pointer.
>>> (Well, I suppose it theoretically could, but that might require some
>>> searching).
>>>
>>> Please correct me if I'm wrong. (Honestly. This isn't sarcasm.)
>>
>> Yes that sort of describes the problem. The pointer to member needs to 
>> be able to refer to a virtual function, or even a function that is a 
>> member of a virtual base class, and handling that sort of stuff is 
>> what forces the extra memory in the pointer-to-member.
> 
> That's incorrect in general. Again, using x86 as a counterexample:
> 
> Implementations that use different representations for virtual vs. 
> non-virtual member pointers (GCC, Clang) manage to stuff that extra 
> information into the "original" memory - they use the lowest bit of the 
> pointer as a flag.
> 
> Implementations that use identical representations for virtual vs. 
> non-virtual member pointers (MSVC) naturally don't have this issue at all.
> 
> So, no, the need to handle virtual functions does not require extra 
> memory in pointer-to-member function.

But not all processors have a LSB to use for that sort of flag.

Some have instructions that can be at any address, because the 
instruction set has single byte opcodes (They could just force all 
functions to be at addresses with lower order zeros, but that is wasteful)

Others, like the ARM, use the low order bit to indicate which 
instruction set the function is written in, so it isn't available for 
such a use.

Thus, there sometimes IS a need for an additional flag bit to indicate 
if the pointer is an index into the v-table or the actual address of the 
function.

> 
>> I remember seeing documentation in one compiler that gave options for 
>> "optimizations" where you could limit what sort of member functions 
>> you could take the address of to make the pointers smaller. Omitting 
>> the handling of Virtual Base classes, or those and virtual functions 
>> (I forget if there was some other level).
> 
> Well, you forget.
> 
> These optimizations have absolutely nothing to do with virtual 
> functions. They have everything to do with the hierarchy structure. They 
> depend on the answer to one question: is it necessary to correct the 
> `this` pointer value when making a call?

Not to my knowledge. That adjustment tends to be done by a "Thunk" to 
correct the base address of "this" for non-first base classes. This 
needs to happen even without pointer-to-member functions, as if you call 
a virtual function from a pointer to a non-first base that is 
over-ridden after the multiple-inheritance. The virtual-function table 
for that non-first base will point to the thunk doing the adjustment, 
will the same class will have another virtual-function table for uses 
that refer to the class or its first base class.

Deciding if the "pointer" is an actual address of the function or an 
offest into the virtual function table is orthogonal to that, and is an 
issue even with multiple inheretance that needs that fixup.

> 
> Correction to `this` might be necessary in case of multiple inheritance, 
> in case of virtual inheritance and in some other niche cases. This 
> correction is what the extra memory in pointer-to-member function 
> stores. This is why pointer-to-member-function is larger than an 
> ordinary pointer.
> 
> But it has nothing to do with calls to virtual functions.
> 

Nope, because the pointer-to-member function pointer has no idea of the 
type of the object that it will be used on, so CAN'T store the full 
offset. (That is the job of the Thunk).

But it may be needed for a pointer-to-member function pointer which 
doesn't always hold the address of the member function to call but 
sometimes an offset/index into the vtable.

As an example:

class B {
public:
             void f1();
     virtual void f2();
}

class D : public B {
public:
     virtual void f2();
}

we take the member-function-pointers

void (B::*ptr1)() = &B::f1;
void (B::*ptr2)() = &B::f2;


B b;
D d;

b*->ptr1() vs b *->ptr2() vs d*->ptr1() vs d*->ptr2()

There is a need to know that the "value" stored in ptr1 is the actual 
address of B:f1(), while ptr2 is storing an offset into the vtable of B 
for the member function.

ptr2 can NOT just have the address of B::f2() as if derived class D 
overrides f2(), then the call via ptr2 needs to go to D::f2().

[toc] | [prev] | [next] | [standalone]

#84196 — Re: Function pointer size (was Re: nullptr is not 0)

On 5/19/2022 10:36 AM, Richard Damon wrote:
>>>
>>> Yes that sort of describes the problem. The pointer to member needs 
>>> to be able to refer to a virtual function, or even a function that is 
>>> a member of a virtual base class, and handling that sort of stuff is 
>>> what forces the extra memory in the pointer-to-member.
>>
>> That's incorrect in general. Again, using x86 as a counterexample:
>>
>> Implementations that use different representations for virtual vs. 
>> non-virtual member pointers (GCC, Clang) manage to stuff that extra 
>> information into the "original" memory - they use the lowest bit of 
>> the pointer as a flag.
>>
>> Implementations that use identical representations for virtual vs. 
>> non-virtual member pointers (MSVC) naturally don't have this issue at 
>> all.
>>
>> So, no, the need to handle virtual functions does not require extra 
>> memory in pointer-to-member function.
> 
> But not all processors have a LSB to use for that sort of flag.
> 
> Some have instructions that can be at any address, because the 
> instruction set has single byte opcodes (They could just force all 
> functions to be at addresses with lower order zeros, but that is wasteful)
> 
> Others, like the ARM, use the low order bit to indicate which 
> instruction set the function is written in, so it isn't available for 
> such a use.
> 
> Thus, there sometimes IS a need for an additional flag bit to indicate 
> if the pointer is an index into the v-table or the actual address of the 
> function.

Well, that's a completely different claim. "Sometimes", huh...

Firstly, as MSVC implementation demonstrates that the flag is not 
necessary at all.

Secondly, quite possibly it is the case "sometimes". However, one might 
argue that if the implementation had to increase pointer size just for 
the sake of that single flag, it would probably abandon that approach 
altogether and switch to MSVC-like approach, which requires no flag.

>>
>>> I remember seeing documentation in one compiler that gave options for 
>>> "optimizations" where you could limit what sort of member functions 
>>> you could take the address of to make the pointers smaller. Omitting 
>>> the handling of Virtual Base classes, or those and virtual functions 
>>> (I forget if there was some other level).
>>
>> Well, you forget.
>>
>> These optimizations have absolutely nothing to do with virtual 
>> functions. They have everything to do with the hierarchy structure. 
>> They depend on the answer to one question: is it necessary to correct 
>> the `this` pointer value when making a call?
> 
> Not to my knowledge. That adjustment tends to be done by a "Thunk" to 
> correct the base address of "this" for non-first base classes. 

No, no, no. That cannot be done by a thunk. The adjustment value is a 
run-time value, which changes as you convert the pointer up and down the 
hierarchy.

Theoretically, it can be probably implemented somehow by a bunch of 
thunks and a heap of compiler magic: say, by generating as many 
different thunks as there are different correction values. But I haven't 
seen any real-life implementations that do that. And it seems unreasonable.

GCC, Clang and MSVC store the `this` correction offset inside the 
pointer. That is exactly why in these compilers the pointer takes two 
words instead of one.

>> Correction to `this` might be necessary in case of multiple 
>> inheritance, in case of virtual inheritance and in some other niche 
>> cases. This correction is what the extra memory in pointer-to-member 
>> function stores. This is why pointer-to-member-function is larger than 
>> an ordinary pointer.
>>
>> But it has nothing to do with calls to virtual functions.
>>
> 
> Nope, because the pointer-to-member function pointer has no idea of the 
> type of the object that it will be used on, so CAN'T store the full 
> offset. (That is the job of the Thunk).

This is completely incorrect.

(Once again, let me reiterate that what I'm stating here is not offered 
for debate. These are hard facts about how modern implementations work. 
This is learning material for those willing to learn.)

The pointer has to store the _delta_ value that corrects the `this` 
pointer between the static pointer type and the dynamic type of the 
pointed object.

Here's a little illustrative sketch

    struct B {};
    struct D : /* some other bases */, B { void foo() {} };
    // Base subobject `B` is offset inside `D`

    void (B::*pf)() = static_cast<void (B::*)()>(&D::foo);
    // Here the upper word of `pf` will store the correction
    // delta, which is needed to down-convert `B *` to `D *`

    D *pd = new D;
    B *pb = pd;
    // `pb` is not the same numerically as `pd`

    (pb->*pf)();
    // Correctly calls `D::foo`
    // Before the actual call value of `pb` is adjusted by the
    // correction delta stored in `pf`, thus providing us with
    // the proper `D *this` pointer

    (pd->*pf)();
    // Correctly calls `D::foo`
    // Since `pf` is declared as `void (B::*)()`, the `pd` is
    // first converted to `B *` (offsetting it), and then
    // adjusted by the correction value stored in `pf`
    // (un-offsetting it). This results in `pd` again, which
    // is the proper `D *this` pointer

As you can see in the example above, even though it "has no idea of the 
type of the object that it will be used on", everything works out nicely 
and correctly.

> But it may be needed for a pointer-to-member function pointer which 
> doesn't always hold the address of the member function to call but 
> sometimes an offset/index into the vtable.
> 
> As an example:
> 
> class B {
> public:
>              void f1();
>      virtual void f2();
> }
> 
> class D : public B {
> public:
>      virtual void f2();
> }
> 
> we take the member-function-pointers
> 
> void (B::*ptr1)() = &B::f1;
> void (B::*ptr2)() = &B::f2;
> 
> 
> B b;
> D d;
> 
> b*->ptr1() vs b *->ptr2() vs d*->ptr1() vs d*->ptr2()
> 
> There is a need to know that the "value" stored in ptr1 is the actual 
> address of B:f1(), while ptr2 is storing an offset into the vtable of B 
> for the member function.
> 
> ptr2 can NOT just have the address of B::f2() as if derived class D 
> overrides f2(), then the call via ptr2 needs to go to D::f2().

Nobody says that `ptr2` has to store address of `B::f2()`. But your 
claim that there is a need to distinguish between `ptr1 and `ptr2` at 
run-time is incorrect.

This is what MSVC will translate your example into:

   class B {
   public:
             void f1();
     virtual void f2();
             void f2_thunk() { return f2(); }
   };

   class D : public B {
   public:
     virtual void f2();
   }

   ...
   void (B::*ptr1)() = &B::f1;
   void (B::*ptr2)() = &B::f2_thunk;

   B b;
   D d;

   //b*->ptr1() vs b *->ptr2() vs d*->ptr1() vs d*->ptr2()

And suddenly everything works correctly, like magic! No need "to know" 
which pointer is virtual and which pointer is not. Very simple.

Again, in this case `f2_thunk` takes advantage of the already 
implemented virtual dispatch mechanism. The pointer simply re-uses it, 
instead of re-implementing it.

It is quite possible that GCC's approach is more efficient, despite 
being more bulky: branch predictions, cache behavior and so on... But 
there's no denying that MSVC's approach is far more elegant.

-- 
Best regards,
Andrey

[toc] | [prev] | [next] | [standalone]

#84201 — Re: Function pointer size (was Re: nullptr is not 0)

On 5/19/22 2:24 PM, Andrey Tarasevich wrote:
> On 5/19/2022 10:36 AM, Richard Damon wrote:
>>>>
>>>> Yes that sort of describes the problem. The pointer to member needs 
>>>> to be able to refer to a virtual function, or even a function that 
>>>> is a member of a virtual base class, and handling that sort of stuff 
>>>> is what forces the extra memory in the pointer-to-member.
>>>
>>> That's incorrect in general. Again, using x86 as a counterexample:
>>>
>>> Implementations that use different representations for virtual vs. 
>>> non-virtual member pointers (GCC, Clang) manage to stuff that extra 
>>> information into the "original" memory - they use the lowest bit of 
>>> the pointer as a flag.
>>>
>>> Implementations that use identical representations for virtual vs. 
>>> non-virtual member pointers (MSVC) naturally don't have this issue at 
>>> all.
>>>
>>> So, no, the need to handle virtual functions does not require extra 
>>> memory in pointer-to-member function.
>>
>> But not all processors have a LSB to use for that sort of flag.
>>
>> Some have instructions that can be at any address, because the 
>> instruction set has single byte opcodes (They could just force all 
>> functions to be at addresses with lower order zeros, but that is 
>> wasteful)
>>
>> Others, like the ARM, use the low order bit to indicate which 
>> instruction set the function is written in, so it isn't available for 
>> such a use.
>>
>> Thus, there sometimes IS a need for an additional flag bit to indicate 
>> if the pointer is an index into the v-table or the actual address of 
>> the function.
> 
> Well, that's a completely different claim. "Sometimes", huh...
> 
> Firstly, as MSVC implementation demonstrates that the flag is not 
> necessary at all.

At the cost of an addition layer of calling and more thunks.

> 
> Secondly, quite possibly it is the case "sometimes". However, one might 
> argue that if the implementation had to increase pointer size just for 
> the sake of that single flag, it would probably abandon that approach 
> altogether and switch to MSVC-like approach, which requires no flag.

The question comes can it find a place to hide it. As you have 
mentioned, if the class the function is in isn't at the same base 
address as the class the pointer is based on, you need some offsets, and 
perhaps you can find a place in there.

Especially for the offset cause by virtual bases, which isn't fixed, so 
DOES need to be a lookup table index which can be of limited size, 
leaving space for the direct/virtual function flag.

Note, for a processor with a conditional skip, that flag doesn't even 
cost you a branch predicition slot.

> 
>>>
>>>> I remember seeing documentation in one compiler that gave options 
>>>> for "optimizations" where you could limit what sort of member 
>>>> functions you could take the address of to make the pointers 
>>>> smaller. Omitting the handling of Virtual Base classes, or those and 
>>>> virtual functions (I forget if there was some other level).
>>>
>>> Well, you forget.
>>>
>>> These optimizations have absolutely nothing to do with virtual 
>>> functions. They have everything to do with the hierarchy structure. 
>>> They depend on the answer to one question: is it necessary to correct 
>>> the `this` pointer value when making a call?
>>
>> Not to my knowledge. That adjustment tends to be done by a "Thunk" to 
>> correct the base address of "this" for non-first base classes. 
> 
> No, no, no. That cannot be done by a thunk. The adjustment value is a 
> run-time value, which changes as you convert the pointer up and down the 
> hierarchy.
> 
> Theoretically, it can be probably implemented somehow by a bunch of 
> thunks and a heap of compiler magic: say, by generating as many 
> different thunks as there are different correction values. But I haven't 
> seen any real-life implementations that do that. And it seems unreasonable.
> 
> GCC, Clang and MSVC store the `this` correction offset inside the 
> pointer. That is exactly why in these compilers the pointer takes two 
> words instead of one.

I was thinking of the adjustment for the class of the object that the 
pointer is being applied to, that can't be stored in the pointer 
(because it doesn't know what it is).

I was forgetting the adjustment if the function is in a class with a 
different base than the pointer-to-function type.
> 
>>> Correction to `this` might be necessary in case of multiple 
>>> inheritance, in case of virtual inheritance and in some other niche 
>>> cases. This correction is what the extra memory in pointer-to-member 
>>> function stores. This is why pointer-to-member-function is larger 
>>> than an ordinary pointer.
>>>
>>> But it has nothing to do with calls to virtual functions.
>>>
>>
>> Nope, because the pointer-to-member function pointer has no idea of 
>> the type of the object that it will be used on, so CAN'T store the 
>> full offset. (That is the job of the Thunk).
> 
> This is completely incorrect.
> 
> (Once again, let me reiterate that what I'm stating here is not offered 
> for debate. These are hard facts about how modern implementations work. 
> This is learning material for those willing to learn.)
> 
> The pointer has to store the _delta_ value that corrects the `this` 
> pointer between the static pointer type and the dynamic type of the 
> pointed object.
> 
> Here's a little illustrative sketch
> 
>     struct B {};
>     struct D : /* some other bases */, B { void foo() {} };
>     // Base subobject `B` is offset inside `D`
> 
>     void (B::*pf)() = static_cast<void (B::*)()>(&D::foo);
>     // Here the upper word of `pf` will store the correction
>     // delta, which is needed to down-convert `B *` to `D *`

THAT is dangerous. You have now made a (B::*)() pointer that can only be 
successsfully applied to  D object, but can legally applied to an object 
that is just a B.

It is legal with the cast, because sometimes that is useful.

A Better example would be a (D::*) member-function-pointer being set 
from a (B::*)() member-function pointer. THAT is safe, as it can only be 
applied to a D object (without playing games) and all D's are also B's, 
so the offset is good.

> 
>     D *pd = new D;
>     B *pb = pd;
>     // `pb` is not the same numerically as `pd`
> 
>     (pb->*pf)();
>     // Correctly calls `D::foo`
>     // Before the actual call value of `pb` is adjusted by the
>     // correction delta stored in `pf`, thus providing us with
>     // the proper `D *this` pointer
> 
>     (pd->*pf)();
>     // Correctly calls `D::foo`
>     // Since `pf` is declared as `void (B::*)()`, the `pd` is
>     // first converted to `B *` (offsetting it), and then
>     // adjusted by the correction value stored in `pf`
>     // (un-offsetting it). This results in `pd` again, which
>     // is the proper `D *this` pointer
> 
> As you can see in the example above, even though it "has no idea of the 
> type of the object that it will be used on", everything works out nicely 
> and correctly.
> 
>> But it may be needed for a pointer-to-member function pointer which 
>> doesn't always hold the address of the member function to call but 
>> sometimes an offset/index into the vtable.
>>
>> As an example:
>>
>> class B {
>> public:
>>              void f1();
>>      virtual void f2();
>> }
>>
>> class D : public B {
>> public:
>>      virtual void f2();
>> }
>>
>> we take the member-function-pointers
>>
>> void (B::*ptr1)() = &B::f1;
>> void (B::*ptr2)() = &B::f2;
>>
>>
>> B b;
>> D d;
>>
>> b*->ptr1() vs b *->ptr2() vs d*->ptr1() vs d*->ptr2()
>>
>> There is a need to know that the "value" stored in ptr1 is the actual 
>> address of B:f1(), while ptr2 is storing an offset into the vtable of 
>> B for the member function.
>>
>> ptr2 can NOT just have the address of B::f2() as if derived class D 
>> overrides f2(), then the call via ptr2 needs to go to D::f2().
> 
> Nobody says that `ptr2` has to store address of `B::f2()`. But your 
> claim that there is a need to distinguish between `ptr1 and `ptr2` at 
> run-time is incorrect.
> 
> This is what MSVC will translate your example into:
> 
>    class B {
>    public:
>              void f1();
>      virtual void f2();
>              void f2_thunk() { return f2(); }
>    };
> 
>    class D : public B {
>    public:
>      virtual void f2();
>    }
> 
>    ...
>    void (B::*ptr1)() = &B::f1;
>    void (B::*ptr2)() = &B::f2_thunk;
> 
>    B b;
>    D d;
> 
>    //b*->ptr1() vs b *->ptr2() vs d*->ptr1() vs d*->ptr2()

Actually, if B isn't the first base, then D also needs a f2_thunk_B 
function that takes a B* as its object pointer and adjusts it to a D* 
pointer to call D:f2(), and the vtable that is in D's B sub-object 
points to that thunk.

You could also increase the size of ALL vtables to add an offset, but 
that slows down ALL virtual calls, not just ones that use non-first base 
class object pointers.

IF D derives from B1 and B2, a B1* -> f2() or a  D* -> f2() would lookup 
f2 in the B1/D vtable, while a B2* -> f2() would look up in the B2/D 
vtable using the pointer in the B2 sub-object.

The tricky part is that if you have a virtual base, you don't know the 
offset to that sub-object just based on the type of pointer you have, 
but you need to look it up in the vtable,

> 
> And suddenly everything works correctly, like magic! No need "to know" 
> which pointer is virtual and which pointer is not. Very simple.
> 
> Again, in this case `f2_thunk` takes advantage of the already 
> implemented virtual dispatch mechanism. The pointer simply re-uses it, 
> instead of re-implementing it.
> 
> It is quite possible that GCC's approach is more efficient, despite 
> being more bulky: branch predictions, cache behavior and so on... But 
> there's no denying that MSVC's approach is far more elegant.
> 

Yes, the difference is that the MSVC thunking method needs to figure out 
which thunks are needed. It can make one for EVERY virtual function as 
part of the class definition, and trust the linker to remove unused 
code. The alternative is to generate the thunk in every module when the 
address of a member function is taken, and truct the linker to merge 
duplicate thunks.

I think that once you handle the need to handle the correction of object 
for a virtual base, the cost isn't that high, and you likely have space 
to hide the flag.

This is for the case of a (D::*)() pointer being set to a (VB::*)() 
non-virtual member function (so you can't just try to arrange the 
virtual function thunk to make the correction).

You CAN'T create thunks for that, since when you first set the (VB::*)() 
pointer, you don't know about D, and when you convert the (VB::*)() to a 
(D::*)(() pointer, you would somehow need to translate every possible VB 
member function to is D equivalent.

[toc] | [prev] | [next] | [standalone]

#84202 — Re: Function pointer size (was Re: nullptr is not 0)

On 5/19/2022 6:32 PM, Richard Damon wrote:
> On 5/19/22 2:24 PM, Andrey Tarasevich wrote:
>> On 5/19/2022 10:36 AM, Richard Damon wrote:
>>>>>
>>>>> Yes that sort of describes the problem. The pointer to member needs 
>>>>> to be able to refer to a virtual function, or even a function that 
>>>>> is a member of a virtual base class, and handling that sort of 
>>>>> stuff is what forces the extra memory in the pointer-to-member.
>>>>
>>>> That's incorrect in general. Again, using x86 as a counterexample:
>>>>
>>>> Implementations that use different representations for virtual vs. 
>>>> non-virtual member pointers (GCC, Clang) manage to stuff that extra 
>>>> information into the "original" memory - they use the lowest bit of 
>>>> the pointer as a flag.
>>>>
>>>> Implementations that use identical representations for virtual vs. 
>>>> non-virtual member pointers (MSVC) naturally don't have this issue 
>>>> at all.
>>>>
>>>> So, no, the need to handle virtual functions does not require extra 
>>>> memory in pointer-to-member function.
>>>
>>> But not all processors have a LSB to use for that sort of flag.
>>>
>>> Some have instructions that can be at any address, because the 
>>> instruction set has single byte opcodes (They could just force all 
>>> functions to be at addresses with lower order zeros, but that is 
>>> wasteful)
>>>
>>> Others, like the ARM, use the low order bit to indicate which 
>>> instruction set the function is written in, so it isn't available for 
>>> such a use.
>>>
>>> Thus, there sometimes IS a need for an additional flag bit to 
>>> indicate if the pointer is an index into the v-table or the actual 
>>> address of the function.
>>
>> Well, that's a completely different claim. "Sometimes", huh...
>>
>> Firstly, as MSVC implementation demonstrates that the flag is not 
>> necessary at all.
> 
> At the cost of an addition layer of calling and more thunks.

Yes, versus the cost of "two representations" approach, which involves 
rather ugly branching to two fundamentally different calling methods in 
every call.

As for "more thunks", see below.

>>>
>>> Nope, because the pointer-to-member function pointer has no idea of 
>>> the type of the object that it will be used on, so CAN'T store the 
>>> full offset. (That is the job of the Thunk).
>>
>> This is completely incorrect.
>>
>> (Once again, let me reiterate that what I'm stating here is not 
>> offered for debate. These are hard facts about how modern 
>> implementations work. This is learning material for those willing to 
>> learn.)
>>
>> The pointer has to store the _delta_ value that corrects the `this` 
>> pointer between the static pointer type and the dynamic type of the 
>> pointed object.
>>
>> Here's a little illustrative sketch
>>
>>     struct B {};
>>     struct D : /* some other bases */, B { void foo() {} };
>>     // Base subobject `B` is offset inside `D`
>>
>>     void (B::*pf)() = static_cast<void (B::*)()>(&D::foo);
>>     // Here the upper word of `pf` will store the correction
>>     // delta, which is needed to down-convert `B *` to `D *`
> 
> THAT is dangerous. You have now made a (B::*)() pointer that can only be 
> successsfully applied to  D object, but can legally applied to an object 
> that is just a B.

Yes, it is dangerous. Yet, this is the kind of "dangerous" that is 
supported by `static_cast`. Co-variant object pointers support dangerous 
downcasts the same way contra-variant member-pointers support dangerous 
upcasts. That's just something C++ permits, as long as you know what you 
are doing.

> It is legal with the cast, because sometimes that is useful.
> 
> A Better example would be a (D::*) member-function-pointer being set 
> from a (B::*)() member-function pointer. THAT is safe, as it can only be 
> applied to a D object (without playing games) 

As far as static typing is concerned, a (D::*) member-function-pointer 
can _only_ be applied to a D object (or a derived from D). There's no 
way around that, with games or without. It simply won't compile otherwise.

Note that in my example from the static typing point of view a (B::*) 
member-function-pointer is applied to a B object. So, everything looks 
fine. However, language allows (B::*) pointers to point to exclusive 
members of D (through `static_cast`) and be applied to B objects as long 
as these B objects are actually D objects from the dynamic typing point 
of view.

> and all D's are also B's, 
> so the offset is good.

Yes, perhaps. Is it a better example though? The "shock value" of mine 
is higher :O)

>> ...
>> And suddenly everything works correctly, like magic! No need "to know" 
>> which pointer is virtual and which pointer is not. Very simple.
>>
>> Again, in this case `f2_thunk` takes advantage of the already 
>> implemented virtual dispatch mechanism. The pointer simply re-uses it, 
>> instead of re-implementing it.
>>
>> It is quite possible that GCC's approach is more efficient, despite 
>> being more bulky: branch predictions, cache behavior and so on... But 
>> there's no denying that MSVC's approach is far more elegant.
>>
> 
> Yes, the difference is that the MSVC thunking method needs to figure out 
> which thunks are needed. It can make one for EVERY virtual function as 
> part of the class definition, and trust the linker to remove unused 
> code. The alternative is to generate the thunk in every module when the 
> address of a member function is taken, and truct the linker to merge 
> duplicate thunks.

Well, once one's not restricted by the rules of the language, i.e. 
starts thinking in terms of generated machine commands, one realizes 
that a thunk created for the "function #1 in the VMT" is immediately 
applicable to _all_ polymorphic classes, which greatly increases the 
utility of this approach

For example

   struct A
   {
     virtual int foo(int a)
       { std::cout << a << std::endl; return 42; }
   };

   struct B
   {
     virtual void bar(double x, double y)
       { std::cout << x << " " << y << std::endl; }
   };

   int main()
   {
     int (A::*pa)(int) = &A::foo;
     void (B::*pb)(double, double) = &B::bar;
   }

Both pointers in this example will point to the same thunk, even though 
the classes are unrelated and the function types are completely different.

> I think that once you handle the need to handle the correction of object 
> for a virtual base, the cost isn't that high, and you likely have space 
> to hide the flag.
> 
> This is for the case of a (D::*)() pointer being set to a (VB::*)() 
> non-virtual member function (so you can't just try to arrange the 
> virtual function thunk to make the correction).
> 
> You CAN'T create thunks for that, since when you first set the (VB::*)() 
> pointer, you don't know about D, and when you convert the (VB::*)() to a 
> (D::*)(() pointer, you would somehow need to translate every possible VB 
> member function to is D equivalent.

Virtual bases are a separate can of worms... I'm surprised they don't 
require an even larger member function pointer. I remember that  old 
versions of MSVC generated 4-word pointers in "virtual bases allowed" 
mode. But apparently it is possible to get around them with two.

-- 
Best regards,
Andrey

[toc] | [prev] | [next] | [standalone]

#84223 — Re: Function pointer size (was Re: nullptr is not 0)

On 5/19/22 11:28 PM, Andrey Tarasevich wrote:
> On 5/19/2022 6:32 PM, Richard Damon wrote:
>> On 5/19/22 2:24 PM, Andrey Tarasevich wrote:
>>> On 5/19/2022 10:36 AM, Richard Damon wrote:
>>>>>>
>>>>>> Yes that sort of describes the problem. The pointer to member 
>>>>>> needs to be able to refer to a virtual function, or even a 
>>>>>> function that is a member of a virtual base class, and handling 
>>>>>> that sort of stuff is what forces the extra memory in the 
>>>>>> pointer-to-member.
>>>>>
>>>>> That's incorrect in general. Again, using x86 as a counterexample:
>>>>>
>>>>> Implementations that use different representations for virtual vs. 
>>>>> non-virtual member pointers (GCC, Clang) manage to stuff that extra 
>>>>> information into the "original" memory - they use the lowest bit of 
>>>>> the pointer as a flag.
>>>>>
>>>>> Implementations that use identical representations for virtual vs. 
>>>>> non-virtual member pointers (MSVC) naturally don't have this issue 
>>>>> at all.
>>>>>
>>>>> So, no, the need to handle virtual functions does not require extra 
>>>>> memory in pointer-to-member function.
>>>>
>>>> But not all processors have a LSB to use for that sort of flag.
>>>>
>>>> Some have instructions that can be at any address, because the 
>>>> instruction set has single byte opcodes (They could just force all 
>>>> functions to be at addresses with lower order zeros, but that is 
>>>> wasteful)
>>>>
>>>> Others, like the ARM, use the low order bit to indicate which 
>>>> instruction set the function is written in, so it isn't available 
>>>> for such a use.
>>>>
>>>> Thus, there sometimes IS a need for an additional flag bit to 
>>>> indicate if the pointer is an index into the v-table or the actual 
>>>> address of the function.
>>>
>>> Well, that's a completely different claim. "Sometimes", huh...
>>>
>>> Firstly, as MSVC implementation demonstrates that the flag is not 
>>> necessary at all.
>>
>> At the cost of an addition layer of calling and more thunks.
> 
> Yes, versus the cost of "two representations" approach, which involves 
> rather ugly branching to two fundamentally different calling methods in 
> every call.

It doesn't require "Branching", it just requires a conditional skip of 
the load of the function address from the vtable.
> 
> As for "more thunks", see below.
> 
>>>>
>>>> Nope, because the pointer-to-member function pointer has no idea of 
>>>> the type of the object that it will be used on, so CAN'T store the 
>>>> full offset. (That is the job of the Thunk).
>>>
>>> This is completely incorrect.
>>>
>>> (Once again, let me reiterate that what I'm stating here is not 
>>> offered for debate. These are hard facts about how modern 
>>> implementations work. This is learning material for those willing to 
>>> learn.)
>>>
>>> The pointer has to store the _delta_ value that corrects the `this` 
>>> pointer between the static pointer type and the dynamic type of the 
>>> pointed object.
>>>
>>> Here's a little illustrative sketch
>>>
>>>     struct B {};
>>>     struct D : /* some other bases */, B { void foo() {} };
>>>     // Base subobject `B` is offset inside `D`
>>>
>>>     void (B::*pf)() = static_cast<void (B::*)()>(&D::foo);
>>>     // Here the upper word of `pf` will store the correction
>>>     // delta, which is needed to down-convert `B *` to `D *`
>>
>> THAT is dangerous. You have now made a (B::*)() pointer that can only 
>> be successsfully applied to  D object, but can legally applied to an 
>> object that is just a B.
> 
> Yes, it is dangerous. Yet, this is the kind of "dangerous" that is 
> supported by `static_cast`. Co-variant object pointers support dangerous 
> downcasts the same way contra-variant member-pointers support dangerous 
> upcasts. That's just something C++ permits, as long as you know what you 
> are doing.

Yes, there are uses when you KNOW things about your code that the 
complier/language isn't allowed to assume. Like you have a B* pointer, 
but have verified that it actually points to a D.

> 
>> It is legal with the cast, because sometimes that is useful.
>>
>> A Better example would be a (D::*) member-function-pointer being set 
>> from a (B::*)() member-function pointer. THAT is safe, as it can only 
>> be applied to a D object (without playing games) 
> 
> As far as static typing is concerned, a (D::*) member-function-pointer 
> can _only_ be applied to a D object (or a derived from D). There's no 
> way around that, with games or without. It simply won't compile otherwise.

Right, and since all D's have a B sub-object, the allowed operation is 
always safe.

> 
> Note that in my example from the static typing point of view a (B::*) 
> member-function-pointer is applied to a B object. So, everything looks 
> fine. However, language allows (B::*) pointers to point to exclusive 
> members of D (through `static_cast`) and be applied to B objects as long 
> as these B objects are actually D objects from the dynamic typing point 
> of view.

Right, you told the complier I know what I am doing when I do the 
"dangerous" cast from (D::*)() to (B::*)() and then your kept your word 
and only used it on an object that was actually a D.

Apply that pointer to an object that is actually just a B, and problems 
occur.

My comment is using "Dangerous" code in an example, without even 
including about why it is dangerous, isn't a good example, especially 
when a safe alternate that shows exactly the same need exist.

> 
>> and all D's are also B's, so the offset is good.
> 
> Yes, perhaps. Is it a better example though? The "shock value" of mine 
> is higher :O)

I prefer to avoid leading people to the dark side of flirting with UB if 
you aren't careful. Teaching people that it is ok to use a static_cast 
without explanation is like playing with guns like they were toys. The 
expert may still be safe, but others may hurt themselves.

My coding practice says that unsafe casts should have a comment near 
them explaining why this is ok here.

> 
>>> ...
>>> And suddenly everything works correctly, like magic! No need "to 
>>> know" which pointer is virtual and which pointer is not. Very simple.
>>>
>>> Again, in this case `f2_thunk` takes advantage of the already 
>>> implemented virtual dispatch mechanism. The pointer simply re-uses 
>>> it, instead of re-implementing it.
>>>
>>> It is quite possible that GCC's approach is more efficient, despite 
>>> being more bulky: branch predictions, cache behavior and so on... But 
>>> there's no denying that MSVC's approach is far more elegant.
>>>
>>
>> Yes, the difference is that the MSVC thunking method needs to figure 
>> out which thunks are needed. It can make one for EVERY virtual 
>> function as part of the class definition, and trust the linker to 
>> remove unused code. The alternative is to generate the thunk in every 
>> module when the address of a member function is taken, and truct the 
>> linker to merge duplicate thunks.
> 
> Well, once one's not restricted by the rules of the language, i.e. 
> starts thinking in terms of generated machine commands, one realizes 
> that a thunk created for the "function #1 in the VMT" is immediately 
> applicable to _all_ polymorphic classes, which greatly increases the 
> utility of this approach
> 
> For example
> 
>    struct A
>    {
>      virtual int foo(int a)
>        { std::cout << a << std::endl; return 42; }
>    };
> 
>    struct B
>    {
>      virtual void bar(double x, double y)
>        { std::cout << x << " " << y << std::endl; }
>    };
> 
>    int main()
>    {
>      int (A::*pa)(int) = &A::foo;
>      void (B::*pb)(double, double) = &B::bar;
>    }
> 
> Both pointers in this example will point to the same thunk, even though 
> the classes are unrelated and the function types are completely different.

It is NOT true that it works on all polymorphic classes (maybe most though).

It only works if the vtbl pointer is at the same offset (typically 0) to 
the class pointer for those two classes. If ever you derive from a class 
with no virtual functions, and add a virtual function, that is no longer 
true.

> 
>> I think that once you handle the need to handle the correction of 
>> object for a virtual base, the cost isn't that high, and you likely 
>> have space to hide the flag.
>>
>> This is for the case of a (D::*)() pointer being set to a (VB::*)() 
>> non-virtual member function (so you can't just try to arrange the 
>> virtual function thunk to make the correction).
>>
>> You CAN'T create thunks for that, since when you first set the 
>> (VB::*)() pointer, you don't know about D, and when you convert the 
>> (VB::*)() to a (D::*)(() pointer, you would somehow need to translate 
>> every possible VB member function to is D equivalent.
> 
> Virtual bases are a separate can of worms... I'm surprised they don't 
> require an even larger member function pointer. I remember that  old 
> versions of MSVC generated 4-word pointers in "virtual bases allowed" 
> mode. But apparently it is possible to get around them with two.
> 

Not sure how they do it either, unless it is more thunks, and these will 
not compress nearly as well.

[toc] | [prev] | [next] | [standalone]

#84189 — Re: Function pointer size (was Re: nullptr is not 0)

On 5/19/2022 7:53 AM, Juha Nieminen wrote:
> Andrey Tarasevich <andreytarasevich@hotmail.com> wrote:
>>> If you have a reference or pointer to an object of type class A, and a
>>> member pointer to a member function of class A, which is virtual, if
>>> the actual object pointed to is of some derived type B which has its
>>> own specialization of that virtual function, calling the function using
>>> the pointer will call the derived implementation not, the one in class A.
>>>
>>> For this to be possible the pointer to member function needs additional
>>> data.
>>
>> No, that's incorrect.
>>
>> This has nothing to do with virtual functions. On the contrary,
>> pointers-to-virtual-functions are "easy": regular virtual call mechanism
>> itself is already required to incorporate all necessary mechanics to
>> properly invoke virtual functions across the entire hierarchy of
>> classes. This mechanics is already present in "regular" virtual calls
>> (without involving any pointers-to-members).
> 
> I'm not sure that's correct.

I am.

> When you have a pointer-to-member, in the location of the call the compiler
> doesn't know *which* member function it's pointing to. It only knows its
> signature, not its name. The class may have several differently-named
> member fuctions with the same signature. I don't think the compiler can
> even know (from the source code alone) if the pointed-to function is
> virtual or not.

That's true. However, some implementations do use different internal 
pointer representations for pointers that happen to point to virtual 
functions, while some other implementations don't. It depends on the 
chosen approach.

* In GCC on x86 (x64) the lower 8-byte word of a pointer that happens to 
point a regular function is simply a pointer. At the same time a pointer 
that happens to point to a virtual function represents an offset into 
virtual table +1. These are two completely different representations.

GCC distinguishes the former from the latter by fact that the former is 
always even and the latter is always odd, i.e. it uses the lowest bit as 
a distinguishing flag. It can do that because function entry points on 
x86 are always even.

This means that GCC cannot just "call" the "address" stored in such a 
pointer. Every time it makes a call through such a pointer, GCC has to 
analyze the pointer's value every and branch to a proper dispatch method.

This is a strange approach, but apparently GCC likes it.

Here's an example for you

   http://coliru.stacked-crooked.com/a/568ab87c2741f22e

See the output

   01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
   09 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
   02 0d 40 00 00 00 00 00 00 00 00 00 00 00 00 00

The first one is 1 (odd) represents virtual function `foo` with offset 
1-1=0 in the VMT.
The second one is 9 (odd) represents virtual function `baz` with offset 
9-1=8 in the VMT.
The third one (even) represents regular function `bar`. This is just a 
callable address.

Note that in all cases the upper 8-byte word is zero. The upper word is 
not used to distinguish "virtual" pointers from "regular" ones.

* MSVC uses a completely different approach. The lower 8-byte word of a 
pointer to a member function is always simply a pointer to an entry 
point into "some" function. It can always be called right away.

MSVC generates non-virtual middle-man caller stubs for each virtual 
function. A pointer to a virtual member function will point to that stub 
instead of pointing to the virtual function itself. In MSVC a pointer to 
a member function always physically points to some non-virtual function.

This is a different approach and MSVC sticks to it.

Here's a sketch: every time you write something like this

   struct S
   {
     virtual int foo() { ... }
   };

   int (S::*p) = &S::foo;

MSVC compiler actually generates something like this

   struct S
   {
     virtual int foo() { ... }
     int foo_dispatch() { return foo(); }
   };

   int (S::*p) = &S::foo_dispatch();

See? Everything works as it is supposed to, yet there's no need to take 
any extra steps to distinguish virtual pointers from non-virtual ones, 
because such pointers always physically point to "regular" functions.

> Thus, the compiler needs to create code that somehow figures out if
> that member function is virtual, and which function it is, 

No, it depends on the chosen approach. See above. In GCC it does. In 
MSVC it doesn't.

> and then
> use the vtable as normal to jump to the actual most-derived implementation.

But that is how a virtual call always works. This is already built into 
the virtual call mechanism. There's no need to do anything additional 
for pointers to members, as long as you know how to piggyback on the 
already-existing virtual call mechanism "for free". And that is exactly 
what MSVC's approach takes advantage of.

> I don't think it can do this without the extra data in the pointer.

Yes, you can.

As I have shown above, GCC does need "extra data" in the pointer to tell 
one representation from the other. But since all function addresses on 
x86 are even, it can use the lowest bit of the lower 8-byte word for 
that. I.e. it stuffs that data into the original "pointer". No need to 
increase the pointer size.

Meanwhile, MSVC does not need that extra data at all.

> Please correct me if I'm wrong.

As I have already explained in my other answer in this thread, the upper 
8-byte word in member function pointer is used for offsetting `this` 
pointer before the call in situations when offsetting is necessary. This 
is needed with multiple inheritance (and some other more niche cases). 
If `this` pointer needs no offsetting, the upper 8-byte word is zero.

So, again, the upper 8-byte word has nothing to do with implementing 
calls to virtual functions.

This all is, of course, implementation details. I have outlined the most 
popular practical approaches. Some other implementation might invent 
something else.

-- 
Best regards,
Andrey

[toc] | [prev] | [next] | [standalone]

#84191 — Re: Function pointer size (was Re: nullptr is not 0)

On 5/19/2022 9:35 AM, Andrey Tarasevich wrote:
> 
> As I have already explained in my other answer in this thread, the upper 
> 8-byte word in member function pointer is used for offsetting `this` 
> pointer before the call in situations when offsetting is necessary. This 
> is needed with multiple inheritance (and some other more niche cases). 
> If `this` pointer needs no offsetting, the upper 8-byte word is zero.
> 

To put it into a more compact and concise form:

Consider this little piece of code

   // Class `B` is a base class of `D`

   Derived *d = new D;
   Base *b = d;

   // Is `b` the same as `d` numerically?
   // Is (std::uintptr_t) b == (std::uintptr_t) d?

If we could always guarantee that `b` has the same numerical value as 
`d`, then we would be able to get away with pointers-to-member-functions 
having the same size as ordinary pointers.

But in real life this does not hold. In general case `b` and `d` point 
to different spots in memory. And that is really why 
pointers-to-member-functions have to be larger than ordinary pointers.

P.S. And, once again, virtual functions have nothing to do with it.

-- 
Best regards,
Andrey

[toc] | [prev] | [next] | [standalone]

#84206 — Re: Function pointer size (was Re: nullptr is not 0)

Andrey Tarasevich <andreytarasevich@hotmail.com> wrote:
> * In GCC on x86 (x64) the lower 8-byte word of a pointer that happens to 
> point a regular function is simply a pointer. At the same time a pointer 
> that happens to point to a virtual function represents an offset into 
> virtual table +1. These are two completely different representations.

I think I get now what you are saying.

A pointer-to-virtual-member doesn't have to point directly to the function
itself (as normal function pointers do). Instead, it points to a memory
location that has the actual memory address of the function (in other
words, the pointer points to a position in a vtable).

And the code generated by the compiler can make that distinction by
looking at the least-significant bit of the pointer (which can be used
to indicate the "type" of the pointer because functions always start
at even memory addresses, at least in x86, and thus this least-significant
bit is free space that can be used to store this flag.)

[toc] | [prev] | [next] | [standalone]

#84171 — Re: Function pointer size (was Re: nullptr is not 0)

On 5/17/2022 1:30 PM, Vir Campestris wrote:
> On 09/05/2022 22:56, Scott Lurndal wrote:
>> Vir Campestris <vir.campestris@invalid.invalid> writes:
>>> On 09/05/2022 06:06, Juha Nieminen wrote:
>>>> Member function pointers shouldn't really be thought of as normal 
>>>> pointers.
>>>> They are rather different beasts, behave differently in many 
>>>> respects, and
>>>> you can't even convert from member-function-pointer to 
>>>> regular-pointer and
>>>> back safely because they are completely incompatible with each other.
>>>> They are likely not even the same size.
>>>
>>> _Likely_ not even the same size?
>>
>> With g++ on linux, sizeof(member function pointer) returns
>> 16 bytes.
>>
> 
> :O
> 
> I just experimented. Yes, it's 16 bytes, while an ordinary function 
> pointer is only 8 (Linux 64, g++).
> 
> But the top 8 bytes always seem to be zero.
> 
> When are they anything else? Because if they aren't there's no reason 
> for the different size.

Iirc, even if nullptr is defined to be something like 0xDEADBEEF, it 
will still work when compared to 0. So:

void* p0 = nullptr;
void* p1 = 0;

p0 == p1

and (p0 == 0 && p1 == 0) is true.

[toc] | [prev] | [next] | [standalone]

#84172 — Re: Function pointer size (was Re: nullptr is not 0)

On 5/18/2022 11:50 AM, Chris M. Thomasson wrote:
> 
> Iirc, even if nullptr is defined to be something like 0xDEADBEEF, it 
> will still work when compared to 0. So:
> 
> void* p0 = nullptr;
> void* p1 = 0;
> 
> p0 == p1

The `p0 == p1` comparison does not really demonstrate anything, since 
both pointers are `0xDEADBEEF` at this point anyway.

> and (p0 == 0 && p1 == 0) is true.

Yes, but that a consequence of the same process that turned `0` to 
`0xDEADBEEF` in the initialization. Such comparisons are performed in 
the domain of pointer type, meaning that `0` gets converted to pointer. 
Just as it turned into `0xDEADBEEF` in the initialization, it turns into 
`0xDEADBEEF` here as well. I.e. these comparisons also compare 
`0xDEADBEEF` to `0xDEADBEEF` internally.

-- 
Best regards,
Andrey

[toc] | [prev] | [next] | [standalone]

#84203 — Re: Function pointer size (was Re: nullptr is not 0)

On 5/18/2022 3:46 PM, Andrey Tarasevich wrote:
> On 5/18/2022 11:50 AM, Chris M. Thomasson wrote:
>>
>> Iirc, even if nullptr is defined to be something like 0xDEADBEEF, it 
>> will still work when compared to 0. So:
>>
>> void* p0 = nullptr;
>> void* p1 = 0;
>>
>> p0 == p1
> 
> The `p0 == p1` comparison does not really demonstrate anything, since 
> both pointers are `0xDEADBEEF` at this point anyway.
> 
>> and (p0 == 0 && p1 == 0) is true.
> 
> Yes, but that a consequence of the same process that turned `0` to 
> `0xDEADBEEF` in the initialization. Such comparisons are performed in 
> the domain of pointer type, meaning that `0` gets converted to pointer. 
> Just as it turned into `0xDEADBEEF` in the initialization, it turns into 
> `0xDEADBEEF` here as well. I.e. these comparisons also compare 
> `0xDEADBEEF` to `0xDEADBEEF` internally.
> 

Correct. For some damn reason this thread is making me think of an older 
quick and dirty OOP thing for C I did a while back:

https://pastebin.com/raw/QPssvGJR

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | comp.lang.c++

csiph-web