Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #94806 > unrolled thread

UB of memcpy(NULL, NULL, 0)

Back to article view | Back to comp.lang.c

Contents

  UB of memcpy(NULL, NULL, 0) Noob <root@127.0.0.1> - 2016-10-13 12:42 +0200
    Re: UB of memcpy(NULL, NULL, 0) Richard Heathfield <rjh@cpax.org.uk> - 2016-10-13 11:55 +0100
      Re: UB of memcpy(NULL, NULL, 0) Noob <root@127.0.0.1> - 2016-10-13 13:09 +0200
        Re: UB of memcpy(NULL, NULL, 0) Richard Heathfield <rjh@cpax.org.uk> - 2016-10-13 12:22 +0100
          Re: UB of memcpy(NULL, NULL, 0) supercat@casperkitty.com - 2016-10-13 12:22 -0700
            Re: UB of memcpy(NULL, NULL, 0) "Rick C. Hodgin" <rick.c.hodgin@gmail.com> - 2016-10-13 12:32 -0700
              Re: UB of memcpy(NULL, NULL, 0) "Rick C. Hodgin" <rick.c.hodgin@gmail.com> - 2016-10-13 12:34 -0700
      Re: UB of memcpy(NULL, NULL, 0) Joe Pfeiffer <pfeiffer@cs.nmsu.edu> - 2016-10-13 08:24 -0600
        Re: UB of memcpy(NULL, NULL, 0) Richard Heathfield <rjh@cpax.org.uk> - 2016-10-13 16:22 +0100
        Re: UB of memcpy(NULL, NULL, 0) Noob <root@127.0.0.1> - 2016-10-13 17:49 +0200
          Re: UB of memcpy(NULL, NULL, 0) Joe Pfeiffer <pfeiffer@cs.nmsu.edu> - 2016-10-13 12:02 -0600
    Re: UB of memcpy(NULL, NULL, 0) Lőrinczy Zsigmond <nospam@for.me> - 2016-10-15 20:56 +0200
      Re: UB of memcpy(NULL, NULL, 0) supercat@casperkitty.com - 2016-10-15 12:52 -0700
      Re: UB of memcpy(NULL, NULL, 0) "James R. Kuyper" <jameskuyper@verizon.net> - 2016-10-15 15:55 -0400
      Re: UB of memcpy(NULL, NULL, 0) Keith Thompson <kst-u@mib.org> - 2016-10-15 13:41 -0700
        Re: UB of memcpy(NULL, NULL, 0) Lőrinczy Zsigmond <nospam@for.me> - 2016-10-16 11:17 +0200
          Re: UB of memcpy(NULL, NULL, 0) Richard Heathfield <rjh@cpax.org.uk> - 2016-10-16 10:26 +0100
          Re: UB of memcpy(NULL, NULL, 0) raltbos@xs4all.nl (Richard Bos) - 2016-10-16 09:55 +0000
            Re: UB of memcpy(NULL, NULL, 0) Jerry Stuckle <jstucklex@attglobal.net> - 2016-10-16 08:44 -0400
              Re: UB of memcpy(NULL, NULL, 0) David Brown <david.brown@hesbynett.no> - 2016-10-17 10:06 +0200
                Re: UB of memcpy(NULL, NULL, 0) Jerry Stuckle <jstucklex@attglobal.net> - 2016-10-17 09:06 -0400
                  Re: UB of memcpy(NULL, NULL, 0) David Brown <david.brown@hesbynett.no> - 2016-10-17 16:29 +0200
                    Re: UB of memcpy(NULL, NULL, 0) Jerry Stuckle <jstucklex@attglobal.net> - 2016-10-17 17:05 -0400
                      Re: UB of memcpy(NULL, NULL, 0) David Brown <david.brown@hesbynett.no> - 2016-10-18 11:19 +0200
                        Re: UB of memcpy(NULL, NULL, 0) supercat@casperkitty.com - 2016-10-18 08:04 -0700
                        Re: UB of memcpy(NULL, NULL, 0) Jerry Stuckle <jstucklex@attglobal.net> - 2016-10-18 16:27 -0400
              Re: UB of memcpy(NULL, NULL, 0) supercat@casperkitty.com - 2016-10-17 08:59 -0700
                Re: UB of memcpy(NULL, NULL, 0) Keith Thompson <kst-u@mib.org> - 2016-10-17 11:21 -0700
                  Re: UB of memcpy(NULL, NULL, 0) supercat@casperkitty.com - 2016-10-17 13:42 -0700
                    Re: UB of memcpy(NULL, NULL, 0) Keith Thompson <kst-u@mib.org> - 2016-10-17 14:46 -0700
                      Re: UB of memcpy(NULL, NULL, 0) supercat@casperkitty.com - 2016-10-17 15:52 -0700
                        Re: UB of memcpy(NULL, NULL, 0) Keith Thompson <kst-u@mib.org> - 2016-10-17 16:43 -0700
                          Re: UB of memcpy(NULL, NULL, 0) supercat@casperkitty.com - 2016-10-18 08:00 -0700
                            Re: UB of memcpy(NULL, NULL, 0) Keith Thompson <kst-u@mib.org> - 2016-10-18 09:14 -0700
                              Re: UB of memcpy(NULL, NULL, 0) supercat@casperkitty.com - 2016-10-18 09:43 -0700
                                Re: UB of memcpy(NULL, NULL, 0) Keith Thompson <kst-u@mib.org> - 2016-10-18 11:31 -0700
                                  Re: UB of memcpy(NULL, NULL, 0) supercat@casperkitty.com - 2016-10-18 12:49 -0700
                                Re: UB of memcpy(NULL, NULL, 0) Keith Thompson <kst-u@mib.org> - 2016-10-18 12:03 -0700
                Re: UB of memcpy(NULL, NULL, 0) Jerry Stuckle <jstucklex@attglobal.net> - 2016-10-17 17:06 -0400
                  Re: UB of memcpy(NULL, NULL, 0) supercat@casperkitty.com - 2016-10-17 14:55 -0700
          Re: UB of memcpy(NULL, NULL, 0) Keith Thompson <kst-u@mib.org> - 2016-10-16 11:23 -0700
        Re: UB of memcpy(NULL, NULL, 0) supercat@casperkitty.com - 2016-10-17 08:49 -0700
          Re: UB of memcpy(NULL, NULL, 0) raltbos@xs4all.nl (Richard Bos) - 2016-10-18 16:54 +0000
            Re: UB of memcpy(NULL, NULL, 0) supercat@casperkitty.com - 2016-10-18 11:09 -0700
              Re: UB of memcpy(NULL, NULL, 0) raltbos@xs4all.nl (Richard Bos) - 2016-10-23 13:24 +0000
                Re: UB of memcpy(NULL, NULL, 0) supercat@casperkitty.com - 2016-10-24 09:28 -0700
                Re: UB of memcpy(NULL, NULL, 0) <william@wilbur.25thandClement.com> - 2016-10-29 11:32 -0700
                  Re: UB of memcpy(NULL, NULL, 0) supercat@casperkitty.com - 2016-10-29 14:12 -0700
      Re: UB of memcpy(NULL, NULL, 0) Ben Bacarisse <ben.usenet@bsb.me.uk> - 2016-10-15 22:00 +0100
        Re: UB of memcpy(NULL, NULL, 0) Malcolm McLean <malcolm.mclean5@btinternet.com> - 2016-10-16 02:51 -0700
          Re: UB of memcpy(NULL, NULL, 0) Ben Bacarisse <ben.usenet@bsb.me.uk> - 2016-10-16 11:20 +0100
            Re: UB of memcpy(NULL, NULL, 0) Malcolm McLean <malcolm.mclean5@btinternet.com> - 2016-10-16 04:14 -0700
              Re: UB of memcpy(NULL, NULL, 0) BartC <bc@freeuk.com> - 2016-10-16 12:22 +0100
              Re: UB of memcpy(NULL, NULL, 0) Richard Heathfield <rjh@cpax.org.uk> - 2016-10-16 13:01 +0100
                Re: UB of memcpy(NULL, NULL, 0) Malcolm McLean <malcolm.mclean5@btinternet.com> - 2016-10-16 05:29 -0700
                  Re: UB of memcpy(NULL, NULL, 0) Ben Bacarisse <ben.usenet@bsb.me.uk> - 2016-10-16 20:47 +0100
                    Re: UB of memcpy(NULL, NULL, 0) Malcolm McLean <malcolm.mclean5@btinternet.com> - 2016-10-17 02:08 -0700
                  Re: UB of memcpy(NULL, NULL, 0) David Brown <david.brown@hesbynett.no> - 2016-10-17 10:24 +0200
                    Re: UB of memcpy(NULL, NULL, 0) Malcolm McLean <malcolm.mclean5@btinternet.com> - 2016-10-17 08:05 -0700
                      Re: UB of memcpy(NULL, NULL, 0) David Brown <david.brown@hesbynett.no> - 2016-10-17 17:25 +0200
    Re: UB of memcpy(NULL, NULL, 0) Lőrinczy Zsigmond <nospam@for.me> - 2016-10-17 17:08 +0200
      Re: UB of memcpy(NULL, NULL, 0) Lorinczy Zsigmond <zsiga@nospam.for.me> - 2016-10-21 12:46 +0200
        Re: UB of memcpy(NULL, NULL, 0) supercat@casperkitty.com - 2016-10-21 08:06 -0700
          Re: UB of memcpy(NULL, NULL, 0) Lőrinczy Zsigmond <nospam@for.me> - 2016-10-21 18:30 +0200
            Re: UB of memcpy(NULL, NULL, 0) supercat@casperkitty.com - 2016-10-21 09:34 -0700
              Re: UB of memcpy(NULL, NULL, 0) Lőrinczy Zsigmond <nospam@for.me> - 2016-10-22 07:42 +0200
                Re: UB of memcpy(NULL, NULL, 0) James Kuyper <jameskuyper@verizon.net> - 2016-10-22 10:31 -0400
                  Re: UB of memcpy(NULL, NULL, 0) Lőrinczy Zsigmond <nospam@for.me> - 2016-10-22 17:49 +0200
                    Re: UB of memcpy(NULL, NULL, 0) Tim Rentsch <txr@alumni.caltech.edu> - 2016-10-22 13:43 -0700
                      Re: UB of memcpy(NULL, NULL, 0) Lőrinczy Zsigmond <nospam@for.me> - 2016-10-23 07:47 +0200
                        Re: UB of memcpy(NULL, NULL, 0) "James R. Kuyper" <jameskuyper@verizon.net> - 2016-10-23 11:07 -0400
                          Re: UB of memcpy(NULL, NULL, 0) Lőrinczy Zsigmond <nospam@for.me> - 2016-10-23 17:45 +0200
                            Re: UB of memcpy(NULL, NULL, 0) "James R. Kuyper" <jameskuyper@verizon.net> - 2016-10-23 12:16 -0400
                            Re: UB of memcpy(NULL, NULL, 0) supercat@casperkitty.com - 2016-10-24 07:38 -0700
                              Re: UB of memcpy(NULL, NULL, 0) Keith Thompson <kst-u@mib.org> - 2016-10-24 08:38 -0700
                                Re: UB of memcpy(NULL, NULL, 0) supercat@casperkitty.com - 2016-10-24 09:44 -0700
                        Re: UB of memcpy(NULL, NULL, 0) Keith Thompson <kst-u@mib.org> - 2016-10-23 12:22 -0700
                          Re: UB of memcpy(NULL, NULL, 0) Malcolm McLean <malcolm.mclean5@btinternet.com> - 2016-10-23 13:37 -0700
                            Re: UB of memcpy(NULL, NULL, 0) Keith Thompson <kst-u@mib.org> - 2016-10-23 14:15 -0700
                            Re: UB of memcpy(NULL, NULL, 0) Ike Naar <ike@iceland.freeshell.org> - 2016-10-24 06:11 +0000
                              Re: UB of memcpy(NULL, NULL, 0) Malcolm McLean <malcolm.mclean5@btinternet.com> - 2016-10-24 00:58 -0700
                        Re: UB of memcpy(NULL, NULL, 0) Tim Rentsch <txr@alumni.caltech.edu> - 2016-10-23 14:07 -0700
                      Re: UB of memcpy(NULL, NULL, 0) supercat@casperkitty.com - 2016-10-24 07:48 -0700
                        Re: UB of memcpy(NULL, NULL, 0) Keith Thompson <kst-u@mib.org> - 2016-10-24 08:40 -0700
                          Re: UB of memcpy(NULL, NULL, 0) supercat@casperkitty.com - 2016-10-24 10:03 -0700
                            Re: UB of memcpy(NULL, NULL, 0) Keith Thompson <kst-u@mib.org> - 2016-10-24 12:06 -0700
                              Re: UB of memcpy(NULL, NULL, 0) supercat@casperkitty.com - 2016-10-24 13:36 -0700
                                Re: UB of memcpy(NULL, NULL, 0) Keith Thompson <kst-u@mib.org> - 2016-10-24 13:57 -0700
                        Re: UB of memcpy(NULL, NULL, 0) Tim Rentsch <txr@alumni.caltech.edu> - 2016-10-24 14:16 -0700
                          Re: UB of memcpy(NULL, NULL, 0) supercat@casperkitty.com - 2016-10-24 17:23 -0700
                            Re: UB of memcpy(NULL, NULL, 0) Tim Rentsch <txr@alumni.caltech.edu> - 2016-10-25 07:57 -0700
                    Re: UB of memcpy(NULL, NULL, 0) James Kuyper <jameskuyper@verizon.net> - 2016-10-22 18:23 -0400
                  Re: UB of memcpy(NULL, NULL, 0) "James R. Kuyper" <jameskuyper@verizon.net> - 2016-10-23 10:44 -0400
                    Re: UB of memcpy(NULL, NULL, 0) Richard Heathfield <rjh@cpax.org.uk> - 2016-10-23 16:24 +0100
                      Re: Verbing Noob <root@127.0.0.1> - 2016-10-23 18:02 +0200
          Re: UB of memcpy(NULL, NULL, 0) raltbos@xs4all.nl (Richard Bos) - 2016-10-23 13:26 +0000
            Re: UB of memcpy(NULL, NULL, 0) Malcolm McLean <malcolm.mclean5@btinternet.com> - 2016-10-23 07:40 -0700
        Re: UB of memcpy(NULL, NULL, 0) asetofsymbols@gmail.com - 2016-10-21 23:56 -0700
          Re: UB of memcpy(NULL, NULL, 0) Lőrinczy Zsigmond <nospam@for.me> - 2016-10-22 12:21 +0200
            Re: UB of memcpy(NULL, NULL, 0) Keith Thompson <kst-u@mib.org> - 2016-10-22 16:33 -0700

Page 3 of 5 — ← Prev page 1 2 [3] 4 5  Next page →

#94931

Lőrinczy Zsigmond <nospam@for.me> writes:
>> As a programmer, how much of a burden is it to avoid calling memcpy()
>> with a null pointer?

I wrote the above.  Please don't delete attribution lines when you post
a followup.

> It would add a redundant 'if' to the code, increasing complexity
>
> - memcpy (to, from, len)
> + if (len) memcpy (to, from, len)

That's only necessary when you don't already know that `to` and
`from` are non-null.

Copying zero bytes between actual objects might be reasonably common.
Copying zero bytes from one nonexistent object to another *might*
be valid, depending on the program's logic, but it's likely to
indicate a bug.

I have no strong opinion on whether memcpy(NULL, NULL, 0) *should*
have defined behavior.  The fact is that it doesn't.  I don't
think I've ever seen real-world code where it matters.  If it does,
adding an `if` is trivial.

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Working, but not speaking, for JetHead Development, Inc.
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

[toc] | [prev] | [next] | [standalone]

#94994

On Saturday, October 15, 2016 at 3:40:50 PM UTC-5, Keith Thompson wrote:
> As a programmer, how much of a burden is it to avoid calling memcpy()
> with a null pointer?

It adds a wasted comparison when implementing an API that can accept
optional data given a pointer and length, and specifies that the pointer
is ignored if the length is zero.  In many cases, an optimizer would not
be able to eliminate that comparison unless it knew that memcpy would be
harmless in all cases where the length is zero (and in that case, does
it makes sense to require a programmer to write an unwanted check in the
hopes that the optimizer will eliminate it?)

For how many real-world implementations of memcpy would a requirement that
memcpy(anything, anything, 0) be regarded as a no-op impose *any burden
whatsoever*?

[toc] | [prev] | [next] | [standalone]

#95071

supercat@casperkitty.com wrote:

> On Saturday, October 15, 2016 at 3:40:50 PM UTC-5, Keith Thompson wrote:
> > As a programmer, how much of a burden is it to avoid calling memcpy()
> > with a null pointer?
> 
> It adds a wasted comparison when implementing an API that can accept
> optional data given a pointer and length, and specifies that the pointer
> is ignored if the length is zero.

No, it doesn't, because if you arrange your code properly that
comparison would have been done in any case, not just for memcpy() but
for all the other code that uses that pointer. At least, I've never
written code that allocates a pointer, memcpy()s to it, and immediately
deallocates it; I've always done more with any single pointer, and
therefore, have always had to check for null in any case. The test is
not done specifically for the memcpy(); therefore, it is adding the test
inside memcpy() which wastes a comparison.

> For how many real-world implementations of memcpy

Your favourite straw man again? Knock it off, already.

Richard

[toc] | [prev] | [next] | [standalone]

#95072

On Tuesday, October 18, 2016 at 11:55:09 AM UTC-5, Richard Bos wrote:
> No, it doesn't, because if you arrange your code properly that
> comparison would have been done in any case, not just for memcpy() but
> for all the other code that uses that pointer. At least, I've never
> written code that allocates a pointer, memcpy()s to it, and immediately
> deallocates it; I've always done more with any single pointer, and
> therefore, have always had to check for null in any case. The test is
> not done specifically for the memcpy(); therefore, it is adding the test
> inside memcpy() which wastes a comparison.

Obviously real code would do something with the object besides allocate it,
use memcpy, and free it, but such other actions are irrelevant to the
question of whether the memcpy cases we're discussing should be treated
as defined.  In cases where code is building an object of size N+constant
by combining a fixed part with N bytes read from somewhere else, there
may not be any reason to regard the N==0 case as special.

> > For how many real-world implementations of memcpy
> 
> Your favourite straw man again? Knock it off, already.

There are cases where treating memcpy(any,any,0) as a no-op regardless of
the values of the pointers is useful.  I would think it should be reasonable
for programmers to expect that writers of quality compilers won't go out of
their way to make their implementations less useful than they otherwise
could be.

Therefore, I think it should be reasonable for programmers to expect that
compilers will treat that case as a no-op unless either:

  1. They are using a platform where treating it as a no-op would
     require effort on the part of the implementation's creator.

  2. There is some reason that treating it as anything other than a
     no-op would make an implementation more useful than it otherwise
     would be.

Perhaps you think programmers should expect compiler writers to go out
of their way to make implementations less useful.  Certainly that would
be the case if one doesn't limit oneself to quality compilers.  Otherwise
do you see anything wrong with my logic if restricted to compilers whose
authors are trying to make them as useful as practical?

[toc] | [prev] | [next] | [standalone]

#95162

supercat@casperkitty.com wrote:

> On Tuesday, October 18, 2016 at 11:55:09 AM UTC-5, Richard Bos wrote:
> > No, it doesn't, because if you arrange your code properly that
> > comparison would have been done in any case, not just for memcpy() but
> > for all the other code that uses that pointer. At least, I've never
> > written code that allocates a pointer, memcpy()s to it, and immediately
> > deallocates it; I've always done more with any single pointer, and
> > therefore, have always had to check for null in any case. The test is
> > not done specifically for the memcpy(); therefore, it is adding the test
> > inside memcpy() which wastes a comparison.
> 
> Obviously real code would do something with the object besides allocate it,
> use memcpy, and free it, but such other actions are irrelevant to the
> question of whether the memcpy cases we're discussing should be treated
> as defined. 

IMAO I just demonstrated why they're _not_ irrelevant: because they
consist of the majority of cases, and incur a potential (if uncommon)
penalty under your demands.

> In cases where code is building an object of size N+constant
> by combining a fixed part with N bytes read from somewhere else, there
> may not be any reason to regard the N==0 case as special.

And guess what? If you actually are building an _object_, so the
pointers are non-null, that case is perfectly defined already! The only
problem is when you're pretending to build a _non_-object, and handling
null pointers; but as I wrote earlier, you shouldn't be doing that in
the first place. You should be catching your nulls before you get to
memcpy().

> > > For how many real-world implementations of memcpy
> > 
> > Your favourite straw man again? Knock it off, already.
> 
> There are cases where treating memcpy(any,any,0) as a no-op regardless of
> the values of the pointers is useful.

Demonstrate one. One realistic(!) case where the _null_ pointer being
passed to memcpy() is better than the null pointer being caught early.

Remember, for valid non-null pointers the case is already defined, and
for _invalid_ non-null pointers your proposal would leave the behaviour
as undefined as it is now, so don't try to confuse the matter by
involving them. We're only interested in the importance of late-caught
null pointers.

> I would think it should be reasonable for programmers to expect that
> writers of quality compilers won't go out of their way to make their
> implementations less useful than they otherwise could be.

Nobody is suggesting that compiler writers write their optimisers
specifically to piss you off, except you.

> Perhaps you think programmers should expect compiler writers to go out
> of their way to make implementations less useful.

Yet another beautiful man of straw. You're good at these, aren't you?

No, I think programmers should write their code properly, so that it
doesn't matter.

> Otherwise do you see anything wrong with my logic if restricted to compilers
> whose authors are trying to make them as useful as practical?

Straw, straw, straw. Did your father immigrate from the land of Oz?

Richard

[toc] | [prev] | [next] | [standalone]

#95232

On Sunday, October 23, 2016 at 8:25:04 AM UTC-5, Richard Bos wrote:
> supercat wrote:
> > Obviously real code would do something with the object besides allocate it,
> > use memcpy, and free it, but such other actions are irrelevant to the
> > question of whether the memcpy cases we're discussing should be treated
> > as defined. 
> 
> IMAO I just demonstrated why they're _not_ irrelevant: because they
> consist of the majority of cases, and incur a potential (if uncommon)
> penalty under your demands.

I haven't seen any realistic "penalty" worth mentioning.  Any programmer who
wants the compiler to optimize on the assumption that neither pointer can
be null would be free to write "if (!ptr) 1/0;" [probably via macro which
could expand to something else on implementations which would favor some
other pattern].


> > In cases where code is building an object of size N+constant
> > by combining a fixed part with N bytes read from somewhere else, there
> > may not be any reason to regard the N==0 case as special.
> 
> And guess what? If you actually are building an _object_, so the
> pointers are non-null, that case is perfectly defined already! The only
> problem is when you're pretending to build a _non_-object, and handling
> null pointers; but as I wrote earlier, you shouldn't be doing that in
> the first place. You should be catching your nulls before you get to
> memcpy().

The *destination* object will always exist in the indicated pattern, but
the source object may or may not.  On many platforms, code to pass a null
pointer to a function will be smaller and faster than code to pass a dummy
object.  If a compiler supported an __UNSPECIFIED intrinsic, passing
__UNSPECIFIED could be smaller and faster yet, but only if the function
can safely pass to memcpy an unspecified pointer received from its caller
when the size is zero.

> > > > For how many real-world implementations of memcpy
> > > 
> > > Your favourite straw man again? Knock it off, already.
> > 
> > There are cases where treating memcpy(any,any,0) as a no-op regardless of
> > the values of the pointers is useful.
> 
> Demonstrate one. One realistic(!) case where the _null_ pointer being
> passed to memcpy() is better than the null pointer being caught early.

I already have.  An API which builds an object with an optional data field
which will be included verbatim in an object being built.

There's no benefit to "catching" the null pointer because in the cases
where the caller would pass null, the code receiving the pointer would be
required to ignore it (because size is zero).  The only question is whether
to require special handling for the size-zero case to ensure that code which
has no reason to do anything with a pointer ignores it.

> Remember, for valid non-null pointers the case is already defined, and
> for _invalid_ non-null pointers your proposal would leave the behaviour
> as undefined as it is now, so don't try to confuse the matter by
> involving them. We're only interested in the importance of late-caught
> null pointers.

Passing a an invalid-but-not-null pointer to *any* function yields UB.

> > I would think it should be reasonable for programmers to expect that
> > writers of quality compilers won't go out of their way to make their
> > implementations less useful than they otherwise could be.
> 
> Nobody is suggesting that compiler writers write their optimisers
> specifically to piss you off, except you.

On a typical platfomrs, the *only* effect of mandating that memcpy ignore
its arguments when the size is zero would be to block inferences the
compiler would otherwise be entitled to make about the values of the
arguments.  A compiler isn't going to make such inferences unless the
compiler writer explicitly includes code to make them.

I would measure the quality of a compiler based upon the quality of machine
code it can produce given a source code which is designed to run optimally
on that machine but would be usable (though not necessarily optimal) on
many others.  A programmer who wanted a compiler to optimize on the
assumption that memcpy(x,y,z) would only be invoked when neither x nor y is
zero could replace that with memcpy(x,y,((x&&y)||(1/0),z)) and achieve that
effect.  By contrast, if memcpy(any,any,0) isn't a guaranteed nop, then a
programmer may have to add extra code for the size-zero case which a compiler
might not be able to optimize out.

> > Perhaps you think programmers should expect compiler writers to go out
> > of their way to make implementations less useful.
> 
> Yet another beautiful man of straw. You're good at these, aren't you?

I should have said you think programmers should expect that compiler writers
might go out of their way to make implementations less useful.

On the vast majority of platforms [I know of no remotely-current exceptions],
a compiler writer would have to go out of the way to make memcpy(any,any,0)
behave as anything other than a nop-op for any combination of pointer values
it receives.  Any compiler for any remotely-modern platform whose writer does
not go out of the way to make memcpy(any,any,0) behave as anything other than
a no-op will treat it as one.  Are you not saying that programmers should
expect that compiler writers might go out of their way to treat size-zero
memcpy as something other than a no-op?

Is there any circumstance in which the "optimization" will would improve
the quality of code a programmer could generate using a compiler.  Given
that it will reduce the fraction of the existing corpus of C code that
the compiler can handle, and increase the amount of source code necessary
to accomplish some tasks, I see zero upside to the optimization and a
noticeable downside.  Even in scenarios where one wants to get the most
efficient machine code from a given source text without adding any
optimization tweaks, cases where a compiler could make useful inferences
based upon assuming memcpy's source operand will never be null *but couldn't
make those inferences based upon anything else in the code* would seem
vanishingly rare.

Do you disagree with my above assessments?  If one replaces "will" with
"might", in what way would my statement not follow from those two points?

> No, I think programmers should write their code properly, so that it
> doesn't matter.

If compiler writers do not go out of their way to make inferences based
upon the arguments of memcpy, and if code is never run on any platform
where a machine-code implementation of memcpy that didn't behave as a
no-op in all size-zero cases wouldn't be more expensive than one that
behaved as something other than a no-op in size-zero cases where one of
the pointers was null, what advantage would there be to including an
explicit size-zero check in source code in cases where either the code
will be called most often with a non-zero size, or where the performance
cost of a function call in the zero-size case would not be objectionable?

[toc] | [prev] | [next] | [standalone]

#95395

Richard Bos <raltbos@xs4all.nl> wrote:
>> On Tuesday, October 18, 2016 at 11:55:09 AM UTC-5, Richard Bos wrote:
<snip>
>> There are cases where treating memcpy(any,any,0) as a no-op regardless of
>> the values of the pointers is useful.
> 
> Demonstrate one. One realistic(!) case where the _null_ pointer being
> passed to memcpy() is better than the null pointer being caught early.

What's your opinion on the semantics of free? Have you ever found it
convenient that you could pass NULL to free? I find it incredibly
convenient, and that it greatly simplifies my code.

Likewise, IME, I find being able to pass NULL to memcpy (or a memcpy
wrapper) very convenient. I already track the size of referenced objects
(usually data buffers in most contexts where memcpy is used). Adding
additional logic to check for the special case of NULL and/or size == 0 is
superfluous. I find this even more convenient than free accepting NULL
because there's usually only 1 line of code where I pass a persistent
reference to free, but many lines where I pass that reference to memcpy.

I try to write my code to have as few conditionals as possible--for
performance, but mostly because littering superfluous conditionals
everywhere tends to obfuscate code. Ideally an algorithm should work for an
entire range of inputs of the data type.

Now, if you feed it garbage values the output might be garbage, but a well
written program will ideally be able to take garbage in and produce garbage
out without crashing or otherwise violating an implementation-dependent
invariant. In the real world that's difficult to do, but being able to pass
NULL to functions that logically should behave like a noop when the
specified size of the referenced object is 0 gets you long way toward that
ideal. For similar reasons I prefer unsigned types when signedness is
irrelevant specifically because overflow is well defined; garbage in will be
garbage out, and even when the value is the size of an object, it still
reduces by half the number of conditional expressions required to ensure the
invariant (that is, I only need to check whether n <= some_size, rather than
n <= size_size && n >= 0).

I realize that NULL is an odd sort of value in C, giving ample room for
disagreement about whether NULL is a legal value for a particular type in
various contexts, including memcpy (for which, in any event, it is illegal
by definition). But if the question is whether memcpy _should_ accept NULL,
I have no doubt in my mind about the convenience that allowance could
afford.

What I do doubt are the reasons for disallowing it. Is it to permit
implementations to do speculative writes? That doesn't seem to make sense.
Speculative reads, maybe, but these days implementations have to be careful
not to cross page boundaries when doing a speculative read, so the algorithm
either needs to be inherently resilient to that (my preferred solution as a
general matter), or explicitly check for that condition before the loop.

[toc] | [prev] | [next] | [standalone]

#95399

On Saturday, October 29, 2016 at 1:45:14 PM UTC-5, wil...@wilbur.25thandclement.com wrote:
> What's your opinion on the semantics of free? Have you ever found it
> convenient that you could pass NULL to free? I find it incredibly
> convenient, and that it greatly simplifies my code.

Prior to the C Standard, there were some implementations where malloc(0)
would return null; the caller would naturally not be required to pass
the returned pointer to "free".  There were other implementations which
would treat malloc(0) as equivalent to malloc(1), and which would guarantee
that each pointer returned from "malloc(0)" would have a distinct value,
but which consequently required that pointers returned from "malloc(0)"
would need to be freed.

I think the main purpose behind making free(0) a no-op was to uphold the
principle that anything returned from malloc may be passed, once, to free().

> Now, if you feed it garbage values the output might be garbage, but a well
> written program will ideally be able to take garbage in and produce garbage
> out without crashing or otherwise violating an implementation-dependent
> invariant. In the real world that's difficult to do, but being able to pass
> NULL to functions that logically should behave like a noop when the
> specified size of the referenced object is 0 gets you long way toward that
> ideal.

Besides, if the most natural way of handling some operation will work for
all n>=0 and for 99% of possible programs the most efficient way which
would work for all n>0 would also work for n>=0, requiring that the
programmer add explicit handling to all programs (even those where the
natural implementation would have handled the n==0 case) doesn't seem like
a recipe for efficiency.

> I realize that NULL is an odd sort of value in C, giving ample room for
> disagreement about whether NULL is a legal value for a particular type in
> various contexts, including memcpy (for which, in any event, it is illegal
> by definition). But if the question is whether memcpy _should_ accept NULL,
> I have no doubt in my mind about the convenience that allowance could
> afford.

There may be some question as to whether the authors of the Standard
intended that an implementation where memcpy() would normally start by
loading a pointer into a register that would trap when loading null, would
not be required to prevent that trap from occuring in the memcpy(x,0,0)
case.  I don't think there's any reason to believe they thought programmers
who knew their machines had no such traps should not be entitled to expect
memcpy(x,y,0) to be a no-op for all values of x and y.

> What I do doubt are the reasons for disallowing it. Is it to permit
> implementations to do speculative writes? That doesn't seem to make sense.
> Speculative reads, maybe, but these days implementations have to be careful
> not to cross page boundaries when doing a speculative read, so the algorithm
> either needs to be inherently resilient to that (my preferred solution as a
> general matter), or explicitly check for that condition before the loop.

The only kind of "optimization" I can think of where it would matter would
be situations where a pointer which is passed to memcpy() is also compared
to NULL (possibly in a different function).  If an API specifies that a
pointer/size combo must either identify an object with a certain header, or
else be 0/0, part of the code for that API might check the pointer against
null before working with the header, while another part might pass it to
memcpy.  If memcpy(x,0,0) is a no-op, there would be no problem, but if it
causes the compiler to decide there's no need for the null check before it
operates upon the object header, kaboom.

Personally, I subscribe to the believe that a good language implementation
should not require edge-case checks in source code that won't be needed in
the machine code.  But at least around here such a viewpoint doesn't seem
very popular.

[toc] | [prev] | [next] | [standalone]

#94879

Lőrinczy Zsigmond <nospam@for.me> writes:

> Nice. A broken standard and a broken optimizer join forces to make
> programmers' life miserable...
>
> Standard should mandate that if len==0,
> memcpy has to return without checking
> any of the pointers,

As a general rule C has functions with minimal argument checking since
it's easier to add the test than it is to remove it:

  inline
  void *nzmemcpy(void *restrict p1, const void *restrict p2, size_t len)
  {
      return len ? memcpy(p1, p2, len) : p1;
  }

Is this really making programmers' lives miserable?

(Whilst it's possible that the implementation's version of memcpy could
incorporate this test more efficiently than the above on-liner, but it
would still probably incur some cost.)

> also compiler should do what it was told to,
> not something else it thinks is 'optimal'.

It is usually possible to tell the compiler to have a go at doing that
(-O0 for example) but, when the language standard does not define the
meaning of the program, it's not always clear what the compiler has been
told to do.

-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#94911

On Saturday, October 15, 2016 at 10:00:54 PM UTC+1, Ben Bacarisse wrote:
> Lőrinczy Zsigmond <nospam@for.me> writes:
> 

>   inline
>   void *nzmemcpy(void *restrict p1, const void *restrict p2, size_t len)
>   {
>       return len ? memcpy(p1, p2, len) : p1;
>   }
> 
> Is this really making programmers' lives miserable?
> 
Yes, The logic is  "copy the contents of p2 to p1". And it just gets buried in minutae
if you manually inline that function, and makes functions non-leaf if you supply
it. 
Leaf functions ("Malcolm functions") with no dependencies at all are the most
portable, most reusable C constructs you can have, and so you shouldn't surrender
that characteristic lightly. Then dependencies only on malloc(), free(), dependencies
only on other trivial, supplied subroutines, and dependencies only on the standard
library are the next best.So nzmemcpy() is not a disaster. But it's still not desireable.
  

[toc] | [prev] | [next] | [standalone]

#94916

Malcolm McLean <malcolm.mclean5@btinternet.com> writes:

> On Saturday, October 15, 2016 at 10:00:54 PM UTC+1, Ben Bacarisse wrote:
>> Lőrinczy Zsigmond <nospam@for.me> writes:
>> 
>
>>   inline
>>   void *nzmemcpy(void *restrict p1, const void *restrict p2, size_t len)
>>   {
>>       return len ? memcpy(p1, p2, len) : p1;
>>   }
>> 
>> Is this really making programmers' lives miserable?
>> 
> Yes, The logic is "copy the contents of p2 to p1". And it just gets
> buried in minutae
> if you manually inline that function, and makes functions non-leaf if you supply
> it. 
> Leaf functions ("Malcolm functions") with no dependencies at all are the most
> portable, most reusable C constructs you can have, and so you shouldn't surrender
> that characteristic lightly. Then dependencies only on malloc(), free(), dependencies
> only on other trivial, supplied subroutines, and dependencies only on the standard
> library are the next best.So nzmemcpy() is not a disaster. But it's
> still not desireable.

A C program that calls memcpy is no less portable (in 2016) than one
that does not, and may well be more efficient than one that uses a
portable copy function written in C.  Your own obsessions need not be
shared (and in my view should not be shared) by other C programmers.

-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#94920

On Sunday, October 16, 2016 at 11:20:28 AM UTC+1, Ben Bacarisse wrote:
> Malcolm McLean <malcolm.mclean5@btinternet.com> writes:
> 

> A C program that calls memcpy is no less portable (in 2016) than one
> that does not, and may well be more efficient than one that uses a
> portable copy function written in C.  Your own obsessions need not be
> shared (and in my view should not be shared) by other C programmers.
> 
Microsoft have deprecated strcpy. A program that calls strcpy will not compile
under Windows without passing a special define to the compiler. A manager
who insists that this define not be used is not technically very well-informed,
but from his own perspective he is acting reasonably. So in such circumstances
any code that calls strcpy must be rewritten.

If we supply mystrcpy, with exactly the same interface, we hardly gain anything
in terms of security, but the problem doesn't arise.

[toc] | [prev] | [next] | [standalone]

#94921

On 16/10/2016 12:14, Malcolm McLean wrote:
> On Sunday, October 16, 2016 at 11:20:28 AM UTC+1, Ben Bacarisse wrote:
>> Malcolm McLean <malcolm.mclean5@btinternet.com> writes:
>>
>
>> A C program that calls memcpy is no less portable (in 2016) than one
>> that does not, and may well be more efficient than one that uses a
>> portable copy function written in C.  Your own obsessions need not be
>> shared (and in my view should not be shared) by other C programmers.
>>
> Microsoft have deprecated strcpy. A program that calls strcpy will not compile
> under Windows without passing a special define to the compiler.

I haven't seen that. Do you mean under any compiler or just MSVC?

MSVC doesn't complain either, not on the -2008 version anyway.

-- 
Bartc

[toc] | [prev] | [next] | [standalone]

#94922

On 16/10/16 12:14, Malcolm McLean wrote:
> On Sunday, October 16, 2016 at 11:20:28 AM UTC+1, Ben Bacarisse wrote:
>> Malcolm McLean <malcolm.mclean5@btinternet.com> writes:
>>
>
>> A C program that calls memcpy is no less portable (in 2016) than one
>> that does not, and may well be more efficient than one that uses a
>> portable copy function written in C.  Your own obsessions need not be
>> shared (and in my view should not be shared) by other C programmers.
>>
> Microsoft have deprecated strcpy. A program that calls strcpy will not compile
> under Windows without passing a special define to the compiler.

Then, without that special define, the compiler is no longer a C compiler.

The strcpy function can, like many functions, be abused. So can 
Microsoft's string-copying function of choice. Here are its claimed 
features:

* The size of the destination buffer is always provided to the function 
to ensure that the function does not write past the end of the buffer.

Exercise for the reader: come up with a call that will render this claim 
false. (It's not hard.)

* Buffers are guaranteed to be null-terminated, even if the operation 
truncates the intended result.

Exercise for the reader: spot the obvious data integrity problem.

* All functions return an HRESULT value, with only one possible success 
code (S_OK).

Exercise for the reader: explain how one would use this function in 
portable code.

> A manager
> who insists that this define not be used is not technically very well-informed,
> but from his own perspective he is acting reasonably. So in such circumstances
> any code that calls strcpy must be rewritten.

Or a new manager found.

> If we supply mystrcpy, with exactly the same interface, we hardly gain anything
> in terms of security, but the problem doesn't arise.

I don't see that we gain anything at all. Any replacement for strcpy is 
going to have exactly the same problem as with strcpy itself, which is 
that it works just fine until you screw up, and then things start to go 
wrong.

-- 
Richard Heathfield
Email: rjh at cpax dot org dot uk
"Usenet is a strange place" - dmr 29 July 1999
Sig line 4 vacant - apply within

[toc] | [prev] | [next] | [standalone]

#94923

On Sunday, October 16, 2016 at 1:01:19 PM UTC+1, Richard Heathfield wrote:
> On 16/10/16 12:14, Malcolm McLean wrote:
> 
> > A manager
> > who insists that this define not be used is not technically very well-informed,
> > but from his own perspective he is acting reasonably. So in such circumstances
> > any code that calls strcpy must be rewritten.
> 
> Or a new manager found.
> 
We're getting an engineering manager, a new role as we expand. We drew up a person
specification and it was agreed that we were looking for a manager with a bit
of software engineering knowledge, not a software engineer looking to move into
a mangement role. There are lots of reasons why, if a big a company like Microsoft
provide supposedly secure features, a small company like ours might have to use 
them. If a bug is found and it's traced to the fact that Microsoft's secure features aren't
really as secure as claimed, that's one thing. If it's traced to the fact that Microsoft
supplied secure features and we turned them off, that could put us in quite a different
position. It's all management rather than engineering, but that's why we want 
someone with management experience.
>
> > If we supply mystrcpy, with exactly the same interface, we hardly gain anything
> > in terms of security, but the problem doesn't arise.
> 
> I don't see that we gain anything at all. Any replacement for strcpy is 
> going to have exactly the same problem as with strcpy itself, which is 
> that it works just fine until you screw up, and then things start to go 
> wrong.
> 
An attacker might search through looking for the calls to strcpy, knowing that that's
where the security vulnerabilites might be. If he's reading this, then obviously he 
will also search for the symbol "mystrcpy" and we'll only slow him down by a few
seconds. But probably he's not reading this.

[toc] | [prev] | [next] | [standalone]

#94934

Malcolm McLean <malcolm.mclean5@btinternet.com> writes:

> On Sunday, October 16, 2016 at 1:01:19 PM UTC+1, Richard Heathfield wrote:
>> On 16/10/16 12:14, Malcolm McLean wrote:
>> 
>> > A manager
>> > who insists that this define not be used is not technically very
>> > well-informed,
>> > but from his own perspective he is acting reasonably. So in such
>> > circumstances
>> > any code that calls strcpy must be rewritten.
>> 
>> Or a new manager found.
>> 
> We're getting an engineering manager, a new role as we expand. We drew
> up a person
> specification and it was agreed that we were looking for a manager with a bit
> of software engineering knowledge, not a software engineer looking to move into
> a mangement role. There are lots of reasons why, if a big a company
> like Microsoft
> provide supposedly secure features, a small company like ours might have to use 
> them. If a bug is found and it's traced to the fact that Microsoft's
> secure features aren't
> really as secure as claimed, that's one thing. If it's traced to the
> fact that Microsoft
> supplied secure features and we turned them off, that could put us in
> quite a different
> position.

What position would the company be in if it's traced back to your
solution: a hand coded replacement written for the sole purposes of
making it a "Malcolm-function"?  In other words, have you just changed
horses mid race?

<snip>
-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#94963

On Sunday, October 16, 2016 at 8:47:40 PM UTC+1, Ben Bacarisse wrote:
> Malcolm McLean <malcolm.mclean5@btinternet.com> writes:
> 
> > If it's traced to the fact that Microsoft supplied secure features and 
> > we turned them off, that could put us in quite a different
> > position.
> 
> What position would the company be in if it's traced back to your
> solution: a hand coded replacement written for the sole purposes of
> making it a "Malcolm-function"?  In other words, have you just changed
> horses mid race?
>
We didn't receive secure features from Microsoft, then switch them off.
The engineering manager gets it in the neck for that, because he must have
authorised that. It's a routine programming error by a developer, who happened
to be me. That happens all the time, even in the best companies.

[toc] | [prev] | [next] | [standalone]

#94960

On 16/10/16 14:29, Malcolm McLean wrote:
> On Sunday, October 16, 2016 at 1:01:19 PM UTC+1, Richard Heathfield wrote:
>> On 16/10/16 12:14, Malcolm McLean wrote:
>>
>>> A manager
>>> who insists that this define not be used is not technically very well-informed,
>>> but from his own perspective he is acting reasonably. So in such circumstances
>>> any code that calls strcpy must be rewritten.
>>
>> Or a new manager found.
>>
> We're getting an engineering manager, a new role as we expand. We drew up a person
> specification and it was agreed that we were looking for a manager with a bit
> of software engineering knowledge, not a software engineer looking to move into
> a mangement role. There are lots of reasons why, if a big a company like Microsoft
> provide supposedly secure features, a small company like ours might have to use 
> them. If a bug is found and it's traced to the fact that Microsoft's secure features aren't
> really as secure as claimed, that's one thing. If it's traced to the fact that Microsoft
> supplied secure features and we turned them off, that could put us in quite a different
> position. It's all management rather than engineering, but that's why we want 
> someone with management experience.

And how well-off will you be when the bug is traced to a blind reliance
on the security advice of /Microsoft/, rather than a proper
understanding of the actual code?  Any knowledgeable judge/jury/customer
would mark you as "guilty" with that defence, once they had stopped
laughing.

It would be even worse if your plan was to leave Microsoft's "security
features" enabled and then manually duplicate exactly the claimed
security fault by using a "mystrcpy" function.

[toc] | [prev] | [next] | [standalone]

#94984

On Monday, October 17, 2016 at 9:24:41 AM UTC+1, David Brown wrote:
> On 16/10/16 14:29, Malcolm McLean wrote:
>
> And how well-off will you be when the bug is traced to a blind reliance
> on the security advice of /Microsoft/, rather than a proper
> understanding of the actual code?  Any knowledgeable judge/jury/customer
> would mark you as "guilty" with that defence, once they had stopped
> laughing.
> 
> It would be even worse if your plan was to leave Microsoft's "security
> features" enabled and then manually duplicate exactly the claimed
> security fault by using a "mystrcpy" function.
>
You don't get it.
The world is not comprised of informed C developers. If it was, then providing
mystrcpy has exactly the same effect as turning stdlib deprecation off and
calling strcpy. We all understand that here.
But that's not how the world works, because judges in trade dispute cases,
managers and so on have to take decisions on the basis of not being expert
programmers. 

[toc] | [prev] | [next] | [standalone]

#94988

On 17/10/16 17:05, Malcolm McLean wrote:
> On Monday, October 17, 2016 at 9:24:41 AM UTC+1, David Brown wrote:
>> On 16/10/16 14:29, Malcolm McLean wrote:
>>
>> And how well-off will you be when the bug is traced to a blind reliance
>> on the security advice of /Microsoft/, rather than a proper
>> understanding of the actual code?  Any knowledgeable judge/jury/customer
>> would mark you as "guilty" with that defence, once they had stopped
>> laughing.
>>
>> It would be even worse if your plan was to leave Microsoft's "security
>> features" enabled and then manually duplicate exactly the claimed
>> security fault by using a "mystrcpy" function.
>>
> You don't get it.
> The world is not comprised of informed C developers. If it was, then providing
> mystrcpy has exactly the same effect as turning stdlib deprecation off and
> calling strcpy. We all understand that here.
> But that's not how the world works, because judges in trade dispute cases,
> managers and so on have to take decisions on the basis of not being expert
> programmers. 
> 

Yes, I know that judges don't understand programming - any more than
programmers understand law.  That's why people use "expert witnesses" in
cases, and the expert witnesses /do/ understand programming and security.

And judges and lawyers usually have a reasonable understanding of who is
supposed to know about what.  The choice of enabling or disabling a
switch in the compiler is a technical decision.  Either the manager is
technically competent and can make that decision, or he should be
managerially competent and ask his technical staff.  So the choice here
should /always/ be made by someone who understands the coding and
security issues involved.

This all boils down to your company being at fault if there is a
security flaw in your code, which you could reasonably have foreseen and
prevented.  And if the cause of the flaw is in Microsoft's code that
your company chose to use, then you better have a /technical/
justification for that choice - "Microsoft said it was better than the
function everyone else uses" is not good enough.

But IANAL, and I don't have to deal with Microsoft's daft ideas about
what is and is not a good idea in security.

[toc] | [prev] | [next] | [standalone]

Page 3 of 5 — ← Prev page 1 2 [3] 4 5  Next page →

Back to top | Article view | comp.lang.c

csiph-web