Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #152174 > unrolled thread

How many wide characters may mbstowcs store?

Back to article view | Back to comp.lang.c

Contents

  How many wide characters may mbstowcs store? Philipp Klaus Krause <pkk@spth.de> - 2020-05-11 13:30 +0200
    Re: How many wide characters may mbstowcs store? Manfred <noname@add.invalid> - 2020-05-11 13:55 +0200
      Re: How many wide characters may mbstowcs store? Philipp Klaus Krause <pkk@spth.de> - 2020-05-11 14:01 +0200
        Re: How many wide characters may mbstowcs store? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-05-11 09:08 -0400
          Re: How many wide characters may mbstowcs store? Philipp Klaus Krause <pkk@spth.de> - 2020-05-11 15:19 +0200
          Re: How many wide characters may mbstowcs store? Manfred <noname@add.invalid> - 2020-05-11 18:32 +0200
          Re: How many wide characters may mbstowcs store? Philipp Klaus Krause <pkk@spth.de> - 2020-05-11 18:59 +0200
            Re: How many wide characters may mbstowcs store? scott@slp53.sl.home (Scott Lurndal) - 2020-05-11 17:42 +0000
              Re: How many wide characters may mbstowcs store? Philipp Klaus Krause <pkk@spth.de> - 2020-05-11 20:30 +0200
              Re: How many wide characters may mbstowcs store? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-05-12 00:29 -0400
                Re: How many wide characters may mbstowcs store? Manfred <noname@add.invalid> - 2020-05-12 14:41 +0200
                Re: How many wide characters may mbstowcs store? Bonita Montero <Bonita.Montero@gmail.com> - 2020-05-12 16:19 +0200
    Re: How many wide characters may mbstowcs store? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2020-05-11 13:03 +0100
      Re: How many wide characters may mbstowcs store? Philipp Klaus Krause <pkk@spth.de> - 2020-05-11 14:07 +0200
    Re: How many wide characters may mbstowcs store? Philipp Klaus Krause <pkk@spth.de> - 2020-05-11 15:20 +0200
      Re: How many wide characters may mbstowcs store? Bonita Montero <Bonita.Montero@gmail.com> - 2020-05-11 16:31 +0200
      Re: How many wide characters may mbstowcs store? Barry Schwarz <schwarzb@delq.com> - 2020-05-11 10:06 -0700
    Re: How many wide characters may mbstowcs store? Philipp Klaus Krause <pkk@spth.de> - 2020-05-11 15:58 +0200
      Re: How many wide characters may mbstowcs store? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-05-11 10:24 -0400
      Re: How many wide characters may mbstowcs store? Philipp Klaus Krause <pkk@spth.de> - 2020-05-11 16:52 +0200
        Re: How many wide characters may mbstowcs store? Manfred <noname@add.invalid> - 2020-05-11 18:55 +0200
      Re: How many wide characters may mbstowcs store? richard@cogsci.ed.ac.uk (Richard Tobin) - 2020-05-11 15:51 +0000
        Re: How many wide characters may mbstowcs store? Philipp Klaus Krause <pkk@spth.de> - 2020-05-11 19:01 +0200
          Re: How many wide characters may mbstowcs store? scott@slp53.sl.home (Scott Lurndal) - 2020-05-11 17:33 +0000
            Re: How many wide characters may mbstowcs store? Philipp Klaus Krause <pkk@spth.de> - 2020-05-11 20:57 +0200
              Re: How many wide characters may mbstowcs store? scott@slp53.sl.home (Scott Lurndal) - 2020-05-11 19:17 +0000
                Re: How many wide characters may mbstowcs store? richard@cogsci.ed.ac.uk (Richard Tobin) - 2020-05-11 19:41 +0000
                  Re: How many wide characters may mbstowcs store? scott@slp53.sl.home (Scott Lurndal) - 2020-05-11 20:01 +0000
              Re: How many wide characters may mbstowcs store? scott@slp53.sl.home (Scott Lurndal) - 2020-05-11 19:19 +0000
      Re: How many wide characters may mbstowcs store? Florian Weimer <fw@deneb.enyo.de> - 2020-05-11 20:24 +0200
        Re: How many wide characters may mbstowcs store? Philipp Klaus Krause <pkk@spth.de> - 2020-05-11 20:59 +0200
          Re: How many wide characters may mbstowcs store? scott@slp53.sl.home (Scott Lurndal) - 2020-05-11 19:17 +0000
            Re: How many wide characters may mbstowcs store? Philipp Klaus Krause <pkk@spth.de> - 2020-05-11 21:24 +0200
          Re: How many wide characters may mbstowcs store? Florian Weimer <fw@deneb.enyo.de> - 2020-05-11 22:30 +0200
    Re: How many wide characters may mbstowcs store? Bonita Montero <Bonita.Montero@gmail.com> - 2020-05-11 16:44 +0200
      Re: How many wide characters may mbstowcs store? Philipp Klaus Krause <pkk@spth.de> - 2020-05-11 16:54 +0200
        Re: How many wide characters may mbstowcs store? Bonita Montero <Bonita.Montero@gmail.com> - 2020-05-11 16:57 +0200
          Re: How many wide characters may mbstowcs store? Philipp Klaus Krause <pkk@spth.de> - 2020-05-11 17:07 +0200
            Re: How many wide characters may mbstowcs store? Bonita Montero <Bonita.Montero@gmail.com> - 2020-05-11 17:08 +0200
            Re: How many wide characters may mbstowcs store? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-05-11 11:25 -0400
        Re: How many wide characters may mbstowcs store? Tim Rentsch <tr.17687@z991.linuxsc.com> - 2020-05-11 09:06 -0700
          Re: How many wide characters may mbstowcs store? Philipp Klaus Krause <pkk@spth.de> - 2020-05-11 19:05 +0200
            Re: How many wide characters may mbstowcs store? Philipp Klaus Krause <pkk@spth.de> - 2020-05-11 19:19 +0200
              Re: How many wide characters may mbstowcs store? Tim Rentsch <tr.17687@z991.linuxsc.com> - 2020-05-23 07:51 -0700
                Re: How many wide characters may mbstowcs store? Philipp Klaus Krause <pkk@spth.de> - 2020-05-23 20:27 +0200
                  Re: How many wide characters may mbstowcs store? Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2020-05-23 14:25 -0700
                    Re: How many wide characters may mbstowcs store? Tim Rentsch <tr.17687@z991.linuxsc.com> - 2020-05-26 07:09 -0700
                  Re: How many wide characters may mbstowcs store? Tim Rentsch <tr.17687@z991.linuxsc.com> - 2020-05-26 07:14 -0700
                    Re: How many wide characters may mbstowcs store? Spiros Bousbouras <spibou@gmail.com> - 2020-05-26 16:00 +0000
                      Re: How many wide characters may mbstowcs store? Tim Rentsch <tr.17687@z991.linuxsc.com> - 2020-05-29 21:23 -0700
                        Re: How many wide characters may mbstowcs store? Spiros Bousbouras <spibou@gmail.com> - 2020-05-30 20:08 +0000
                          Re: How many wide characters may mbstowcs store? Tim Rentsch <tr.17687@z991.linuxsc.com> - 2020-06-03 08:46 -0700
                            Re: How many wide characters may mbstowcs store? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-06-03 10:18 -0700
                              Re: How many wide characters may mbstowcs store? Tim Rentsch <tr.17687@z991.linuxsc.com> - 2020-06-23 05:35 -0700
                                Re: How many wide characters may mbstowcs store? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-06-26 06:32 -0700
                                  Re: How many wide characters may mbstowcs store? Tim Rentsch <tr.17687@z991.linuxsc.com> - 2020-09-02 09:19 -0700
                                    Re: How many wide characters may mbstowcs store? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-09-02 19:50 -0700
                            Re: How many wide characters may mbstowcs store? raltbos@xs4all.nl (Richard Bos) - 2020-06-04 20:34 +0000
          Re: How many wide characters may mbstowcs store? scott@slp53.sl.home (Scott Lurndal) - 2020-05-11 17:39 +0000
            Re: How many wide characters may mbstowcs store? Autist <autist69@gmail.com> - 2020-05-11 19:42 +0200
            Re: How many wide characters may mbstowcs store? Philipp Klaus Krause <pkk@spth.de> - 2020-05-11 20:28 +0200
              Re: How many wide characters may mbstowcs store? scott@slp53.sl.home (Scott Lurndal) - 2020-05-11 18:37 +0000
                Re: How many wide characters may mbstowcs store? Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2020-05-11 11:50 -0700
                  Re: How many wide characters may mbstowcs store? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2020-05-12 20:02 +0100
                    Re: How many wide characters may mbstowcs store? Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2020-05-12 13:12 -0700
            Re: How many wide characters may mbstowcs store? richard@cogsci.ed.ac.uk (Richard Tobin) - 2020-05-11 19:56 +0000
            Re: How many wide characters may mbstowcs store? Tim Rentsch <tr.17687@z991.linuxsc.com> - 2020-05-24 16:49 -0700
        Re: How many wide characters may mbstowcs store? Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2020-05-11 14:19 -0700
          Re: How many wide characters may mbstowcs store? Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2020-05-11 14:22 -0700
            Re: How many wide characters may mbstowcs store? Philipp Klaus Krause <pkk@spth.de> - 2020-05-12 09:17 +0200
          Re: How many wide characters may mbstowcs store? raltbos@xs4all.nl (Richard Bos) - 2020-05-24 16:12 +0000
            Re: How many wide characters may mbstowcs store? Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2020-05-24 15:10 -0700
            Re: How many wide characters may mbstowcs store? James Kuyper <jameskuyper@alumni.caltech.edu> - 2020-05-24 22:58 -0400
    Re: How many wide characters may mbstowcs store? richard@cogsci.ed.ac.uk (Richard Tobin) - 2020-05-11 20:07 +0000
    Re: How many wide characters may mbstowcs store? Andrey Tarasevich <andreytarasevich@hotmail.com> - 2020-06-25 21:42 -0700
      Re: How many wide characters may mbstowcs store? Tim Rentsch <tr.17687@z991.linuxsc.com> - 2020-06-28 06:32 -0700

Page 3 of 4 — ← Prev page 1 2 [3] 4  Next page →

#152211

Philipp Klaus Krause <pkk@spth.de> writes:

> Am 11.05.20 um 16:44 schrieb Bonita Montero:
>
>> There's a POSIX-extension that if you pass nullptr for s, you get
>> the size of the buffer needed for s.  Maybe this will help you.
>> Otherwise: multibyte-characters are usually UTF-8-characters and
>> it should be easy to find code to convert these charaters into
>> wide-characters; but it should be also easy to write this yourself
>> in 20min.
>
> At the moment I want to figure out what to do about the problem.  File a
> bug against GCC in Ubuntu?  File a defect report / clarification request
> with WG14?

File a gcc bug report.  The gnu/gcc folks have misunderstood the
standard, and they are shooting their users in the foot.  Your
support/regression tests deserve thanks, and have provided a
public service.

It might also be useful to file a DR/CR, mainly to beat the gcc
people over the head when they push back and say it isn't a bug.
It is.

Here are the clues.

Your first posting asked about mbstowcs, but in the followup to 
Manfred you said the report you got was on wcstombs.  The wording
in the Standard for wcstombs is definite that conversion stops
when a null is encountered.  (I confirm that wcstombs exhibits
the erroneous behavior on Ubuntu 18.04 under gcc -O1 or -O2,
but not with clang, and not with gcc -O0.)

Second clue:  compare the man pages for mbstowcs and wcstombs.
The wording they give for what the length of the array should be
is essentially identical.  That description is obviously wrong
for wcstombs, and probably was simply copied from the wrong
wording in mbstowcs.  (Note that the Standard lists these
functions in the order mbstowcs then wcstombs, so the man pages
were probably written in that order.)

Third clue:  under -O0, gcc calls wcstombs, but under -O1 or -O2
gcc calls __wcstombs_chk.  Not only that, but the call passes an
extra argument, which is (surprise, surprise) the length of the
array into which the output is going.  Mostly likely the _chk
function simply compares the actual length against the requested
length, and then gives a buffer overflow message (as happens on
my Ubuntu system) because actual < requested.  Brilliant!

Fourth clue:  if the call to wcstombs is simply wrapped in another
function, so at the call site it isn't known how large the output
buffer is, gcc works fine.  The past-terminating-null array
positions are left unaffected.

Editorial comment:  I suspect, with no particular proof, that we
are seeing another manifestation of "overzealous optimization",
aka "anything that might possibly be undefined behavior gives us
license to screw things up however we want."  If that is the case
then gcc developers deserve to have their noses rubbed in this,
ahem, "helpful" interpretation.

[toc] | [prev] | [next] | [standalone]

#152216

Am 11.05.20 um 18:06 schrieb Tim Rentsch:
> Philipp Klaus Krause <pkk@spth.de> writes:
> 
>> Am 11.05.20 um 16:44 schrieb Bonita Montero:
>>
>>> There's a POSIX-extension that if you pass nullptr for s, you get
>>> the size of the buffer needed for s.  Maybe this will help you.
>>> Otherwise: multibyte-characters are usually UTF-8-characters and
>>> it should be easy to find code to convert these charaters into
>>> wide-characters; but it should be also easy to write this yourself
>>> in 20min.
>>
>> At the moment I want to figure out what to do about the problem.  File a
>> bug against GCC in Ubuntu?  File a defect report / clarification request
>> with WG14?
> 
> File a gcc bug report.  The gnu/gcc folks have misunderstood the
> standard, and they are shooting their users in the foot.  Your
> support/regression tests deserve thanks, and have provided a
> public service.

However, with GCC having the bug on Ubuntu, but not on Debian, I wonder
if the GCC developers are really the source of the bug. Maybe the bug is
in some patches Ubuntu adds to the upstream debian packages?

[toc] | [prev] | [next] | [standalone]

#152218

Am 11.05.20 um 19:05 schrieb Philipp Klaus Krause:
> Am 11.05.20 um 18:06 schrieb Tim Rentsch:
>> Philipp Klaus Krause <pkk@spth.de> writes:
>>
>>> Am 11.05.20 um 16:44 schrieb Bonita Montero:
>>>
>>>> There's a POSIX-extension that if you pass nullptr for s, you get
>>>> the size of the buffer needed for s.  Maybe this will help you.
>>>> Otherwise: multibyte-characters are usually UTF-8-characters and
>>>> it should be easy to find code to convert these charaters into
>>>> wide-characters; but it should be also easy to write this yourself
>>>> in 20min.
>>>
>>> At the moment I want to figure out what to do about the problem.  File a
>>> bug against GCC in Ubuntu?  File a defect report / clarification request
>>> with WG14?
>>
>> File a gcc bug report.  The gnu/gcc folks have misunderstood the
>> standard, and they are shooting their users in the foot.  Your
>> support/regression tests deserve thanks, and have provided a
>> public service.
> 
> However, with GCC having the bug on Ubuntu, but not on Debian, I wonder
> if the GCC developers are really the source of the bug. Maybe the bug is
> in some patches Ubuntu adds to the upstream debian packages?
> 

I don't really undestand the patches Ubuntu applies to upstream Debian
GCC packages, but there is something about

+ __interceptor_mbstowcs@Base 4.9

so maybe Ubuntu enables some experimental optimization, that upstream
Debian and GCC considered to not be ready yet to rill out to all users?

[toc] | [prev] | [next] | [standalone]

#152445

Philipp Klaus Krause <pkk@spth.de> writes:

> Am 11.05.20 um 19:05 schrieb Philipp Klaus Krause:
>
>> Am 11.05.20 um 18:06 schrieb Tim Rentsch:
>>
>>> Philipp Klaus Krause <pkk@spth.de> writes:
>>>
>>>> Am 11.05.20 um 16:44 schrieb Bonita Montero:
>>>>
>>>>> There's a POSIX-extension that if you pass nullptr for s, you get
>>>>> the size of the buffer needed for s.  Maybe this will help you.
>>>>> Otherwise: multibyte-characters are usually UTF-8-characters and
>>>>> it should be easy to find code to convert these charaters into
>>>>> wide-characters; but it should be also easy to write this yourself
>>>>> in 20min.
>>>>
>>>> At the moment I want to figure out what to do about the problem.  File a
>>>> bug against GCC in Ubuntu?  File a defect report / clarification request
>>>> with WG14?
>>>
>>> File a gcc bug report.  The gnu/gcc folks have misunderstood the
>>> standard, and they are shooting their users in the foot.  Your
>>> support/regression tests deserve thanks, and have provided a
>>> public service.
>>
>> However, with GCC having the bug on Ubuntu, but not on Debian, I wonder
>> if the GCC developers are really the source of the bug.  Maybe the bug is
>> in some patches Ubuntu adds to the upstream debian packages?
>
> I don't really undestand the patches Ubuntu applies to upstream Debian
> GCC packages, but there is something about
>
> + __interceptor_mbstowcs@Base 4.9
>
> so maybe Ubuntu enables some experimental optimization, that upstream
> Debian and GCC considered to not be ready yet to rill out to all users?

Looking at other messages I think you have identified what is
going on.  Doing some web searches, it does seem like what is
happening is triggered, at least partially, by the Ubuntu
choice to check the arguments more strictly than the Standard
allows.  Incidentally, there was some indication that the same
issue exists in Android, in case you care about that.

Even so, I think it would be good to file a bug report against
gcc.  It is gcc that chooses to call the overly strict, and also
non-standard, interface that does the wrong thing.  If gcc had
simply called the standard interface wcstombs then everything
would be fine.  There should at least be an option on gcc that
prevents it from using the non-standard and not conforming
interface.

[toc] | [prev] | [next] | [standalone]

#152446

Am 23.05.20 um 16:51 schrieb Tim Rentsch:
> 
> Even so, I think it would be good to file a bug report against
> gcc.  It is gcc that chooses to call the overly strict, and also
> non-standard, interface that does the wrong thing.  If gcc had
> simply called the standard interface wcstombs then everything
> would be fine.  There should at least be an option on gcc that
> prevents it from using the non-standard and not conforming
> interface.
> 

The correct thing to do would be to file a bug against the gcc package
in Ubuntu (after all it is an Ubunt-specific patch that makes gcc
non-stdandard-compliant - on upstream GNU GCC the non-compliant
behaviour would need to be explicitly requested by the user). However, I
guess the bug report would be rejected, as Ubuntu considers
non-standard-compliance in this respect a feature (much like OpenBSD
with their non-standard-compliant rand()).
I'll just leave the filing of a bug report to a real Ubuntu user.

[toc] | [prev] | [next] | [standalone]

#152447

Philipp Klaus Krause <pkk@spth.de> writes:
[...]
> The correct thing to do would be to file a bug against the gcc package
> in Ubuntu (after all it is an Ubunt-specific patch that makes gcc
> non-stdandard-compliant - on upstream GNU GCC the non-compliant
> behaviour would need to be explicitly requested by the user). However, I
> guess the bug report would be rejected, as Ubuntu considers
> non-standard-compliance in this respect a feature (much like OpenBSD
> with their non-standard-compliant rand()).
> I'll just leave the filing of a bug report to a real Ubuntu user.

I don't think I had heard this about OpenBSD's rand().

Here's the man page: https://man.openbsd.org/rand

    Standards insist that this interface return deterministic
    results. Unsafe usage is very common, so OpenBSD changed the
    subsystem to return non-deterministic results by default.

They add a non-standard srand_deterministic() that causes rand() to
yield the required deterministic results.

Personally, I think this was a very bad idea.  Portable code can rely on
rand() yielding *repeatable* results, which can be important in some
contexts; this breaks such code.  Most rand() implementations are known
to be fairly low quality, and not suitable for cryptography, for
example.  This encourages programmers to assume that rand() is
non-deterministic and having their code break when ported to anything
other than Open BSD.

-- 
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Working, but not speaking, for Philips Healthcare
void Void(void) { Void(); } /* The recursive call of the void */

[toc] | [prev] | [next] | [standalone]

#152484

Keith Thompson <Keith.S.Thompson+u@gmail.com> writes:

> Philipp Klaus Krause <pkk@spth.de> writes:
> [...]
>
>> The correct thing to do would be to file a bug against the gcc package
>> in Ubuntu (after all it is an Ubunt-specific patch that makes gcc
>> non-stdandard-compliant - on upstream GNU GCC the non-compliant
>> behaviour would need to be explicitly requested by the user).  However, I
>> guess the bug report would be rejected, as Ubuntu considers
>> non-standard-compliance in this respect a feature (much like OpenBSD
>> with their non-standard-compliant rand()).
>> I'll just leave the filing of a bug report to a real Ubuntu user.
>
> I don't think I had heard this about OpenBSD's rand().
>
> Here's the man page:  https://man.openbsd.org/rand
>
>     Standards insist that this interface return deterministic
>     results.  Unsafe usage is very common, so OpenBSD changed the
>     subsystem to return non-deterministic results by default.
>
> They add a non-standard srand_deterministic() that causes rand() to
> yield the required deterministic results.

Thank you for posting this.  Yet another reason never to use
rand().

> Personally, I think this was a very bad idea.

I absolutely, categorically, emphatically and unequivocally
completely agree.  What were they thinking???

[toc] | [prev] | [next] | [standalone]

#152485

Philipp Klaus Krause <pkk@spth.de> writes:

> Am 23.05.20 um 16:51 schrieb Tim Rentsch:
>
>> Even so, I think it would be good to file a bug report against
>> gcc.  It is gcc that chooses to call the overly strict, and also
>> non-standard, interface that does the wrong thing.  If gcc had
>> simply called the standard interface wcstombs then everything
>> would be fine.  There should at least be an option on gcc that
>> prevents it from using the non-standard and not conforming
>> interface.
>
> The correct thing to do would be to file a bug against the gcc package
> in Ubuntu (after all it is an Ubunt-specific patch that makes gcc
> non-stdandard-compliant - on upstream GNU GCC the non-compliant
> behaviour would need to be explicitly requested by the user).  However, I
> guess the bug report would be rejected, as Ubuntu considers
> non-standard-compliance in this respect a feature (much like OpenBSD
> with their non-standard-compliant rand()).
> I'll just leave the filing of a bug report to a real Ubuntu user.

I still think a bug report should be filed against gcc itself.
If gcc is going to allow outside forces (meaning ones completely
outside its control) to alter its behavior so it is no longer
conforming, there should at least be an option to prevent such
alterations.

That said, it's your bug to file, and I defer to your judgment
as to how and where it might be reported.

[toc] | [prev] | [next] | [standalone]

#152489

On Tue, 26 May 2020 07:14:01 -0700
Tim Rentsch <tr.17687@z991.linuxsc.com> wrote:
> I still think a bug report should be filed against gcc itself.
> If gcc is going to allow outside forces (meaning ones completely
> outside its control) to alter its behavior so it is no longer
> conforming, there should at least be an option to prevent such
> alterations.

The whole point of GPL is to allow anyone to make modifications to the GPL
software which in their view are improvements. "anyone" includes "forces
completely outside the control" of the people who wrote the software. So I
think that a bug report against upstream  gcc  would be a waste of time.

[toc] | [prev] | [next] | [standalone]

#152531

Spiros Bousbouras <spibou@gmail.com> writes:

> On Tue, 26 May 2020 07:14:01 -0700
> Tim Rentsch <tr.17687@z991.linuxsc.com> wrote:
>
>> I still think a bug report should be filed against gcc itself.
>> If gcc is going to allow outside forces (meaning ones completely
>> outside its control) to alter its behavior so it is no longer
>> conforming, there should at least be an option to prevent such
>> alterations.
>
> The whole point of GPL is to allow anyone to make modifications to the GPL
> software which in their view are improvements.  "anyone" includes "forces
> completely outside the control" of the people who wrote the software.  So I
> think that a bug report against upstream  gcc  would be a waste of time.

AFAICT it is always true that filing a bug report against gcc
might be a waste of time.

OTOH, if it is the case, which I suspect it is, that gcc supplies
an optional capability to enable this non-coforming behavior, and
the capability is merely enabled by a feature setting, users of
gcc might agree that there should be a compile-time option not to
use it.  Certainly I would fall in that camp, and my guess is so
would many others.  If there is enough user sentiment that such
an option should be supplied (and especially if someone just goes
ahead and submits a patch to supply one), then I think there is a
good chance that submitting a bug report would start the ball
rolling on that.  Conversely, if no one makes an effort to bring
the gcc developers into the conversation, then the chance of
something being done about it seems rather remote.  So, yes, they
might just blow it off, but then again there is a reasonable
chance that something good will come of it.

[toc] | [prev] | [next] | [standalone]

#152543

On Fri, 29 May 2020 21:23:00 -0700
Tim Rentsch <tr.17687@z991.linuxsc.com> wrote:
> Spiros Bousbouras <spibou@gmail.com> writes:
> 
> > On Tue, 26 May 2020 07:14:01 -0700
> > Tim Rentsch <tr.17687@z991.linuxsc.com> wrote:
> >
> >> I still think a bug report should be filed against gcc itself.
> >> If gcc is going to allow outside forces (meaning ones completely
> >> outside its control) to alter its behavior so it is no longer
> >> conforming, there should at least be an option to prevent such
> >> alterations.
> >
> > The whole point of GPL is to allow anyone to make modifications to the GPL
> > software which in their view are improvements.  "anyone" includes "forces
> > completely outside the control" of the people who wrote the software.  So I
> > think that a bug report against upstream  gcc  would be a waste of time.
> 
> AFAICT it is always true that filing a bug report against gcc
> might be a waste of time.

I didn't mean it in such a general sense but rather than the chances of getting
the  gcc  people to accept responsibility for something the Ubuntu people did
is very low.

> OTOH, if it is the case, which I suspect it is, that gcc supplies
> an optional capability to enable this non-coforming behavior, and
> the capability is merely enabled by a feature setting, users of
> gcc might agree that there should be a compile-time option not to
> use it.  Certainly I would fall in that camp, and my guess is so
> would many others.

From what has been said in the thread I got the impression that if one compiles
one's code with   -D _FORTIFY_SOURCE=0   or   -U _FORTIFY_SOURCE   or put at
the top of one's source code    #define _FORTIFY_SOURCE 0    or
#undef _FORTIFY_SOURCE  , it will solve the problem. Defining it with value 1
might also work. But I don't run Ubuntu so I can't check. For those using
Ubuntu if might be worthwhile doing   man feature_test_macros   to see if it
mentions that the macro is defined to 2 as a default.

-- 
vlaho.ninja/prog

[toc] | [prev] | [next] | [standalone]

#152646

Spiros Bousbouras <spibou@gmail.com> writes:

> On Fri, 29 May 2020 21:23:00 -0700
> Tim Rentsch <tr.17687@z991.linuxsc.com> wrote:
>
>> Spiros Bousbouras <spibou@gmail.com> writes:
>>
>>> On Tue, 26 May 2020 07:14:01 -0700
>>> Tim Rentsch <tr.17687@z991.linuxsc.com> wrote:
>>>
>>>> I still think a bug report should be filed against gcc itself.
>>>> If gcc is going to allow outside forces (meaning ones completely
>>>> outside its control) to alter its behavior so it is no longer
>>>> conforming, there should at least be an option to prevent such
>>>> alterations.
>>>
>>> The whole point of GPL is to allow anyone to make modifications to the GPL
>>> software which in their view are improvements.  "anyone" includes "forces
>>> completely outside the control" of the people who wrote the software.  So I
>>> think that a bug report against upstream  gcc  would be a waste of time.
>>
>> AFAICT it is always true that filing a bug report against gcc
>> might be a waste of time.
>
> I didn't mean it in such a general sense but rather than the chances
> of getting the gcc people to accept responsibility for something the
> Ubuntu people did is very low.

Perhaps so but that isn't what I'd be asking.  What I would be
asking is for gcc people to be responsible for something gcc is
doing.  Clearly this choice is one that is under gcc's control,
because in some circumstances gcc emits code that behaves
conformingly, even on Ubuntu.  So clearly gcc can do the right
thing, it is simply choosing not to.

>> OTOH, if it is the case, which I suspect it is, that gcc supplies
>> an optional capability to enable this non-coforming behavior, and
>> the capability is merely enabled by a feature setting, users of
>> gcc might agree that there should be a compile-time option not to
>> use it.  Certainly I would fall in that camp, and my guess is so
>> would many others.
>
> From what has been said in the thread I got the impression
> that if one compiles one's code with -D _FORTIFY_SOURCE=0
> or -U _FORTIFY_SOURCE or put at the top of one's source
> code #define _FORTIFY_SOURCE 0 or #undef _FORTIFY_SOURCE ,
> it will solve the problem.  Defining it with value 1 might
> also work.  But I don't run Ubuntu so I can't check.  [...]

Thank you for the info.  There still should be a compiler option
but until then this is something.

[toc] | [prev] | [next] | [standalone]

#152648

On Wednesday, June 3, 2020 at 11:46:21 AM UTC-4, Tim Rentsch wrote:
> Spiros Bousbouras <spibou@gmail.com> writes:
> 
> > On Fri, 29 May 2020 21:23:00 -0700
> > Tim Rentsch <tr.17687@z991.linuxsc.com> wrote:
> >
> >> Spiros Bousbouras <spibou@gmail.com> writes:
> >>
> >>> On Tue, 26 May 2020 07:14:01 -0700
> >>> Tim Rentsch <tr.17687@z991.linuxsc.com> wrote:
> >>>
> >>>> I still think a bug report should be filed against gcc itself.
> >>>> If gcc is going to allow outside forces (meaning ones completely
> >>>> outside its control) to alter its behavior so it is no longer
> >>>> conforming, there should at least be an option to prevent such
> >>>> alterations.

As a user, you have that option: install a version of gcc that was not
so patched.

Or are you saying that the Ubuntu-patched code should provide an option
to turn off that feature? If so, gcc is unambiguosly not a useful place
to go to make such a request - such a request should go to Ubuntu. The
code whose execution you want to make optional doesn't exist in the gcc
version of the code, and even if it were possible to put such an option
into the gcc version of the code,  Ubuntu would be free to remove it.

Or are you suggesting that gcc should do something that would prevent
Ubuntu from patching its code? Good luck with that - creating a system
that others are free to modify is the fundamental purpose of gcc.

> >>> The whole point of GPL is to allow anyone to make modifications to the GPL
> >>> software which in their view are improvements.  "anyone" includes "forces
> >>> completely outside the control" of the people who wrote the software.  So I
> >>> think that a bug report against upstream  gcc  would be a waste of time.
> >>
> >> AFAICT it is always true that filing a bug report against gcc
> >> might be a waste of time.
> >
> > I didn't mean it in such a general sense but rather than the chances
> > of getting the gcc people to accept responsibility for something the
> > Ubuntu people did is very low.
> 
> Perhaps so but that isn't what I'd be asking.  What I would be
> asking is for gcc people to be responsible for something gcc is
> doing.  Clearly this choice is one that is under gcc's control,
> because in some circumstances gcc emits code that behaves
> conformingly, even on Ubuntu.  So clearly gcc can do the right
> thing, it is simply choosing not to.

How does a code patch made by Ubuntu count as something that gcc is
doing?

[toc] | [prev] | [next] | [standalone]

#152891

James Kuyper <jameskuyper@alumni.caltech.edu> writes:

> On Wednesday, June 3, 2020 at 11:46:21 AM UTC-4, Tim Rentsch wrote:
>
>> Spiros Bousbouras <spibou@gmail.com> writes:
>>
>>> On Fri, 29 May 2020 21:23:00 -0700
>>> Tim Rentsch <tr.17687@z991.linuxsc.com> wrote:
>>>
>>>> Spiros Bousbouras <spibou@gmail.com> writes:
>>>>
>>>>> On Tue, 26 May 2020 07:14:01 -0700
>>>>> Tim Rentsch <tr.17687@z991.linuxsc.com> wrote:
>>>>>
>>>>>> I still think a bug report should be filed against gcc
>>>>>> itself.  If gcc is going to allow outside forces (meaning
>>>>>> ones completely outside its control) to alter its behavior
>>>>>> so it is no longer conforming, there should at least be an
>>>>>> option to prevent such alterations.
>
> As a user, you have that option:  install a version of gcc that
> was not so patched.

Apparently I haven't made myself clear.  I see no reason to
suppose that gcc has been patched by anyone outside the gcc
developer group.  It seems more likely that they put in a
capability for enabling these transformations, triggered by a
setting in library header files.  The people who put together the
Ubuntu distribution simply enabled the setting in the library
headers;  they didn't any changes to gcc itself.  So gcc should
have a way of turning off that outside interference, since they
are the ones who enabled it in the first place.

Disclaimer:  I have no direct knowledge of who made what changes
to gcc source.  I am simply making an educated guess based on
whatever miscellaneous evidence has been discovered during my
various perusings.

[toc] | [prev] | [next] | [standalone]

#152902

On Tuesday, June 23, 2020 at 8:35:14 AM UTC-4, Tim Rentsch wrote:
> James Kuyper <jameskuyper@alumni.caltech.edu> writes:
> 
> > On Wednesday, June 3, 2020 at 11:46:21 AM UTC-4, Tim Rentsch wrote:
> >
> >> Spiros Bousbouras <spibou@gmail.com> writes:
> >>
> >>> On Fri, 29 May 2020 21:23:00 -0700
> >>> Tim Rentsch <tr.17687@z991.linuxsc.com> wrote:
> >>>
> >>>> Spiros Bousbouras <spibou@gmail.com> writes:
> >>>>
> >>>>> On Tue, 26 May 2020 07:14:01 -0700
> >>>>> Tim Rentsch <tr.17687@z991.linuxsc.com> wrote:
> >>>>>
> >>>>>> I still think a bug report should be filed against gcc
> >>>>>> itself.  If gcc is going to allow outside forces (meaning
> >>>>>> ones completely outside its control) to alter its behavior
> >>>>>> so it is no longer conforming, there should at least be an
> >>>>>> option to prevent such alterations.
> >
> > As a user, you have that option:  install a version of gcc that
> > was not so patched.
> 
> Apparently I haven't made myself clear.  I see no reason to
> suppose that gcc has been patched by anyone outside the gcc
> developer group.  It seems more likely that they put in a
> capability for enabling these transformations, triggered by a
> setting in library header files.  The people who put together the
> Ubuntu distribution simply enabled the setting in the library
> headers;  they didn't any changes to gcc itself.


On Tuesday, May 12, 2020 at 3:17:15 AM UTC-4, Philipp Klaus Krause wrote:
> And indeed the problem is apparently in the gcc package for Ubuntu:
> Their patch to the upstream Debian gcc package predefines
> _FORTIFY_SOURCE to 2, which makes glibc non-compliant. 

Since he describes it as a patch to the gcc package, I feel
comfortable describing it in the same fashion.

> ...  So gcc should
> have a way of turning off that outside interference, since they
> are the ones who enabled it in the first place.

The only way I could see them doing that is to drop the currently
optional and non-default support for _FORTIFY_SOURCE. Why should they
do that, when they have users who actively want that feature? And even
if they did, what's to prevent Ubuntu from reinstating that support, and
continuing to have it default to 2, since they obviously want it to be 2?

I reiterate that any defect report should be filed against Ubuntu, not gcc.

[toc] | [prev] | [next] | [standalone]

#154389

James Kuyper <jameskuyper@alumni.caltech.edu> writes:

> On Tuesday, June 23, 2020 at 8:35:14 AM UTC-4, Tim Rentsch wrote:
>
>> James Kuyper <jameskuyper@alumni.caltech.edu> writes:
>>
>>> On Wednesday, June 3, 2020 at 11:46:21 AM UTC-4, Tim Rentsch wrote:
>>>
>>>> Spiros Bousbouras <spibou@gmail.com> writes:
>>>>
>>>>> On Fri, 29 May 2020 21:23:00 -0700
>>>>> Tim Rentsch <tr.17687@z991.linuxsc.com> wrote:
>>>>>
>>>>>> Spiros Bousbouras <spibou@gmail.com> writes:
>>>>>>
>>>>>>> On Tue, 26 May 2020 07:14:01 -0700
>>>>>>> Tim Rentsch <tr.17687@z991.linuxsc.com> wrote:
>>>>>>>
>>>>>>>> I still think a bug report should be filed against gcc
>>>>>>>> itself.  If gcc is going to allow outside forces (meaning
>>>>>>>> ones completely outside its control) to alter its behavior
>>>>>>>> so it is no longer conforming, there should at least be an
>>>>>>>> option to prevent such alterations.
>>>
>>> As a user, you have that option:  install a version of gcc that
>>> was not so patched.
>>
>> Apparently I haven't made myself clear.  I see no reason to
>> suppose that gcc has been patched by anyone outside the gcc
>> developer group.  It seems more likely that they put in a
>> capability for enabling these transformations, triggered by a
>> setting in library header files.  The people who put together the
>> Ubuntu distribution simply enabled the setting in the library
>> headers;  they didn't any changes to gcc itself.
>
> On Tuesday, May 12, 2020 at 3:17:15 AM UTC-4, Philipp Klaus Krause wrote:
>
>> And indeed the problem is apparently in the gcc package for Ubuntu:
>> Their patch to the upstream Debian gcc package predefines
>> _FORTIFY_SOURCE to 2, which makes glibc non-compliant.
>
> Since he describes it as a patch to the gcc package, I feel
> comfortable describing it in the same fashion.

Exactly.  The gcc /package/ was changed, but the gcc /compiler/
was not changed.

>> ...  So gcc should
>> have a way of turning off that outside interference, since
>> they are the ones who enabled it in the first place.
>
> The only way I could see them doing that is to drop the currently
> optional and non-default support for _FORTIFY_SOURCE.

If you really think that then either you don't have much
imagination or you haven't been paying attention.  First the
non-conforming transformation is not unconditional, which means
it is under the compiler's control, even if _FORTIFY_SOURCE has
been predefined to be 2.  Second there was posted upthread a way
of circumventing the transformation using a -D option to gcc,
which again works even if _FORTIFY_SOURCE has been predefined to
be 2.  Clearly gcc could add an option -fno-fortify that would
accomplish exactly what I have suggested.

> Why should
> they do that, when they have users who actively want that feature?
> And even if they did, what's to prevent Ubuntu from reinstating
> that support, and continuing to have it default to 2, since they
> obviously want it to be 2?

A gcc option -fno-fortify has none of those problems.

> I reiterate that any defect report should be filed against Ubuntu,
> not gcc.

Then I think you are misunderstanding the problem I want addressed.
The problem is not Ubuntu arranging that _FORTIFY_SOURCE is
pre-defined to be 2.  The problem is that gcc doesn't provide an
easy and obvious way to disable the non-conforming transformation,
which after all they themselves enabled.  Simply adding an option
such as -fno-fortify would easily accomplish that, and the regular
gcc team is in an ideal position to do that.  The people who work
on Ubuntu are not.

[toc] | [prev] | [next] | [standalone]

#154413

On Wednesday, September 2, 2020 at 12:19:56 PM UTC-4, Tim Rentsch wrote:
> James Kuyper <jameskuyper@alumni.caltech.edu> writes:
> 
> > On Tuesday, June 23, 2020 at 8:35:14 AM UTC-4, Tim Rentsch wrote:
> >
> >> James Kuyper <jameskuyper@alumni.caltech.edu> writes:
...
> >>> As a user, you have that option:  install a version of gcc that
> >>> was not so patched.
> >>
> >> Apparently I haven't made myself clear.  I see no reason to
> >> suppose that gcc has been patched by anyone outside the gcc
> >> developer group.  It seems more likely that they put in a
> >> capability for enabling these transformations, triggered by a
> >> setting in library header files.  The people who put together the
> >> Ubuntu distribution simply enabled the setting in the library
> >> headers;  they didn't any changes to gcc itself.
> >
> > On Tuesday, May 12, 2020 at 3:17:15 AM UTC-4, Philipp Klaus Krause wrote:
> >
> >> And indeed the problem is apparently in the gcc package for Ubuntu:
> >> Their patch to the upstream Debian gcc package predefines
> >> _FORTIFY_SOURCE to 2, which makes glibc non-compliant.
> >
> > Since he describes it as a patch to the gcc package, I feel
> > comfortable describing it in the same fashion.
> 
> Exactly.  The gcc /package/ was changed, but the gcc /compiler/
> was not changed.

Whether it's the gcc package or the gcc compiler that they patched is
irrelevant - the only thing that's relevant to the point I was raising
is that they patched something which affected how the code got
translated into an executable - and if it didn't have such an effect, no
one would be complaining about it.

I can't claim that failing to specify whether I was talking about the
package or the compiler was deliberate, I wasn't thinking about such a
distinction when I wrote that sentence - but I did conveniently fail to
specify which of the two I was talking about, rendering your objection
irrelevant.

> >> ...  So gcc should
> >> have a way of turning off that outside interference, since
> >> they are the ones who enabled it in the first place.
> >
> > The only way I could see them doing that is to drop the currently
> > optional and non-default support for _FORTIFY_SOURCE.
> 
> If you really think that then either you don't have much
> imagination or you haven't been paying attention.  First the
> non-conforming transformation is not unconditional, which means
> it is under the compiler's control, even if _FORTIFY_SOURCE has
> been predefined to be 2.  Second there was posted upthread a way
> of circumventing the transformation using a -D option to gcc,
> which again works even if _FORTIFY_SOURCE has been predefined to
> be 2.  Clearly gcc could add an option -fno-fortify that would
> accomplish exactly what I have suggested.
> 
> > Why should
> > they do that, when they have users who actively want that feature?
> > And even if they did, what's to prevent Ubuntu from reinstating
> > that support, and continuing to have it default to 2, since they
> > obviously want it to be 2?
> 
> A gcc option -fno-fortify has none of those problems.

Yes, it would. Ubuntu would still be free to either disable or even
remove the -fno-fortify option, if they wanted to.

> > I reiterate that any defect report should be filed against Ubuntu,
> > not gcc.
> 
> Then I think you are misunderstanding the problem I want addressed.
> The problem is not Ubuntu arranging that _FORTIFY_SOURCE is
> pre-defined to be 2.  The problem is that gcc doesn't provide an
> easy and obvious way to disable the non-conforming transformation,
> which after all they themselves enabled.

It's disabled by default, why would they need an option to disable it?
The option to disable it should be added by the same people who made it
enabled by default - which was Ubuntu, not gcc.

>  Simply adding an option
> such as -fno-fortify would easily accomplish that, ...

Well, they choose to allow -D_FORTIFY_SOURCE=0 have that effect instead.
Does the exact spelling of the option really matter?

> ... and the regular
> gcc team is in an ideal position to do that. The people who work
> on Ubuntu are not.

As far as I can tell, -D_FORTIFY_SOURCE=0 is already enabled on Ubuntu.
However, even if it weren't enabled in either version of gcc, and gcc
added that option, the Ubuntu team would be free to disable it, if they
wanted. So it really would still be Ubuntu's decision that matters, not
gcc's.

[toc] | [prev] | [next] | [standalone]

#152660

Tim Rentsch <tr.17687@z991.linuxsc.com> wrote:

> Perhaps so but that isn't what I'd be asking.  What I would be
> asking is for gcc people to be responsible

*Snigger*

Richard

[toc] | [prev] | [next] | [standalone]

#152220

Tim Rentsch <tr.17687@z991.linuxsc.com> writes:
>Philipp Klaus Krause <pkk@spth.de> writes:
>
>> Am 11.05.20 um 16:44 schrieb Bonita Montero:
>>
>>> There's a POSIX-extension that if you pass nullptr for s, you get
>>> the size of the buffer needed for s.  Maybe this will help you.
>>> Otherwise: multibyte-characters are usually UTF-8-characters and
>>> it should be easy to find code to convert these charaters into
>>> wide-characters; but it should be also easy to write this yourself
>>> in 20min.
>>
>> At the moment I want to figure out what to do about the problem.  File a
>> bug against GCC in Ubuntu?  File a defect report / clarification request
>> with WG14?
>
>File a gcc bug report.  The gnu/gcc folks have misunderstood the
>standard, and they are shooting their users in the foot.  Your
>support/regression tests deserve thanks, and have provided a
>public service.

It's not a bug.  The phrase

    "No characters that follow a null byte (which is converted into
     a wide-character code with value 0) shall be examined or converted."

Is there to ensure that no bytes beyond the null are _read_ from the source
string (thus ensuring that no page fault, for example, occurs because a byte
beyond the nul is on the next (unallocated) page).   It has no bearing on
whether the function is allowed to write element 'n-1' of the destination operand
which is allowed explictly by the standard regardless of the length of
the input string.

An application that provides an 'n' parameter larger than the allocated
space of the destination buffer is _BROKEN_.

[toc] | [prev] | [next] | [standalone]

#152221

> An application that provides an 'n' parameter larger than the allocated
> space of the destination buffer is _BROKEN_.

No, that's perfectly o.k.!

[toc] | [prev] | [next] | [standalone]

Page 3 of 4 — ← Prev page 1 2 [3] 4  Next page →

Back to top | Article view | comp.lang.c

csiph-web