Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #99309 > unrolled thread

Why is this outputting alternate blank lines?

Back to article view | Back to comp.lang.c

Contents

  Why is this outputting alternate blank lines? DFS <nospam@dfs.com> - 2016-12-13 23:18 -0500
    Re: Why is this outputting alternate blank lines? Melzzzzz <mel@zzzzz.com> - 2016-12-14 05:30 +0100
    Re: Why is this outputting alternate blank lines? mark.bluemel@gmail.com - 2016-12-14 00:59 -0800
    Re: Why is this outputting alternate blank lines? Richard Heathfield <rjh@cpax.org.uk> - 2016-12-14 10:38 +0000
      Re: Why is this outputting alternate blank lines? DFS <nospam@dfs.com> - 2016-12-14 12:53 -0500
        Re: Why is this outputting alternate blank lines? Richard Heathfield <rjh@cpax.org.uk> - 2016-12-14 18:05 +0000
          Re: Why is this outputting alternate blank lines? Keith Thompson <kst-u@mib.org> - 2016-12-14 11:57 -0800
            Re: Why is this outputting alternate blank lines? Richard Heathfield <rjh@cpax.org.uk> - 2016-12-14 20:05 +0000
              Re: Why is this outputting alternate blank lines? "James R. Kuyper" <jameskuyper@verizon.net> - 2016-12-14 15:21 -0500
              Re: Why is this outputting alternate blank lines? Keith Thompson <kst-u@mib.org> - 2016-12-14 12:28 -0800
                Re: Why is this outputting alternate blank lines? supercat@casperkitty.com - 2016-12-14 13:09 -0800
                Re: Why is this outputting alternate blank lines? Jean-Marc Bourguet <jm@bourguet.org> - 2016-12-14 22:53 +0100
                  Re: Why is this outputting alternate blank lines? <william@wilbur.25thandClement.com> - 2016-12-14 15:47 -0800
              Re: Why is this outputting alternate blank lines? Joe Pfeiffer <pfeiffer@cs.nmsu.edu> - 2016-12-14 15:41 -0700
                Re: Why is this outputting alternate blank lines? Richard Heathfield <rjh@cpax.org.uk> - 2016-12-14 23:26 +0000
                  Re: Why is this outputting alternate blank lines? Joe Pfeiffer <pfeiffer@cs.nmsu.edu> - 2016-12-14 17:33 -0700
                    Re: Why is this outputting alternate blank lines? Richard Heathfield <rjh@cpax.org.uk> - 2016-12-15 00:42 +0000
                      Re: Why is this outputting alternate blank lines? supercat@casperkitty.com - 2016-12-14 17:05 -0800
                        Re: Why is this outputting alternate blank lines? Richard Heathfield <rjh@cpax.org.uk> - 2016-12-15 08:21 +0000
                          Re: Why is this outputting alternate blank lines? supercat@casperkitty.com - 2016-12-15 09:15 -0800
                            Re: Why is this outputting alternate blank lines? Richard Heathfield <rjh@cpax.org.uk> - 2016-12-15 17:50 +0000
                              Re: Why is this outputting alternate blank lines? supercat@casperkitty.com - 2016-12-15 12:15 -0800
                                Re: Why is this outputting alternate blank lines? Richard Heathfield <rjh@cpax.org.uk> - 2016-12-15 20:37 +0000
                            Re: Why is this outputting alternate blank lines? raltbos@xs4all.nl (Richard Bos) - 2016-12-15 20:08 +0000
                      Re: Why is this outputting alternate blank lines? Joe Pfeiffer <pfeiffer@cs.nmsu.edu> - 2016-12-14 20:40 -0700
                        Re: Why is this outputting alternate blank lines? Richard Heathfield <rjh@cpax.org.uk> - 2016-12-15 08:36 +0000
                          Re: Why is this outputting alternate blank lines? Joe Pfeiffer <pfeiffer@cs.nmsu.edu> - 2016-12-15 17:43 -0700
                            Re: Why is this outputting alternate blank lines? Richard Heathfield <rjh@cpax.org.uk> - 2016-12-16 01:05 +0000
                              Re: Why is this outputting alternate blank lines? Joe Pfeiffer <pfeiffer@cs.nmsu.edu> - 2016-12-15 18:34 -0700
                  Re: Why is this outputting alternate blank lines? <william@wilbur.25thandClement.com> - 2016-12-15 10:14 -0800
                    Re: Why is this outputting alternate blank lines? raltbos@xs4all.nl (Richard Bos) - 2016-12-15 20:15 +0000
              Re: Why is this outputting alternate blank lines? Tim Rentsch <txr@alumni.caltech.edu> - 2016-12-15 01:51 -0800
                Re: Why is this outputting alternate blank lines? asetofsymbols@gmail.com - 2016-12-15 06:09 -0800
                  Re: Why is this outputting alternate blank lines? Tim Rentsch <txr@alumni.caltech.edu> - 2016-12-15 17:47 -0800
            Re: Why is this outputting alternate blank lines? Keith Thompson <kst-u@mib.org> - 2016-12-14 12:15 -0800
              Re: Why is this outputting alternate blank lines? <william@wilbur.25thandClement.com> - 2016-12-15 10:24 -0800
            Re: overcommit is the plague Noob <root@127.0.0.1> - 2016-12-17 01:17 +0100
        Re: Why is this outputting alternate blank lines? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2016-12-14 20:19 +0000
          Re: Why is this outputting alternate blank lines? DFS <nospam@dfs.com> - 2016-12-14 17:44 -0500
            Re: Why is this outputting alternate blank lines? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2016-12-15 00:21 +0000
              Re: Why is this outputting alternate blank lines? mark.bluemel@gmail.com - 2016-12-15 00:56 -0800
                Re: Why is this outputting alternate blank lines? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2016-12-15 22:08 +0000
        Re: Why is this outputting alternate blank lines? Geoff <geoff@invalid.invalid> - 2016-12-14 13:39 -0800
        Re: Why is this outputting alternate blank lines? Ike Naar <ike@iceland.freeshell.org> - 2016-12-16 07:12 +0000

Page 2 of 3 — ← Prev page 1 [2] 3  Next page →

#99541

On 15/12/16 17:15, supercat@casperkitty.com wrote:
> On Thursday, December 15, 2016 at 2:21:11 AM UTC-6, Richard Heathfield wrote:
>> On 15/12/16 01:05, supercat wrote:
>>> There are many kinds of application that won't be able to do anything useful
>>> unless all of their malloc() calls succeed.
>>
>> I think that depends on what you mean by "application". If you just mean
>> "program", I agree. But nowadays, it's probably fair to say that
>> "application" means "event-driven, interactive program" - word
>> processors, spreadsheets, browsers, desktop publishing programs, music
>> editors, arts packages, photographic editors, and all the rest of it.
>> And these programs certainly can continue to be useful even if some of
>> their malloc calls fail.
>
> Not all applications are interactive programs whose memory usage will vary
> based upon user input.  There are many other kinds which either do not
> operate interactively, or which pre-allocate the storage they're ever going
> to need, and for those kinds of programs memory will either be adequate or
> it won't.

Right. I would say that it is reasonable for a program to "bomb out" on 
insufficient memory if and only if both the following conditions are 
observed:

1) it is non-interactive;
2) it can be re-started, either from scratch or from a checkpoint, 
without data corruption.

The kinds of programs that would fall into this category would be things 
like grep, sed, ImageMagick's "convert" program, compilers, linkers, and 
a great many batch processes.

>> If a photo editor, say, can't open a photograph
>> because it's too darn huge, I'd appreciate being told that in a dialog
>> window that I can dismiss to carry on working on some other photos,
>> rather than being told it by the crashing of the application before I
>> have so much as a chance to save my work.
>
> In order for that to work robustly, the program would have to have different
> means of requesting allocations for things which are allowed to fail, and
> for things which aren't.  If all of the allocations made by a program have
> succeeded up until the user selects "Save as...", but there isn't enough
> memory left to display the dialog box, what should happen?

The programmer should be re-trained. The right approach here is to 
reserve memory for vital resources while it is still available, and 
/before/ the user has invested time in using the program.

Take "Save as...", which is pretty essential as part of an emergency 
backing-down procedure, as you have to allow your user to save the work 
he or she has created or modified before you land (so to speak). A "Save 
as..." dialog can be created without having to display it. So you create 
it /first/. Then, if everything starts to go pear-shaped, you can offer 
the user the chance to save created or modified documents.

> What's needed is not to say that the program needs to prepare for every
> allocation request to fail, but rather that a program needs to have a means
> of ensuring that certain requests *won't* fail, and that requests which
> could fail safely will only succeed in cases that might cause other, more
> important, requests to fail.

ITYM s/might/will not/

But the programmer /still/ needs to prepare for /every/ allocation 
request to fail, because to do this:

p = malloc(n * sizeof *p);
memcpy(p, source, lots);

is asking for a crash, and the whole thrust of my argument is that 
applications *must not crash*. They might crash-land, so to speak, but 
they mustn't crash. By "crash-land", I mean that they take all necessary 
steps to ensure that the program continues in a stable state for long 
enough that the user can save any new or modified data.

>> Useful and convenient to the programmer, maybe, but not even remotely
>> useful to the user, and it is the user to whom the program is /supposed/
>> to be useful. As for checking the result of malloc and then deliberately
>> aborting if it's NULL, well, that's precisely the kind of thinking that
>> leads to the flaky applications we have to use every day because we lack
>> the several hundred person-years of spare time it would take to go and
>> fix all the source code.
>
> How should the code for a "Save as..." dialog box behave if it runs out of
> memory after creating a few of the graphical resources it requires?  Is it
> more helpful to check each individual allocation for success, or instead
> focusing on ways of ensuring that memory doesn't get committed to "optional"
> purposes unless there's certain to be enough to satisfy all necessary ones?

Every time the programmer fails to check that the allocation succeeded, 
he or she risks the possibility that the program will crash without 
warning. It's slip-shod, it's tacky, it's broken programming.

>>> Abnormal program termination is hardly elegant, but if a program isn't going
>>> to be able to behave usefully, the value of avoiding an APE may be limited.
>>
>> Abnormal program termination isn't just inelegant. It's very often a
>> sign of lazy and thoughtless programming. It's fine for debugging and
>> testing, but it has no place in production code.
>
> That depends upon the application.  There are certainly some kinds that can
> and must continue functioning usefully even after an allocation failure,

Absolutely. I listed a few. I would only except filter programs that can 
be re-started without data corruption.

> but
> if an application knows in advance of receiving input how much memory it is
> going to need, and can't do anything useful without it, having the program
> die with a system-generated "Out of memory error" isn't much less useful than
> having a program-generated "Sorry, but I can't allocate enough memory."

I disagree. A program-generated "Sorry, but I can't allocate enough 
memory for opening a new image. In fact, I think I need to close, but 
before I do, let's try and save what you've done so far" is a damn sight 
more useful than a syste^\<#!+89E$-+q"|(NO CARRIER

[toc] | [prev] | [next] | [standalone]

#99559

On Thursday, December 15, 2016 at 11:50:59 AM UTC-6, Richard Heathfield wrote:
> On 15/12/16 17:15, supercat wrote:
> Right. I would say that it is reasonable for a program to "bomb out" on 
> insufficient memory if and only if both the following conditions are 
> observed:
> 
> 1) it is non-interactive;
> 2) it can be re-started, either from scratch or from a checkpoint, 
> without data corruption.
> 
> The kinds of programs that would fall into this category would be things 
> like grep, sed, ImageMagick's "convert" program, compilers, linkers, and 
> a great many batch processes.

Yup.  I didn't claim all, or even most, applications fit into that category,
but a substantial number of applications do.

> > In order for that to work robustly, the program would have to have different
> > means of requesting allocations for things which are allowed to fail, and
> > for things which aren't.  If all of the allocations made by a program have
> > succeeded up until the user selects "Save as...", but there isn't enough
> > memory left to display the dialog box, what should happen?
> 
> The programmer should be re-trained. The right approach here is to 
> reserve memory for vital resources while it is still available, and 
> /before/ the user has invested time in using the program.

Agreed that one should endeavor to ensure things can't run out of memory.
The question is what level of detailed handling is deeded if they do.

> Take "Save as...", which is pretty essential as part of an emergency 
> backing-down procedure, as you have to allow your user to save the work 
> he or she has created or modified before you land (so to speak). A "Save 
> as..." dialog can be created without having to display it. So you create 
> it /first/. Then, if everything starts to go pear-shaped, you can offer 
> the user the chance to save created or modified documents.
> 
> > What's needed is not to say that the program needs to prepare for every
> > allocation request to fail, but rather that a program needs to have a means
> > of ensuring that certain requests *won't* fail, and that requests which
> > could fail safely will only succeed in cases that might cause other, more
> > important, requests to fail.
> 
> ITYM s/might/will not/

Yeah.

> But the programmer /still/ needs to prepare for /every/ allocation 
> request to fail, because to do this:
> 
> p = malloc(n * sizeof *p);
> memcpy(p, source, lots);
> 
> is asking for a crash, and the whole thrust of my argument is that 
> applications *must not crash*. They might crash-land, so to speak, but 
> they mustn't crash. By "crash-land", I mean that they take all necessary 
> steps to ensure that the program continues in a stable state for long 
> enough that the user can save any new or modified data.

I'd suggest that user-facing applications probably should generally acquire
memory in ways that offer more control than malloc() does, and should when
possible map malloc() to a routine that allocates memory from a pool which
is large enough to handle the needs of all code that's going to use malloc().
In many cases, there may be calls to malloc() that are outside the control
of the application programmer, and some code that uses malloc() may have no
way to handle failure [e.g. some compilers may use malloc() for variable-
length arrays, and the Standard specifies no way to recover from a failed
malloc() in such a case].  The clean way to handle that would be to have
distinct semantics for "give me memory or report failure" or "give me memory
or trap in an implementation-defined fashion"; code using the latter would
then not need to check for success, because the fact the function returns
would imply that it didn't fail.

> > How should the code for a "Save as..." dialog box behave if it runs out of
> > memory after creating a few of the graphical resources it requires?  Is it
> > more helpful to check each individual allocation for success, or instead
> > focusing on ways of ensuring that memory doesn't get committed to "optional"
> > purposes unless there's certain to be enough to satisfy all necessary ones?
> 
> Every time the programmer fails to check that the allocation succeeded, 
> he or she risks the possibility that the program will crash without 
> warning. It's slip-shod, it's tacky, it's broken programming.

A programmer who isn't prepared to do anything useful in case of allocation
failure should use forms of allocation request which are guaranteed never
to return anything but a valid pointer.  Duplicating error handling code
at every allocation request is only useful for those requests which can do
something beyond what a unified "allocation failure" handler could do.

> > but
> > if an application knows in advance of receiving input how much memory it is
> > going to need, and can't do anything useful without it, having the program
> > die with a system-generated "Out of memory error" isn't much less useful than
> > having a program-generated "Sorry, but I can't allocate enough memory."
> 
> I disagree. A program-generated "Sorry, but I can't allocate enough 
> memory for opening a new image. In fact, I think I need to close, but 
> before I do, let's try and save what you've done so far" is a damn sight 
> more useful than a syste^\<#!+89E$-+q"|(NO CARRIER

If the program attempts unsuccessfully to acquire the memory it needs
in advance of receiving input, how much work can a user have done by that
point?

[toc] | [prev] | [next] | [standalone]

#99567

On 15/12/16 20:15, supercat@casperkitty.com wrote:
> On Thursday, December 15, 2016 at 11:50:59 AM UTC-6, Richard Heathfield wrote:
>> On 15/12/16 17:15, supercat wrote:

<snip>
>
>>> but
>>> if an application knows

I read this bit...

>>> in advance of receiving input

...and I didn't read this bit...

>>> how much memory it is
>>> going to need, and can't do anything useful without it, having the program
>>> die with a system-generated "Out of memory error" isn't much less useful than
>>> having a program-generated "Sorry, but I can't allocate enough memory."

...and I read this bit. Consequently, I wrote:

>>
>> I disagree. A program-generated "Sorry, but I can't allocate enough
>> memory for opening a new image. In fact, I think I need to close, but
>> before I do, let's try and save what you've done so far" is a damn sight
>> more useful than a syste^\<#!+89E$-+q"|(NO CARRIER
>
> If the program attempts unsuccessfully to acquire the memory it needs
> in advance of receiving input, how much work can a user have done by that
> point?

If I'd read the bit I didn't read, I'd have given a different answer. :-)

-- 
Richard Heathfield
Email: rjh at cpax dot org dot uk
"Usenet is a strange place" - dmr 29 July 1999
Sig line 4 vacant - apply within

[toc] | [prev] | [next] | [standalone]

#99557

supercat@casperkitty.com wrote:

> On Thursday, December 15, 2016 at 2:21:11 AM UTC-6, Richard Heathfield wrote:
> > On 15/12/16 01:05, supercat wrote:

> > > Abnormal program termination is hardly elegant, but if a program isn't going
> > > to be able to behave usefully, the value of avoiding an APE may be limited.
> > 
> > Abnormal program termination isn't just inelegant. It's very often a 
> > sign of lazy and thoughtless programming. It's fine for debugging and 
> > testing, but it has no place in production code.
> 
> That depends upon the application.  There are certainly some kinds that can
> and must continue functioning usefully even after an allocation failure, but
> if an application knows in advance of receiving input how much memory it is
> going to need, and can't do anything useful without it, having the program
> die with a system-generated "Out of memory error" isn't much less useful than
> having a program-generated "Sorry, but I can't allocate enough memory."

That's not the problem, though. The problem is the _ubiquitous_ reliance
on overcommit even where it isn't warranted, and the equally ubiquitous
expectation that an ABEND will do just fine. It won't. An abnormal
termination often will - but a controlled one. A core dump is useless to
anyone except a hardcore hacker. 
If Linux wants to be taken seriously as an alternative to M$ Windows -
as an OS for Auntie Dora and Mr. Thompson the bookkeeper, not just for
you, me, and companies with real sysadmins - its programmers will have
to ditch this habit, and accept that a core dump is _not_ acceptable in
an end-user system. It's bad enough trying to have your computer
illiterate granny report the error message when it _is_ legible; forget
about it when it says "ERRNOMEM - core dumped #*(&$#*(@(@(@".

Richard

[toc] | [prev] | [next] | [standalone]

#99437

Richard Heathfield <rjh@cpax.org.uk> writes:

> On 15/12/16 00:33, Joe Pfeiffer wrote:
>> Richard Heathfield <rjh@cpax.org.uk> writes:
>>
>>> I did, and practically every application I tried to use would fall
>>> over within a few minutes (or, sometimes, within a few seconds).
>>
>> That's weird.  How much swap did you have?
>
> Several tons. Truckloads.

Wow.  All I can say is that's surprising.

>> Well, they're expecting overcommit.
>
> Well, they shouldn't.

You have to have expectations.  If 99+% of the systems you're running on
have overcommit, it's not a bad assumption.  The only other real
possibility is to have some sort of developer-accessible way to check
the memory pool to decide whether you really need to crash, or if you
can continue.  If that exists, I'm not aware of it.

>> It's pretty close to true
>> (especially on a 64 bit system) with overcommit that the only time
>> malloc() will fail is if something has corrupted the memory pool; in
>> that case, you might as well crash now since you're going to very soon
>> (or worse).
>
> Or you might as well not bother testing, which is what I suspect
> happens most of the time. And that's why Linux software is so
> hopeless. Not as hopeless as Windows software, obviously --- but still
> pretty hopeless. Look! Shiny!

Well, yeah.  Most programmers expect all library and system calls (even
open()!) to succeed and don't check anything.  The question here is, for
that tiny fraction who check at all, what should they do?  I try to do
something sane if it's likely to be a user error, and crash with an
error message if it's mine.  If malloc() fails, it's almost certain
that I've got a wild pointer someplace.

[toc] | [prev] | [next] | [standalone]

#99457

On 15/12/16 03:40, Joe Pfeiffer wrote:
> Richard Heathfield <rjh@cpax.org.uk> writes:
>
>> On 15/12/16 00:33, Joe Pfeiffer wrote:
>>> Richard Heathfield <rjh@cpax.org.uk> writes:
>>>
>>>> I did, and practically every application I tried to use would fall
>>>> over within a few minutes (or, sometimes, within a few seconds).
>>>
>>> That's weird.  How much swap did you have?
>>
>> Several tons. Truckloads.
>
> Wow.  All I can say is that's surprising.

I was surprised, too, at first, and then later on I realised that 
actually it wasn't so surprising after all. Because the big problem is...

>
>>> Well, they're expecting overcommit.

...that.

>>
>> Well, they shouldn't.
>
> You have to have expectations.  If 99+% of the systems you're running on
> have overcommit, it's not a bad assumption.

Yes it is! It's always a bad assumption! It's an assumption that leads 
to sloppy memory management. And I do realise I'm railing against the 
inevitable, and while we're at it let's bring back the birch and get the 
hell out of Europe and put Churchill back in No 10, and several other 
unattainable ideals which I only mention because, after years and years 
of persistent campaigning, one of them turned out not to be so 
unattainable after all. If someone has got into the terrible habit of 
being lazy about memory management, it may well be too late to change 
his or her ways. But let us at least set a good example and write code 
that uses memory properly.

> The only other real
> possibility is to have some sort of developer-accessible way to check
> the memory pool to decide whether you really need to crash, or if you
> can continue.  If that exists, I'm not aware of it.

"Need to crash"? Thank heaven you're not an airline pilot.

It's a good analogy, actually. (And an illustration, not a proof.)

When things start to go wrong in the air, a good airline pilot 
continually monitors the state of his or her aircraft, warns his or her 
passengers that they may need to fasten their seatbelts, attempts at all 
times to maintain control of the plane, and --- if it turns out that 
there is no way he or she is going to keep it in the air for much longer 
--- try to find a nearby airfield on which to attempt a runway landing 
or, failing that, a road or a bit of flat grass or, in extreme cases, 
the Hudson River. The point is, he or she doesn't just let go of the 
controls and wait for the aircraft to spiral down to the ground and be 
destroyed in a ball of flame that kills everyone on board.

"Need to crash" shows a misunderstanding, and admittedly it is a 
/widespread/ misunderstanding, of what computer programming is actually 
/for/.

-- 
Richard Heathfield
Email: rjh at cpax dot org dot uk
"Usenet is a strange place" - dmr 29 July 1999
Sig line 4 vacant - apply within

[toc] | [prev] | [next] | [standalone]

#99590

Richard Heathfield <rjh@cpax.org.uk> writes:

> On 15/12/16 03:40, Joe Pfeiffer wrote:
>>
>> You have to have expectations.  If 99+% of the systems you're running on
>> have overcommit, it's not a bad assumption.
>
> Yes it is! It's always a bad assumption! It's an assumption that leads
> to sloppy memory management. And I do realise I'm railing against the
> inevitable, and while we're at it let's bring back the birch and get
> the hell out of Europe and put Churchill back in No 10, and several
> other unattainable ideals which I only mention because, after years
> and years of persistent campaigning, one of them turned out not to be
> so unattainable after all. If someone has got into the terrible habit
> of being lazy about memory management, it may well be too late to
> change his or her ways. But let us at least set a good example and
> write code that uses memory properly.

It needn't lead to poor memory management practices -- but it certainly
influences a reasonable response to malloc() failing.

>> The only other real
>> possibility is to have some sort of developer-accessible way to check
>> the memory pool to decide whether you really need to crash, or if you
>> can continue.  If that exists, I'm not aware of it.
>
> "Need to crash"? Thank heaven you're not an airline pilot.
>
> It's a good analogy, actually. (And an illustration, not a proof.)
>
> When things start to go wrong in the air, a good airline pilot
> continually monitors the state of his or her aircraft, warns his or
> her passengers that they may need to fasten their seatbelts, attempts
> at all times to maintain control of the plane, and --- if it turns out
> that there is no way he or she is going to keep it in the air for much
> longer --- try to find a nearby airfield on which to attempt a runway
> landing or, failing that, a road or a bit of flat grass or, in extreme
> cases, the Hudson River. The point is, he or she doesn't just let go
> of the controls and wait for the aircraft to spiral down to the ground
> and be destroyed in a ball of flame that kills everyone on board.
>
> "Need to crash" shows a misunderstanding, and admittedly it is a
> /widespread/ misunderstanding, of what computer programming is
> actually /for/.

A vivid analogy, but not an especially apt one.  I suppose you can hope
your data structures are in good enough shape to try to dump them to a
checkpoint file, but in my experience that's not likely to get very far
before segfault.

[toc] | [prev] | [next] | [standalone]

#99591

On 16/12/16 00:43, Joe Pfeiffer wrote:
> Richard Heathfield <rjh@cpax.org.uk> writes:
>
<snip>
>>
>> When things start to go wrong in the air, a good airline pilot
>> continually monitors the state of his or her aircraft, warns his or
>> her passengers that they may need to fasten their seatbelts, attempts
>> at all times to maintain control of the plane, and --- if it turns out
>> that there is no way he or she is going to keep it in the air for much
>> longer --- try to find a nearby airfield on which to attempt a runway
>> landing or, failing that, a road or a bit of flat grass or, in extreme
>> cases, the Hudson River. The point is, he or she doesn't just let go
>> of the controls and wait for the aircraft to spiral down to the ground
>> and be destroyed in a ball of flame that kills everyone on board.
>>
>> "Need to crash" shows a misunderstanding, and admittedly it is a
>> /widespread/ misunderstanding, of what computer programming is
>> actually /for/.
>
> A vivid analogy,

It was, wasn't it? :-)

> but not an especially apt one.

I disagree, obviously.

> I suppose you can hope
> your data structures are in good enough shape to try to dump them to a
> checkpoint file, but in my experience that's not likely to get very far
> before segfault.

If you can't leave the data in a consistent state, then at the very 
least you should tell your user that before the program dies. And, if 
possible, you should tell them in what way it might be corrupted and 
what reasonable steps they can take in an attempt to fix the problem.

-- 
Richard Heathfield
Email: rjh at cpax dot org dot uk
"Usenet is a strange place" - dmr 29 July 1999
Sig line 4 vacant - apply within

[toc] | [prev] | [next] | [standalone]

#99593

Richard Heathfield <rjh@cpax.org.uk> writes:

> On 16/12/16 00:43, Joe Pfeiffer wrote:
>> Richard Heathfield <rjh@cpax.org.uk> writes:
>>
> <snip>
>>>
>>> When things start to go wrong in the air, a good airline pilot
>>> continually monitors the state of his or her aircraft, warns his or
>>> her passengers that they may need to fasten their seatbelts, attempts
>>> at all times to maintain control of the plane, and --- if it turns out
>>> that there is no way he or she is going to keep it in the air for much
>>> longer --- try to find a nearby airfield on which to attempt a runway
>>> landing or, failing that, a road or a bit of flat grass or, in extreme
>>> cases, the Hudson River. The point is, he or she doesn't just let go
>>> of the controls and wait for the aircraft to spiral down to the ground
>>> and be destroyed in a ball of flame that kills everyone on board.
>>>
>>> "Need to crash" shows a misunderstanding, and admittedly it is a
>>> /widespread/ misunderstanding, of what computer programming is
>>> actually /for/.
>>
>> A vivid analogy,
>
> It was, wasn't it? :-)
>
>> but not an especially apt one.
>
> I disagree, obviously.
>
>> I suppose you can hope
>> your data structures are in good enough shape to try to dump them to a
>> checkpoint file, but in my experience that's not likely to get very far
>> before segfault.
>
> If you can't leave the data in a consistent state, then at the very
> least you should tell your user that before the program dies. And, if
> possible, you should tell them in what way it might be corrupted and
> what reasonable steps they can take in an attempt to fix the problem.

Here we partially agree -- you should certainly tell the user what
happened.  I have a lot of calls like

error_at_line(1, errno, __FILE__, __LINE__, "Internal error: calloc() returned NULL");

in my code.  It doesn't give the user much help on resolving the issue,
but at least it tells what happened and where.

[toc] | [prev] | [next] | [standalone]

#99544

Richard Heathfield <rjh@cpax.org.uk> wrote:
> On 14/12/16 22:41, Joe Pfeiffer wrote:
> <snip>
>>
>> You can turn off overcommit, however.
> 
> Have you ever tried?
> 
> I did, and practically every application I tried to use would fall over 
> within a few minutes (or, sometimes, within a few seconds).

I disable overcommit on all my Linux app servers _and_ I never add swap.

But I don't run Linux on the desktop. Popular desktop toolkits like glib and
Gtk+ use malloc wrappers that simply abort the process on malloc failure.
glib is used by some server software, but I avoid any server software
that uses glib.

Windows doesn't have overcommit and works just fine. See

  https://lwn.net/Articles/627725/

It's ridiculous that Linux developers rely on it so heavily.

> I do worry sometimes. So many people think that features are more 
> important than robustness. The prevailing attitude seems to be that, if 
> there isn't enough memory to do /everything/, we'll damn well do 
> nothing. Not enough memory for a new tab in the browser, say? What 
> should we do? Put up a dialog saying "sorry, can't do that, but we'll 
> leave the other tabs open so you can keep on doing whatever you were 
> doing"? No, that sounds too sensible. Let's just crash and burn, and to 
> hell with the user.
> 
> Oh well. C'est la vie.
> 

[toc] | [prev] | [next] | [standalone]

#99558

<william@wilbur.25thandClement.com> wrote:

> Windows doesn't have overcommit and works just fine. See
> 
>   https://lwn.net/Articles/627725/
> 
> It's ridiculous that Linux developers rely on it so heavily.

IMAO it's a sad but somewhat inevitable outcome of the Linux mindset:
"Our system is better than any, safer than any, harder than any, and
_cannot fail_. So are we, and neither can we!"

They are, of course, wrong. But many of them do display that attitude.
Say what you will about Windows programmers, but at least most of them
are aware that "Windoze sux".
What Linux programmers are unwilling to admit is that, though there are
certainly differences in degree, All Hardware Sucks, All Software Sucks.
No exceptions. That Windoze sux harder than some doesn't mean Linux'
over-committed arse doesn't suck chunks as well. But you try and get a
GNUhead to face that truth...

Richard

[toc] | [prev] | [next] | [standalone]

#99470

Richard Heathfield <rjh@cpax.org.uk> writes:

> On 14/12/16 19:57, Keith Thompson wrote:
>> Richard Heathfield <rjh@cpax.org.uk> writes:
>>> On 14/12/16 17:53, DFS wrote:
>>
>> [...]
>>
>>>> Any idea why it crashes above 1 million?
>>>
>>> Do you remember when I suggested checking the return value of
>>> malloc?
>>>
>>> I again suggest that you check the return value of malloc.
>>>
>>> The malloc function returns NULL when it fails to allocate memory.
>>> Since you're not checking for this, if ever it /does/ return NULL
>>> you will be scribbling all over memory you don't own, hence the
>>> segfault.
>>
>> On the other hand, on some systems malloc() generally doesn't report
>> failure;  rather if you allocate too much memory, a crash occurs when
>> you try to access it.  (This is arguably non-conforming.)
>
> Arguably?  I'd have said it's absolutely definitely non-conforming.
> The malloc function is /required/ to report failure if the memory
> can't be allocated. [...]

I will offer a different view, namely, that neither "conforming"
nor "non-conforming" is apt in this case, because what happens is
determined by factors outside the implementation.  Rather, the
behavior in question is in the province of "a data-processing
system that supports an implementation", which under section 1
paragraph 2 is outside the realm of what the Standard specifies,
and also (I opine) meant to be a separate and distinct element,
ie, /not/ part of whatever it is that makes up a conforming
implementation.

[toc] | [prev] | [next] | [standalone]

#99495

It is useful malloc Return 0 if
fail, because the programmer in this
way can choose better what to
do in this case of malloc failure

[toc] | [prev] | [next] | [standalone]

#99594

asetofsymbols@gmail.com writes:

> It is useful malloc Return 0 if
> fail, because the programmer in this
> way can choose better what to
> do in this case of malloc failure

I won't disagree, but that is incidental to what
I was saying.

[toc] | [prev] | [next] | [standalone]

#99399

Keith Thompson <kst-u@mib.org> writes:
> Richard Heathfield <rjh@cpax.org.uk> writes:
>> On 14/12/16 17:53, DFS wrote:
> [...]
>>> Any idea why it crashes above 1 million?
>>
>> Do you remember when I suggested checking the return value of malloc?
>>
>> I again suggest that you check the return value of malloc.
>>
>> The malloc function returns NULL when it fails to allocate memory. Since 
>> you're not checking for this, if ever it /does/ return NULL you will be 
>> scribbling all over memory you don't own, hence the segfault.
>
> On the other hand, on some systems malloc() generally doesn't report
> failure; rather if you allocate too much memory, a crash occurs when you
> try to access it.  (This is arguably non-conforming.)

Which is not to say you shouldn't check the value returned by malloc(),
even on systems that behave this way.

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Working, but not speaking, for JetHead Development, Inc.
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

[toc] | [prev] | [next] | [standalone]

#99546

Keith Thompson <kst-u@mib.org> wrote:
> Keith Thompson <kst-u@mib.org> writes:
>> Richard Heathfield <rjh@cpax.org.uk> writes:
>>> On 14/12/16 17:53, DFS wrote:
>> [...]
>>>> Any idea why it crashes above 1 million?
>>>
>>> Do you remember when I suggested checking the return value of malloc?
>>>
>>> I again suggest that you check the return value of malloc.
>>>
>>> The malloc function returns NULL when it fails to allocate memory. Since 
>>> you're not checking for this, if ever it /does/ return NULL you will be 
>>> scribbling all over memory you don't own, hence the segfault.
>>
>> On the other hand, on some systems malloc() generally doesn't report
>> failure; rather if you allocate too much memory, a crash occurs when you
>> try to access it.  (This is arguably non-conforming.)
> 
> Which is not to say you shouldn't check the value returned by malloc(),
> even on systems that behave this way.

malloc can and will fail on Linux even with overcommit enabled. Linux still
obeys process resource limits. Per-process limits are often set to infinity
by default, but not always. Some distributions will have finite defaults
especially for non-root, interactive sessions. And it's always smart to set
specially tailored limits for service daemons to prevent, e.g., DoS attacks
on the system. This is also why I never use swap space on my servers, Linux
or not.

[toc] | [prev] | [next] | [standalone]

#99715 — Re: overcommit is the plague

On 14/12/2016 20:57, Keith Thompson wrote:

> On the other hand, on some systems malloc() generally doesn't report
> failure; rather if you allocate too much memory, a crash occurs when you
> try to access it.  (This is arguably non-conforming.)

Restore sanity to your Linux system, disable memory overcommit.

https://www.kernel.org/doc/Documentation/vm/overcommit-accounting
https://www.kernel.org/doc/Documentation/sysctl/vm.txt

sysctl vm.overcommit_memory=2

[toc] | [prev] | [next] | [standalone]

#99401

DFS <nospam@dfs.com> writes:
<snip>
> readdates.c
> ================================================================
> #include <stdio.h>
> #include <stdlib.h>
> #include <string.h>
>
> int main(int argc, char *argv[])
> {
>
> 	int i=0,n=0,linewidth=13,linecount=0;
>   char line[linewidth];
> 	char txtfile[strlen(argv[1])];

These arrays are called variable length arrays because the size is not a
compile time constant (despite linewidth looking very predictable).
Nothing wrong with that but it's useful to know since this feature has
been made optional in the latest C standard.

The second one (txtfile) is not needed.

> 	FILE *fp; char c;
>
> 	//open file
> 	strcpy(txtfile,argv[1]);
> 	fp=fopen(txtfile,"r");

Just fp = fopen(argv[1], "r");

> 	if(fp==NULL){
> 	  printf("Failed to open file\n");
> 	  return -1;
> 	}

You test fp, but not whether argv[1] exists.  For simple programs, an
easy option is to provide a default:

  FILE *fp = argc > 1 ? fopen(argv[1], "r") : stdin;

(You still have to test fp.)

> 	//count lines in file
> 	c=getc(fp);
> 	while (c!=EOF){
> 	  if(c=='\n'){linecount+=1;}
> 	  c=getc(fp);
> 	}
> 	fclose(fp);

It's better just to use rewind(fp) here.  That sets the reading position
back to the start.

Also, if a file contains "abc\ndef" (and nothing else) how many lines do
you want to count?

> 	printf("%d lines in file '%s'\n",linecount,txtfile);
>
> 	//copy data into array
> 	char *datestr[linecount];

This is likely to be the source of your trouble.  Most systems limit the
amount of space that this sort of object can occupy so that
million-element arrays are likely to be a problem (look up "stack space"
to find out more).  Most space is usually available via malloc, so
instead you can write

  char *datestr = malloc(linecount * sizeof *datestr);

> 	fp = fopen(txtfile,"r");

If you rewind, no need to reopen.

> 	while(fgets(line,linewidth,fp)!=NULL) {
> 	 datestr[i]=malloc(linewidth);
> 	 strcpy(datestr[i],line);
> 	 i++;n++;
> 	}

There are some possible problems here though you may choose to disregard
them for the moment.  Currently, you'll count one line for the example I
gave above, but fgets will read two lines (one with "abc\n" and the
second with "def" and no newline).  You will access datestr outside it's
bound.  Secondly, if there are long line, fgets will return them in two
or more parts, again making you access datestr outside its bounds.

> 	fclose(fp);

and if you do as I suggested above you probably want to write

  if (fp != stdin) fclose(fp);
>
> 	//print array
>   for (i=0;i<n;i++){printf("%s", datestr[i]);}
> }
> ================================================================

-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#99424

On 12/14/2016 03:19 PM, Ben Bacarisse wrote:
> DFS <nospam@dfs.com> writes:
> <snip>
>> readdates.c
>> ================================================================
>> #include <stdio.h>
>> #include <stdlib.h>
>> #include <string.h>
>>
>> int main(int argc, char *argv[])
>> {
>>
>> 	int i=0,n=0,linewidth=13,linecount=0;
>>   char line[linewidth];
>> 	char txtfile[strlen(argv[1])];
>
> These arrays are called variable length arrays because the size is not a
> compile time constant (despite linewidth looking very predictable).
> Nothing wrong with that but it's useful to know since this feature has
> been made optional in the latest C standard.
>
> The second one (txtfile) is not needed.
>
>> 	FILE *fp; char c;
>>
>> 	//open file
>> 	strcpy(txtfile,argv[1]);
>> 	fp=fopen(txtfile,"r");
>
> Just fp = fopen(argv[1], "r");
>
>> 	if(fp==NULL){
>> 	  printf("Failed to open file\n");
>> 	  return -1;
>> 	}
>
> You test fp, but not whether argv[1] exists.  For simple programs, an
> easy option is to provide a default:
>
>   FILE *fp = argc > 1 ? fopen(argv[1], "r") : stdin;
>
> (You still have to test fp.)
>
>> 	//count lines in file
>> 	c=getc(fp);
>> 	while (c!=EOF){
>> 	  if(c=='\n'){linecount+=1;}
>> 	  c=getc(fp);
>> 	}
>> 	fclose(fp);
>
> It's better just to use rewind(fp) here.  That sets the reading position
> back to the start.
>
> Also, if a file contains "abc\ndef" (and nothing else) how many lines do
> you want to count?
>
>> 	printf("%d lines in file '%s'\n",linecount,txtfile);
>>
>> 	//copy data into array
>> 	char *datestr[linecount];
>
> This is likely to be the source of your trouble.  Most systems limit the
> amount of space that this sort of object can occupy so that
> million-element arrays are likely to be a problem (look up "stack space"
> to find out more).  Most space is usually available via malloc, so
> instead you can write
>
>   char *datestr = malloc(linecount * sizeof *datestr);
>
>> 	fp = fopen(txtfile,"r");
>
> If you rewind, no need to reopen.
>
>> 	while(fgets(line,linewidth,fp)!=NULL) {
>> 	 datestr[i]=malloc(linewidth);
>> 	 strcpy(datestr[i],line);
>> 	 i++;n++;
>> 	}
>
> There are some possible problems here though you may choose to disregard
> them for the moment.  Currently, you'll count one line for the example I
> gave above, but fgets will read two lines (one with "abc\n" and the
> second with "def" and no newline).  You will access datestr outside it's
> bound.  Secondly, if there are long line, fgets will return them in two
> or more parts, again making you access datestr outside its bounds.

That's for another day, but thanks.

Today I just want to efficiently read into an array a list of date 
strings (14-Dec-2016 format), and use mktime or some other function to 
convert the strings to and from ints.

Using your recommendations:

//copy file data into array
char *datestr=malloc(linecount*sizeof(*datestr));
rewind(fp);
while(fgets(line,linewidth,fp)!=NULL) {
  strcpy(datestr[i],line);
  i++;n++;
}


Gives me a warning and a note:

readdates.c:46:10: warning: passing argument 1 of ‘strcpy’ makes pointer 
from integer without a cast [-Wint-conversion]
    strcpy(datestr[i],line);
           ^~~~~~~

In file included from readdates.c:3:0:
/usr/include/string.h:125:14: note: expected ‘char * restrict’ but 
argument is of type ‘char’
  extern char *strcpy (char *__restrict __dest, const char *__restrict 
__src)
               ^~~~~~





>> 	fclose(fp);
>
> and if you do as I suggested above you probably want to write
>
>   if (fp != stdin) fclose(fp);
>>
>> 	//print array
>>   for (i=0;i<n;i++){printf("%s", datestr[i]);}
>> }
>> ================================================================


Thanks for the tips, Ben.

[toc] | [prev] | [next] | [standalone]

#99427

I'm correcting a mistake I made and adding something I missed...

DFS <nospam@dfs.com> writes:

> On 12/14/2016 03:19 PM, Ben Bacarisse wrote:
>> DFS <nospam@dfs.com> writes:
>> <snip>
>>> readdates.c
>>> ================================================================
>>> #include <stdio.h>
>>> #include <stdlib.h>
>>> #include <string.h>
>>>
>>> int main(int argc, char *argv[])
>>> {
>>>
>>> 	int i=0,n=0,linewidth=13,linecount=0;
>>>   char line[linewidth];
>>> 	char txtfile[strlen(argv[1])];
>>
>> These arrays are called variable length arrays because the size is not a
>> compile time constant (despite linewidth looking very predictable).
>> Nothing wrong with that but it's useful to know since this feature has
>> been made optional in the latest C standard.
>>
>> The second one (txtfile) is not needed.
>>
>>> 	FILE *fp; char c;

Making c a char is going to cause problems on some systems.  C's
character input functions return an int so that they can return a
special value that is not equal to any value of type char.  Therefore c
should be int here.

>>>
>>> 	//open file
>>> 	strcpy(txtfile,argv[1]);
>>> 	fp=fopen(txtfile,"r");
>>
>> Just fp = fopen(argv[1], "r");
>>
>>> 	if(fp==NULL){
>>> 	  printf("Failed to open file\n");
>>> 	  return -1;
>>> 	}
>>
>> You test fp, but not whether argv[1] exists.  For simple programs, an
>> easy option is to provide a default:
>>
>>   FILE *fp = argc > 1 ? fopen(argv[1], "r") : stdin;

Given that you open the file twice (or rewind it) this won't work.  I
should have suggested simply a default file name:

  FILE *fp = fopen(argc > 1 ? argv[1] : "dates.txt", "r");

>> (You still have to test fp.)
>>
>>> 	//count lines in file
>>> 	c=getc(fp);
>>> 	while (c!=EOF){
>>> 	  if(c=='\n'){linecount+=1;}
>>> 	  c=getc(fp);
>>> 	}
>>> 	fclose(fp);

Because of the char type for c, on some systems this loop may never end!
The idiomatic way to write this loop is:

  while ((c = getc(fp)) != EOF)
     if (c == '\n') linecount += 1;

(you can add {}s -- I'm commenting on revode the duplicate c=getc(fp);)

>> It's better just to use rewind(fp) here.  That sets the reading position
>> back to the start.
>>
>> Also, if a file contains "abc\ndef" (and nothing else) how many lines do
>> you want to count?
>>
>>> 	printf("%d lines in file '%s'\n",linecount,txtfile);
>>>
>>> 	//copy data into array
>>> 	char *datestr[linecount];
>>
>> This is likely to be the source of your trouble.  Most systems limit the
>> amount of space that this sort of object can occupy so that
>> million-element arrays are likely to be a problem (look up "stack space"
>> to find out more).  Most space is usually available via malloc, so
>> instead you can write
>>
>>   char *datestr = malloc(linecount * sizeof *datestr);
>>
>>> 	fp = fopen(txtfile,"r");
>>
>> If you rewind, no need to reopen.
>>
>>> 	while(fgets(line,linewidth,fp)!=NULL) {
>>> 	 datestr[i]=malloc(linewidth);
>>> 	 strcpy(datestr[i],line);
>>> 	 i++;n++;
>>> 	}
>>
>> There are some possible problems here though you may choose to disregard
>> them for the moment.  Currently, you'll count one line for the example I
>> gave above, but fgets will read two lines (one with "abc\n" and the
>> second with "def" and no newline).  You will access datestr outside it's
>> bound.  Secondly, if there are long line, fgets will return them in two
>> or more parts, again making you access datestr outside its bounds.
>
> That's for another day, but thanks.
>
> Today I just want to efficiently read into an array a list of date
> strings (14-Dec-2016 format), and use mktime or some other function to
> convert the strings to and from ints.
>
> Using your recommendations:
>
> //copy file data into array
> char *datestr=malloc(linecount*sizeof(*datestr));
> rewind(fp);
> while(fgets(line,linewidth,fp)!=NULL) {
>  strcpy(datestr[i],line);
>  i++;n++;
> }
>
>
> Gives me a warning and a note:
>
> readdates.c:46:10: warning: passing argument 1 of ‘strcpy’ makes
> pointer from integer without a cast [-Wint-conversion]
>    strcpy(datestr[i],line);
>           ^~~~~~~

Sorry, my mistake

  char **datestr = malloc(linecount * sizeof *datestr);

You need datestr to be a pointer to the first element of an allocated
array of character pointers.

<snip>
>>> 	fclose(fp);
>>
>> and if you do as I suggested above you probably want to write
>>
>>   if (fp != stdin) fclose(fp);
>>>
>>> 	//print array
>>>   for (i=0;i<n;i++){printf("%s", datestr[i]);}
>>> }
>>> ================================================================
>
>
> Thanks for the tips, Ben.

-- 
Ben.

[toc] | [prev] | [next] | [standalone]

Page 2 of 3 — ← Prev page 1 [2] 3  Next page →

Back to top | Article view | comp.lang.c

csiph-web