Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.lang.c++ > #121358 > unrolled thread
| Started by | Michael S <already5chosen@yahoo.com> |
|---|---|
| First post | 2025-01-01 18:25 +0200 |
| Last post | 2025-01-02 16:47 +0200 |
| Articles | 13 on this page of 73 — 14 participants |
Back to article view | Back to comp.lang.c++
This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by
below is the oldest one visible, not the original post.
Re: We have a new standard! Michael S <already5chosen@yahoo.com> - 2025-01-01 18:25 +0200
Re: We have a new standard! Muttley@dastardlyhq.com - 2025-01-01 16:33 +0000
Re: We have a new standard! Ross Finlayson <ross.a.finlayson@gmail.com> - 2025-01-01 09:41 -0800
Re: We have a new standard! David Brown <david.brown@hesbynett.no> - 2025-01-02 08:06 +0100
Re: We have a new standard! Muttley@dastardlyhq.com - 2025-01-02 09:23 +0000
Re: We have a new standard! David Brown <david.brown@hesbynett.no> - 2025-01-02 13:51 +0100
Re: We have a new standard! Muttley@DastardlyHQ.org - 2025-01-02 14:07 +0000
Re: We have a new standard! David Brown <david.brown@hesbynett.no> - 2025-01-02 17:54 +0100
Re: We have a new standard! Muttley@DastardlyHQ.org - 2025-01-02 17:15 +0000
Re: We have a new standard! Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2025-01-02 13:58 -0800
Re: We have a new standard! Muttley@DastardlyHQ.org - 2025-01-03 10:13 +0000
Re: We have a new standard! Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2025-01-03 11:22 -0800
Re: We have a new standard! Muttley@dastardlyhq.com - 2025-01-04 10:17 +0000
Re: We have a new standard! David Brown <david.brown@hesbynett.no> - 2025-01-04 12:31 +0100
Re: We have a new standard! Muttley@dastardlyhq.com - 2025-01-04 11:34 +0000
Re: We have a new standard! David Brown <david.brown@hesbynett.no> - 2025-01-04 14:01 +0100
Re: We have a new standard! Muttley@dastardlyhq.com - 2025-01-04 15:49 +0000
Re: We have a new standard! Ross Finlayson <ross.a.finlayson@gmail.com> - 2025-01-03 17:42 -0800
Re: We have a new standard! Sam <sam@email-scan.com> - 2025-01-02 15:20 -0500
Re: We have a new standard! David Brown <david.brown@hesbynett.no> - 2025-01-03 13:43 +0100
Re: We have a new standard! Sam <sam@email-scan.com> - 2025-01-03 08:11 -0500
Re: We have a new standard! Paavo Helde <eesnimi@osa.pri.ee> - 2025-01-03 16:33 +0200
Re: We have a new standard! Sam <sam@email-scan.com> - 2025-01-03 10:50 -0500
Re: We have a new standard! Muttley@DastardlyHQ.org - 2025-01-03 16:09 +0000
Re: We have a new standard! Sam <sam@email-scan.com> - 2025-01-03 11:53 -0500
Re: We have a new standard! Muttley@dastardlyhq.com - 2025-01-03 17:10 +0000
Re: We have a new standard! Sam <sam@email-scan.com> - 2025-01-03 12:14 -0500
Re: We have a new standard! Muttley@dastardlyhq.com - 2025-01-04 10:13 +0000
Re: We have a new standard! David Brown <david.brown@hesbynett.no> - 2025-01-03 18:17 +0100
Re: We have a new standard! Sam <sam@email-scan.com> - 2025-01-03 13:58 -0500
Re: We have a new standard! David Brown <david.brown@hesbynett.no> - 2025-01-04 12:39 +0100
Re: We have a new standard! Sam <sam@email-scan.com> - 2025-01-04 09:56 -0500
Re: We have a new standard! Paavo Helde <eesnimi@osa.pri.ee> - 2025-01-03 21:46 +0200
Re: We have a new standard! Sam <sam@email-scan.com> - 2025-01-03 21:06 -0500
Re: We have a new standard! Ross Finlayson <ross.a.finlayson@gmail.com> - 2025-01-03 18:32 -0800
Re: We have a new standard! Paavo Helde <eesnimi@osa.pri.ee> - 2025-01-04 20:08 +0200
Re: We have a new standard! Sam <sam@email-scan.com> - 2025-01-04 14:46 -0500
Re: We have a new standard! wij <wyniijj5@gmail.com> - 2025-01-05 07:14 +0800
Re: We have a new standard! Muttley@DastardlyHQ.org - 2025-01-06 08:38 +0000
Re: We have a new standard! Chris Ahlstrom <OFeem1987@teleworm.us> - 2025-01-07 08:40 -0500
Re: We have a new standard! Muttley@DastardlyHQ.org - 2025-01-07 14:04 +0000
Re: We have a new standard! Chris Ahlstrom <OFeem1987@teleworm.us> - 2025-01-08 14:52 -0500
Re: We have a new standard! Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2025-01-08 13:31 -0800
Re: We have a new standard! Muttley@DastardlyHQ.org - 2025-01-09 08:39 +0000
Re: We have a new standard! Sam <sam@email-scan.com> - 2025-01-07 21:32 -0500
Re: We have a new standard! Ross Finlayson <ross.a.finlayson@gmail.com> - 2025-01-03 17:49 -0800
Re: We have a new standard! David Brown <david.brown@hesbynett.no> - 2025-01-03 17:12 +0100
Re: We have a new standard! Sam <sam@email-scan.com> - 2025-01-03 11:56 -0500
Re: We have a new standard! Michael S <already5chosen@yahoo.com> - 2025-01-02 17:35 +0200
Re: We have a new standard! David Brown <david.brown@hesbynett.no> - 2025-01-02 17:57 +0100
Re: We have a new standard! Michael S <already5chosen@yahoo.com> - 2025-01-02 19:55 +0200
Re: We have a new standard! Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2025-01-02 13:40 -0800
Re: We have a new standard! David Brown <david.brown@hesbynett.no> - 2025-01-03 09:42 +0100
Re: We have a new standard! Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2025-01-02 13:38 -0800
Re: We have a new standard! Muttley@DastardlyHQ.org - 2025-01-03 10:09 +0000
Re: We have a new standard! Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2025-01-03 11:19 -0800
Re: We have a new standard! Muttley@dastardlyhq.com - 2025-01-04 10:15 +0000
Re: We have a new standard! Paavo Helde <eesnimi@osa.pri.ee> - 2025-01-01 22:43 +0200
Re: We have a new standard! David Brown <david.brown@hesbynett.no> - 2025-01-02 07:56 +0100
Re: We have a new standard! Tim Rentsch <tr.17687@z991.linuxsc.com> - 2025-01-02 00:39 -0800
Re: We have a new standard! Paavo Helde <eesnimi@osa.pri.ee> - 2025-01-02 12:31 +0200
Re: We have a new standard! Michael S <already5chosen@yahoo.com> - 2025-01-02 14:59 +0200
Re: We have a new standard! Paavo Helde <eesnimi@osa.pri.ee> - 2025-01-02 15:50 +0200
Re: We have a new standard! Tim Rentsch <tr.17687@z991.linuxsc.com> - 2025-01-07 21:15 -0800
Re: We have a new standard! Paavo Helde <eesnimi@osa.pri.ee> - 2025-01-08 08:44 +0200
Re: We have a new standard! Michael S <already5chosen@yahoo.com> - 2025-01-08 11:44 +0200
Re: We have a new standard! James Kuyper <jameskuyper@alumni.caltech.edu> - 2025-01-08 15:04 -0500
Re: We have a new standard! Tim Rentsch <tr.17687@z991.linuxsc.com> - 2025-01-14 08:49 -0800
Re: We have a new standard! Tim Rentsch <tr.17687@z991.linuxsc.com> - 2025-01-14 09:12 -0800
Re: We have a new standard! Paavo Helde <eesnimi@osa.pri.ee> - 2025-01-14 19:53 +0200
Re: We have a new standard! "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-01-08 14:43 -0800
Re: We have a new standard! scott@slp53.sl.home (Scott Lurndal) - 2025-01-02 20:51 +0000
Re: We have a new standard! Michael S <already5chosen@yahoo.com> - 2025-01-02 16:47 +0200
Page 4 of 4 — ← Prev page 1 2 3 [4]
| From | Paavo Helde <eesnimi@osa.pri.ee> |
|---|---|
| Date | 2025-01-02 12:31 +0200 |
| Message-ID | <vl5pte$3a2v3$1@dont-email.me> |
| In reply to | #121366 |
On 02.01.2025 10:39, Tim Rentsch wrote: > Paavo Helde <eesnimi@osa.pri.ee> writes: > > [...] > >> Half of the slowness of both in C printf() and C++ streams comes >> from the locale support, [...] > > I find this assertion very hard to believe (specifically, for > printf(); I might guess that C++ streams have similar > performance characteristics, but I don't have nearly as much > experience with C++ streams as I do with printf()). > > Is there anything else you can say about it besides an > unsupported assertion? Probably you were misled by my vague word usage. I said "half of the slowness", not "half of the total time". As "slowness" is nowhere exactly defined, this was just a figure of speech, no need to get overly upset about that. Anyway, I have profiled the writing of large CSV tables (multi-GB) which was deemed too slow. 90% of time was spent in sprintf() calls, from which 10% was spent in locale-related routines, presumably for checking that the global process locale has not changed during the last microsecond. Considering that our CSV file format had to always use the fixed C-locale format anyway, this locale checking was something which was absolutely unneeded. We tried to parallelize the writing, expecting to increase the speed N times by using N threads, but alas, as the locale is a global thread-shared object, now the threads started to fight over it and the performance did not scale nowhere near to our expectations. (The thread-specific locales were not available an all needed platforms). We resolved it finally by using std::to_chars() for integer data and Google's double-conversion library for floating-point data (our C++ implementations did not support std::to_chars() on floating-point data on all needed platforms at that time).
[toc] | [prev] | [next] | [standalone]
| From | Michael S <already5chosen@yahoo.com> |
|---|---|
| Date | 2025-01-02 14:59 +0200 |
| Message-ID | <20250102145952.00000d83@yahoo.com> |
| In reply to | #121368 |
On Thu, 2 Jan 2025 12:31:08 +0200 Paavo Helde <eesnimi@osa.pri.ee> wrote: > > We resolved it finally by using std::to_chars() for integer data and > Google's double-conversion library for floating-point data (our C++ > implementations did not support std::to_chars() on floating-point > data on all needed platforms at that time). > What do you consider "resolved" ? Suppose, we write to modern M.2 SSD. Not top speed, like Corsair MP700 Pro, but something more modest, like Samsung 990 Pro. Sequential write speed of this popular SSD is close to 7 GB/s. Is the problem considered "resolved" when you are able to write your CVS at 2 GB/s, i.e. ~30% of what is theoretically possible? Me thinks, not quite. Client that paid for SSD would still be disappointed.
[toc] | [prev] | [next] | [standalone]
| From | Paavo Helde <eesnimi@osa.pri.ee> |
|---|---|
| Date | 2025-01-02 15:50 +0200 |
| Message-ID | <vl65j5$3c5s7$1@dont-email.me> |
| In reply to | #121370 |
On 02.01.2025 14:59, Michael S wrote: > On Thu, 2 Jan 2025 12:31:08 +0200 > Paavo Helde <eesnimi@osa.pri.ee> wrote: >> >> We resolved it finally by using std::to_chars() for integer data and >> Google's double-conversion library for floating-point data (our C++ >> implementations did not support std::to_chars() on floating-point >> data on all needed platforms at that time). >> > > What do you consider "resolved" ? > Suppose, we write to modern M.2 SSD. Not top speed, like Corsair MP700 > Pro, but something more modest, like Samsung 990 Pro. Sequential write > speed of this popular SSD is close to 7 GB/s. > Is the problem considered "resolved" when you are able to write your > CVS at 2 GB/s, i.e. ~30% of what is theoretically possible? > Me thinks, not quite. Client that paid for SSD would still be > disappointed. The case is resolved when it gets closed in the issue tracker :-) And there is always the law of diminishing returns, meaning that there is only so much effort which we can spend on a specific case. A performance case like that gets closed when I say I do not see any further feasible possibilities for improvement, and my team leader agrees. If the file writing is still perceived as too slow, then one would need to consider radically different alternatives (which would be then tracked as new cases in the issue tracker ;-) In this case of CSV serialization, an obvious speedup would be to write binary files instead of text files, but this would complicate interoperability and disrupt all existing data processing pipelines involving both multiple departments in our company and also our customers, so most probably this would never happen. In other more contained situations, what we have done e.g. in the past, was to use shared memory instead of HTTP downloads for file transfer from one process to another. I suspect the HTTP slowness was actually caused by some overeager antivirus scanner, but I think they never figured it out for sure. And finally, if all else fails, then it will be the task for the customer support to explain the customers that processing their always growing amounts of data needs some actual work and does not happen instantly.
[toc] | [prev] | [next] | [standalone]
| From | Tim Rentsch <tr.17687@z991.linuxsc.com> |
|---|---|
| Date | 2025-01-07 21:15 -0800 |
| Message-ID | <86a5c1antv.fsf@linuxsc.com> |
| In reply to | #121368 |
Paavo Helde <eesnimi@osa.pri.ee> writes: > On 02.01.2025 10:39, Tim Rentsch wrote: > >> Paavo Helde <eesnimi@osa.pri.ee> writes: >> >> [...] >> >>> Half of the slowness of both in C printf() and C++ streams comes >>> from the locale support, [...] >> >> I find this assertion very hard to believe (specifically, for >> printf(); I might guess that C++ streams have similar >> performance characteristics, but I don't have nearly as much >> experience with C++ streams as I do with printf()). >> >> Is there anything else you can say about it besides an >> unsupported assertion? > > Probably you were misled by my vague word usage. I said "half of the > slowness", not "half of the total time". As "slowness" is nowhere > exactly defined, this was just a figure of speech, no need to get > overly upset about that. I'm not upset, just disappointed. I consider you one of the better contributors in the newsgroups, and so it's surprising to see a statement from you with zero information content. > Anyway, I have profiled the writing of large CSV tables (multi-GB) > which was deemed too slow. 90% of time was spent in sprintf() calls, > from which 10% was spent in locale-related routines, presumably for > checking that the global process locale has not changed during the > last microsecond. Considering that our CSV file format had to always > use the fixed C-locale format anyway, this locale checking was > something which was absolutely unneeded. The 10% number is interesting, and disappointing. Surely the authors of the standard library implementation could have done better. Incidentally, I did a measurement of the cost of localeconv() on my test server (running Ubuntu), and that gave a result in line with your 10% number. > We tried to parallelize the writing, expecting to increase the speed N > times by using N threads, but alas, as the locale is a global > thread-shared object, now the threads started to fight over it and the > performance did not scale nowhere near to our expectations. (The > thread-specific locales were not available an all needed platforms). Does the C standard (or the C++ standard) allow locale information to be thread-specific? I don't see anything in the C standard that looks like the <locale.h> functions can be anything but global. I haven't looked at what C++ allows. > We resolved it finally by using std::to_chars() for integer data and > Google's double-conversion library for floating-point data (our C++ > implementations did not support std::to_chars() on floating-point data > on all needed platforms at that time). It's disappointing that the C standard (and I assume also the C++ standard) doesn't provide conversion functions that don't use global locale information. Oh, come to think of it, I think I saw some C++ functions (in which C++ standard I don't know) that pass locale information in explicitly, which is probably what you were alluding to above.
[toc] | [prev] | [next] | [standalone]
| From | Paavo Helde <eesnimi@osa.pri.ee> |
|---|---|
| Date | 2025-01-08 08:44 +0200 |
| Message-ID | <vll6rn$2lt27$1@dont-email.me> |
| In reply to | #121424 |
On 08.01.2025 07:15, Tim Rentsch wrote: > Paavo Helde <eesnimi@osa.pri.ee> writes: > >> On 02.01.2025 10:39, Tim Rentsch wrote: >> >>> Paavo Helde <eesnimi@osa.pri.ee> writes: >>> >>> [...] >>> >>>> Half of the slowness of both in C printf() and C++ streams comes >>>> from the locale support, [...] >>> >>> I find this assertion very hard to believe (specifically, for >>> printf(); I might guess that C++ streams have similar >>> performance characteristics, but I don't have nearly as much >>> experience with C++ streams as I do with printf()). >>> >>> Is there anything else you can say about it besides an >>> unsupported assertion? >> >> Probably you were misled by my vague word usage. I said "half of the >> slowness", not "half of the total time". As "slowness" is nowhere >> exactly defined, this was just a figure of speech, no need to get >> overly upset about that. > > I'm not upset, just disappointed. I consider you one of the better > contributors in the newsgroups, and so it's surprising to see a > statement from you with zero information content. > >> Anyway, I have profiled the writing of large CSV tables (multi-GB) >> which was deemed too slow. 90% of time was spent in sprintf() calls, >> from which 10% was spent in locale-related routines, presumably for >> checking that the global process locale has not changed during the >> last microsecond. Considering that our CSV file format had to always >> use the fixed C-locale format anyway, this locale checking was >> something which was absolutely unneeded. > > The 10% number is interesting, and disappointing. Surely the authors > of the standard library implementation could have done better. > Incidentally, I did a measurement of the cost of localeconv() on my > test server (running Ubuntu), and that gave a result in line with > your 10% number. > >> We tried to parallelize the writing, expecting to increase the speed N >> times by using N threads, but alas, as the locale is a global >> thread-shared object, now the threads started to fight over it and the >> performance did not scale nowhere near to our expectations. (The >> thread-specific locales were not available an all needed platforms). > > Does the C standard (or the C++ standard) allow locale information to > be thread-specific? I don't see anything in the C standard that looks > like the <locale.h> functions can be anything but global. I haven't > looked at what C++ allows. _configthreadlocale() is a MS extension. "https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/configthreadlocale" > >> We resolved it finally by using std::to_chars() for integer data and >> Google's double-conversion library for floating-point data (our C++ >> implementations did not support std::to_chars() on floating-point data >> on all needed platforms at that time). > > It's disappointing that the C standard (and I assume also the C++ > standard) doesn't provide conversion functions that don't use global > locale information. Oh, come to think of it, I think I saw some C++ > functions (in which C++ standard I don't know) that pass locale > information in explicitly, which is probably what you were alluding > to above. MSVC has _sprintf_l() which takes a locale explicitly, but this is again a non-standard extension. Regardless if the locale is passed in implicitly or explicitly, the function would still need to process it, which might take non-zero time. We were aiming to use functions which would not consider locales at all. C++ actually delivers that with std::to_chars() since C++17. I have not measured how fast it is nowadays, compared to Google's double-conversion. Fast *and* accurate printing of floating-point numbers is not a trivial task. AFAIK last significant advances in this area were done as late as in 2010. See e.g. "http://www.serpentine.com/blog/2011/06/29/here-be-dragons-advances-in-problems-you-didnt-even-know-you-had/"
[toc] | [prev] | [next] | [standalone]
| From | Michael S <already5chosen@yahoo.com> |
|---|---|
| Date | 2025-01-08 11:44 +0200 |
| Message-ID | <20250108114450.000028dd@yahoo.com> |
| In reply to | #121425 |
On Wed, 8 Jan 2025 08:44:05 +0200 Paavo Helde <eesnimi@osa.pri.ee> wrote: > > Fast *and* accurate printing of floating-point numbers is not a > trivial task. AFAIK last significant advances in this area were done > as late as in 2010. See e.g. > "http://www.serpentine.com/blog/2011/06/29/here-be-dragons-advances-in-problems-you-didnt-even-know-you-had/" > I didn't read it, but I want to point that even the question of what is 'accurate' is not trivial when we consider conversion in non-default rounding modes. I am not sure that C++ standard contains sufficient information about it.
[toc] | [prev] | [next] | [standalone]
| From | James Kuyper <jameskuyper@alumni.caltech.edu> |
|---|---|
| Date | 2025-01-08 15:04 -0500 |
| Message-ID | <vlmloj$2t9n2$3@dont-email.me> |
| In reply to | #121426 |
On 1/8/25 04:44, Michael S wrote: > On Wed, 8 Jan 2025 08:44:05 +0200 > Paavo Helde <eesnimi@osa.pri.ee> wrote: > >> >> Fast *and* accurate printing of floating-point numbers is not a >> trivial task. AFAIK last significant advances in this area were done >> as late as in 2010. See e.g. >> "http://www.serpentine.com/blog/2011/06/29/here-be-dragons-advances-in-problems-you-didnt-even-know-you-had/" >> > > I didn't read it, but I want to point that even the question of what is > 'accurate' is not trivial when we consider conversion in > non-default rounding modes. I am not sure that C++ standard contains > sufficient information about it. Printing of floating-point values in C++ traces to to_chars(), which is defined as converting the value to a string in accordance with the C rules for printf(). The C standard says, for printf(), in the "Recommended practice" section, that "f the number of significant decimal digits is at most the maximum value M of the T_DECIMAL_DIG macros (defined in <float.h>), then the result should be correctly rounded.346) If the number of significant decimal digits is more than M but the source value is exactly representable with M digits, then the result should be an exact representation with trailing zeros. Otherwise, the source value is bounded by two adjacent decimal strings L < U, both having M significant digits; the value of the resultant decimal string D should satisfy L ≤ D ≤ U, with the extra stipulation that the error should have a correct sign for the current rounding direction." (7.23.6.1p13). Note, however, that this is only recommended practice, not a requirement. Here's what it says about the minimum requirements: "The accuracy of the floating-point operations (+, -, *, /) and of most of the library functions in <math.h> and <complex.h> that return floating-point results is implementation-defined, as is the accuracy of the conversion between floating-point internal representations and string representations performed by the library functions in <stdio.h>, <stdlib.h>, and <wchar.h>. The implementation may state that the accuracy is unknown." (5.2.4.2.2p14) "conversion between floating-point internal representations and string representations" is what you're talking about. Yes, that means that a fully conforming implementation of C++ may implement floating point subtraction with such poor precision that the expression LDBL_MAX-LDBL_MIN < LDBL_MIN - LDBL_MAX will evaluate to true, just so long as they document the fact that it is in fact that inaccurate. An implementation that pre#defines __STDC_IEC_60559_BFP__ is required to conform to ISO/IEC 60559 (equivalent to IEEE 754) requirements for the precision of such conversions, which are much stricter.
[toc] | [prev] | [next] | [standalone]
| From | Tim Rentsch <tr.17687@z991.linuxsc.com> |
|---|---|
| Date | 2025-01-14 08:49 -0800 |
| Message-ID | <86h6615oje.fsf@linuxsc.com> |
| In reply to | #121426 |
Michael S <already5chosen@yahoo.com> writes: > On Wed, 8 Jan 2025 08:44:05 +0200 > Paavo Helde <eesnimi@osa.pri.ee> wrote: > >> Fast *and* accurate printing of floating-point numbers is not a >> trivial task. AFAIK last significant advances in this area were done >> as late as in 2010. See e.g. >> "http://www.serpentine.com/blog/2011/06/29/ >> here-be-dragons-advances-in-problems-you-didnt-even-know-you-had/" > > I didn't read it, but I want to point that even the question of what is > 'accurate' is not trivial when we consider conversion in > non-default rounding modes. I am not sure that C++ standard contains > sufficient information about it. IIANM the default rounding mode (at least in C) is round to nearest, and even just that is not trivial. (For michael: I sent you an email about the start of 2025; no reply as yet.)
[toc] | [prev] | [next] | [standalone]
| From | Tim Rentsch <tr.17687@z991.linuxsc.com> |
|---|---|
| Date | 2025-01-14 09:12 -0800 |
| Message-ID | <86cygp5ngg.fsf@linuxsc.com> |
| In reply to | #121425 |
Paavo Helde <eesnimi@osa.pri.ee> writes: > On 08.01.2025 07:15, Tim Rentsch wrote: > >> Paavo Helde <eesnimi@osa.pri.ee> writes: >> >>> On 02.01.2025 10:39, Tim Rentsch wrote: >>> >>>> Paavo Helde <eesnimi@osa.pri.ee> writes: >>>> >>>> [...] >>>> >>>>> Half of the slowness of both in C printf() and C++ streams comes >>>>> from the locale support, [...] >>>> >>>> I find this assertion very hard to believe (specifically, for >>>> printf(); I might guess that C++ streams have similar >>>> performance characteristics, but I don't have nearly as much >>>> experience with C++ streams as I do with printf()). >>>> >>>> Is there anything else you can say about it besides an >>>> unsupported assertion? >>> >>> Probably you were misled by my vague word usage. I said "half of the >>> slowness", not "half of the total time". As "slowness" is nowhere >>> exactly defined, this was just a figure of speech, no need to get >>> overly upset about that. >> >> I'm not upset, just disappointed. I consider you one of the better >> contributors in the newsgroups, and so it's surprising to see a >> statement from you with zero information content. >> >>> Anyway, I have profiled the writing of large CSV tables (multi-GB) >>> which was deemed too slow. 90% of time was spent in sprintf() calls, >>> from which 10% was spent in locale-related routines, presumably for >>> checking that the global process locale has not changed during the >>> last microsecond. Considering that our CSV file format had to always >>> use the fixed C-locale format anyway, this locale checking was >>> something which was absolutely unneeded. >> >> The 10% number is interesting, and disappointing. Surely the authors >> of the standard library implementation could have done better. >> Incidentally, I did a measurement of the cost of localeconv() on my >> test server (running Ubuntu), and that gave a result in line with >> your 10% number. >> >>> We tried to parallelize the writing, expecting to increase the speed N >>> times by using N threads, but alas, as the locale is a global >>> thread-shared object, now the threads started to fight over it and the >>> performance did not scale nowhere near to our expectations. (The >>> thread-specific locales were not available an all needed platforms). >> >> Does the C standard (or the C++ standard) allow locale information to >> be thread-specific? I don't see anything in the C standard that looks >> like the <locale.h> functions can be anything but global. I haven't >> looked at what C++ allows. > > _configthreadlocale() is a MS extension. > > "https://learn.microsoft.com/en-us/cpp/ > c-runtime-library/reference/configthreadlocale" > >>> We resolved it finally by using std::to_chars() for integer data and >>> Google's double-conversion library for floating-point data (our C++ >>> implementations did not support std::to_chars() on floating-point data >>> on all needed platforms at that time). >> >> It's disappointing that the C standard (and I assume also the C++ >> standard) doesn't provide conversion functions that don't use global >> locale information. Oh, come to think of it, I think I saw some C++ >> functions (in which C++ standard I don't know) that pass locale >> information in explicitly, which is probably what you were alluding >> to above. > > MSVC has _sprintf_l() which takes a locale explicitly, but this is > again a non-standard extension. > > Regardless if the locale is passed in implicitly or explicitly, the > function would still need to process it, which might take non-zero > time. I'm pretty sure the amount of locale processing needed by *printf() functions is minimal, and most of the cost is the effort needed to put together the whole locale information structure, most of which is not used. > We were aiming to use functions which would not consider locales > at all. Understood. And there may be some other *printf() functionality that you don't need either. The *printf() functions are useful, but the full generality is quite a terror. > C++ actually delivers that with std::to_chars() since C++17. I have > not measured how fast it is nowadays, compared to Google's > double-conversion. > > Fast *and* accurate printing of floating-point numbers is not a > trivial task. AFAIK last significant advances in this area were done > as late as in 2010. See > e.g. "http://www.serpentine.com/blog/2011/06/29/ > here-be-dragons-advances-in-problems-you-didnt-even-know-you-had/" Yes, I am well aware of the difficulties of formatting floating point output. Does your output need to be human readable, or is program readable good enough? If the consumers can deal with non-decimal formatting, you could use %a or %A, which is easier and faster to produce.
[toc] | [prev] | [next] | [standalone]
| From | Paavo Helde <eesnimi@osa.pri.ee> |
|---|---|
| Date | 2025-01-14 19:53 +0200 |
| Message-ID | <vm68as$2gvle$1@dont-email.me> |
| In reply to | #121436 |
On 14.01.2025 19:12, Tim Rentsch wrote: > > Does your output need to be human readable, or is program readable > good enough? If the consumers can deal with non-decimal formatting, > you could use %a or %A, which is easier and faster to produce. As the files are sometimes gigabytes in size, for sure they are for programmatic access mostly. But this does not mean that a human might not want to take a first look at the data, or to study portions of it. About consumers, this would include various usage both in-house and at customer sites. To be honest I think the whole diaspora of consumers is not known to anybody. The most advanced common format which we were able to use is tab-separated ASCII text, misnamed as a .csv file (IIRC .tsv was not acceptable by some reason in some scenarios). The CSV/TSV format is over 40 years old. Anyway, I just tried to import 0x1.91eb86p+1 into Excel and it failed to recognize this as a number, so obviously the %a format would not fly. Maybe in another 40 years...
[toc] | [prev] | [next] | [standalone]
| From | "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> |
|---|---|
| Date | 2025-01-08 14:43 -0800 |
| Message-ID | <vlmv2e$3023s$4@dont-email.me> |
| In reply to | #121424 |
On 1/7/2025 9:15 PM, Tim Rentsch wrote: > Paavo Helde <eesnimi@osa.pri.ee> writes: > >> On 02.01.2025 10:39, Tim Rentsch wrote: >> >>> Paavo Helde <eesnimi@osa.pri.ee> writes: >>> >>> [...] >>> >>>> Half of the slowness of both in C printf() and C++ streams comes >>>> from the locale support, [...] >>> >>> I find this assertion very hard to believe (specifically, for >>> printf(); I might guess that C++ streams have similar >>> performance characteristics, but I don't have nearly as much >>> experience with C++ streams as I do with printf()). >>> >>> Is there anything else you can say about it besides an >>> unsupported assertion? >> >> Probably you were misled by my vague word usage. I said "half of the >> slowness", not "half of the total time". As "slowness" is nowhere >> exactly defined, this was just a figure of speech, no need to get >> overly upset about that. > > I'm not upset, just disappointed. I consider you one of the better > contributors in the newsgroups, and so it's surprising to see a > statement from you with zero information content. > >> Anyway, I have profiled the writing of large CSV tables (multi-GB) >> which was deemed too slow. 90% of time was spent in sprintf() calls, >> from which 10% was spent in locale-related routines, presumably for >> checking that the global process locale has not changed during the >> last microsecond. Considering that our CSV file format had to always >> use the fixed C-locale format anyway, this locale checking was >> something which was absolutely unneeded. > > The 10% number is interesting, and disappointing. Surely the authors > of the standard library implementation could have done better. > Incidentally, I did a measurement of the cost of localeconv() on my > test server (running Ubuntu), and that gave a result in line with > your 10% number. > >> We tried to parallelize the writing, expecting to increase the speed N >> times by using N threads, but alas, as the locale is a global >> thread-shared object, now the threads started to fight over it and the >> performance did not scale nowhere near to our expectations. (The >> thread-specific locales were not available an all needed platforms). > > Does the C standard (or the C++ standard) allow locale information to > be thread-specific? I don't see anything in the C standard that looks > like the <locale.h> functions can be anything but global. I haven't > looked at what C++ allows. C: https://en.cppreference.com/w/c/thread/tss_set https://en.cppreference.com/w/c/thread/tss_get C++: Either use C, or perhaps: thread_local https://en.cppreference.com/w/cpp/language/storage_duration > >> We resolved it finally by using std::to_chars() for integer data and >> Google's double-conversion library for floating-point data (our C++ >> implementations did not support std::to_chars() on floating-point data >> on all needed platforms at that time). > > It's disappointing that the C standard (and I assume also the C++ > standard) doesn't provide conversion functions that don't use global > locale information. Oh, come to think of it, I think I saw some C++ > functions (in which C++ standard I don't know) that pass locale > information in explicitly, which is probably what you were alluding > to above.
[toc] | [prev] | [next] | [standalone]
| From | scott@slp53.sl.home (Scott Lurndal) |
|---|---|
| Date | 2025-01-02 20:51 +0000 |
| Message-ID | <1%CdP.49607$HxS1.31348@fx39.iad> |
| In reply to | #121362 |
Paavo Helde <eesnimi@osa.pri.ee> writes: >On 01.01.2025 18:25, Michael S wrote: >> On 27 Dec 2024 14:47:51 GMT >> ram@zedat.fu-berlin.de (Stefan Ram) wrote: >> >> Nevertheless, it is a step in right direction. >> Introduction of format() already showed that C++ committee is aware of >> of the fact that "Stroustrup streams" are crap not only relatively to >> format/printing facilities of more modern languages, but relatively >> to what we have in C as well. std::print() proves that committee is not >> only aware of the fact, but finally willing to consider fixes. > >Half of the slowness of both in C printf() and C++ streams comes from Don't lump them together. C printf/snprintf is rather efficient when using the POSIX C locale. Faster by orders of magnitude when comparing with C++ output streams. C++ streams, on the other hand, cannot be made efficient without completely replacing the paradigm. To that end, std::print is a start - although being built on the standard C++ streams paradigm inevitiably leads to sub-optimal performance.
[toc] | [prev] | [next] | [standalone]
| From | Michael S <already5chosen@yahoo.com> |
|---|---|
| Date | 2025-01-02 16:47 +0200 |
| Message-ID | <20250102164750.00000977@yahoo.com> |
| In reply to | #121358 |
On Wed, 1 Jan 2025 18:25:27 +0200 Michael S <already5chosen@yahoo.com> wrote: > > Now to why, despite said above, I wouldn't use std::print() in its > current incarnation neither in production nor in hobby programs: > because compilation is too slow. ~4 seconds on the old home PC. I > didn't try on newer machines yet, but would be surprised if any of > them beats 2 seconds. Which is way above my threshold of > inconvenience. > More experiments showed that slowness of compilation is caused by C++20 part (format()) rather than by C++23 addition. It's somewhat less bad with -O1, instead of original -O2 and yet less bad with -Og or -O0. But even for -O0 compilation takes over 2 sec. Still above my personal threshold for annoyance.
[toc] | [prev] | [standalone]
Page 4 of 4 — ← Prev page 1 2 3 [4]
Back to top | Article view | comp.lang.c++
csiph-web