Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #126692 > unrolled thread

Text Files

Back to article view | Back to comp.lang.c

Contents

  Text Files G B <gb@gb.com> - 2018-02-14 11:47 -0400
    Re: Text Files Ed Prochak <edprochak@gmail.com> - 2018-02-14 08:45 -0800
    Re: Text Files Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-02-14 08:47 -0800
      Re: Text Files already5chosen@yahoo.com - 2018-02-15 01:17 -0800
    Re: Text Files gordonb.ea2mq@burditt.org (Gordon Burditt) - 2018-02-14 11:10 -0600
    Re: Text Files "James R. Kuyper" <jameskuyper@verizon.net> - 2018-02-14 12:28 -0500
    Re: Text Files jt@toerring.de (Jens Thoms Toerring) - 2018-02-14 21:07 +0000
    Re: Text Files Joe Pfeiffer <pfeiffer@cs.nmsu.edu> - 2018-02-14 15:31 -0700
      Re: Text Files Jorgen Grahn <grahn+nntp@snipabacken.se> - 2018-02-15 06:59 +0000
        Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-15 10:20 +0000
    Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-14 22:51 +0000
      Re: Text Files David Brown <david.brown@hesbynett.no> - 2018-02-15 11:04 +0100
        Re: Text Files Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-02-15 06:16 -0800
          Re: Text Files "Mr. Man-wai Chang" <toylet.toylet@gmail.com> - 2018-02-15 22:22 +0800
          Re: Text Files David Brown <david.brown@hesbynett.no> - 2018-02-15 15:58 +0100
            Re: Text Files bartc <bc@freeuk.com> - 2018-02-15 16:31 +0000
              Re: Text Files David Brown <david.brown@hesbynett.no> - 2018-02-15 21:31 +0100
                Re: Text Files bartc <bc@freeuk.com> - 2018-02-15 21:21 +0000
                  Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-15 22:17 +0000
                    Re: Text Files bartc <bc@freeuk.com> - 2018-02-15 23:36 +0000
                      Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-15 23:48 +0000
                  Re: Text Files David Brown <david.brown@hesbynett.no> - 2018-02-16 10:09 +0100
                    Re: Text Files bartc <bc@freeuk.com> - 2018-02-16 12:57 +0000
                      Re: Text Files David Brown <david.brown@hesbynett.no> - 2018-02-16 15:50 +0100
                        Re: Text Files bartc <bc@freeuk.com> - 2018-02-16 15:50 +0000
                          Re: Text Files David Brown <david.brown@hesbynett.no> - 2018-02-16 17:07 +0100
                            Re: Text Files bartc <bc@freeuk.com> - 2018-02-16 20:44 +0000
                              Re: Text Files Ian Collins <ian-news@hotmail.com> - 2018-02-17 11:22 +1300
                          Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-16 17:48 +0000
                        Re: Text Files GOTHIER Nathan <nathan.gothier@gmail.com> - 2018-02-16 17:06 +0100
                      Re: Text Files Ian Collins <ian-news@hotmail.com> - 2018-02-17 11:10 +1300
                      Re: Text Files supercat@casperkitty.com - 2018-02-16 14:48 -0800
                        Re: Text Files Keith Thompson <kst-u@mib.org> - 2018-02-16 15:14 -0800
                          Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-16 23:25 +0000
                          Re: Text Files supercat@casperkitty.com - 2018-02-16 15:50 -0800
            Re: Text Files already5chosen@yahoo.com - 2018-02-16 07:19 -0800
              Re: Text Files David Brown <david.brown@hesbynett.no> - 2018-02-16 17:13 +0100
                Re: Text Files Jorgen Grahn <grahn+nntp@snipabacken.se> - 2018-02-19 17:55 +0000
                  Re: Text Files David Brown <david.brown@hesbynett.no> - 2018-02-19 19:14 +0100
                  Re: Text Files Keith Thompson <kst-u@mib.org> - 2018-02-19 11:48 -0800
              Re: Text Files Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-02-16 08:14 -0800
                Re: Text Files Robert Wessel <robertwessel2@yahoo.com> - 2018-02-17 18:32 -0600
      Re: Text Files jacobnavia <jacob@jacob.remcomp.fr> - 2018-02-15 22:57 +0100
        Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-15 22:58 +0000
          Re: Text Files bartc <bc@freeuk.com> - 2018-02-16 00:19 +0000
            Re: Text Files bartc <bc@freeuk.com> - 2018-02-16 02:26 +0000
              Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-16 02:57 +0000
                Re: Text Files asetofsymbols@gmail.com - 2018-02-16 00:21 -0800
                  Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-16 11:30 +0000
            Re: Text Files David Brown <david.brown@hesbynett.no> - 2018-02-16 10:36 +0100
              Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-16 11:36 +0000
                Re: Text Files David Brown <david.brown@hesbynett.no> - 2018-02-16 13:10 +0100
                  Re: Text Files bartc <bc@freeuk.com> - 2018-02-16 12:25 +0000
                    Re: Text Files Ike Naar <ike@iceland.freeshell.org> - 2018-03-06 19:33 +0000
                      Re: Text Files bartc <bc@freeuk.com> - 2018-03-06 20:55 +0000
                  Re: Text Files Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-02-16 04:42 -0800
                    Re: Text Files David Brown <david.brown@hesbynett.no> - 2018-02-16 16:58 +0100
                      Re: Text Files bartc <bc@freeuk.com> - 2018-02-16 20:02 +0000
                        Re: Text Files David Brown <david.brown@hesbynett.no> - 2018-02-16 21:43 +0100
                      Re: Text Files Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-02-17 04:01 -0800
                        Re: Text Files bartc <bc@freeuk.com> - 2018-02-17 13:10 +0000
                          Re: Text Files Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-02-17 05:27 -0800
                            Re: Text Files bartc <bc@freeuk.com> - 2018-02-17 14:20 +0000
                    Re: Text Files Joe Pfeiffer <pfeiffer@cs.nmsu.edu> - 2018-02-16 09:04 -0700
                  Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-16 14:31 +0000
                    Re: Text Files Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-02-16 06:42 -0800
                      Re: Text Files bartc <bc@freeuk.com> - 2018-02-16 14:58 +0000
                        Re: Text Files Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-02-16 07:23 -0800
                      Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-16 15:24 +0000
                Re: Text Files Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-02-16 04:13 -0800
                  Re: Text Files Siri Cruise <chine.bleu@yahoo.com> - 2018-02-16 04:29 -0800
                    Re: Text Files Jorgen Grahn <grahn+nntp@snipabacken.se> - 2018-02-20 16:41 +0000
                      Re: Text Files Siri Cruise <chine.bleu@yahoo.com> - 2018-02-20 09:39 -0800
                        Re: Text Files Jorgen Grahn <grahn+nntp@snipabacken.se> - 2018-02-20 19:16 +0000
    Re: Text Files bartc <bc@freeuk.com> - 2018-02-15 12:46 +0000
      Re: Text Files GOTHIER Nathan <nathan.gothier@gmail.com> - 2018-02-15 14:52 +0100
      Re: Text Files Anton Shepelev <anton.txt@g{oogle}mail.com> - 2018-02-15 17:07 +0300
        Re: Text Files bartc <bc@freeuk.com> - 2018-02-15 14:23 +0000
          Re: Text Files Anton Shepelev <anton.txt@g{oogle}mail.com> - 2018-02-15 18:52 +0300
      Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-15 14:39 +0000
        Re: Text Files bartc <bc@freeuk.com> - 2018-02-15 15:43 +0000
          Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-15 20:32 +0000
            Re: Text Files bartc <bc@freeuk.com> - 2018-02-15 22:06 +0000
              Re: Text Files GOTHIER Nathan <nathan.gothier@gmail.com> - 2018-02-15 23:12 +0100
              Re: Text Files Keith Thompson <kst-u@mib.org> - 2018-02-15 15:12 -0800
                Re: Text Files bartc <bc@freeuk.com> - 2018-02-15 23:43 +0000
                  Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-15 23:54 +0000
              Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-15 23:35 +0000
            Re: Text Files Tim Rentsch <txr@alumni.caltech.edu> - 2018-02-16 17:07 -0800
              Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-17 02:38 +0000
                Re: Text Files Ian Collins <ian-news@hotmail.com> - 2018-02-17 17:26 +1300
                  Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-17 11:22 +0000
                  Re: Text Files bartc <bc@freeuk.com> - 2018-02-17 12:07 +0000
                    Re: Text Files Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-02-17 04:17 -0800
                      Re: Text Files Anton Shepelev <anton.txt@gmail.com> - 2018-02-17 16:25 +0300
                        Re: Text Files Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-02-17 07:35 -0800
                      Re: Text Files GOTHIER Nathan <nathan.gothier@gmail.com> - 2018-02-17 15:24 +0100
                    Re: Text Files Ian Collins <ian-news@hotmail.com> - 2018-02-18 11:10 +1300
                Re: Text Files Tim Rentsch <txr@alumni.caltech.edu> - 2018-02-19 12:05 -0800
                  Re: Text Files already5chosen@yahoo.com - 2018-02-19 15:32 -0800
                    Re: Text Files Tim Rentsch <txr@alumni.caltech.edu> - 2018-02-21 10:34 -0800
                      Re: Text Files already5chosen@yahoo.com - 2018-02-21 15:35 -0800
                        Re: Text Files bartc <bc@freeuk.com> - 2018-02-22 00:11 +0000
                      Re: Text Files Robert Wessel <robertwessel2@yahoo.com> - 2018-02-21 18:26 -0600
                        Re: Text Files Tim Rentsch <txr@alumni.caltech.edu> - 2018-02-21 21:39 -0800
                          Re: Text Files Robert Wessel <robertwessel2@yahoo.com> - 2018-02-23 03:08 -0600
                            Re: Text Files supercat@casperkitty.com - 2018-02-23 16:16 -0800
                              Re: Text Files Keith Thompson <kst-u@mib.org> - 2018-02-23 16:46 -0800
                                Re: Text Files Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-02-24 02:49 -0800
                                  Re: Text Files asetofsymbols@gmail.com - 2018-02-24 05:16 -0800
                                    Re: Text Files "F. Russell" <fr@random.info> - 2018-02-24 14:35 +0000
                                      Re: Text Files Joe Pfeiffer <pfeiffer@cs.nmsu.edu> - 2018-02-24 08:48 -0700
                                Re: Text Files supercat@casperkitty.com - 2018-02-25 15:09 -0800
                                  Re: Text Files Keith Thompson <kst-u@mib.org> - 2018-02-25 15:54 -0800
                                    Re: Text Files supercat@casperkitty.com - 2018-02-26 11:04 -0800
                                      Re: Text Files Keith Thompson <kst-u@mib.org> - 2018-02-26 11:52 -0800
                                    Re: Text Files supercat@casperkitty.com - 2018-02-26 14:12 -0800
                                      Re: Text Files Keith Thompson <kst-u@mib.org> - 2018-02-26 14:40 -0800
                        Re: Text Files already5chosen@yahoo.com - 2018-02-22 04:58 -0800
                          Re: Text Files Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-02-22 05:33 -0800
                          Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-22 13:59 +0000
                            Re: Text Files Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-02-22 07:24 -0800
                Re: Text Files supercat@casperkitty.com - 2018-02-20 12:14 -0800
          Re: Text Files David Brown <david.brown@hesbynett.no> - 2018-02-15 21:37 +0100
            Re: Text Files bartc <bc@freeuk.com> - 2018-02-15 21:34 +0000
              Re: Text Files Keith Thompson <kst-u@mib.org> - 2018-02-15 14:15 -0800
              Re: Text Files David Brown <david.brown@hesbynett.no> - 2018-02-16 10:47 +0100
    Re: Text Files "Mr. Man-wai Chang" <toylet.toylet@gmail.com> - 2018-02-15 22:20 +0800
    Re: Text Files "F. Russell" <fr@random.info> - 2018-02-15 15:24 +0000
      Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-15 16:05 +0000
      Re: Text Files "Mr. Man-wai Chang" <toylet.toylet@gmail.com> - 2018-02-16 00:22 +0800
      Re: Text Files Joe Pfeiffer <pfeiffer@cs.nmsu.edu> - 2018-02-15 13:35 -0700
    Re: Text Files Anton Shepelev <anton.txt@gmail.com> - 2018-02-25 00:13 +0300
      Re: Text Files bartc <bc@freeuk.com> - 2018-02-24 21:24 +0000
        Re: Text Files Anton Shepelev <anton.txt@gmail.com> - 2018-02-25 14:55 +0300
          Re: Text Files bartc <bc@freeuk.com> - 2018-02-25 13:00 +0000
      Re: Text Files Real Troll <Real.Troll@Trolls.com> - 2018-02-24 17:35 -0400
      Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-24 21:36 +0000
        Re: Text Files Real Troll <Real.Troll@Trolls.com> - 2018-02-24 17:45 -0400
        Re: Text Files Jorgen Grahn <grahn+nntp@snipabacken.se> - 2018-02-24 22:51 +0000
          Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-24 23:56 +0000
            Re: Text Files bartc <bc@freeuk.com> - 2018-02-25 00:14 +0000
              Re: Text Files bartc <bc@freeuk.com> - 2018-02-25 00:22 +0000
              Re: Text Files Anton Shepelev <anton.txt@gmail.com> - 2018-02-25 15:16 +0300
                Re: Text Files Keith Thompson <kst-u@mib.org> - 2018-02-25 15:20 -0800
                  Re: Text Files Joe Pfeiffer <pfeiffer@cs.nmsu.edu> - 2018-02-25 20:27 -0700
                  Re: Text Files gazelle@shell.xmission.com (Kenny McCormack) - 2018-02-26 06:19 +0000
                  Re: Text Files Anton Shepelev <anton.txt@g{oogle}mail.com> - 2018-02-26 15:35 +0300
                    Re: Text Files Reinhardt Behm <rbehm@hushmail.com> - 2018-02-26 20:53 +0800
                      Grammar Patrol (Was: Text Files) gazelle@shell.xmission.com (Kenny McCormack) - 2018-02-26 17:54 +0000
                    Re: Text Files Jorgen Grahn <grahn+nntp@snipabacken.se> - 2018-02-26 15:27 +0000
                      Re: Text Files Anton Shepelev <anton.txt@g{oogle}mail.com> - 2018-02-26 18:50 +0300
                        Re: Text Files Keith Thompson <kst-u@mib.org> - 2018-02-26 09:26 -0800
                          Re: Text Files scott@slp53.sl.home (Scott Lurndal) - 2018-02-26 19:46 +0000
                          Re: Text Files David Brown <david.brown@hesbynett.no> - 2018-02-27 09:04 +0100
                            Re: Text Files already5chosen@yahoo.com - 2018-02-27 00:56 -0800
                              Re: Text Files Ian Collins <ian-news@hotmail.com> - 2018-02-27 22:07 +1300
                                Re: Text Files David Brown <david.brown@hesbynett.no> - 2018-02-27 14:37 +0100
                                  Re: Text Files already5chosen@yahoo.com - 2018-02-27 07:38 -0800
                                    Re: Text Files supercat@casperkitty.com - 2018-02-27 09:18 -0800
                              Re: Text Files David Brown <david.brown@hesbynett.no> - 2018-02-27 10:15 +0100
                              Re: Text Files Keith Thompson <kst-u@mib.org> - 2018-02-27 08:53 -0800
                        Re: Text Files Tim Rentsch <txr@alumni.caltech.edu> - 2018-02-26 23:14 -0800
                    Re: Text Files Joe Pfeiffer <pfeiffer@cs.nmsu.edu> - 2018-02-26 08:32 -0700
                    Re: Text Files Keith Thompson <kst-u@mib.org> - 2018-02-26 09:17 -0800
                    Re: Text Files Tim Rentsch <txr@alumni.caltech.edu> - 2018-02-26 22:55 -0800
        Re: Text Files Anton Shepelev <anton.txt@gmail.com> - 2018-02-25 15:02 +0300
          Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-25 20:12 +0000

Page 3 of 9 — ← Prev page 1 2 [3] 4 5 6 7 8 9  Next page →

#126802

On Friday, February 16, 2018 at 3:19:34 PM UTC, already...@yahoo.com wrote:
> On Thursday, February 15, 2018 at 4:58:29 PM UTC+2, David Brown wrote:
> >
> > That is all you need.  No faffing about with fgets, or which strtol
> > function to use.  No worrying about sizes, overflows, or how big your
> > buffers should be.  No concerns about line endings, or if the file uses
> > tabs or spaces.  No worries about whether MS in their infinite wisdom
> > have decided to deprecate the C standard library functions you want to
> > use, so that you have to figure out different compiler switches to use.
> 
> When MS deprecate something, the code still compiles and links and runs. It just produces a warning.
> 
No it doesn't. It will refuse to compile unless you define CRC_SECURE_NO_WARNINGS to the preprocessor. Nowadays it prompts that this
is necessary, but originally you had to do a Google search to find out
what to do.
The virus problem on Windows is unfortunately very serious.

[toc] | [prev] | [next] | [standalone]

#126841

On Fri, 16 Feb 2018 08:14:51 -0800 (PST), Malcolm McLean
<malcolm.arthur.mclean@gmail.com> wrote:

>On Friday, February 16, 2018 at 3:19:34 PM UTC, already...@yahoo.com wrote:
>> On Thursday, February 15, 2018 at 4:58:29 PM UTC+2, David Brown wrote:
>> >
>> > That is all you need.  No faffing about with fgets, or which strtol
>> > function to use.  No worrying about sizes, overflows, or how big your
>> > buffers should be.  No concerns about line endings, or if the file uses
>> > tabs or spaces.  No worries about whether MS in their infinite wisdom
>> > have decided to deprecate the C standard library functions you want to
>> > use, so that you have to figure out different compiler switches to use.
>> 
>> When MS deprecate something, the code still compiles and links and runs. It just produces a warning.
>> 
>No it doesn't. It will refuse to compile unless you define CRC_SECURE_NO_WARNINGS to the preprocessor. Nowadays it prompts that this
>is necessary, but originally you had to do a Google search to find out
>what to do.
>The virus problem on Windows is unfortunately very serious.


Unless they've changed it, the "deprecated" functions just throw a
warning in MSVC.  While it's good practice to treat (most) warnings as
something to be fixed, it doesn't actually prevent compilation unless
you also add -Wx to the build.

[toc] | [prev] | [next] | [standalone]

#126758

Le 14/02/2018 à 23:51, Ben Bacarisse a écrit :
> You've had some C answers but I'd raise another question -- why C?

Why C?

Because C allows you a fine control of the machine, algorithms, and has 
all characteristics of a high level language:

o Portability
o Easy to read, once you get used to it... :-)
o Speed

That last one is difficult to beat.

C is fast. Fast because it is just an abstraction of the machine.

C doesn't impose you anything. You can do OO programming with C, real 
time programming, event oriented programming; whatever. C doesn't have 
any pre-conceived way of organizing your data.

Other languages took other choices. For instance, APL, "A Programming 
language" by Iverson.

APL is an interpreted language, that uses greek letters to name the 
basic operations. (Besides +, -, etc).  In APL everything is a vector, 
of several dimensions.

The ideas of Iverson were so radical and beautiful that APL became a 
jewel. But C is more practical, I thought.

Well, for the kind of programming I want to do C is better. APL is not 
compiled and suffers from a heavy interpreter.

Compiled code is faster. Faster so that you can write good APL 
interpreters with it. Many languages are written with C as a back-end. 
Eiffel, for instance.

C is the language to use when you are doing languages precisely because 
it doesn't have any special way of dealing with bits.

What bothers me are the preachers of the good advise here, that do not 
know enough about C.

They say for instance:

<quote>
If this is just a programming exercise, then fine, but if it's "for
real" it's very unlikely that C is going to be the tool for this job.
<end quote>

C has an enormous set of followers in the scientific community. Why? you 
would ask?

Because C is fast.

Before you tell people nonsense here about C not doing the job, I may 
remind you that


<quote>
Currently the programming language that is most commonly used in 
scientific applications is C++, Python, and Julia (which is a newcomer 
but swiftly gaining ground). One of the main reasons behind this choice 
is their high-level abstractions. However, GNU Astronomy Utilities is 
fully written in the C programming language. The reasons can be 
summarized with simplicity, portability and efficiency/speed. All three 
are very important in a scientific software and we will discuss them below.
https://www.gnu.org/software/gnuastro/manual/html_node/Why-C.html
<end quote>

That is exactly the attitude of C people. I am not a C "fan". I use C 
and I think it is a good language, without implying that C is the best 
tool for the job everywhere OF COURSE!

Only if you want:

o SIMPLICITY
o PORTABILITY
o SPEED

If you want that, you use C.

[toc] | [prev] | [next] | [standalone]

#126763

jacobnavia <jacob@jacob.remcomp.fr> writes:

> Le 14/02/2018 à 23:51, Ben Bacarisse a écrit :
>> You've had some C answers but I'd raise another question -- why C?
>
> Why C?
>
> Because C allows you a fine control of the machine, algorithms, and
> has all characteristics of a high level language:
>
> o Portability
> o Easy to read, once you get used to it... :-)
> o Speed
>
> That last one is difficult to beat.

Yes, I know what C's good at, but I was asking the OP.  It's possible
that none of that matters to them.  It could be a quick throwaway
program to find the line with the widest range:

  awk 'NF==6 { if ($6-$1 > r) { r = $6-$1; ln = NR }} END { print ln }'

I'm shameless enough to say that I'm a pretty fast C programmer but,
none the less, I'd write that sort of thing in AWK every time.

> C is fast. Fast because it is just an abstraction of the machine.

I changed my C version to do this calculation and found that the AWK
version was faster.  AWK wins by not having to convert all the fields
from strings to numbers so I had to write a specific version to beat
AWK, one that looked only at the first and last numbers.  That version
wins on speed by a factor of 10 (but it does not check for 6 numbers).

Both versions took way more time to write than that AWK line, but I do
have an advantage here -- I've been using AWK for almost as long as I've
been using C.

<snip>
> What bothers me are the preachers of the good advise here, that do not
> know enough about C.

You often make these sorts of un-targeted remarks.  Who do you mean?
Specifically, who here does know enough about C to give what you call
good advice?

> They say for instance:
>
> <quote>
> If this is just a programming exercise, then fine, but if it's "for
> real" it's very unlikely that C is going to be the tool for this job.
> <end quote>

Ah, I see.  It seems I do not know enough about C.  Oh well, maybe I
will one day...

> Before you tell people nonsense here about C not doing the job,

Specifically, what nonsense did I tell people about C, please?

<snip>
-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#126770

On 15/02/2018 22:58, Ben Bacarisse wrote:
> jacobnavia <jacob@jacob.remcomp.fr> writes:

>> That last one is difficult to beat.
> 
> Yes, I know what C's good at, but I was asking the OP.  It's possible
> that none of that matters to them.  It could be a quick throwaway
> program to find the line with the widest range:
> 
>    awk 'NF==6 { if ($6-$1 > r) { r = $6-$1; ln = NR }} END { print ln }'
> 
> I'm shameless enough to say that I'm a pretty fast C programmer but,
> none the less, I'd write that sort of thing in AWK every time.
> 
>> C is fast. Fast because it is just an abstraction of the machine.
> 
> I changed my C version to do this calculation and found that the AWK
> version was faster.

So, the task is to find the line where column 6 is bigger than column 1 
by the maximum amount?

I tried your code in gawk, on a 1-million line input file containing 6 
columns of random numbers from 1 to 100,000.

Gawk took 1.64 seconds.

Then I threw something together with my scripting language, and it took 
1.50 seconds at the first attempt [code shown in my sig].

Then, since you said that C was slower, I tweaked my demo program of 
this morning, and it did the test in 0.67 seconds.

(Oddly, that was using my own slow C compiler; optimised gcc took 0.88 
seconds.)

All three versions gave the same result: line 289900 out of 100000, 
which was this line:

  27 84309 80881 33587 88056 99998

> AWK wins by not having to convert all the fields
> from strings to numbers

It has to at some point. Or do you mean it compares strings, and 
converts them if numbers?

> Both versions took way more time to write than that AWK line,

Perhaps your AWK is just very fast. I tried MAWK (which didn't work) 
then GAWK, but I know nothing about it, whether it's considered fast or 
slow.

-- 
Bartc

proc start=
     lines := readtextfile("input")

     lineno := 0
     maxdiff := 0

     forall i, line in lines do
         sreadln(line)
         read a, b, c, d, e, f
         if f-a > maxdiff then
             maxdiff := f-a
             lineno := i
         fi
     od

     println =lineno
end

[toc] | [prev] | [next] | [standalone]

#126773

On 16/02/2018 00:19, bartc wrote:
> On 15/02/2018 22:58, Ben Bacarisse wrote:
>> jacobnavia <jacob@jacob.remcomp.fr> writes:
> 
>>> That last one is difficult to beat.
>>
>> Yes, I know what C's good at, but I was asking the OP.  It's possible
>> that none of that matters to them.  It could be a quick throwaway
>> program to find the line with the widest range:
>>
>>    awk 'NF==6 { if ($6-$1 > r) { r = $6-$1; ln = NR }} END { print ln }'
>>
>> I'm shameless enough to say that I'm a pretty fast C programmer but,
>> none the less, I'd write that sort of thing in AWK every time.
>>
>>> C is fast. Fast because it is just an abstraction of the machine.
>>
>> I changed my C version to do this calculation and found that the AWK
>> version was faster.

> Then, since you said that C was slower, I tweaked my demo program of 
> this morning, and it did the test in 0.67 seconds.

Best timing I could get out of C was 0.22 seconds (compiled with 
gcc-O3), using a modified parser that reads the whole file into memory.

With full error-checking, about 0.25, but I don't have time to make it 
efficient. About 4M lines per second, not unreasonable.

What sort of timings between AWK and C were you getting?

-- 
bartc

[toc] | [prev] | [next] | [standalone]

#126774

bartc <bc@freeuk.com> writes:

> On 16/02/2018 00:19, bartc wrote:
>> On 15/02/2018 22:58, Ben Bacarisse wrote:
>>> jacobnavia <jacob@jacob.remcomp.fr> writes:
>>
>>>> That last one is difficult to beat.
>>>
>>> Yes, I know what C's good at, but I was asking the OP.  It's possible
>>> that none of that matters to them.  It could be a quick throwaway
>>> program to find the line with the widest range:
>>>
>>>    awk 'NF==6 { if ($6-$1 > r) { r = $6-$1; ln = NR }} END { print ln }'
>>>
>>> I'm shameless enough to say that I'm a pretty fast C programmer but,
>>> none the less, I'd write that sort of thing in AWK every time.
>>>
>>>> C is fast. Fast because it is just an abstraction of the machine.
>>>
>>> I changed my C version to do this calculation and found that the AWK
>>> version was faster.
>
>> Then, since you said that C was slower, I tweaked my demo program of this morning, and it did the test in 0.67 seconds.
>
> Best timing I could get out of C was 0.22 seconds (compiled with gcc-O3), using a modified parser that reads the whole file into memory.
>
> With full error-checking, about 0.25, but I don't have time to make it efficient. About 4M lines per second, not unreasonable.
>
> What sort of timings between AWK and C were you getting?

With a 2,000,000 line file, AWK takes about 1.2 seconds.  C using sscanf
to parse all six numbers: 2 seconds.  A C program that uses strtol on
the first and last part of the line: 0.2 seconds.  Both C programs read
line by line and are compiled -O3.

-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#126776

Problem of input is not "How fast I can get them" but "they are in range?" 
And all language I have seen follow the wrong direction on that
only strtod strtol in C are good and right. 

[toc] | [prev] | [next] | [standalone]

#126781

asetofsymbols@gmail.com writes:

> Problem of input is not "How fast I can get them" but "they are in
> range?" And all language I have seen follow the wrong direction on that
> only strtod strtol in C are good and right.

Correctness is important, yes, and none of the examples checked the
input for range errors (and my example using sscanf can't do that at
all).  But not all languages follow what you call the wrong direction.
For example, the Python code won't fail due to the range of the inputs
(except in extraordinary circumstances) because it uses
multiple-precision integers.

In this particular case, we have little idea what's important.  If the
OP has very large amounts of data, and all the input number are small
like the ones shown, then checking for out of range inputs is going to
be wasted effort.

-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#126778

On 16/02/18 01:19, bartc wrote:
> On 15/02/2018 22:58, Ben Bacarisse wrote:
>> jacobnavia <jacob@jacob.remcomp.fr> writes:
> 
>>> That last one is difficult to beat.
>>
>> Yes, I know what C's good at, but I was asking the OP.  It's possible
>> that none of that matters to them.  It could be a quick throwaway
>> program to find the line with the widest range:
>>
>>    awk 'NF==6 { if ($6-$1 > r) { r = $6-$1; ln = NR }} END { print ln }'
>>
>> I'm shameless enough to say that I'm a pretty fast C programmer but,
>> none the less, I'd write that sort of thing in AWK every time.
>>
>>> C is fast. Fast because it is just an abstraction of the machine.
>>
>> I changed my C version to do this calculation and found that the AWK
>> version was faster.
> 
> So, the task is to find the line where column 6 is bigger than column 1
> by the maximum amount?
> 
> I tried your code in gawk, on a 1-million line input file containing 6
> columns of random numbers from 1 to 100,000.
> 
> Gawk took 1.64 seconds.
> 
> Then I threw something together with my scripting language, and it took
> 1.50 seconds at the first attempt [code shown in my sig].
> 

Since we are having fun,  I tested a simple python implementation.

awk took 0.44s.
Python2 and Python3 both took 0.9 seconds.
pypy (JIT compiled python) took 0.37 seconds.

All this shows that for a task like this, many languages are within
spitting distance of the speed of C, while being vastly simpler, clearer
and easier to write.  (With the details there depending on the
programmer and the task in question.)

[toc] | [prev] | [next] | [standalone]

#126782

David Brown <david.brown@hesbynett.no> writes:

> On 16/02/18 01:19, bartc wrote:
>> On 15/02/2018 22:58, Ben Bacarisse wrote:
>>> jacobnavia <jacob@jacob.remcomp.fr> writes:
>> 
>>>> That last one is difficult to beat.
>>>
>>> Yes, I know what C's good at, but I was asking the OP.  It's possible
>>> that none of that matters to them.  It could be a quick throwaway
>>> program to find the line with the widest range:
>>>
>>>    awk 'NF==6 { if ($6-$1 > r) { r = $6-$1; ln = NR }} END { print ln }'
>>>
>>> I'm shameless enough to say that I'm a pretty fast C programmer but,
>>> none the less, I'd write that sort of thing in AWK every time.
>>>
>>>> C is fast. Fast because it is just an abstraction of the machine.
>>>
>>> I changed my C version to do this calculation and found that the AWK
>>> version was faster.
>> 
>> So, the task is to find the line where column 6 is bigger than column 1
>> by the maximum amount?
>> 
>> I tried your code in gawk, on a 1-million line input file containing 6
>> columns of random numbers from 1 to 100,000.
>> 
>> Gawk took 1.64 seconds.
>> 
>> Then I threw something together with my scripting language, and it took
>> 1.50 seconds at the first attempt [code shown in my sig].
>
> Since we are having fun,  I tested a simple python implementation.
>
> awk took 0.44s.
> Python2 and Python3 both took 0.9 seconds.
> pypy (JIT compiled python) took 0.37 seconds.
>
> All this shows that for a task like this, many languages are within
> spitting distance of the speed of C,

I don't see how you can say that without a C data point on the same
machine.  And, being very flexible, C code comes in with very different
timings depending on the exact job being done (for the "max range"
problem I got a fact of 10 between two simple C programs).

> while being vastly simpler, clearer
> and easier to write.  (With the details there depending on the
> programmer and the task in question.)

-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#126783

On 16/02/18 12:36, Ben Bacarisse wrote:
> David Brown <david.brown@hesbynett.no> writes:
> 
>> On 16/02/18 01:19, bartc wrote:
>>> On 15/02/2018 22:58, Ben Bacarisse wrote:
>>>> jacobnavia <jacob@jacob.remcomp.fr> writes:
>>>
>>>>> That last one is difficult to beat.
>>>>
>>>> Yes, I know what C's good at, but I was asking the OP.  It's possible
>>>> that none of that matters to them.  It could be a quick throwaway
>>>> program to find the line with the widest range:
>>>>
>>>>    awk 'NF==6 { if ($6-$1 > r) { r = $6-$1; ln = NR }} END { print ln }'
>>>>
>>>> I'm shameless enough to say that I'm a pretty fast C programmer but,
>>>> none the less, I'd write that sort of thing in AWK every time.
>>>>
>>>>> C is fast. Fast because it is just an abstraction of the machine.
>>>>
>>>> I changed my C version to do this calculation and found that the AWK
>>>> version was faster.
>>>
>>> So, the task is to find the line where column 6 is bigger than column 1
>>> by the maximum amount?
>>>
>>> I tried your code in gawk, on a 1-million line input file containing 6
>>> columns of random numbers from 1 to 100,000.
>>>
>>> Gawk took 1.64 seconds.
>>>
>>> Then I threw something together with my scripting language, and it took
>>> 1.50 seconds at the first attempt [code shown in my sig].
>>
>> Since we are having fun,  I tested a simple python implementation.
>>
>> awk took 0.44s.
>> Python2 and Python3 both took 0.9 seconds.
>> pypy (JIT compiled python) took 0.37 seconds.
>>
>> All this shows that for a task like this, many languages are within
>> spitting distance of the speed of C,
> 
> I don't see how you can say that without a C data point on the same
> machine.  And, being very flexible, C code comes in with very different
> timings depending on the exact job being done (for the "max range"
> problem I got a fact of 10 between two simple C programs).

"Within spitting distance" does not mean the same speed.  It means fast
enough to be usable for the same sort of task.  I haven't bothered to
write C code for the job or test posted C code on the same system - it
is good enough for me to use the figures others have posted with speeds
faster than awk, but not massively faster.  There is no clearly
specified problem here, so we are making guesses and approximations.  If
we can rely on all numbers in the file being two digit and having 4
spaces between them, you can make a much faster C program for the job.
But if the layout varies and integer sizes exceed 64 bits, you'll have a
hard job significantly beating pypy with C (noting that most of the work
done during the python execution is from very optimised C code).

What we have shown in these few posts is that when you want to analyse
lines of a text file with integer data, a language like Python (or awk,
if the problem is simple enough, or even Bart's scripting language if he
is the one solving the problem) is likely to give you a solid fraction
of the speed of C with vastly less development effort.

> 
>> while being vastly simpler, clearer
>> and easier to write.  (With the details there depending on the
>> programmer and the task in question.)
> 

[toc] | [prev] | [next] | [standalone]

#126785

On 16/02/2018 12:10, David Brown wrote:
> On 16/02/18 12:36, Ben Bacarisse wrote:

>> I don't see how you can say that without a C data point on the same
>> machine.  And, being very flexible, C code comes in with very different
>> timings depending on the exact job being done (for the "max range"
>> problem I got a fact of 10 between two simple C programs).
> 
> "Within spitting distance" does not mean the same speed.  It means fast
> enough to be usable for the same sort of task.  I haven't bothered to
> write C code for the job or test posted C code on the same system - it
> is good enough for me to use the figures others have posted with speeds
> faster than awk, but not massively faster.  There is no clearly
> specified problem here, so we are making guesses and approximations.  If
> we can rely on all numbers in the file being two digit and having 4
> spaces between them, you can make a much faster C program for the job.
> But if the layout varies and integer sizes exceed 64 bits, you'll have a
> hard job significantly beating pypy with C (noting that most of the work
> done during the python execution is from very optimised C code).

For me, a first attempt in Python took 3.5 seconds, or 1.7 seconds in pypy.

A second, tweaked version took 1.45 seconds in pypy. (Actually, the same 
speed as my language. Although my pypy isn't that recent.)

My fastest C version (or pseudo-C transpiled from my own language in 
this case), was 0.25 seconds. (The main loop of this, and my Python 
effort, shown below.)

I haven't tried a dedicated C version; but the first C version (0.7 
seconds) took one minute to adapt, the second 5 minutes. So it was easy 
to beat pypy without much effort. Unless you have a particularly 
efficient algorithm for Python.


--------------------------
      maxdiff = lineno = n = 0;
      lex();
      while (lxsymbol != eofsym) {
          ++n;
          a = lxvalue;
          lex();
          lex();
          lex();
          lex();
          lex();
          f = lxvalue;
          lex();
          if (lxsymbol != eolsym) {
              serror("SYNC ERROR");
          }
          if (f-a > maxdiff) {
              maxdiff = f-a;
              lineno = n;
          }
          lex();
      }
      printf("%s %d","LINENO=",lineno);
      exit(0);
---------------------------------
def fn():    # code runs faster inside a function
	lineno=0
	maxdiff=0
	n=0

	for ln in open("input").readlines() :
		n+=1
		s=ln.split()
		a=int(s[0])
		f=int(s[5])
		if f-a>maxdiff:
			lineno=n
			maxdiff=f-a

	print(lineno)

fn()
----------------------------

-- 
bartc

[toc] | [prev] | [next] | [standalone]

#127404

On 2018-02-16, bartc <bc@freeuk.com> wrote:
> On 16/02/2018 12:10, David Brown wrote:
>> On 16/02/18 12:36, Ben Bacarisse wrote:
>
>>> I don't see how you can say that without a C data point on the same
>>> machine.  And, being very flexible, C code comes in with very different
>>> timings depending on the exact job being done (for the "max range"
>>> problem I got a fact of 10 between two simple C programs).
>> 
>> "Within spitting distance" does not mean the same speed.  It means fast
>> enough to be usable for the same sort of task.  I haven't bothered to
>> write C code for the job or test posted C code on the same system - it
>> is good enough for me to use the figures others have posted with speeds
>> faster than awk, but not massively faster.  There is no clearly
>> specified problem here, so we are making guesses and approximations.  If
>> we can rely on all numbers in the file being two digit and having 4
>> spaces between them, you can make a much faster C program for the job.
>> But if the layout varies and integer sizes exceed 64 bits, you'll have a
>> hard job significantly beating pypy with C (noting that most of the work
>> done during the python execution is from very optimised C code).
>
> For me, a first attempt in Python took 3.5 seconds, or 1.7 seconds in pypy.
>
> A second, tweaked version took 1.45 seconds in pypy. (Actually, the same 
> speed as my language. Although my pypy isn't that recent.)
>
> My fastest C version (or pseudo-C transpiled from my own language in 
> this case), was 0.25 seconds. (The main loop of this, and my Python 
> effort, shown below.)
>
> I haven't tried a dedicated C version; but the first C version (0.7 
> seconds) took one minute to adapt, the second 5 minutes. So it was easy 
> to beat pypy without much effort. Unless you have a particularly 
> efficient algorithm for Python.
>
>
> --------------------------
>       maxdiff = lineno = n = 0;
>       lex();
>       while (lxsymbol != eofsym) {
>           ++n;
>           a = lxvalue;
>           lex();
>           lex();
>           lex();
>           lex();
>           lex();
>           f = lxvalue;
>           lex();
>           if (lxsymbol != eolsym) {
>               serror("SYNC ERROR");
>           }
>           if (f-a > maxdiff) {
>               maxdiff = f-a;
>               lineno = n;
>           }
>           lex();
>       }
>       printf("%s %d","LINENO=",lineno);
>       exit(0);
> ---------------------------------
> def fn():    # code runs faster inside a function
> 	lineno=0
> 	maxdiff=0
> 	n=0
>
> 	for ln in open("input").readlines() :
> 		n+=1
> 		s=ln.split()
> 		a=int(s[0])
> 		f=int(s[5])
> 		if f-a>maxdiff:
> 			lineno=n
> 			maxdiff=f-a
>
> 	print(lineno)
>
> fn()
> ----------------------------

How much of your lifetime was wasted on this?

[toc] | [prev] | [next] | [standalone]

#127406

On 06/03/2018 19:33, Ike Naar wrote:
> On 2018-02-16, bartc <bc@freeuk.com> wrote:

> How much of your lifetime was wasted on this?

Why, is that a big concern of yours how I choose to spend my time?

How you waste /your/ life (apart from making personal digs at strangers 
on the internet), I don't know, I don't care and it's none of my business.

[toc] | [prev] | [next] | [standalone]

#126787

On Friday, February 16, 2018 at 12:10:54 PM UTC, David Brown wrote:
> On 16/02/18 12:36, Ben Bacarisse wrote:
> > David Brown <david.brown@hesbynett.no> writes:
> > 
> >> On 16/02/18 01:19, bartc wrote:
> >>> On 15/02/2018 22:58, Ben Bacarisse wrote:
> >>>> jacobnavia <jacob@jacob.remcomp.fr> writes:
> >>>
> >>>>> That last one is difficult to beat.
> >>>>
> >>>> Yes, I know what C's good at, but I was asking the OP.  It's possible
> >>>> that none of that matters to them.  It could be a quick throwaway
> >>>> program to find the line with the widest range:
> >>>>
> >>>>    awk 'NF==6 { if ($6-$1 > r) { r = $6-$1; ln = NR }} END { print ln }'
> >>>>
> >>>> I'm shameless enough to say that I'm a pretty fast C programmer but,
> >>>> none the less, I'd write that sort of thing in AWK every time.
> >>>>
> >>>>> C is fast. Fast because it is just an abstraction of the machine.
> >>>>
> >>>> I changed my C version to do this calculation and found that the AWK
> >>>> version was faster.
> >>>
> >>> So, the task is to find the line where column 6 is bigger than column 1
> >>> by the maximum amount?
> >>>
> >>> I tried your code in gawk, on a 1-million line input file containing 6
> >>> columns of random numbers from 1 to 100,000.
> >>>
> >>> Gawk took 1.64 seconds.
> >>>
> >>> Then I threw something together with my scripting language, and it took
> >>> 1.50 seconds at the first attempt [code shown in my sig].
> >>
> >> Since we are having fun,  I tested a simple python implementation.
> >>
> >> awk took 0.44s.
> >> Python2 and Python3 both took 0.9 seconds.
> >> pypy (JIT compiled python) took 0.37 seconds.
> >>
> >> All this shows that for a task like this, many languages are within
> >> spitting distance of the speed of C,
> > 
> > I don't see how you can say that without a C data point on the same
> > machine.  And, being very flexible, C code comes in with very different
> > timings depending on the exact job being done (for the "max range"
> > problem I got a fact of 10 between two simple C programs).
> 
> "Within spitting distance" does not mean the same speed.  It means fast
> enough to be usable for the same sort of task.  I haven't bothered to
> write C code for the job or test posted C code on the same system - it
> is good enough for me to use the figures others have posted with speeds
> faster than awk, but not massively faster.  There is no clearly
> specified problem here, so we are making guesses and approximations.  If
> we can rely on all numbers in the file being two digit and having 4
> spaces between them, you can make a much faster C program for the job.
> But if the layout varies and integer sizes exceed 64 bits, you'll have a
> hard job significantly beating pypy with C (noting that most of the work
> done during the python execution is from very optimised C code).
> 
> What we have shown in these few posts is that when you want to analyse
> lines of a text file with integer data, a language like Python (or awk,
> if the problem is simple enough, or even Bart's scripting language if he
> is the one solving the problem) is likely to give you a solid fraction
> of the speed of C with vastly less development effort.
> 
Yes, if we want a really fast C program and can trust the input

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

int main(void)
{
  FILE *fp = fopen("numbers.txt", "r");
  char buff[32] = {0};
  char bestline[32];
  int low, high;
  int best = -1;
  long linenumber = 0;
  long bestlineno = -1;

  while(fread(buff,1, 18, fp) == 18)
  {
    linenumber++;
    low = (buff[0]-'0')*10 + buff[1] - '0';
    high = (buff[15]-'0')*10 + buff[16] - '0';
    if(high - low > best)
    {
      bestlineno = linenumber;
      strcpy(bestline, buff);
      best = high - low;
    }  
  }
  fclose(fp);

  printf("bestline %d (%s)\n", bestlineno, bestline);
}

Now it runs in 0.080 seconds with no optimisation, 0.050
seconds on gcc - O3.

Note also that the C is more self-documenting. The Python is
essentially meaningless to a non-Python programmer, just a series
of square brackets with a contained for loop which could mean
anything. The C should be more or less readable to anyone who
knows any programming language - the exception being the fread
call.

[toc] | [prev] | [next] | [standalone]

#126797

On 16/02/18 13:42, Malcolm McLean wrote:

> 
> Now it runs in 0.080 seconds with no optimisation, 0.050
> seconds on gcc - O3.

As I said - with different requirements, timing will be different.  More
stringent formats will favour C (a Python version that works like your
code - but with the correct offsets - runs about twice as fast as the
original Python).

> 
> Note also that the C is more self-documenting. 

Bollocks.

> The Python is
> essentially meaningless to a non-Python programmer, just a series
> of square brackets with a contained for loop which could mean
> anything. The C should be more or less readable to anyone who
> knows any programming language - the exception being the fread
> call.
> 

Drivel.

The C code is easier for people familiar with C or related programming
languages, the Python code is easier for people familiar with Python.
It is true that list comprehension is a bit odd to people who have not
seen it before - it is easy for anyone who has worked with functional
programming languages or mathematics (it's just like set comprehension).

Plus, you are comparing apples to oranges here.  The Python code posted
earlier was for summing arbitrary sized integers with arbitrary spacing
- and must be compared to the original C code versions posted here
(which are a lot bigger, and therefore harder to comprehend).

The Python code I tested corresponding to the limited problem with
fixed-format files is:

lines = open("numbers.txt").readlines()
lineno = 0
maxdiff = 0
zero = ord('0')

for i, line in enumerate(lines) :
	a = (ord(line[0]) - zero) * 10 + ord(line[1]) - zero
	f = (ord(line[30]) - zero) * 10 + ord(line[31]) - zero
	if (f - a) > maxdiff :
		maxdiff = f - a
		lineno = i

print(lineno)


Are you really going to claim that the C code is "self-documenting"
while the Python code is meaningless?

Now, if I had written it as:

print(-max([(ord(line[30]) * 10 + ord(line[31]) - ord(line[0]) * 10 -
      ord(line[1]), -i) for i, line in enumerate(open(
      "samples.txt").readlines())])[1])

/then/ you could complain!



The best I got from pypy on my system was about 0.07 seconds from:

data = open("samples.txt").read()
linecount = len(data) // 33
lineno = 0
maxdiff = 0
offset = 0

for i in range(linecount) :
	d = ord(data[offset + 30]) * 10 + ord(data[offset + 31]) -
ord(data[offset + 0]) * 10 - ord(data[offset + 1])
	offset += 33
	if d > maxdiff :
		maxdiff = d
		lineno = i

print(lineno)



Your C code, after bug-fixes and tweaks to get the line lengths correct,
took 0.05 seconds.


We are at the stage here where timing jitter is more significant than
the differences in speed between the C code and the Python code.

[toc] | [prev] | [next] | [standalone]

#126810

On 16/02/2018 15:58, David Brown wrote:
> On 16/02/18 13:42, Malcolm McLean wrote:
> 
>>
>> Now it runs in 0.080 seconds with no optimisation, 0.050
>> seconds on gcc - O3.
> 
> As I said - with different requirements, timing will be different.  More
> stringent formats will favour C (a Python version that works like your
> code - but with the correct offsets - runs about twice as fast as the
> original Python).
> 
>>
>> Note also that the C is more self-documenting.
> 
> Bollocks.

I think he means that C can require you to use simple constructs that 
are universally understood. Lines aren't just mysterious chains of 
method calls.

> The Python code I tested corresponding to the limited problem with
> fixed-format files is:

> The best I got from pypy on my system was about 0.07 seconds from:
> 
> data = open("samples.txt").read()
> linecount = len(data) // 33
> lineno = 0
> maxdiff = 0
> offset = 0
> 
> for i in range(linecount) :
> 	d = ord(data[offset + 30]) * 10 + ord(data[offset + 31]) -
> ord(data[offset + 0]) * 10 - ord(data[offset + 1])
> 	offset += 33
> 	if d > maxdiff :
> 		maxdiff = d
> 		lineno = i
> 
> print(lineno)
> 
> 
> 
> Your C code, after bug-fixes and tweaks to get the line lengths correct,
> took 0.05 seconds.
> 
> 
> We are at the stage here where timing jitter is more significant than
> the differences in speed between the C code and the Python code.

You had to resort to using low-level idioms as used in C to get the best 
results.

Which also has a benefit, as I could directly transcribe the algorithm 
to my languages, which is not usually the case with Python code.

Using a 10-million line, 32-character per line file (with LF 
line-endings otherwise the Python is slowed down significantly 
translating CRLF to LF), I got only 2.5 seconds for PyPy.

My script language managed 1.6 seconds.

While my compiled language (I didn't need to put it through C/gcc) 
managed 0.45 seconds.

So a big difference.

However, it seems PyPy spent nearly all of its time reading in the file, 
and very little on the loop. So perhaps its file handling is less 
efficient on Windows than on its native Unix. Although that didn't 
affect the 0.45 timing much. Normal Python took 10 seconds.

I've learnt however not to give much significance to PyPy timings for 
small benchmarks (like gcc-O3 really); its big thing is loops, and here 
the program largely consists of a loop.

-- 
bartc

[toc] | [prev] | [next] | [standalone]

#126812

On 16/02/18 21:02, bartc wrote:
> On 16/02/2018 15:58, David Brown wrote:
>> On 16/02/18 13:42, Malcolm McLean wrote:
>>
>>>
>>> Now it runs in 0.080 seconds with no optimisation, 0.050
>>> seconds on gcc - O3.
>>
>> As I said - with different requirements, timing will be different.  More
>> stringent formats will favour C (a Python version that works like your
>> code - but with the correct offsets - runs about twice as fast as the
>> original Python).
>>
>>>
>>> Note also that the C is more self-documenting.
>>
>> Bollocks.
> 
> I think he means that C can require you to use simple constructs that 
> are universally understood. Lines aren't just mysterious chains of 
> method calls.
> 
>> The Python code I tested corresponding to the limited problem with
>> fixed-format files is:
> 
>> The best I got from pypy on my system was about 0.07 seconds from:
>>
>> data = open("samples.txt").read()
>> linecount = len(data) // 33
>> lineno = 0
>> maxdiff = 0
>> offset = 0
>>
>> for i in range(linecount) :
>>     d = ord(data[offset + 30]) * 10 + ord(data[offset + 31]) -
>> ord(data[offset + 0]) * 10 - ord(data[offset + 1])
>>     offset += 33
>>     if d > maxdiff :
>>         maxdiff = d
>>         lineno = i
>>
>> print(lineno)
>>
>>
>>
>> Your C code, after bug-fixes and tweaks to get the line lengths correct,
>> took 0.05 seconds.
>>
>>
>> We are at the stage here where timing jitter is more significant than
>> the differences in speed between the C code and the Python code.
> 
> You had to resort to using low-level idioms as used in C to get the best 
> results.
> 

It is taking advantage of the format of the input (in this case, 
restricted to 2 digit numbers).  This is part of the description of the 
modified problem, and it's not surprising that the same structure is 
used in C and Python (and any other language).

> Which also has a benefit, as I could directly transcribe the algorithm 
> to my languages, which is not usually the case with Python code.
> 
> Using a 10-million line, 32-character per line file (with LF 
> line-endings otherwise the Python is slowed down significantly 
> translating CRLF to LF), I got only 2.5 seconds for PyPy.
> 
> My script language managed 1.6 seconds.
> 
> While my compiled language (I didn't need to put it through C/gcc) 
> managed 0.45 seconds.
> 
> So a big difference.
> 

Overall, no, it is not a big difference.  Usually figures for Python 
versus C speeds are far bigger than that - a couple of orders of magnitude.

> However, it seems PyPy spent nearly all of its time reading in the file, 
> and very little on the loop. So perhaps its file handling is less 
> efficient on Windows than on its native Unix. Although that didn't 
> affect the 0.45 timing much. Normal Python took 10 seconds.

/All/ file handling is less efficient on Windows than Linux.  But 
perhaps pypy's is particularly slow on Windows, or there is something 
else going on like data in file caches.  (For my tests, I had the data 
file on a ram filesystem.)

> 
> I've learnt however not to give much significance to PyPy timings for 
> small benchmarks (like gcc-O3 really); its big thing is loops, and here 
> the program largely consists of a loop.
> 

In most code, most of the time is spend in loops.

To be honest, I haven't used pypy much to see what it does well or not. 
Usually my Python code is not speed critical.

[toc] | [prev] | [next] | [standalone]

#126827

On Friday, February 16, 2018 at 3:59:04 PM UTC, David Brown wrote:
> On 16/02/18 13:42, Malcolm McLean wrote:
> 
> The C code is easier for people familiar with C or related programming
> languages, the Python code is easier for people familiar with Python.
> It is true that list comprehension is a bit odd to people who have not
> seen it before - it is easy for anyone who has worked with functional
> programming languages or mathematics (it's just like set comprehension).
> 
> Plus, you are comparing apples to oranges here.  The Python code posted
> earlier was for summing arbitrary sized integers with arbitrary spacing
> - and must be compared to the original C code versions posted here
> (which are a lot bigger, and therefore harder to comprehend).
> 
> The Python code I tested corresponding to the limited problem with
> fixed-format files is:
> 
> lines = open("numbers.txt").readlines()
> lineno = 0
> maxdiff = 0
> zero = ord('0')
> 
> for i, line in enumerate(lines) :
> 	a = (ord(line[0]) - zero) * 10 + ord(line[1]) - zero
> 	f = (ord(line[30]) - zero) * 10 + ord(line[31]) - zero
> 	if (f - a) > maxdiff :
> 		maxdiff = f - a
> 		lineno = i
> 
> print(lineno)
> 
> 
> Are you really going to claim that the C code is "self-documenting"
> while the Python code is meaningless?
> 
> Now, if I had written it as:
> 
> print(-max([(ord(line[30]) * 10 + ord(line[31]) - ord(line[0]) * 10 -
>       ord(line[1]), -i) for i, line in enumerate(open(
>       "samples.txt").readlines())])[1])
> 
> /then/ you could complain!
> 
Exactly.
The first Python function is loading an entire file into memory, which
is probably why it takes nearly twice as long to execute. The second
Python function, I couldn't tell you. 

If you write Python like C, however, you get a reasonably fast and
comprehensible function.

The OP had data in regular columns. Having fixed-size records is a
fairly standard technique for allowing programs to optimise. If you
make to easy to read variable-width fields and difficult to read
fixed-width fields then people are less likely to take this
optimisation.

[toc] | [prev] | [next] | [standalone]

Page 3 of 9 — ← Prev page 1 2 [3] 4 5 6 7 8 9  Next page →

Back to top | Article view | comp.lang.c

csiph-web