Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.lang.c > #126810
| Subject | Re: Text Files |
|---|---|
| Newsgroups | comp.lang.c |
| References | (5 earlier) <p668md$gp3$1@dont-email.me> <87lgft4145.fsf@bsb.me.uk> <p66ho1$c2r$1@dont-email.me> <2d7a5d27-c106-4bbf-96a5-a23fa0a2e4dc@googlegroups.com> <p66v3u$bcv$1@dont-email.me> |
| From | bartc <bc@freeuk.com> |
| Message-ID | <vNGhC.52751$dR6.19830@fx09.am4> (permalink) |
| Organization | virginmedia.com |
| Date | 2018-02-16 20:02 +0000 |
On 16/02/2018 15:58, David Brown wrote:
> On 16/02/18 13:42, Malcolm McLean wrote:
>
>>
>> Now it runs in 0.080 seconds with no optimisation, 0.050
>> seconds on gcc - O3.
>
> As I said - with different requirements, timing will be different. More
> stringent formats will favour C (a Python version that works like your
> code - but with the correct offsets - runs about twice as fast as the
> original Python).
>
>>
>> Note also that the C is more self-documenting.
>
> Bollocks.
I think he means that C can require you to use simple constructs that
are universally understood. Lines aren't just mysterious chains of
method calls.
> The Python code I tested corresponding to the limited problem with
> fixed-format files is:
> The best I got from pypy on my system was about 0.07 seconds from:
>
> data = open("samples.txt").read()
> linecount = len(data) // 33
> lineno = 0
> maxdiff = 0
> offset = 0
>
> for i in range(linecount) :
> d = ord(data[offset + 30]) * 10 + ord(data[offset + 31]) -
> ord(data[offset + 0]) * 10 - ord(data[offset + 1])
> offset += 33
> if d > maxdiff :
> maxdiff = d
> lineno = i
>
> print(lineno)
>
>
>
> Your C code, after bug-fixes and tweaks to get the line lengths correct,
> took 0.05 seconds.
>
>
> We are at the stage here where timing jitter is more significant than
> the differences in speed between the C code and the Python code.
You had to resort to using low-level idioms as used in C to get the best
results.
Which also has a benefit, as I could directly transcribe the algorithm
to my languages, which is not usually the case with Python code.
Using a 10-million line, 32-character per line file (with LF
line-endings otherwise the Python is slowed down significantly
translating CRLF to LF), I got only 2.5 seconds for PyPy.
My script language managed 1.6 seconds.
While my compiled language (I didn't need to put it through C/gcc)
managed 0.45 seconds.
So a big difference.
However, it seems PyPy spent nearly all of its time reading in the file,
and very little on the loop. So perhaps its file handling is less
efficient on Windows than on its native Unix. Although that didn't
affect the 0.45 timing much. Normal Python took 10 seconds.
I've learnt however not to give much significance to PyPy timings for
small benchmarks (like gcc-O3 really); its big thing is loops, and here
the program largely consists of a loop.
--
bartc
Back to comp.lang.c | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
Text Files G B <gb@gb.com> - 2018-02-14 11:47 -0400
Re: Text Files Ed Prochak <edprochak@gmail.com> - 2018-02-14 08:45 -0800
Re: Text Files Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-02-14 08:47 -0800
Re: Text Files already5chosen@yahoo.com - 2018-02-15 01:17 -0800
Re: Text Files gordonb.ea2mq@burditt.org (Gordon Burditt) - 2018-02-14 11:10 -0600
Re: Text Files "James R. Kuyper" <jameskuyper@verizon.net> - 2018-02-14 12:28 -0500
Re: Text Files jt@toerring.de (Jens Thoms Toerring) - 2018-02-14 21:07 +0000
Re: Text Files Joe Pfeiffer <pfeiffer@cs.nmsu.edu> - 2018-02-14 15:31 -0700
Re: Text Files Jorgen Grahn <grahn+nntp@snipabacken.se> - 2018-02-15 06:59 +0000
Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-15 10:20 +0000
Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-14 22:51 +0000
Re: Text Files David Brown <david.brown@hesbynett.no> - 2018-02-15 11:04 +0100
Re: Text Files Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-02-15 06:16 -0800
Re: Text Files "Mr. Man-wai Chang" <toylet.toylet@gmail.com> - 2018-02-15 22:22 +0800
Re: Text Files David Brown <david.brown@hesbynett.no> - 2018-02-15 15:58 +0100
Re: Text Files bartc <bc@freeuk.com> - 2018-02-15 16:31 +0000
Re: Text Files David Brown <david.brown@hesbynett.no> - 2018-02-15 21:31 +0100
Re: Text Files bartc <bc@freeuk.com> - 2018-02-15 21:21 +0000
Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-15 22:17 +0000
Re: Text Files bartc <bc@freeuk.com> - 2018-02-15 23:36 +0000
Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-15 23:48 +0000
Re: Text Files David Brown <david.brown@hesbynett.no> - 2018-02-16 10:09 +0100
Re: Text Files bartc <bc@freeuk.com> - 2018-02-16 12:57 +0000
Re: Text Files David Brown <david.brown@hesbynett.no> - 2018-02-16 15:50 +0100
Re: Text Files bartc <bc@freeuk.com> - 2018-02-16 15:50 +0000
Re: Text Files David Brown <david.brown@hesbynett.no> - 2018-02-16 17:07 +0100
Re: Text Files bartc <bc@freeuk.com> - 2018-02-16 20:44 +0000
Re: Text Files Ian Collins <ian-news@hotmail.com> - 2018-02-17 11:22 +1300
Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-16 17:48 +0000
Re: Text Files GOTHIER Nathan <nathan.gothier@gmail.com> - 2018-02-16 17:06 +0100
Re: Text Files Ian Collins <ian-news@hotmail.com> - 2018-02-17 11:10 +1300
Re: Text Files supercat@casperkitty.com - 2018-02-16 14:48 -0800
Re: Text Files Keith Thompson <kst-u@mib.org> - 2018-02-16 15:14 -0800
Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-16 23:25 +0000
Re: Text Files supercat@casperkitty.com - 2018-02-16 15:50 -0800
Re: Text Files already5chosen@yahoo.com - 2018-02-16 07:19 -0800
Re: Text Files David Brown <david.brown@hesbynett.no> - 2018-02-16 17:13 +0100
Re: Text Files Jorgen Grahn <grahn+nntp@snipabacken.se> - 2018-02-19 17:55 +0000
Re: Text Files David Brown <david.brown@hesbynett.no> - 2018-02-19 19:14 +0100
Re: Text Files Keith Thompson <kst-u@mib.org> - 2018-02-19 11:48 -0800
Re: Text Files Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-02-16 08:14 -0800
Re: Text Files Robert Wessel <robertwessel2@yahoo.com> - 2018-02-17 18:32 -0600
Re: Text Files jacobnavia <jacob@jacob.remcomp.fr> - 2018-02-15 22:57 +0100
Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-15 22:58 +0000
Re: Text Files bartc <bc@freeuk.com> - 2018-02-16 00:19 +0000
Re: Text Files bartc <bc@freeuk.com> - 2018-02-16 02:26 +0000
Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-16 02:57 +0000
Re: Text Files asetofsymbols@gmail.com - 2018-02-16 00:21 -0800
Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-16 11:30 +0000
Re: Text Files David Brown <david.brown@hesbynett.no> - 2018-02-16 10:36 +0100
Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-16 11:36 +0000
Re: Text Files David Brown <david.brown@hesbynett.no> - 2018-02-16 13:10 +0100
Re: Text Files bartc <bc@freeuk.com> - 2018-02-16 12:25 +0000
Re: Text Files Ike Naar <ike@iceland.freeshell.org> - 2018-03-06 19:33 +0000
Re: Text Files bartc <bc@freeuk.com> - 2018-03-06 20:55 +0000
Re: Text Files Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-02-16 04:42 -0800
Re: Text Files David Brown <david.brown@hesbynett.no> - 2018-02-16 16:58 +0100
Re: Text Files bartc <bc@freeuk.com> - 2018-02-16 20:02 +0000
Re: Text Files David Brown <david.brown@hesbynett.no> - 2018-02-16 21:43 +0100
Re: Text Files Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-02-17 04:01 -0800
Re: Text Files bartc <bc@freeuk.com> - 2018-02-17 13:10 +0000
Re: Text Files Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-02-17 05:27 -0800
Re: Text Files bartc <bc@freeuk.com> - 2018-02-17 14:20 +0000
Re: Text Files Joe Pfeiffer <pfeiffer@cs.nmsu.edu> - 2018-02-16 09:04 -0700
Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-16 14:31 +0000
Re: Text Files Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-02-16 06:42 -0800
Re: Text Files bartc <bc@freeuk.com> - 2018-02-16 14:58 +0000
Re: Text Files Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-02-16 07:23 -0800
Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-16 15:24 +0000
Re: Text Files Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-02-16 04:13 -0800
Re: Text Files Siri Cruise <chine.bleu@yahoo.com> - 2018-02-16 04:29 -0800
Re: Text Files Jorgen Grahn <grahn+nntp@snipabacken.se> - 2018-02-20 16:41 +0000
Re: Text Files Siri Cruise <chine.bleu@yahoo.com> - 2018-02-20 09:39 -0800
Re: Text Files Jorgen Grahn <grahn+nntp@snipabacken.se> - 2018-02-20 19:16 +0000
Re: Text Files bartc <bc@freeuk.com> - 2018-02-15 12:46 +0000
Re: Text Files GOTHIER Nathan <nathan.gothier@gmail.com> - 2018-02-15 14:52 +0100
Re: Text Files Anton Shepelev <anton.txt@g{oogle}mail.com> - 2018-02-15 17:07 +0300
Re: Text Files bartc <bc@freeuk.com> - 2018-02-15 14:23 +0000
Re: Text Files Anton Shepelev <anton.txt@g{oogle}mail.com> - 2018-02-15 18:52 +0300
Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-15 14:39 +0000
Re: Text Files bartc <bc@freeuk.com> - 2018-02-15 15:43 +0000
Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-15 20:32 +0000
Re: Text Files bartc <bc@freeuk.com> - 2018-02-15 22:06 +0000
Re: Text Files GOTHIER Nathan <nathan.gothier@gmail.com> - 2018-02-15 23:12 +0100
Re: Text Files Keith Thompson <kst-u@mib.org> - 2018-02-15 15:12 -0800
Re: Text Files bartc <bc@freeuk.com> - 2018-02-15 23:43 +0000
Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-15 23:54 +0000
Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-15 23:35 +0000
Re: Text Files Tim Rentsch <txr@alumni.caltech.edu> - 2018-02-16 17:07 -0800
Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-17 02:38 +0000
Re: Text Files Ian Collins <ian-news@hotmail.com> - 2018-02-17 17:26 +1300
Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-17 11:22 +0000
Re: Text Files bartc <bc@freeuk.com> - 2018-02-17 12:07 +0000
Re: Text Files Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-02-17 04:17 -0800
Re: Text Files Anton Shepelev <anton.txt@gmail.com> - 2018-02-17 16:25 +0300
Re: Text Files Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-02-17 07:35 -0800
Re: Text Files GOTHIER Nathan <nathan.gothier@gmail.com> - 2018-02-17 15:24 +0100
Re: Text Files Ian Collins <ian-news@hotmail.com> - 2018-02-18 11:10 +1300
Re: Text Files Tim Rentsch <txr@alumni.caltech.edu> - 2018-02-19 12:05 -0800
Re: Text Files already5chosen@yahoo.com - 2018-02-19 15:32 -0800
Re: Text Files Tim Rentsch <txr@alumni.caltech.edu> - 2018-02-21 10:34 -0800
Re: Text Files already5chosen@yahoo.com - 2018-02-21 15:35 -0800
Re: Text Files bartc <bc@freeuk.com> - 2018-02-22 00:11 +0000
Re: Text Files Robert Wessel <robertwessel2@yahoo.com> - 2018-02-21 18:26 -0600
Re: Text Files Tim Rentsch <txr@alumni.caltech.edu> - 2018-02-21 21:39 -0800
Re: Text Files Robert Wessel <robertwessel2@yahoo.com> - 2018-02-23 03:08 -0600
Re: Text Files supercat@casperkitty.com - 2018-02-23 16:16 -0800
Re: Text Files Keith Thompson <kst-u@mib.org> - 2018-02-23 16:46 -0800
Re: Text Files Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-02-24 02:49 -0800
Re: Text Files asetofsymbols@gmail.com - 2018-02-24 05:16 -0800
Re: Text Files "F. Russell" <fr@random.info> - 2018-02-24 14:35 +0000
Re: Text Files Joe Pfeiffer <pfeiffer@cs.nmsu.edu> - 2018-02-24 08:48 -0700
Re: Text Files supercat@casperkitty.com - 2018-02-25 15:09 -0800
Re: Text Files Keith Thompson <kst-u@mib.org> - 2018-02-25 15:54 -0800
Re: Text Files supercat@casperkitty.com - 2018-02-26 11:04 -0800
Re: Text Files Keith Thompson <kst-u@mib.org> - 2018-02-26 11:52 -0800
Re: Text Files supercat@casperkitty.com - 2018-02-26 14:12 -0800
Re: Text Files Keith Thompson <kst-u@mib.org> - 2018-02-26 14:40 -0800
Re: Text Files already5chosen@yahoo.com - 2018-02-22 04:58 -0800
Re: Text Files Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-02-22 05:33 -0800
Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-22 13:59 +0000
Re: Text Files Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2018-02-22 07:24 -0800
Re: Text Files supercat@casperkitty.com - 2018-02-20 12:14 -0800
Re: Text Files David Brown <david.brown@hesbynett.no> - 2018-02-15 21:37 +0100
Re: Text Files bartc <bc@freeuk.com> - 2018-02-15 21:34 +0000
Re: Text Files Keith Thompson <kst-u@mib.org> - 2018-02-15 14:15 -0800
Re: Text Files David Brown <david.brown@hesbynett.no> - 2018-02-16 10:47 +0100
Re: Text Files "Mr. Man-wai Chang" <toylet.toylet@gmail.com> - 2018-02-15 22:20 +0800
Re: Text Files "F. Russell" <fr@random.info> - 2018-02-15 15:24 +0000
Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-15 16:05 +0000
Re: Text Files "Mr. Man-wai Chang" <toylet.toylet@gmail.com> - 2018-02-16 00:22 +0800
Re: Text Files Joe Pfeiffer <pfeiffer@cs.nmsu.edu> - 2018-02-15 13:35 -0700
Re: Text Files Anton Shepelev <anton.txt@gmail.com> - 2018-02-25 00:13 +0300
Re: Text Files bartc <bc@freeuk.com> - 2018-02-24 21:24 +0000
Re: Text Files Anton Shepelev <anton.txt@gmail.com> - 2018-02-25 14:55 +0300
Re: Text Files bartc <bc@freeuk.com> - 2018-02-25 13:00 +0000
Re: Text Files Real Troll <Real.Troll@Trolls.com> - 2018-02-24 17:35 -0400
Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-24 21:36 +0000
Re: Text Files Real Troll <Real.Troll@Trolls.com> - 2018-02-24 17:45 -0400
Re: Text Files Jorgen Grahn <grahn+nntp@snipabacken.se> - 2018-02-24 22:51 +0000
Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-24 23:56 +0000
Re: Text Files bartc <bc@freeuk.com> - 2018-02-25 00:14 +0000
Re: Text Files bartc <bc@freeuk.com> - 2018-02-25 00:22 +0000
Re: Text Files Anton Shepelev <anton.txt@gmail.com> - 2018-02-25 15:16 +0300
Re: Text Files Keith Thompson <kst-u@mib.org> - 2018-02-25 15:20 -0800
Re: Text Files Joe Pfeiffer <pfeiffer@cs.nmsu.edu> - 2018-02-25 20:27 -0700
Re: Text Files gazelle@shell.xmission.com (Kenny McCormack) - 2018-02-26 06:19 +0000
Re: Text Files Anton Shepelev <anton.txt@g{oogle}mail.com> - 2018-02-26 15:35 +0300
Re: Text Files Reinhardt Behm <rbehm@hushmail.com> - 2018-02-26 20:53 +0800
Grammar Patrol (Was: Text Files) gazelle@shell.xmission.com (Kenny McCormack) - 2018-02-26 17:54 +0000
Re: Text Files Jorgen Grahn <grahn+nntp@snipabacken.se> - 2018-02-26 15:27 +0000
Re: Text Files Anton Shepelev <anton.txt@g{oogle}mail.com> - 2018-02-26 18:50 +0300
Re: Text Files Keith Thompson <kst-u@mib.org> - 2018-02-26 09:26 -0800
Re: Text Files scott@slp53.sl.home (Scott Lurndal) - 2018-02-26 19:46 +0000
Re: Text Files David Brown <david.brown@hesbynett.no> - 2018-02-27 09:04 +0100
Re: Text Files already5chosen@yahoo.com - 2018-02-27 00:56 -0800
Re: Text Files Ian Collins <ian-news@hotmail.com> - 2018-02-27 22:07 +1300
Re: Text Files David Brown <david.brown@hesbynett.no> - 2018-02-27 14:37 +0100
Re: Text Files already5chosen@yahoo.com - 2018-02-27 07:38 -0800
Re: Text Files supercat@casperkitty.com - 2018-02-27 09:18 -0800
Re: Text Files David Brown <david.brown@hesbynett.no> - 2018-02-27 10:15 +0100
Re: Text Files Keith Thompson <kst-u@mib.org> - 2018-02-27 08:53 -0800
Re: Text Files Tim Rentsch <txr@alumni.caltech.edu> - 2018-02-26 23:14 -0800
Re: Text Files Joe Pfeiffer <pfeiffer@cs.nmsu.edu> - 2018-02-26 08:32 -0700
Re: Text Files Keith Thompson <kst-u@mib.org> - 2018-02-26 09:17 -0800
Re: Text Files Tim Rentsch <txr@alumni.caltech.edu> - 2018-02-26 22:55 -0800
Re: Text Files Anton Shepelev <anton.txt@gmail.com> - 2018-02-25 15:02 +0300
Re: Text Files Ben Bacarisse <ben.usenet@bsb.me.uk> - 2018-02-25 20:12 +0000
csiph-web