Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #62146

Re: Mysterious slow IO on fread() to large buffer on large Dell system

Show all headers | View raw

El 16/05/15 a las 16:57, Lew Pitcher escribió:
> If I read that fread() call correctly, you are (atomically) reading the file
> one byte at a time, for as many bytes as will fit in your buffer. That's
> gotta be expensive.
>
> How about reading the file once, in <<buffer_sized>> chunks? That should
> remove the excessive memory-to-memory copy that happens with the other read.
>
> Replace your fread() with
>          rlen = fread(buffer, len_file, 1, fin);
> and see if that improves your throughput.

Your solution seems to be the better.
If I were the OP I'll try this as fast as I could.

I wrote only to tell that I have done similar experiments
with other language, and found that in my PC a lenght of
1024*1024 bytes to read at a time, is nearly as good as
read the entire file "in a stream." But not less than
that, for files that weight gigabytes.

Regards.

-- 
http://www.telecable.es/personales/gamo/
The generation of random numbers is too important to be left to chance

Back to comp.lang.c | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Mysterious slow IO on fread() to large buffer on large Dell system mathog <dmathog@gmail.com> - 2015-05-15 18:45 -0700
  Re: Mysterious slow IO on fread() to large buffer on large Dell system Ian Collins <ian-news@hotmail.com> - 2015-05-16 14:09 +1200
  Re: Mysterious slow IO on fread() to large buffer on large Dell system Siri Cruz <chine.bleu@yahoo.com> - 2015-05-15 19:20 -0700
    Re: Mysterious slow IO on fread() to large buffer on large Dell system mathog <dmathog@gmail.com> - 2015-05-15 22:29 -0700
      Re: Mysterious slow IO on fread() to large buffer on large Dell system Siri Cruz <chine.bleu@yahoo.com> - 2015-05-16 08:47 -0700
    Re: Mysterious slow IO on fread() to large buffer on large Dell system Robert Wessel <robertwessel2@yahoo.com> - 2015-05-16 02:11 -0500
  Re: Mysterious slow IO on fread() to large buffer on large Dell system Malcolm McLean <malcolm.mclean5@btinternet.com> - 2015-05-15 19:23 -0700
  Re: Mysterious slow IO on fread() to large buffer on large Dell system glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2015-05-16 05:05 +0000
    Re: Mysterious slow IO on fread() to large buffer on large Dell system Melzzzzz <mel@zzzzz.com> - 2015-05-16 07:20 +0200
    Re: Mysterious slow IO on fread() to large buffer on large Dell system mathog <dmathog@gmail.com> - 2015-05-15 22:27 -0700
      Re: Mysterious slow IO on fread() to large buffer on large Dell system Melzzzzz <mel@zzzzz.com> - 2015-05-16 07:59 +0200
      Re: Mysterious slow IO on fread() to large buffer on large Dell system Ian Collins <ian-news@hotmail.com> - 2015-05-16 18:31 +1200
  Re: Mysterious slow IO on fread() to large buffer on large Dell system Lew Pitcher <lew.pitcher@digitalfreehold.ca> - 2015-05-16 10:57 -0400
    Re: Mysterious slow IO on fread() to large buffer on large Dell system Joe Pfeiffer <pfeiffer@cs.nmsu.edu> - 2015-05-16 10:51 -0600
    Re: Mysterious slow IO on fread() to large buffer on large Dell system Malcolm McLean <malcolm.mclean5@btinternet.com> - 2015-05-16 11:41 -0700
      Re: Mysterious slow IO on fread() to large buffer on large Dell system Melzzzzz <mel@zzzzz.com> - 2015-05-16 21:01 +0200
        Re: Mysterious slow IO on fread() to large buffer on large Dell system glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2015-05-16 19:37 +0000
          Re: Mysterious slow IO on fread() to large buffer on large Dell system Noob <root@127.0.0.1> - 2015-05-16 22:14 +0200
    Re: Mysterious slow IO on fread() to large buffer on large Dell system gamo <gamo@telecable.es> - 2015-05-17 12:36 +0200
  Re: Mysterious slow IO on fread() to large buffer on large Dell system Paul <nospam@needed.com> - 2015-05-16 11:29 -0400
  Re: Mysterious slow IO on fread() to large buffer on large Dell system mathog <dmathog@gmail.com> - 2015-05-16 10:50 -0700
    Re: Mysterious slow IO on fread() to large buffer on large Dell system Johann Klammer <klammerj@NOSPAM.a1.net> - 2015-05-17 01:43 +0200
  Re: Mysterious slow IO on fread() to large buffer on large Dell system Stephen Sprunk <stephen@sprunk.org> - 2015-05-16 16:40 -0500
  Re: Mysterious slow IO on fread() to large buffer on large Dell system mathog <dmathog@gmail.com> - 2015-05-26 09:58 -0700

csiph-web