Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.ruby > #4352

Optimize write of large file

Show all headers | View raw

Hello,
I have data to process and to write into files progressively. The data
files are in the end very large, but I append to them small strings. I
suppose buffering the strings before apending to the file would be
faster. I don't need the files to be written before the end of the whole
process (i.e. I don't use their content).

I've searched for info about how File buffer its data but it seems we
can not configure anything about this, did I miss something ?
My first idea was to buffer everything myself, appending lines to a
string, or an array of strings and write when I reach a big enough
amount of data. But if File uses a buffer anyway, it would be a waste of
time I suppose ?
Do you have any advice to optimize the writing of large files ?
Thanks !

-- 
Posted via http://www.ruby-forum.com/.

Back to comp.lang.ruby | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

Optimize write of large file "Yoann M." <yoann6@gmail.com> - 2011-05-12 07:58 -0500
  Re: Optimize write of large file Markus Schirp <mbj@seonic.net> - 2011-05-12 08:17 -0500
  Re: Optimize write of large file Jeremy Bopp <jeremy@bopp.net> - 2011-05-12 09:19 -0500
  Re: Optimize write of large file "Yoann M." <yoann6@gmail.com> - 2011-05-12 10:07 -0500
    Re: Optimize write of large file Robert Klemme <shortcutter@googlemail.com> - 2011-05-12 10:25 -0500
    Re: Optimize write of large file Markus Schirp <mbj@seonic.net> - 2011-05-12 10:51 -0500
  Re: Optimize write of large file "Yoann M." <yoann6@gmail.com> - 2011-05-13 02:21 -0500

csiph-web