Path: csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!aioe.org!feeder.news-service.com!feeds.phibee-telecom.net!talisker.lacave.net!lacave.net!not-for-mail From: "Yoann M." Newsgroups: comp.lang.ruby Subject: Optimize write of large file Date: Thu, 12 May 2011 07:58:36 -0500 Organization: Service de news de lacave.net Lines: 19 Message-ID: NNTP-Posting-Host: bristol.highgroove.com Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Trace: talisker.lacave.net 1305205469 57786 65.111.164.187 (12 May 2011 13:04:29 GMT) X-Complaints-To: abuse@lacave.net NNTP-Posting-Date: Thu, 12 May 2011 13:04:29 +0000 (UTC) X-Received-From: This message has been automatically forwarded from the ruby-talk mailing list by a gateway at comp.lang.ruby. If it is SPAM, it did not originate at comp.lang.ruby. Please report the original sender, and not us. Thanks! For more details about this gateway, please visit: http://blog.grayproductions.net/categories/the_gateway X-Mail-Count: 383059 X-Ml-Name: ruby-talk X-Rubymirror: Yes X-Ruby-Talk: Xref: x330-a1.tempe.blueboxinc.net comp.lang.ruby:4352 Hello, I have data to process and to write into files progressively. The data files are in the end very large, but I append to them small strings. I suppose buffering the strings before apending to the file would be faster. I don't need the files to be written before the end of the whole process (i.e. I don't use their content). I've searched for info about how File buffer its data but it seems we can not configure anything about this, did I miss something ? My first idea was to buffer everything myself, appending lines to a string, or an array of strings and write when I reach a big enough amount of data. But if File uses a buffer anyway, it would be a waste of time I suppose ? Do you have any advice to optimize the writing of large files ? Thanks ! -- Posted via http://www.ruby-forum.com/.