Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.programmer > #38704

Re: Massiv creating and copying of files

Show all headers | View raw

Am Donnerstag, 17. Januar 2019 17:30:25 UTC+1 schrieb Eric Sosman:
> On 1/17/2019 9:19 AM, Sandro Leinzinger wrote:
> > I wrote a program which does this:
> > 
> > 1. Create a temp folder
> > 2. Requesting a file from a web server and copy to temp folder
> > 3. Generating some files in the temp folder from 1.
> > 4. Generating a .iso-file from the temp-folder content with mkisofs
> > 5. Zipping the whole content (3GB) of temp
> > 
> > All this operations are happening on the same harddisk array. The porblem is, that the array is getting slower and slower and works on 100% even with only 5 Threads executing this task.
> > 
> > Can some one tell me some optimisation in this task? Maybe create them in ram and then copy the zip to the hdd? But this would also bring the disks to 100% of load?
> 
>      See Arne Vajhøj's response.  I'll just add one possibility: If the
> destination is truly an "array" of multiple disks, you might gain some
> speed by splitting the I/O operations between distinct drives.  For
> example, if the ISO file and the temp folder are on different drives,
> so ISO creation reads from one drive while writing to another, you may
> spend less time in disk seeks.
> 
>      However, if the "array" is already configured as one giant virtual
> drive this strategy is probably inapplicable.
> 
> -- 
> esosman@comcast-dot-net.invalid
> Seven hundred thirty-four days to go.

Thanks for your help ... i could boost the process by creating the files in a ram disk (4 at the same time) and then write it to disk. But you are right ... IO takes time and you have to wait :)

Back to comp.lang.java.programmer | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Massiv creating and copying of files Sandro Leinzinger <leinzinger.sandro@googlemail.com> - 2019-01-17 06:19 -0800
  Re: Massiv creating and copying of files Arne Vajhøj <arne@vajhoej.dk> - 2019-01-17 09:30 -0500
  Re: Massiv creating and copying of files Eric Sosman <esosman@comcast-dot-net.invalid> - 2019-01-17 11:30 -0500
    Re: Massiv creating and copying of files Sandro Leinzinger <leinzinger.sandro@googlemail.com> - 2019-01-17 11:11 -0800
      Re: Massiv creating and copying of files Eric Douglas <e.d.programmer@gmail.com> - 2019-01-17 11:24 -0800
        Re: Massiv creating and copying of files Eric Sosman <esosman@comcast-dot-net.invalid> - 2019-01-17 17:16 -0500
          Re: Massiv creating and copying of files Martin Gregorie <martin@mydomain.invalid> - 2019-01-17 23:33 +0000
  Re: Massiv creating and copying of files Marcel Mueller <news.5.maazl@spamgourmet.org> - 2019-01-18 07:57 +0100

csiph-web