Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.programmer > #38703

Re: Massiv creating and copying of files

Show all headers | View raw

On 1/17/2019 9:19 AM, Sandro Leinzinger wrote:
> I wrote a program which does this:
> 
> 1. Create a temp folder
> 2. Requesting a file from a web server and copy to temp folder
> 3. Generating some files in the temp folder from 1.
> 4. Generating a .iso-file from the temp-folder content with mkisofs
> 5. Zipping the whole content (3GB) of temp
> 
> All this operations are happening on the same harddisk array. The porblem is, that the array is getting slower and slower and works on 100% even with only 5 Threads executing this task.
> 
> Can some one tell me some optimisation in this task? Maybe create them in ram and then copy the zip to the hdd? But this would also bring the disks to 100% of load?

     See Arne Vajhøj's response.  I'll just add one possibility: If the
destination is truly an "array" of multiple disks, you might gain some
speed by splitting the I/O operations between distinct drives.  For
example, if the ISO file and the temp folder are on different drives,
so ISO creation reads from one drive while writing to another, you may
spend less time in disk seeks.

     However, if the "array" is already configured as one giant virtual
drive this strategy is probably inapplicable.

-- 
esosman@comcast-dot-net.invalid
Seven hundred thirty-four days to go.

Back to comp.lang.java.programmer | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Massiv creating and copying of files Sandro Leinzinger <leinzinger.sandro@googlemail.com> - 2019-01-17 06:19 -0800
  Re: Massiv creating and copying of files Arne Vajhøj <arne@vajhoej.dk> - 2019-01-17 09:30 -0500
  Re: Massiv creating and copying of files Eric Sosman <esosman@comcast-dot-net.invalid> - 2019-01-17 11:30 -0500
    Re: Massiv creating and copying of files Sandro Leinzinger <leinzinger.sandro@googlemail.com> - 2019-01-17 11:11 -0800
      Re: Massiv creating and copying of files Eric Douglas <e.d.programmer@gmail.com> - 2019-01-17 11:24 -0800
        Re: Massiv creating and copying of files Eric Sosman <esosman@comcast-dot-net.invalid> - 2019-01-17 17:16 -0500
          Re: Massiv creating and copying of files Martin Gregorie <martin@mydomain.invalid> - 2019-01-17 23:33 +0000
  Re: Massiv creating and copying of files Marcel Mueller <news.5.maazl@spamgourmet.org> - 2019-01-18 07:57 +0100

csiph-web