Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.os.linux.misc > #11853

Re: External hard drive usage. NTFS or other file system?

Show all headers | View raw

On Sat, 30 Aug 2014 14:42:37 +0200, Pascal Hambourg wrote:


> There is a huge gap between "contiguous" and "gazillion of small
> fragments". If fragments are big enough (wrt. the disk sequential speed
> * access time product), the adverse effects of fragmentation should be
> minimized.

It should be minimized, and it is minimized by using buffering. Buffering 
is the mechanism that offsets the effects of fragmentation. However, when 
I run 4 virtual machines simultaneously, each with 2GB or more of memory, 
memory is a precious commodity. I know, my PC needs an upgrade and it 
will get it around Christmas. I don't have enough time until then to plan 
that all.

> 
> In ext4, isn't the use of extents supposed to reduce fragmentation and
> improve large file performance ?

You are, of course, quite right. However, XFS does that better:

mgogala@medo RAC]$ du -sh sharedhd2.vdi 
21G	sharedhd2.vdi
[mgogala@medo RAC]$ filefrag *
sharedhd2.vdi: 1 extent found


[mgogala@medo oradb]$ ls
OraDB.vdi
[mgogala@medo oradb]$ du -sh *
33G	OraDB.vdi
[mgogala@medo oradb]$ filefrag *
OraDB.vdi: 1 extent found
[mgogala@medo oradb]$ 
mgogala@medo oradb]$ file OraDB.vdi
OraDB.vdi: VirtualBox Disk Image, major 1, minor 1 (<<< Oracle VM 
VirtualBox Disk Image >>>), 34359738368 bytes


I spoke of VMDK files because everyone is, in fact, familiar with VMWare. 
On Linux, and I am running Fedora 20, I prefer VirtualBox. VDI files are 
VirtualBox version of VMDK files. Ext4 has a maximum extent size of 128M. 
XFS manages to keep the files contiguous, which means that a sequential 
scan of several consecutive disk blocks will be done by a single multi-
block IO both on the virtual machine and on the actual disk. The machine 
is 4 years old and could use an upgrade, but until then, I will try 
squeezing every last drop of performance, even by de-fragmenting my large 
files, if necessary. Of course, I am using the deadline scheduler, too.

> Besides, shouldn't VM managers be aware of this issue and grow virtual
> disk files by big chunks in order to limit fragmentation too ?

They should and they are, but they could always use some help. Don't get 
me wrong, I have nothing against ext4. I use it on my desktop 
environment, which is Xubuntu 14.04 (XFCE spin off). Ext4 performs better 
when there is a myriad of small files to process. The largest files on my 
desktop are produced by LibreOffice and Evolution and are only few 
megabytes in size. Ext4 with CFQ scheduler is great for that environment, 
but not for my very own Simpana backup suite lab. That's the job for XFS.



-- 
Mladen Gogala
The Oracle Whisperer
http://mgogala.byethost5.com

Back to comp.os.linux.misc | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

External hard drive usage. NTFS or other file system? Evgenii Sputnik <esputnik7788@gmail.com> - 2014-07-31 13:35 -0400
  Re: External hard drive usage. NTFS or other file system? Aragorn <thorongil@telenet.be.invalid> - 2014-07-31 19:44 +0200
    Re: External hard drive usage. NTFS or other file system? Evgenii Sputnik <esputnik7788@gmail.com> - 2014-08-29 07:45 +0700
      Re: External hard drive usage. NTFS or other file system? Aragorn <thorongil@telenet.be.invalid> - 2014-08-29 03:37 +0200
      Re: External hard drive usage. NTFS or other file system? Robert Heller <heller@deepsoft.com> - 2014-08-28 21:09 -0500
        Re: External hard drive usage. NTFS or other file system? David Brown <david.brown@hesbynett.no> - 2014-08-29 21:16 +0200
    Re: External hard drive usage. NTFS or other file system? Mladen Gogala <gogala.mladen@gmail.com> - 2014-08-29 02:46 +0000
      Re: External hard drive usage. NTFS or other file system? Aragorn <thorongil@telenet.be.invalid> - 2014-08-29 05:14 +0200
        Re: External hard drive usage. NTFS or other file system? William Poaster <wp@dev.null> - 2014-08-29 14:45 +0100
          Re: External hard drive usage. NTFS or other file system? Mladen Gogala <gogala.mladen@gmail.com> - 2014-08-29 15:19 +0000
            Re: External hard drive usage. NTFS or other file system? Pascal Hambourg <boite-a-spam@plouf.fr.eu.org> - 2014-08-30 14:42 +0200
              Re: External hard drive usage. NTFS or other file system? Robert Heller <heller@deepsoft.com> - 2014-08-30 09:09 -0500
                Re: External hard drive usage. NTFS or other file system? Pascal Hambourg <boite-a-spam@plouf.fr.eu.org> - 2014-08-30 17:41 +0200
                Re: External hard drive usage. NTFS or other file system? Keith Keller <kkeller-usenet@wombat.san-francisco.ca.us> - 2014-08-30 15:21 -0700
                Re: External hard drive usage. NTFS or other file system? Mladen Gogala <gogala.mladen@gmail.com> - 2014-08-31 17:13 +0000
                Re: External hard drive usage. NTFS or other file system? Robert Heller <heller@deepsoft.com> - 2014-08-31 13:24 -0500
                Is VirtualBox "lame"? (Was: something else...) gazelle@shell.xmission.com (Kenny McCormack) - 2014-08-31 20:23 +0000
                Re: Is VirtualBox "lame"? (Was: something else...) Robert Heller <heller@deepsoft.com> - 2014-08-31 15:51 -0500
                Re: Is VirtualBox "lame"? Rich <rich@example.invalid> - 2014-08-31 23:58 +0000
                Re: Is VirtualBox "lame"? Robert Heller <heller@deepsoft.com> - 2014-08-31 21:54 -0500
                Re: Is VirtualBox "lame"? The Natural Philosopher <tnp@invalid.invalid> - 2014-09-01 07:54 +0100
                Re: Is VirtualBox "lame"? Marc Haber <mh+usenetspam1118@zugschl.us> - 2014-09-01 08:53 +0200
                Re: Is VirtualBox "lame"? (Was: something else...) The Natural Philosopher <tnp@invalid.invalid> - 2014-09-01 07:47 +0100
                Re: External hard drive usage. NTFS or other file system? Mladen Gogala <gogala.mladen@gmail.com> - 2014-08-31 21:00 +0000
                Re: External hard drive usage. NTFS or other file system? Rich <rich@example.invalid> - 2014-08-31 18:50 +0000
                Re: External hard drive usage. NTFS or other file system? Mladen Gogala <gogala.mladen@gmail.com> - 2014-08-31 21:01 +0000
                Re: External hard drive usage. NTFS or other file system? Rich <rich@example.invalid> - 2014-09-01 00:06 +0000
                Re: External hard drive usage. NTFS or other file system? The Natural Philosopher <tnp@invalid.invalid> - 2014-09-01 07:50 +0100
              Re: External hard drive usage. NTFS or other file system? Mladen Gogala <gogala.mladen@gmail.com> - 2014-08-31 03:18 +0000
  Re: External hard drive usage. NTFS or other file system? Robert Heller <heller@deepsoft.com> - 2014-07-31 12:54 -0500
    Re: External hard drive usage. NTFS or other file system? Unknown <dog@gmail.com> - 2014-08-14 12:39 +0000
      Re: External hard drive usage. NTFS or other file system? Keith Keller <kkeller-usenet@wombat.san-francisco.ca.us> - 2014-08-14 08:15 -0700
  Re: External hard drive usage. NTFS or other file system? Tim Watts <tw_usenet@dionic.net> - 2014-07-31 19:08 +0100
    Re: External hard drive usage. NTFS or other file system? Robert Heller <heller@deepsoft.com> - 2014-07-31 14:27 -0500
  Re: External hard drive usage. NTFS or other file system? Bobbie Sellers <bliss-sf4ever@dslextreme.com> - 2014-07-31 12:43 -0700

csiph-web