Groups > comp.sys.mac.system > #71823 > unrolled thread

why does cloned mavericks drive use 5 gb less space than the original?

Started by	Bill F <anon@anonumost.org>
First post	2015-03-22 14:26 -0400
Last post	2015-03-23 16:18 -0700
Articles	20 on this page of 22 — 11 participants

Back to article view | Back to comp.sys.mac.system

Page 1 of 2 [1] 2 Next page →

#71823 — why does cloned mavericks drive use 5 gb less space than the original?

From	Bill F <anon@anonumost.org>
Date	2015-03-22 14:26 -0400
Subject	why does cloned mavericks drive use 5 gb less space than the original?
Message-ID	<2015032214264664806-anon@anonumostorg>

I make a bootable backup of my 120gb MBA SSD to a 120gb external drive 
partition.  The SSD then reports 7gb free space and the external drive 
reports 12gb free.  When I compare all folder sizes their total is 
within 500mb of each other.  I can't understand why the clone has so 
much more free space than the original.  All trashes are empty on both 
drives.

[toc] | [next] | [standalone]

#71827

From	JF Mezei <jfmezei.spamnot@vaxination.ca>
Date	2015-03-22 14:48 -0400
Message-ID	<550f0e97$0$34441$c3e8da3$dbd57e7@news.astraweb.com>
In reply to	#71823

On 15-03-22 14:26, Bill F wrote:
> I make a bootable backup of my 120gb MBA SSD to a 120gb external drive 
> partition.  The SSD then reports 7gb free space and the external drive 
> reports 12gb free.

What type of utility did you use to make the backup ?

If you do a disk repair or fsck on the source disk, does it release 5gb
of storage ?

Not sure about HFS, but on some file systems you can have temporary
files that are not entered in any directory, so they may or may not be
captured by a backup utility depending on how it works.

For instance, on VMS, the BACKUP utility will traverse directory
structure to collect files. But BACKUP/IMAGE will traverse the
INDEXF.SYS catalogue to ensure it backs up all files, including those
with no entries in any directory.

A "smart" BACKUP may also decide to not backup page/swap files if it
knows where they are. On VMS, those files are marked "no backup", biut
BACKUP will recreate an empty file on the target with exact same file
attributes (allocation etc). So it ends up taking the same space. On
VMS, such files are permanent. On BSD/OSX they can be created on the fly
and you don't need to back them up.

look to see if you have files on /var/vm on the target drive compared to
the source drive.

same with /tmp and /var/tmp

The warning: depending on what utility you used, it may have omitted
files locked because they are being written to. This is much rarely the
case on Unix as file locking is far less strict.

Are you able to do a comparison of the 2 disks ? (does OS-X have that
ability ?) This may provide explanation of whether there are any missing
files.

[toc] | [prev] | [next] | [standalone]

#71951

From	Lewis <g.kreme@gmail.com.dontsendmecopies>
Date	2015-03-23 18:29 +0000
Message-ID	<slrnmh0mtq.16dp.g.kreme@kreme.eternal-september.org>
In reply to	#71827

Okay, so one time? In band camp? JF Mezei <jfmezei.spamnot@vaxination.ca> was all, like:
> On 15-03-22 14:26, Bill F wrote:
>> I make a bootable backup of my 120gb MBA SSD to a 120gb external drive 
>> partition.  The SSD then reports 7gb free space and the external drive 
>> reports 12gb free.

> What type of utility did you use to make the backup ?

> If you do a disk repair or fsck on the source disk, does it release 5gb
> of storage ?

> Not sure about HFS, but on some file systems you can have temporary
> files that are not entered in any directory, so they may or may not be
> captured by a backup utility depending on how it works.

I can't imagine any backup utility would backup the vm files and the
sleep image. That would be 5GB right there, probably. No utility should
backup cache files either. I'd expect a bootable backup to be quite a
lot smaller than just 5GB difference.

$ du -shc /var/vm/*
4.0G	/var/vm/sleepimage
1.0G	/var/vm/swapfile0
5.0G	total


-- 
http://2blog.kreme.com
Silence is golden, duct tape is silver.

[toc] | [prev] | [next] | [standalone]

#71831

From	Your Name <YourName@YourISP.com>
Date	2015-03-23 09:13 +1200
Message-ID	<230320150913098187%YourName@YourISP.com>
In reply to	#71823

In article <2015032214264664806-anon@anonumostorg>, Bill F
<anon@anonumost.org> wrote:
> 
> I make a bootable backup of my 120gb MBA SSD to a 120gb external drive 
> partition.  The SSD then reports 7gb free space and the external drive 
> reports 12gb free.  When I compare all folder sizes their total is 
> within 500mb of each other.  I can't understand why the clone has so 
> much more free space than the original.  All trashes are empty on both 
> drives.  

The block size for data can vary for different drives and formats. You
often saw this when copying files between floppy disks and hard drives.

Some drives may use (for example) a 4K block and others a 8K block. If
you save a 3K file onto the second drive it will need to take up a 8K
block, but saved onto the first drive it can fit in a 4K block. It
seems to take up less space on the first drive when you look at the KB
/ MB / GB size in the Get Info window, but if you look at the number of
*bytes* in the Get Info window they're exactly the same size.

[toc] | [prev] | [next] | [standalone]

#71835

From	dempson@actrix.gen.nz (David Empson)
Date	2015-03-23 09:58 +1300
Message-ID	<1m1qd8k.1mqjvx61p4mypcN%dempson@actrix.gen.nz>
In reply to	#71831

Your Name <YourName@YourISP.com> wrote:

> In article <2015032214264664806-anon@anonumostorg>, Bill F
> <anon@anonumost.org> wrote:
> > 
> > I make a bootable backup of my 120gb MBA SSD to a 120gb external drive
> > partition.  The SSD then reports 7gb free space and the external drive
> > reports 12gb free.  When I compare all folder sizes their total is 
> > within 500mb of each other.  I can't understand why the clone has so
> > much more free space than the original.  All trashes are empty on both
> > drives.  
> 
> The block size for data can vary for different drives and formats. You
> often saw this when copying files between floppy disks and hard drives.
> 
> Some drives may use (for example) a 4K block and others a 8K block. If
> you save a 3K file onto the second drive it will need to take up a 8K
> block, but saved onto the first drive it can fit in a 4K block.

That isn't relevant for two drives in this capacity range, assuming both
are using HFS+ (Mac OS Extended) as their file system, which they should
be for a bootable clone.

HFS+ has 32-bit allocation block numbers and standardises on 4 KB
allocation block sizes for drive capacities up to and including low
terabytes. For example, on my 2 TB backup drive (which has 512 byte
physical blocks) HFS+ uses 4 KB allocation blocks, giving it about 500
million allocation blocks.

Above 16 TB, HFS+ will need to switch to 8 KB (or larger) allocation
blocks, and it might do that at an earlier point, e.g. 8 TB to avoid the
possibility of some buggy implementation misinterpreting a 32-bit
unsigned integer greater than 2^31 as a negative signed integer.

I can't check what allocation block size my Drobo (nominally 16 TB) is
using, because it appears that "diskutil info" in Snow Leopard doesn't
reveal the allocation block size (it does in Yosemite, if you enquire
about the volume/partition).

-- 
David Empson
dempson@actrix.gen.nz

[toc] | [prev] | [next] | [standalone]

#71832

From	dempson@actrix.gen.nz (David Empson)
Date	2015-03-23 09:27 +1300
Message-ID	<1m1qc8y.uheta417jeiv4N%dempson@actrix.gen.nz>
In reply to	#71823

Bill F <anon@anonumost.org> wrote:

> I make a bootable backup of my 120gb MBA SSD to a 120gb external drive
> partition.  The SSD then reports 7gb free space and the external drive
> reports 12gb free.  When I compare all folder sizes their total is 
> within 500mb of each other.  I can't understand why the clone has so 
> much more free space than the original.  All trashes are empty on both
> drives.  

How do the volume sizes compare? Nominal "120 GB" drives could differ by
up to 500 MB in capacity, simply due to rounding (assuming both
manufacturers rounded down). The difference might be even more than
that, e.g. if a drive actually had 124 GB of usable capacity it would
probably be advertised as 120 GB.

Depending on which utility you used to clone the drive, the clone might
be missing the recovery partition, which would result in it having 650
MB more space available for its main volume.

Is your folder size comparison including all the invisible folders?

For example, the Spotlight index folder on my main drive (in the root
directory and hidden) is 1.8 GB. If your backup drive hasn't been
indexed yet that would account for a big difference in free space.

A clone will not copy caches and temporary files, many of which are in
hidden folders. For example, on my system, /private/var/folders is 4.3
GB, but on the clone I made yesterday (and then booted from) it is only
150 MB.

Have you booted from your clone yet? If not, some of those caches have
not yet been created on the clone.

-- 
David Empson
dempson@actrix.gen.nz

[toc] | [prev] | [next] | [standalone]

#71837

From	tmo1138 <tmo1138@tmo1138.com>
Date	2015-03-22 21:44 +0000
Message-ID	<slrnmgue04.bk5.tmo1138@davidjr-macpro.local>
In reply to	#71823

On 2015-03-22, Bill F <anon@anonumost.org> wrote:
> I make a bootable backup of my 120gb MBA SSD to a 120gb external drive 
> partition.  The SSD then reports 7gb free space and the external drive 
> reports 12gb free.  When I compare all folder sizes their total is 
> within 500mb of each other.  I can't understand why the clone has so 
> much more free space than the original.  All trashes are empty on both 
> drives.  
>

No recovery partion?

-- 
Perfection is reached, not when there is no longer anything to add, but
when there is no longer anything to take away.
		-- Antoine de Saint-Exupery

[toc] | [prev] | [next] | [standalone]

#71845

From	billy@MIX.COM
Date	2015-03-22 22:30 +0000
Message-ID	<menfqe$ehe$1@reader2.panix.com>
In reply to	#71837

tmo1138 <tmo1138@tmo1138.com> writes, quoting
Bill F <anon@anonumost.org>:

> > I make a bootable backup of my 120gb MBA SSD to a 120gb external drive 
> > partition.  The SSD then reports 7gb free space and the external drive 
> > reports 12gb free.  When I compare all folder sizes their total is 
> > within 500mb of each other.  I can't understand why the clone has so 
> > much more free space than the original.  All trashes are empty on both 
> > drives.  
> 
> No recovery partion?

The recovery partition is only 650MB.  Maybe the external drive is
using a block size smaller than 4096 bytes?

Billy Y..
-- 
        sub     #'9+1   ,r0             ; convert ascii byte
	add     #9.+1   ,r0             ; to an integer
	bcc     20$                     ; not a number

[toc] | [prev] | [next] | [standalone]

#71851

From	Jolly Roger <jollyroger@pobox.com>
Date	2015-03-22 23:49 +0000
Message-ID	<cn92ngF5ssnU11@mid.individual.net>
In reply to	#71823

On 2015-03-22, Bill F <anon@anonumost.org> wrote:
> I make a bootable backup of my 120gb MBA SSD to a 120gb external drive 
> partition.  The SSD then reports 7gb free space and the external drive 
> reports 12gb free.  When I compare all folder sizes their total is 
> within 500mb of each other.  I can't understand why the clone has so 
> much more free space than the original.  All trashes are empty on both 
> drives.  

Probably just cache files that will be re-created to initial small
states during first boot on the new drive, and grow larger from there
over time.

-- 
E-mail sent to this address may be devoured by my ravenous SPAM filter.
I often ignore posts from Google. Use a real news client instead.

JR

[toc] | [prev] | [next] | [standalone]

#71863

From	JF Mezei <jfmezei.spamnot@vaxination.ca>
Date	2015-03-22 20:35 -0400
Message-ID	<550f5fb5$0$41643$c3e8da3$5d8fb80f@news.astraweb.com>
In reply to	#71851

On 15-03-22 19:49, Jolly Roger wrote:

> Probably just cache files that will be re-created to initial small
> states during first boot on the new drive, and grow larger from there
> over time.

If a cache file has an entry in the catalogue, it should be copied over
by the backup program.

Unless of course, the backup program was built with specific knowledge
of what directories not to backup or which files or file types to exclude.

One other possibility, and again, this depends on how the backup
application works, is that the app creates files on target disk based on
space used instead of space allocated.

Say you allicate a 500MB file and your app starts to write to it and has
only writte records in the first 200MB. The "end of file" marker would
be at the 200MB mark. So a backup program COULD choose to create a 200MB
file on target drive and copy all fo the data from the original file.

It really all depends on which application you used to make the backup.

[toc] | [prev] | [next] | [standalone]

#71874

From	Jolly Roger <jollyroger@pobox.com>
Date	2015-03-23 01:21 +0000
Message-ID	<cn984oF6nbtU7@mid.individual.net>
In reply to	#71863

On 2015-03-23, JF Mezei <jfmezei.spamnot@vaxination.ca> wrote:
> On 15-03-22 19:49, Jolly Roger wrote:
>
>> Probably just cache files that will be re-created to initial small
>> states during first boot on the new drive, and grow larger from there
>> over time.
>
> If a cache file has an entry in the catalogue, it should be copied over
> by the backup program.
>
> Unless of course, the backup program was built with specific knowledge
> of what directories not to backup or which files or file types to exclude.

Duh. That's exactly the case.

> It really all depends on which application you used to make the backup.

Yep.

-- 
E-mail sent to this address may be devoured by my ravenous SPAM filter.
I often ignore posts from Google. Use a real news client instead.

JR

[toc] | [prev] | [next] | [standalone]

#71882

From	dempson@actrix.gen.nz (David Empson)
Date	2015-03-23 14:41 +1300
Message-ID	<1m1qp7c.f3pp8u9sabptN%dempson@actrix.gen.nz>
In reply to	#71863

JF Mezei <jfmezei.spamnot@vaxination.ca> wrote:

> On 15-03-22 19:49, Jolly Roger wrote:
> 
> > Probably just cache files that will be re-created to initial small
> > states during first boot on the new drive, and grow larger from there
> > over time.
> 
> If a cache file has an entry in the catalogue, it should be copied over
> by the backup program.
> 
> Unless of course, the backup program was built with specific knowledge
> of what directories not to backup or which files or file types to exclude.

That is exactly what does happen for utilities like SuperDuper! and
Carbon Copy Cloner. They have rules for folders to skip during the copy,
because they know that copying caches will just waste time.

I normally use SuperDuper!, and my most recent copy log shows that it
ignored these:

/cores
/.vol
/private/var/folders/*          User-specific caches and temp items
/private/var/sleepimage         Same size as the computer's memory
/private/var/tmp/(various)
/private/var/db/(various)
/private/var/run/(various)
/private/tmp/(various)
/.MobileBackups                 Time Machine local backups
/.DocumentRevisions-V100        Autosave/Versions revision history
/Volumes/*                      Other mounted volumes

It also special cased /.Spotlight-V100 (Spotlight index)

It didn't mention ignoring any caches in /System, /Library or /Users,
but I'd have to do a comparison to work out whether it did in fact skip
them.

-- 
David Empson
dempson@actrix.gen.nz

[toc] | [prev] | [next] | [standalone]

#71892

From	nmassello@yahoo.com (Neill Massello)
Date	2015-03-22 20:23 -0600
Message-ID	<1m1pcbi.mbl558eelwhoN%nmassello@yahoo.com>
In reply to	#71882

David Empson <dempson@actrix.gen.nz> wrote:

> I normally use SuperDuper!, and my most recent copy log shows that it
> ignored these:
> 
> /cores
> /.vol
> /private/var/folders/*          User-specific caches and temp items

That last one is where most of the missing space is likely to be. On my
boot drive, it's is more than 2GB, whereas the Spotlight folder is less
than 150MB.

[toc] | [prev] | [next] | [standalone]

#71952

From	Lewis <g.kreme@gmail.com.dontsendmecopies>
Date	2015-03-23 18:30 +0000
Message-ID	<slrnmh0n0f.16dp.g.kreme@kreme.eternal-september.org>
In reply to	#71863

Okay, so one time? In band camp? JF Mezei <jfmezei.spamnot@vaxination.ca> was all, like:
> On 15-03-22 19:49, Jolly Roger wrote:

>> Probably just cache files that will be re-created to initial small
>> states during first boot on the new drive, and grow larger from there
>> over time.

> If a cache file has an entry in the catalogue, it should be copied over
> by the backup program.

No it shouldn't. That would be a very poor backup program.

-- 
http://2blog.kreme.com
"Esprit de corpse," said the Senior Wrangler. Ridcully glared at him

[toc] | [prev] | [next] | [standalone]

#71865

From	Bill F <anon@anonumost.org>
Date	2015-03-22 20:40 -0400
Message-ID	<2015032220402161101-anon@anonumostorg>
In reply to	#71823

I made the clone with Superduper.  I have booted fthe MBA from both 
drives.  The two drives are the same useable size,  The MBA SSD is 
128gb which is 120.1gb useable as formatted.  The external drive is a 
120.4gb partitionon a 2tb Samsung P3 external drive.

I can look at both drives using Pathf Fnder 7 which shows both visible 
and invisible files and file sizes.    My private folder is 0.5gb 
larger on the MBA than the P3.  The spotlight file is 1gb on the P3 and 
0gb on the SSD.  The spotlight file on the SSD has a mod date of 131029 
so that can't be the actual spotlight file date and zero can't be the 
correct size.  I suspect the real spotlight file is probably where the 
missing 5gb is hiding but don't know why it does not show up in Path 
Finder.

[toc] | [prev] | [next] | [standalone]

#71883

From	dempson@actrix.gen.nz (David Empson)
Date	2015-03-23 14:41 +1300
Message-ID	<1m1qq4r.1xngn4msyae4tN%dempson@actrix.gen.nz>
In reply to	#71865

Bill F <anon@anonumost.org> wrote:

> I made the clone with Superduper.  I have booted fthe MBA from both 
> drives.  The two drives are the same useable size,  The MBA SSD is 
> 128gb which is 120.1gb useable as formatted.  The external drive is a
> 120.4gb partitionon a 2tb Samsung P3 external drive.

The external partition is smaller than the original, therefore it may
have less space preallocated to store HFS+ directory information. I
doubt that would account for more than about 7% of 8 GB (ballpark 500
MB), but all the differences are starting to add up.

> I can look at both drives using Pathf Fnder 7 which shows both visible
> and invisible files and file sizes.    My private folder is 0.5gb 
> larger on the MBA than the P3.  The spotlight file is 1gb on the P3 and
> 0gb on the SSD.  The spotlight file on the SSD has a mod date of 131029
> so that can't be the actual spotlight file date and zero can't be the
> correct size.

That date was difficult to parse. I assume you meant 2013-10-29?

My /.Spotlight-V100 folder has a similar modification date, which
happens to be the same as its creation date. The top level folder's
modification date isn't updated by the system when the Spotlight index
is updated. I have to go two levels deep into my Spotlight index to find
a folder which shows a current modification date.

> I suspect the real spotlight file is probably where the missing 5gb is
> hiding but don't know why it does not show up in Path Finder.

The /.Spotlight-V100 folder is one of many which only the root user can
access. Unless you've done something to launch Path Finder with root
privileges (which I do NOT recommend), it can't look inside that folder,
even to find its size, so it reports the size as zero bytes.

I was able to find the size of that folder using this command in
Terminal:

sudo du -h -d 0 /.Spotlight-V100

Explanation:
- sudo means "run this command as the root user".
- du calculates and displays disk usage.
- The "-h" option displays the results in human-readible form.
- The "-d 0" option skips displaying partial results for all the folders
inside the top level folder. (If you omit this option, you want the last
line it outputs.)

This is just one example. There will be others, e.g. in
/private/var/folders for the system or other user accounts, which your
user account can't access, therefore Path Finder can't report its actual
size.

-- 
David Empson
dempson@actrix.gen.nz

[toc] | [prev] | [next] | [standalone]

#71893

From	nmassello@yahoo.com (Neill Massello)
Date	2015-03-22 20:23 -0600
Message-ID	<1m1pbu6.ia9cmy1fuc0w0N%nmassello@yahoo.com>
In reply to	#71883

David Empson <dempson@actrix.gen.nz> wrote:

> I was able to find the size of that folder using this command in
> Terminal:
> 
> sudo du -h -d 0 /.Spotlight-V100

The OP can also use Path Finder's "Launch as Root" command with
OmniDiskSweeper to see the size of /.Spotlight-V100 (and other root-only
folders) without resorting to the command line.

[toc] | [prev] | [next] | [standalone]

#72102

From	Bill F <anon@anonumost.org>
Date	2015-03-26 03:00 -0400
Message-ID	<2015032603003575695-anon@anonumostorg>
In reply to	#71893

On 2015-03-23 02:23:12 +0000, Neill Massello said:

> David Empson <dempson@actrix.gen.nz> wrote:
> 
>> I was able to find the size of that folder using this command in
>> Terminal:
>> 
>> sudo du -h -d 0 /.Spotlight-V100
> 
> The OP can also use Path Finder's "Launch as Root" command with
> OmniDiskSweeper to see the size of /.Spotlight-V100 (and other root-only
> folders) without resorting to the command line.

Thanks to everyone who provided answers to my question about clone size 
being smaller than the original.   It was enlightening.

[toc] | [prev] | [next] | [standalone]

#72200

From	Electric Comet <electric-comet@mail.invalid>
Date	2015-03-27 20:56 -0700
Message-ID	<mf58q3$ell$7@dont-email.me>
In reply to	#72102

On Thu, 26 Mar 2015 03:00:35 -0400
Bill F <anon@anonumost.org> wrote:

> size being smaller than the original.   It was enlightening.

enlightening but what was the reason for the difference
was it all the dot files/dirs

[toc] | [prev] | [next] | [standalone]

#72257

From	Jolly Roger <jollyroger@pobox.com>
Date	2015-03-28 21:07 +0000
Message-ID	<cnojgmF5h8sU10@mid.individual.net>
In reply to	#72200

On 2015-03-28, Electric Comet <electric-comet@mail.invalid> wrote:
> On Thu, 26 Mar 2015 03:00:35 -0400
> Bill F <anon@anonumost.org> wrote:
>
>> size being smaller than the original.   It was enlightening.
>
> enlightening but what was the reason for the difference
> was it all the dot files/dirs

Cache files. Please try to keep up.

-- 
E-mail sent to this address may be devoured by my ravenous SPAM filter.
I often ignore posts from Google. Use a real news client instead.

JR

[toc] | [prev] | [next] | [standalone]

Page 1 of 2 [1] 2 Next page →

csiph-web

why does cloned mavericks drive use 5 gb less space than the original?

Contents

#71823 — why does cloned mavericks drive use 5 gb less space than the original?

#71827

#71951

#71831

#71835

#71832

#71837

#71845

#71851

#71863

#71874

#71882

#71892

#71952

#71865

#71883

#71893

#72102

#72200

#72257