Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.sys.mac.system > #107450 > unrolled thread

Mechanics of the Apple File System transition?

Back to article view | Back to comp.sys.mac.system

Contents

  Mechanics of the Apple File System transition? Alan Browne <alan.browne@freelunchvideotron.ca> - 2017-06-05 17:21 -0400
    Re: Mechanics of the Apple File System transition? JF Mezei <jfmezei.spamnot@vaxination.ca> - 2017-06-05 18:01 -0400
    Re: Mechanics of the Apple File System transition? dempson@actrix.gen.nz (David Empson) - 2017-06-06 12:03 +1200
      Re: Mechanics of the Apple File System transition? dempson@actrix.gen.nz (David Empson) - 2017-06-06 13:05 +1200
        Re: Mechanics of the Apple File System transition? JF Mezei <jfmezei.spamnot@vaxination.ca> - 2017-06-05 22:14 -0400
          Re: Mechanics of the Apple File System transition? dempson@actrix.gen.nz (David Empson) - 2017-06-06 14:50 +1200
            Re: Mechanics of the Apple File System transition? JF Mezei <jfmezei.spamnot@vaxination.ca> - 2017-06-05 23:44 -0400
              Re: Mechanics of the Apple File System transition? Barry Margolin <barmar@alum.mit.edu> - 2017-06-07 11:02 -0400
                Re: Mechanics of the Apple File System transition? dempson@actrix.gen.nz (David Empson) - 2017-06-08 08:29 +1200
        Re: Mechanics of the Apple File System transition? Alan Browne <alan.browne@freelunchvideotron.ca> - 2017-06-06 19:36 -0400
          Re: Mechanics of the Apple File System transition? JF Mezei <jfmezei.spamnot@vaxination.ca> - 2017-06-06 23:40 -0400
            Re: Mechanics of the Apple File System transition? dempson@actrix.gen.nz (David Empson) - 2017-06-08 08:23 +1200
              Re: Mechanics of the Apple File System transition? JF Mezei <jfmezei.spamnot@vaxination.ca> - 2017-06-08 10:23 -0400
                Re: Mechanics of the Apple File System transition? Jolly Roger <jollyroger@pobox.com> - 2017-06-08 15:56 +0000
                Re: Mechanics of the Apple File System transition? dempson@actrix.gen.nz (David Empson) - 2017-06-09 09:23 +1200
                  Re: Mechanics of the Apple File System transition? JF Mezei <jfmezei.spamnot@vaxination.ca> - 2017-06-08 21:49 -0400
                    Re: Mechanics of the Apple File System transition? nospam <nospam@nospam.invalid> - 2017-06-09 00:44 -0400
                      Re: Mechanics of the Apple File System transition? JF Mezei <jfmezei.spamnot@vaxination.ca> - 2017-06-09 14:28 -0400
                        Re: Mechanics of the Apple File System transition? JF Mezei <jfmezei.spamnot@vaxination.ca> - 2017-06-09 15:33 -0400
                        Re: Mechanics of the Apple File System transition? dempson@actrix.gen.nz (David Empson) - 2017-06-10 13:38 +1200
                          Re: Mechanics of the Apple File System transition? JF Mezei <jfmezei.spamnot@vaxination.ca> - 2017-06-09 22:24 -0400
                            Re: Mechanics of the Apple File System transition? Alan Baker <alangbaker@telus.net> - 2017-06-09 19:29 -0700
                            Re: Mechanics of the Apple File System transition? dempson@actrix.gen.nz (David Empson) - 2017-06-10 15:59 +1200
                      Re: Mechanics of the Apple File System transition? dempson@actrix.gen.nz (David Empson) - 2017-06-10 15:59 +1200
                        Re: Mechanics of the Apple File System transition? JF Mezei <jfmezei.spamnot@vaxination.ca> - 2017-06-12 23:56 -0400
                  Re: Mechanics of the Apple File System transition? nmassello@yahoo.com (Neill Massello) - 2017-06-09 13:53 -0600
                    Re: Mechanics of the Apple File System transition? Jolly Roger <jollyroger@pobox.com> - 2017-06-09 23:40 +0000
                    Re: Mechanics of the Apple File System transition? dempson@actrix.gen.nz (David Empson) - 2017-06-10 13:38 +1200
                      Re: Mechanics of the Apple File System transition? JF Mezei <jfmezei.spamnot@vaxination.ca> - 2017-06-09 22:27 -0400
                        Re: Mechanics of the Apple File System transition? dempson@actrix.gen.nz (David Empson) - 2017-06-10 15:58 +1200
      Re: Mechanics of the Apple File System transition? Alan Browne <alan.browne@freelunchvideotron.ca> - 2017-06-06 19:29 -0400
        Re: Mechanics of the Apple File System transition? Your Name <YourName@YourISP.com> - 2017-06-07 13:02 +1200

Page 1 of 2  [1] 2  Next page →

#107450 — Mechanics of the Apple File System transition?

I'm wary of fundamental changes such as the new AFS that is coming.

Will files be converted as needed, or in the background?  Or as a part 
of installing prior to booting under the new system?

How about external drives?  Will they need to be re-formatted?

-- 
"If war is God's way of teaching Americans geography, then
recession is His way of teaching everyone a little economics."
   ..Raj Patel, The Value of Nothing.

[toc] | [next] | [standalone]

#107453

On 2017-06-05 17:21, Alan Browne wrote:

> Will files be converted as needed, or in the background?  Or as a part 
> of installing prior to booting under the new system?

This was discussed during the IOS 10.3 released whcih di the conversion
in-situ and quite seamless.

I suspect in a few days, more will be know as WWDC presentations are
bound to happen on APFS.

Normally, this will happen during the time of the upgrade where the
system boots from a temp partition and gets to build a new APFS file
index and structures on your real partition.

The files remain where they are. Only the metadate , and file
structure/index are recreated using the new file system and I suspect
the old ones (catalogue files etc) deleted once all is said and done.

> How about external drives?  Will they need to be re-formatted?

The Macs will retain the ability to deal with HFS+ drives. So you don't
NEED to reformat other drives, but on those you can't take advantage of
the new features.

I *assume* that there will be a tool to convert drives from HFS to APFS
(since that tool exists to convert the system drive).


BTW in the keynote, there was no mention of
Time Machine/backups, which was an unresolved issue as of last year and
I assume is now resolved.

[toc] | [prev] | [next] | [standalone]

#107458

Alan Browne <alan.browne@freelunchvideotron.ca> wrote:

> I'm wary of fundamental changes such as the new AFS that is coming.
> 
> Will files be converted as needed, or in the background?

The file system conversion must be done as a single step, but thanks to
the flexibility of APFS, it only needs to write new directory structures
and possibly fiddle with the partitioning, not touch any file content.

> Or as a part of installing prior to booting under the new system?

Someone has leaked to macrumors.com screen shots of installing the first
developer preview (in comments on the article about the preview). It
shows a screen during installation asking whether to convert the drive
to APFS, with the checkbox off in the picture (no idea if that is the
default, and the details may change by the public release).

On that basis, the conversion will optionally happen as part of
installing High Sierra. I expect there will be a manual method of
converting to APFS later, which will probably involve a restart if you
do it on the startup drive.

The keynote said that APFS would be the default for new machines.

> How about external drives?  Will they need to be re-formatted?

No. I expect there will be an optional "convert to APFS" feature for
external drives, which will probably involve an unmount, directory
rewrite, then remount. It might take a while depending on the number of
files (Time Machine drives especially).

Obviously you should only do this with external drives that are going to
be used exclusively with recent macOS versions (10.12 Sierra or later,
unless Apple ports limited APFS support back to 10.11 El Capitan - at
the moment El Capitan knows to ignore APFS volumes, but earlier OS X
versions will offer to initialize the drive as they have no idea what
the file system is).

We'll probably know more later in WWDC once they have the APFS session,
and there may be more info in the "Platform State of the Union" address,
which I won't have time to watch for the next few hours.

-- 
David Empson
dempson@actrix.gen.nz

[toc] | [prev] | [next] | [standalone]

#107460

David Empson <dempson@actrix.gen.nz> wrote:

> Alan Browne <alan.browne@freelunchvideotron.ca> wrote:
> 
> > I'm wary of fundamental changes such as the new AFS that is coming.
> > 
> > Will files be converted as needed, or in the background?
> 
> The file system conversion must be done as a single step, but thanks to
> the flexibility of APFS, it only needs to write new directory structures
> and possibly fiddle with the partitioning, not touch any file content.
> 
> > Or as a part of installing prior to booting under the new system?
> 
> Someone has leaked to macrumors.com screen shots of installing the first
> developer preview (in comments on the article about the preview). It
> shows a screen during installation asking whether to convert the drive
> to APFS, with the checkbox off in the picture (no idea if that is the
> default, and the details may change by the public release).
> 
> On that basis, the conversion will optionally happen as part of
> installing High Sierra. I expect there will be a manual method of
> converting to APFS later, which will probably involve a restart if you
> do it on the startup drive.
> 
> The keynote said that APFS would be the default for new machines.
> 
> > How about external drives?  Will they need to be re-formatted?
> 
> No. I expect there will be an optional "convert to APFS" feature for
> external drives, which will probably involve an unmount, directory
> rewrite, then remount. It might take a while depending on the number of
> files (Time Machine drives especially).
> 
> Obviously you should only do this with external drives that are going to
> be used exclusively with recent macOS versions (10.12 Sierra or later,
> unless Apple ports limited APFS support back to 10.11 El Capitan - at
> the moment El Capitan knows to ignore APFS volumes, but earlier OS X
> versions will offer to initialize the drive as they have no idea what
> the file system is).
> 
> We'll probably know more later in WWDC once they have the APFS session,
> and there may be more info in the "Platform State of the Union" address,
> which I won't have time to watch for the next few hours.

Further minor details are in this document, updated for macOS 10.13 High
Sierra:

https://developer.apple.com/library/content/documentation/FileManagement/Conceptual/APFS_Guide/Introduction/Introduction.html#//apple_ref/doc/uid/TP40016999

From the FAQ:

[quote]
How do I upgrade to Apple File System?

The macOS 10.13 installer offers nondestructive in-place upgrades from
HFS+ to APFS for bootable volumes. You can use Disk Utility to convert
external volumes from HFS+ to APFS format.

If I convert a volume to APFS, can I later revert to HFS+?

You can use Disk Utility to erase an APFS-formatted volume and reformat
as HFS+. However, your data will not be preserved when you reformat the
volume as HFS+.
[end quote]

I expect that if you skip converting your boot volume during install,
doing the conversion later would require booting from another volume
with High Sierra installed, e.g. the recovery partition.

-- 
David Empson
dempson@actrix.gen.nz

[toc] | [prev] | [next] | [standalone]

#107462

On 2017-06-05 21:05, David Empson wrote:

> https://developer.apple.com/library/content/documentation/FileManagement/Conceptual/APFS_Guide/Introduction/Introduction.html#//apple_ref/doc/uid/TP40016999
##
Does Apple File System support directory hard links?

Directory hard links are not supported by Apple File System. All
directory hard links are converted to symbolic links or aliases when you
convert from HFS+ to APFS volume formats on macOS.
##


In the case of iPhoto and Photos libraries that are correctly using same
storage via hard links, is there a way to predict which of the file
entries will have the "real" file and which will have just a symbolic link?

(aka: once converted to symbilic link, you can't delete 1 version before
you check whether it has the original file or is just a symbolic link)

Or will symbolic links end up using the fancy copy of a file in APFS
where blocks are shared and you can delete either without worrying about
losing the other too?

[toc] | [prev] | [next] | [standalone]

#107463

JF Mezei <jfmezei.spamnot@vaxination.ca> wrote:

> On 2017-06-05 21:05, David Empson wrote:
> 
>>
https://developer.apple.com/library/content/documentation/FileManagement/Conceptual/APFS_Guide/Introduction/Introduction.html#//apple_ref/doc/uid/TP40016999
> ##
> Does Apple File System support directory hard links?
> 
> Directory hard links are not supported by Apple File System. All
> directory hard links are converted to symbolic links or aliases when you
> convert from HFS+ to APFS volume formats on macOS.
> ##
> 
> 
> In the case of iPhoto and Photos libraries that are correctly using same
> storage via hard links, is there a way to predict which of the file
> entries will have the "real" file and which will have just a symbolic link?

Read the text you quoted again.

*Directory* hard links are not supported by Apple File System.

Linked photos between an iPhoto and Photos library does NOT use
directory hard links, just file hard links.

Only Time Machine uses directory hard links.

We haven't seen any more information yet about how APFS will work with
Time Machine, or whether it will be possible to convert a TM backup
drive from HFS+ to APFS.

I'll wait until the WWDC session on the file system (Friday afternoon,
US time) before jumping to conclusions, but the above text suggests that
it may not be possible to convert a TM backup and keep using it.

-- 
David Empson
dempson@actrix.gen.nz

[toc] | [prev] | [next] | [standalone]

#107465

On 2017-06-05 22:50, David Empson wrote:

> *Directory* hard links are not supported by Apple File System.

My bad. I have no excuse , the word "DIRECTORY HARD LINKS" was right
there in my face.


> Linked photos between an iPhoto and Photos library does NOT use
> directory hard links, just file hard links.

> Only Time Machine uses directory hard links.

Which was interestingly not mentioned in the keynote. ( I guess they
don't want to get too much into details).


> We haven't seen any more information yet about how APFS will work with
> Time Machine, or whether it will be possible to convert a TM backup
> drive from HFS+ to APFS.

From the point of view of a utility like Disk Util, would a TM drive
look different from any other HFS+ drive ?

From the point of view of rolling back an upgrade to previous version, I
would think that any TM volume would remain untouched during the upgrade.

Say I go from Yosemite to High Sierra. What is on my TM backup is
software at Yosemite which doesn't understand APFS, so while the boot
block may contain pointer to the right file, that file, once loaded,
wouldn't be able to continue to boot since it wouldnt see the right disk
structures.

Apple could:
	-require TM disk be reformmated from scratch
	-have utility that coverts not only file system but TM structure
	-have TM be able to write backups to either HFS or
	 APFS backups.

If you were able to convert existing TM backup, you might not be able to
roll back the OS, but you would still have access to files.

I guess we just have to be patient and see what Apple reveals about it.

[toc] | [prev] | [next] | [standalone]

#107491

In article <5936250e$0$43862$c3e8da3$5e5e430d@news.astraweb.com>,
 JF Mezei <jfmezei.spamnot@vaxination.ca> wrote:

> > We haven't seen any more information yet about how APFS will work with
> > Time Machine, or whether it will be possible to convert a TM backup
> > drive from HFS+ to APFS.
> 
> From the point of view of a utility like Disk Util, would a TM drive
> look different from any other HFS+ drive ?

I think the question is not whether you can do the conversion, but 
whether it will still work with TM. If TM requires directory hard links 
and APFS doesn't support them, how would it work?

Or does TM in High Sierra use some other mechanism to avoid duplicating 
unchanged files in each backup?

-- 
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***

[toc] | [prev] | [next] | [standalone]

#107496

Barry Margolin <barmar@alum.mit.edu> wrote:

> In article <5936250e$0$43862$c3e8da3$5e5e430d@news.astraweb.com>,
>  JF Mezei <jfmezei.spamnot@vaxination.ca> wrote:
> 
> > > We haven't seen any more information yet about how APFS will work with
> > > Time Machine, or whether it will be possible to convert a TM backup
> > > drive from HFS+ to APFS.
> > 
> > From the point of view of a utility like Disk Util, would a TM drive
> > look different from any other HFS+ drive ?
> 
> I think the question is not whether you can do the conversion, but 
> whether it will still work with TM. If TM requires directory hard links
> and APFS doesn't support them, how would it work?

A TM backup drive formatted as APFS would not require directory hard
links. It can use clones to implement the same behaviour but in a safer
way.

> Or does TM in High Sierra use some other mechanism to avoid duplicating
> unchanged files in each backup?

Based on this document:

https://developer.apple.com/library/content/releasenotes/MacOSX/WhatsNewInOSX/Articles/macOS_10_13_0.html#//apple_ref/doc/uid/TP40017638-SW1

(see "System" at the end)

TM in macOS High Sierra will support backing up from an APFS source
volume to an HFS+ TM backup volume.

-- 
David Empson
dempson@actrix.gen.nz

[toc] | [prev] | [next] | [standalone]

#107478

On 2017-06-05 21:05, David Empson wrote:

> Further minor details are in this document, updated for macOS 10.13 High
> Sierra:
>
> https://developer.apple.com/library/content/documentation/FileManagement/Conceptual/APFS_Guide/Introduction/Introduction.html#//apple_ref/doc/uid/TP40016999
>
> From the FAQ:
>
> [quote]
> How do I upgrade to Apple File System?
>
> The macOS 10.13 installer offers nondestructive in-place upgrades from
> HFS+ to APFS for bootable volumes. You can use Disk Utility to convert
> external volumes from HFS+ to APFS format.

Check!  Great!

>
> If I convert a volume to APFS, can I later revert to HFS+?
>
> You can use Disk Utility to erase an APFS-formatted volume and reformat
> as HFS+. However, your data will not be preserved when you reformat the
> volume as HFS+.
> [end quote]

S'aright.

>
> I expect that if you skip converting your boot volume during install,
> doing the conversion later would require booting from another volume
> with High Sierra installed, e.g. the recovery partition.


Digesting ... don't see why - should be able to convert before boot in 
that case.  TBS.


-- 
"If war is God's way of teaching Americans geography, then
recession is His way of teaching everyone a little economics."
   ..Raj Patel, The Value of Nothing.

[toc] | [prev] | [next] | [standalone]

#107483

On 2017-06-06 19:36, Alan Browne wrote:

> Digesting ... don't see why - should be able to convert before boot in 
> that case.  TBS.

Conversion software requires you to boot enough of an OS to run the
utility that will convert a disk into the new file system, but can't
boot from the disk you will convert. aka: boot from High Sierra recovery
partition to have the software to convert your main disk.


What isn't clear to me is that APFS introduces new partitioning scheme,
but would it be correct to state that the EFI and recovery partitions
remain at the GUID level instead of the APFS partitioning scheme ?

[toc] | [prev] | [next] | [standalone]

#107495

JF Mezei <jfmezei.spamnot@vaxination.ca> wrote:

> On 2017-06-06 19:36, Alan Browne wrote:
> 
> > Digesting ... don't see why - should be able to convert before boot in
> > that case.  TBS.
> 
> Conversion software requires you to boot enough of an OS to run the
> utility that will convert a disk into the new file system, but can't
> boot from the disk you will convert. aka: boot from High Sierra recovery
> partition to have the software to convert your main disk.

Given that the recovery partition will (optionally) be doing the HFS+ to
APFS conversion during an install of High Sierra, I expect the same
method would be used to convert the boot volume after install.

Conversion could be initiated from Disk Utility involving a reboot, but
that reboot would use the recovery partition (without UI apart from a
progress indicator, similar to some stages of the OS install), then boot
from the main partition once the conversion is complete.

It seems unnecessary for Apple to implement a special mechanism to
support partially booting from a volume then doing a self conversion,
when there is a perfectly good recovery partition which can do the
conversion.

> What isn't clear to me is that APFS introduces new partitioning scheme,
> but would it be correct to state that the EFI and recovery partitions
> remain at the GUID level instead of the APFS partitioning scheme ?

There is no "APFS partitioning scheme". If you mean an APFS "container"
which may have multiple volumes with shared free space, that is a single
partition in the GUID Partition Table.

I expect the recovery partition will be the same as before: a separate
partition in the GPT, which is exactly how it is done for plain HFS+, or
CoreStorage with FileVault encryption or a Fusion drive. (I don't have
an unencrypted bootable non-Fusion CoreStorage volume handy to check but
I think it is the same.)

Having the recovery partition outside the APFS container makes it
independent of any APFS encryption, which will be necessary for booting
the encrypted volume via the password prompt presented by the recovery
partition.


This page has some more details about APFS implementation in 10.13,
which answers several questions that have come up in this thread:

https://developer.apple.com/library/content/releasenotes/MacOSX/WhatsNewInOSX/Articles/macOS_10_13_0.html#//apple_ref/doc/uid/TP40017638-SW1

See the "System" section at the end.

EFI (with updates if necessary) supports booting APFS. That means the
option key startup disk selector will recognise a bootable volume within
an APFS container.


Existing TM volumes "should still remain HFS+ in this release". 10.13
will support backing up from an APFS source volume to an HFS+ backup
volume.

What other source/backup file system combinations work is unknown at
this stage. Waiting for the WWDC session which may reveal more detail.

-- 
David Empson
dempson@actrix.gen.nz

[toc] | [prev] | [next] | [standalone]

#107501

On 2017-06-07 16:23, David Empson wrote:

> I expect the recovery partition will be the same as before: a separate
> partition in the GPT, which is exactly how it is done for plain HFS+, 

How will boot proceed?

With VMS, the EFI firmware would load the EFI programme from the EFI
partition, and that program had the code to be able to parse through an
ODS file system to find the VMS boot file. (standard EFI only has FAT
support).

The Apple docs mention that the EFI firmware finds the file:
/System/Library/CoreServices/boot.efi

> https://developer.apple.com/library/content/documentation/Darwin/Conceptual/KernelProgramming/booting/booting.html

The above would imply that the EFI firmware has logic to notony parse
FAT but also HFS+ since it is able to find a file by name in an HFS+


(In other documentation, I had found that the volume header of an HFS
disk contains the block number and size of the boot file so the firmware
need only look at volume header to find the boot file).

Any information on how boot will proceed on APFS?

Will Apple update the EFI firmware of all qualifying Macs so it is able
to read through APFS folumes to find the boot.efi file? Or will it use
the EFI standard mechanism of placing an EFI program in the EFI
partition that has the logic to parse trough the system APFS disk and
find/load the boot file ?

[toc] | [prev] | [next] | [standalone]

#107505

On 2017-06-08, JF Mezei <jfmezei.spamnot@vaxination.ca> wrote:
> On 2017-06-07 16:23, David Empson wrote:
>
>> I expect the recovery partition will be the same as before: a separate
>> partition in the GPT, which is exactly how it is done for plain HFS+, 
>
> How will boot proceed?

Honestly: Why do you even care, as long as it works? Time after time it
seems like your default position is that nothing Apple will work
correctly, so you sit here worrying and wringing your hands about this
or that (all the while spewing misguided misinformation) until someone
finally has enough patience to correct you and hammer it into your head,
by which time you're unfortunately on to the next thing.

-- 
E-mail sent to this address may be devoured by my ravenous SPAM filter.
I often ignore posts from Google. Use a real news client instead.

JR

[toc] | [prev] | [next] | [standalone]

#107517

JF Mezei <jfmezei.spamnot@vaxination.ca> wrote:

> On 2017-06-07 16:23, David Empson wrote:
> 
> > I expect the recovery partition will be the same as before: a separate
> > partition in the GPT, which is exactly how it is done for plain HFS+,
> 
> How will boot proceed?

With a progress bar.

> With VMS, the EFI firmware would load the EFI programme from the EFI
> partition, and that program had the code to be able to parse through an
> ODS file system to find the VMS boot file. (standard EFI only has FAT
> support).

Again with the irrelevant comparison to other systems.

> The Apple docs mention that the EFI firmware finds the file:
> /System/Library/CoreServices/boot.efi
> 
>>https://developer.apple.com/library/content/documentation/Darwin/Conceptual/KernelProgramming/booting/booting.html
> 
> The above would imply that the EFI firmware has logic to notony parse
> FAT but also HFS+ since it is able to find a file by name in an HFS+

and now APFS as well.

> (In other documentation, I had found that the volume header of an HFS
> disk contains the block number and size of the boot file so the firmware
> need only look at volume header to find the boot file).
> 
> Any information on how boot will proceed on APFS?

Details of the APFS file system data structures have not been published
yet, if that is what you are asking.

> Will Apple update the EFI firmware of all qualifying Macs so it is able
> to read through APFS folumes to find the boot.efi file? Or will it use
> the EFI standard mechanism of placing an EFI program in the EFI
> partition that has the logic to parse trough the system APFS disk and
> find/load the boot file ?

Did you even bother to read the document I linked in my previous post,
but you trimmed from quotes, or the point I explicitly stated in my
subsequent comments?

From the document:

"APFS is now supported as a boot volume with full, native EFI support."

-- 
David Empson
dempson@actrix.gen.nz

[toc] | [prev] | [next] | [standalone]

#107522

On 2017-06-08 17:23, David Empson wrote:

> Did you even bother to read the document I linked in my previous post,
> but you trimmed from quotes, or the point I explicitly stated in my
> subsequent comments?

> "APFS is now supported as a boot volume with full, native EFI support."

That actualy says nothing.

It could mean that EFI will first look in the EFI partition for an EFI
program whch, once loaded, will know how to look for OS_X boot file on
an APFS volume.

It could mean that Apple will update firmware for all hardware to give
the firmware EFI ability to parse through APFS to look for "boot.efi" file

Or it could mean that APFS emulates HFS and places a block number and
size of boot file in the same disk block as HFS and EFI only needs to
look there to know wher the boot file is.

Which brings my original question of which of the three is the case ?

[toc] | [prev] | [next] | [standalone]

#107523

In article <5939febf$0$38679$b1db1813$19ace300@news.astraweb.com>, JF
Mezei <jfmezei.spamnot@vaxination.ca> wrote:

> 
> > Did you even bother to read the document I linked in my previous post,
> > but you trimmed from quotes, or the point I explicitly stated in my
> > subsequent comments?
> 
> > "APFS is now supported as a boot volume with full, native EFI support."
> 
> That actualy says nothing.
> 
> It could mean that EFI will first look in the EFI partition for an EFI
> program whch, once loaded, will know how to look for OS_X boot file on
> an APFS volume.
> 
> It could mean that Apple will update firmware for all hardware to give
> the firmware EFI ability to parse through APFS to look for "boot.efi" file
> 
> Or it could mean that APFS emulates HFS and places a block number and
> size of boot file in the same disk block as HFS and EFI only needs to
> look there to know wher the boot file is.
> 
> Which brings my original question of which of the three is the case ?

this may give you some insight, although i suspect it won't.

<https://bombich.com/software/updates/ccc-4.1.16-b1.html>
   €  APFS startup volumes have new, proprietary Recovery, Virtual
  Memory, and Preboot volumes. Adding support for these new volume
  types will take a considerable amount of engineering effort, so
  support for these is not currently available in this release of CCC.
  CCC can make a backup of your user and system files to an APFS
  volume, but it will not produce a bootable backup on an APFS
  destination. Use an HFS+ destination for now if you require a
  bootable backup.

<https://bombich.com/blog/2017/06/07/ccc-support-macos-high-sierra-and-a
pfs>
  Creating a bootable APFS volume, however, is brand-new territory. The
  semantics of starting a Mac from an APFS volume are completely
  different from those of an HFS+ volume. We have established a
  procedure to create an APFS startup volume, though, and we've even
  created a proof-of-concept bootable APFS clone. What lies ahead is a
  massive amount of engineering work to build support for these new
  procedures into CCC. APFS encryption is also handled quite
  differently from CoreStorage encryption, so we have a lot of work to
  do in regard to building in support for automatically unlocking and
  mounting APFS encrypted backup volumes. We're aiming to offer new
  functionality for creating APFS bootable (and optionally encrypted)
  backups by the time Apple ships macOS High Sierra in the Fall.

[toc] | [prev] | [next] | [standalone]

#107530

On 2017-06-09 00:44, nospam wrote:

> this may give you some insight, although i suspect it won't.
> 
> <https://bombich.com/software/updates/ccc-4.1.16-b1.html>

Thanks. Raises another question about encryption.

The doc states for HFS+ that
/System/Library/CoreServices/boot.efi is the one responsible for the UI
to request encryption password at boot.

So this means the firmware EFI does not do decryption so it must be able
to access boot.efi on the system disk without decryption.

If the disk and catalogue are encrypted, the firmware would have no
means to go through the file system to find "boot.efi" in a specific
directory, which tends to point to boot.efi's location and size
conveniently placed in a field on the volume header which means the
firmware can bypass the file system and load X blocks at location Y on disk.

In APFS parlance, it would also mean that a "blind" read of X blocks
starting at a certain block reauires boot.efi to be unencrypted (even if
the catelogue entry is) and that it be contiguous.

The other option is to go via an EFI partition programme which handles
the UI to ask for password, and has the logic to parse and decrypt the
APFS volume to get the boot.efi file


Will look at what Apple has posted for WWDC17 videos later to see if
there is mention of this.

[toc] | [prev] | [next] | [standalone]

#107531

On 2017-06-09 14:28, JF Mezei wrote:
> On 2017-06-09 00:44, nospam wrote:

>> <https://bombich.com/software/updates/ccc-4.1.16-b1.html>
> 


The plot thickens:
##
APFS startup volumes have new, proprietary Recovery, Virtual Memory, and
Preboot volumes.
##

Wonder if those are GPT volumes or volumes inside the APFS container.
"Preboot" might contain un-encrypted boot files with enough logic to
access the encrypted system disk and load the subsequent system files. (
essentially what other systems do with the EFI program in the EFI
partition.)

Putting virtual memory in separate volume, save disk coule make sense if
that volume is not encrypted (less overhead when writing/reading to
disk). But that opens a vulnerability as RAM data would be unencrypted
on the disk for the NSA to access. (especially in the APFS mentality of
never rewriting over data).

Another advantage if having VM on a separate volume, you don't need to
create exceptions to skip over those files during a backup or disk cloning.

Looking forward to seeing details on how all that will work

[toc] | [prev] | [next] | [standalone]

#107542

JF Mezei <jfmezei.spamnot@vaxination.ca> wrote:

> On 2017-06-09 00:44, nospam wrote:
> 
> > this may give you some insight, although i suspect it won't.
> > 
> > <https://bombich.com/software/updates/ccc-4.1.16-b1.html>
> 
> Thanks. Raises another question about encryption.
> 
> The doc states for HFS+ that
> /System/Library/CoreServices/boot.efi is the one responsible for the UI
> to request encryption password at boot.

If this isn't the document you alluded to, please post a link.

https://developer.apple.com/library/content/documentation/Darwin/Conceptual/KernelProgramming/booting/booting.html

It says "In the simplest case, the boot loader can be found in the
/System/Library/CoreServices directory on the root partition, in a file
named boot.efi."

An encrypted volume is not "the simplest case".

> So this means the firmware EFI does not do decryption so it must be able
> to access boot.efi on the system disk without decryption.

No it doesn't.

With FileVault 2, the entire logical HFS+ volume within the CoreStorage
container is encrypted. There is no part, including the volume header,
which can be accessed without the disk encryption key, and getting that
requires one of the user passwords (or the recovery key, or the disk
password if that method was used). Therefore no part of the logical HFS+
volume can be accessed until after the password prompt.

For an encrypted volume, the first boot stage including the password
prompt is achieved by doing a partial startup from the recovery
partition. Once the password is entered and the the main volume
unlocked, boot proceeds from the encrypted main volume.

This can be confirmed easily using this command in Terminal after
setting the startup disk as appropriate:

nvram efi-boot-device

It outputs a block of XML, near the end of which is a bit that looks
like this on my main MacBook Pro:

<key>BLLastBSDName</key><string>disk0s3</string>

You can use "diskutil list" to find which volume is which.

On my MacBook Pro, which is encrypted with FileVault 2, disk0s3 is the
recovery partition.

Checking some other Macs:

1. My Mac Mini with a Fusion Drive but no FileVault encryption has
efi-boot-device referencing a "Boot OS X" partition on the SSD, which is
134.2 MB. (There is also a 650 MB recovery partition on the hard drive,
which is not involved in the boot.)

2. My other MacBook Pro with three bootable partitions has two encrypted
with FileVault using CoreStorage and the third is a plain HFS+
partition. If I set the startup disk to either of the FileVault
encrypted partitions, I see the same pattern as my main MacBook Pro. If
I set the the startup disk to the plain HFS+ partition, efi-boot-device
points directly to the HFS+ partition.

Everything else you wrote is wrong because you started from a false
assumption.

-- 
David Empson
dempson@actrix.gen.nz

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | comp.sys.mac.system

csiph-web