Proposed Enhancements to the EFI booting

Discussion:

Warner Losh

2017-08-11 04:03:57 UTC

Greetings,

I've been circulating a document (the latest version you can find here
https://docs.google.com/document/d/1aK9IqF-60JPEbUeSAUAkYjF2W_8EnmczFs6RqCT90Jg/edit?usp=sharing
)

I'm opening it up for general comments now. I intend to turn this into a
full spec, including the bits in the UEFI standard that I reference by
pointer, and move to implementation soon once any issues are resolved.

Warner

FreeBSD UEFI boot protocol

This document outlines the enhancements to the UEFI Boot Manager Protocol,
as outlined in Version 2.6 of the UEFI spec, chapter 3. It assumes that you
are familiar with that chapter and though a comprehensive doc with our
additions folded into it would be good, it hasn’t been produced.
Current Algorithm

Boot1.efi currently searches for /boot/loader.efi using a fairly
complicated algorithm.

First, we look for any ZFS pool that has a /boot/loader.efi on it,
consistent with its boot environment settings. If we find that, we use it.

Second, we search all of the partitions on the device that we booted
boot1.efi off of for a UFS partition that has a /boot/loader.efi we can
load. If we find that, we use it.

Third, we search all other devices for a partition that is UFS formatted
and has a /boot/loader.efi we can load.

There’s some problems with this algorithm. It’s not possible to specify a
partition to boot from. We boot from the first /boot/loader.efi that’s
found, even if multiple partitions on a drive have one. Second, it’s very
non-deterministic. The second step was added to add some determinism to the
process so if you plugged in a FreeBSD 11.0R release USB stick and
rebooted, you would reboot to the installed system and not the USB
installer. It doesn’t use the ad-hoc method for identifying which FreeBSD
partition to boot off of that we use for BIOS boots since in the UEFI
runtime environment it’s tricky to get the attribute flags of a partition.
Finally, it doesn’t use the standard UEFI boot manager protocol.
Proposed Algorithm

First, the FreeBSD UEFI Boot Loader UUID shall be
cfee69ad-a0de-47a9-93a8-f63106f8ae99 (below as FreeBSD:). The UEFI defined
global variable UUID (EFI_GLOBAL_VARIABLE) is
8BE4DF61-93CA-11d2-AA-0D-00-E0-98-03-2B-8C (below as UEFI:)

Boot1.efi will find what to use with the following algorithm:

1.

If the UEFI boot manager passes an optional parameter to boot1.efi, then
parse it as if it were a EFI Device Path to find the next stage boot loader
to use. If we can use that boot loader, we’re done. Otherwise, ignore it.
2.

Get the value of UEFI:BootCurrent. This will be a 4 digit hex number
XXXX. If FreeBSD:BootLoaderXXXX exists, and specifies a file we can load,
use it. This is in the form of a EFI device path (binary). If it is a File
path that’s ZFS:yyyy, then boot from the Boot environment for ZFS pool
yyyy. If :yyyy is omitted, use the first ZFS pool you find.
3.

If a ZFS pool exists with a bootable environment, boot it.
4.

If a partition with a known filesystem exists on the same device as
boot1.efi was loaded from, and it contains /boot/loader.efi, use it.
5.

Set FreeBSD:BootFailedXXXX to the reason for the failure and return
Failure to UEFI (so UEFI goes to the next item in the list)

If FreeBSD:BootArgumentsXXXX exists, parse it like ASCII text and pass it
to the boot loader found in steps 1-4 above. Otherwise pass nothing.

If the FreeBSD:Update variable is set, the rc system will update the boot
order so that UEFI:BootCurrent is at the start of UEFI:BootOrder. This is
currently done in /etc/rc.d/gptupdate. I propose a /etc/rc.d/efiupdate to
do this, and to report all the BootFailedXXXX variables ala gptupdate.

Discussion of points raised on IRC:

FreeBSD currently implements a ‘boot once’ option that’s similar to
‘BootNext’ but happens only once. In UEFI, BootNext almost implements this.
The BIOS loads that option, then deletes the BootNext env variable. Unless
the OS does something the make it permanent, this is the same as FreeBSD’s
boot once (because the order then reverts to BootOrder). I think that
BootFailed needs to be implemented with the extra protocol outlined in step
5.

Open Issues:

Allan Jude has raised some issues about ZFS. ZFS has it’s own way to deal
with all this and he’s curious how this fits into that. EFI doesn’t know
about ZFS, so at most we can only define what happens when boot1.efi takes
over. In that case, I proposed to him that we do what we do now, and
selection of what ZFS thing can be done either as I proposed, or via the
zfsbootcfg process if nothing further than ZFS is specified (or
automatically selected).

How does loader.efi fit for people that load it directly?

Do we want to enable / disable the automatic looking for boot locations
separately from what to boot being explicitly specified or not? I’m leaning
against.

Can nextboot -k foo be implemented with this? My first notion is that it
would FreeBSD:KernelXXXX to specify the kernel and FreeBSD:NextKernelXXXX
which would load that kernel and then unset FreeBSD:NextKernelXXXX. Ditto
for FreeBSD:KernelFlagsXXXX and FreeBSD:NextKernelFlagsXXXX. nextboot(8)
would set all these. Maybe FreeBSD:LoaderXXXX and FreeBSD:NextLoaderXXXX
too...

Mark Johnston

2017-08-11 19:40:28 UTC

Permalink

Post by Warner Losh
Greetings,
I've been circulating a document (the latest version you can find here
https://docs.google.com/document/d/1aK9IqF-60JPEbUeSAUAkYjF2W_8EnmczFs6RqCT90Jg/edit?usp=sharing
)
I'm opening it up for general comments now. I intend to turn this into a
full spec, including the bits in the UEFI standard that I reference by
pointer, and move to implementation soon once any issues are resolved.
Warner
FreeBSD UEFI boot protocol
This document outlines the enhancements to the UEFI Boot Manager Protocol,
as outlined in Version 2.6 of the UEFI spec, chapter 3. It assumes that you
are familiar with that chapter and though a comprehensive doc with our
additions folded into it would be good, it hasn’t been produced.
Current Algorithm
Boot1.efi currently searches for /boot/loader.efi using a fairly
complicated algorithm.
First, we look for any ZFS pool that has a /boot/loader.efi on it,
consistent with its boot environment settings. If we find that, we use it.
Second, we search all of the partitions on the device that we booted
boot1.efi off of for a UFS partition that has a /boot/loader.efi we can
load. If we find that, we use it.
Third, we search all other devices for a partition that is UFS formatted
and has a /boot/loader.efi we can load.
There’s some problems with this algorithm. It’s not possible to specify a
partition to boot from. We boot from the first /boot/loader.efi that’s
found, even if multiple partitions on a drive have one. Second, it’s very
non-deterministic. The second step was added to add some determinism to the
process so if you plugged in a FreeBSD 11.0R release USB stick and
rebooted, you would reboot to the installed system and not the USB
installer. It doesn’t use the ad-hoc method for identifying which FreeBSD
partition to boot off of that we use for BIOS boots since in the UEFI
runtime environment it’s tricky to get the attribute flags of a partition.
Finally, it doesn’t use the standard UEFI boot manager protocol.
Proposed Algorithm
First, the FreeBSD UEFI Boot Loader UUID shall be
cfee69ad-a0de-47a9-93a8-f63106f8ae99 (below as FreeBSD:). The UEFI defined
global variable UUID (EFI_GLOBAL_VARIABLE) is
8BE4DF61-93CA-11d2-AA-0D-00-E0-98-03-2B-8C (below as UEFI:)
1.
If the UEFI boot manager passes an optional parameter to boot1.efi, then
parse it as if it were a EFI Device Path to find the next stage boot loader
to use. If we can use that boot loader, we’re done. Otherwise, ignore it.
2.
Get the value of UEFI:BootCurrent. This will be a 4 digit hex number
XXXX. If FreeBSD:BootLoaderXXXX exists, and specifies a file we can load,
use it. This is in the form of a EFI device path (binary). If it is a File
path that’s ZFS:yyyy, then boot from the Boot environment for ZFS pool
yyyy. If :yyyy is omitted, use the first ZFS pool you find.
3.
If a ZFS pool exists with a bootable environment, boot it.
4.
If a partition with a known filesystem exists on the same device as
boot1.efi was loaded from, and it contains /boot/loader.efi, use it.
5.
Set FreeBSD:BootFailedXXXX to the reason for the failure and return
Failure to UEFI (so UEFI goes to the next item in the list)
If FreeBSD:BootArgumentsXXXX exists, parse it like ASCII text and pass it
to the boot loader found in steps 1-4 above. Otherwise pass nothing.
If the FreeBSD:Update variable is set, the rc system will update the boot
order so that UEFI:BootCurrent is at the start of UEFI:BootOrder. This is
currently done in /etc/rc.d/gptupdate. I propose a /etc/rc.d/efiupdate to
do this, and to report all the BootFailedXXXX variables ala gptupdate.
FreeBSD currently implements a ‘boot once’ option that’s similar to
‘BootNext’ but happens only once. In UEFI, BootNext almost implements this.
The BIOS loads that option, then deletes the BootNext env variable. Unless
the OS does something the make it permanent, this is the same as FreeBSD’s
boot once (because the order then reverts to BootOrder). I think that
BootFailed needs to be implemented with the extra protocol outlined in step
5.

There is also a "bootme" GPT flag that is currently ignored when booting
using boot1.efi. We use it to avoid in-place upgrades: there are two
root filesystems, only one of which is mounted during boot. Upgrades are
done by installing the to-version of the OS to the inactive root
filesystem, flipping the "bootme" flag off on the active partition,
flipping it on on the inactive partition, and rebooting.

How should this interact with step 4 of your algorithm? Should we prefer
a GPT "bootme" entry over a filesystem on the same device as boot1.efi?

Post by Warner Losh
Allan Jude has raised some issues about ZFS. ZFS has it’s own way to deal
with all this and he’s curious how this fits into that. EFI doesn’t know
about ZFS, so at most we can only define what happens when boot1.efi takes
over. In that case, I proposed to him that we do what we do now, and
selection of what ZFS thing can be done either as I proposed, or via the
zfsbootcfg process if nothing further than ZFS is specified (or
automatically selected).
How does loader.efi fit for people that load it directly?
Do we want to enable / disable the automatic looking for boot locations
separately from what to boot being explicitly specified or not? I’m leaning
against.
Can nextboot -k foo be implemented with this? My first notion is that it
would FreeBSD:KernelXXXX to specify the kernel and FreeBSD:NextKernelXXXX
which would load that kernel and then unset FreeBSD:NextKernelXXXX. Ditto
for FreeBSD:KernelFlagsXXXX and FreeBSD:NextKernelFlagsXXXX. nextboot(8)
would set all these. Maybe FreeBSD:LoaderXXXX and FreeBSD:NextLoaderXXXX
too...
_______________________________________________
https://lists.freebsd.org/mailman/listinfo/freebsd-arch

Warner Losh

2017-08-11 19:48:02 UTC

Permalink

Post by Warner Losh

Post by Warner Losh
Greetings,
I've been circulating a document (the latest version you can find here
https://docs.google.com/document/d/1aK9IqF-60JPEbUeSAUAkYjF2W_

8EnmczFs6RqCT90Jg/edit?usp=sharing

Post by Warner Losh
)
I'm opening it up for general comments now. I intend to turn this into a
full spec, including the bits in the UEFI standard that I reference by
pointer, and move to implementation soon once any issues are resolved.
Warner
FreeBSD UEFI boot protocol
This document outlines the enhancements to the UEFI Boot Manager

Protocol,

Post by Warner Losh
as outlined in Version 2.6 of the UEFI spec, chapter 3. It assumes that

you

Post by Warner Losh
are familiar with that chapter and though a comprehensive doc with our
additions folded into it would be good, it hasn’t been produced.
Current Algorithm
Boot1.efi currently searches for /boot/loader.efi using a fairly
complicated algorithm.
First, we look for any ZFS pool that has a /boot/loader.efi on it,
consistent with its boot environment settings. If we find that, we use

it.

Post by Warner Losh
Second, we search all of the partitions on the device that we booted
boot1.efi off of for a UFS partition that has a /boot/loader.efi we can
load. If we find that, we use it.
Third, we search all other devices for a partition that is UFS formatted
and has a /boot/loader.efi we can load.
There’s some problems with this algorithm. It’s not possible to specify a
partition to boot from. We boot from the first /boot/loader.efi that’s
found, even if multiple partitions on a drive have one. Second, it’s very
non-deterministic. The second step was added to add some determinism to

the

Post by Warner Losh
process so if you plugged in a FreeBSD 11.0R release USB stick and
rebooted, you would reboot to the installed system and not the USB
installer. It doesn’t use the ad-hoc method for identifying which FreeBSD
partition to boot off of that we use for BIOS boots since in the UEFI
runtime environment it’s tricky to get the attribute flags of a

partition.

Post by Warner Losh
Finally, it doesn’t use the standard UEFI boot manager protocol.
Proposed Algorithm
First, the FreeBSD UEFI Boot Loader UUID shall be
cfee69ad-a0de-47a9-93a8-f63106f8ae99 (below as FreeBSD:). The UEFI

defined

Post by Warner Losh
global variable UUID (EFI_GLOBAL_VARIABLE) is
8BE4DF61-93CA-11d2-AA-0D-00-E0-98-03-2B-8C (below as UEFI:)
1.
If the UEFI boot manager passes an optional parameter to boot1.efi,

then

Post by Warner Losh
parse it as if it were a EFI Device Path to find the next stage boot

loader

Post by Warner Losh
to use. If we can use that boot loader, we’re done. Otherwise, ignore

it.

Post by Warner Losh
2.
Get the value of UEFI:BootCurrent. This will be a 4 digit hex number
XXXX. If FreeBSD:BootLoaderXXXX exists, and specifies a file we can

load,

Post by Warner Losh
use it. This is in the form of a EFI device path (binary). If it is a

File

Post by Warner Losh
path that’s ZFS:yyyy, then boot from the Boot environment for ZFS pool
yyyy. If :yyyy is omitted, use the first ZFS pool you find.
3.
If a ZFS pool exists with a bootable environment, boot it.
4.
If a partition with a known filesystem exists on the same device as
boot1.efi was loaded from, and it contains /boot/loader.efi, use it.
5.
Set FreeBSD:BootFailedXXXX to the reason for the failure and return
Failure to UEFI (so UEFI goes to the next item in the list)
If FreeBSD:BootArgumentsXXXX exists, parse it like ASCII text and pass it
to the boot loader found in steps 1-4 above. Otherwise pass nothing.
If the FreeBSD:Update variable is set, the rc system will update the boot
order so that UEFI:BootCurrent is at the start of UEFI:BootOrder. This is
currently done in /etc/rc.d/gptupdate. I propose a /etc/rc.d/efiupdate to
do this, and to report all the BootFailedXXXX variables ala gptupdate.
FreeBSD currently implements a ‘boot once’ option that’s similar to
‘BootNext’ but happens only once. In UEFI, BootNext almost implements

this.

Post by Warner Losh
The BIOS loads that option, then deletes the BootNext env variable.

Unless

Post by Warner Losh
the OS does something the make it permanent, this is the same as

FreeBSD’s

Post by Warner Losh
boot once (because the order then reverts to BootOrder). I think that
BootFailed needs to be implemented with the extra protocol outlined in

step

Post by Warner Losh
5.

Correct. EFI Boot Manager has a concept of profiles being active or not
active. That's how this functionality will be implemented. There's no
standardized notion of a partition being bootable or not. That's a FreeBSD
extension. same with bootnext and bootfail flags.

So upgrades would become flipping the boot order in UEFI:BootOrder and/or
toggling active on the two UEFI:BootXXXX variables. It just changes the
command you need to issue from gpart <mumble> to efibootmgr <mumble>.

Post by Warner Losh
How should this interact with step 4 of your algorithm? Should we prefer
a GPT "bootme" entry over a filesystem on the same device as boot1.efi?

No. We shouldn't. That duplicates functionality, and the flags bits we need
to make these decisions are a pain to get to in boot1.efi. They are FreeBSD
specific, but won't be carried forward since better functionality already
exists in the BootOrder and BootXXX variables.

This is much more flexible as well, since it allows you to have multiple
profiles that point to the same partition, but have different parameters,
only one of which is active at any time, which is impossible with the old
GPT scheme.

Warner

Post by Warner Losh

takes

Post by Warner Losh
over. In that case, I proposed to him that we do what we do now, and
selection of what ZFS thing can be done either as I proposed, or via the
zfsbootcfg process if nothing further than ZFS is specified (or
automatically selected).
How does loader.efi fit for people that load it directly?
Do we want to enable / disable the automatic looking for boot locations
separately from what to boot being explicitly specified or not? I’m

leaning

Post by Warner Losh
against.
Can nextboot -k foo be implemented with this? My first notion is that it
would FreeBSD:KernelXXXX to specify the kernel and FreeBSD:NextKernelXXXX
which would load that kernel and then unset FreeBSD:NextKernelXXXX. Ditto
for FreeBSD:KernelFlagsXXXX and FreeBSD:NextKernelFlagsXXXX. nextboot(8)
would set all these. Maybe FreeBSD:LoaderXXXX and FreeBSD:NextLoaderXXXX
too...
_______________________________________________
https://lists.freebsd.org/mailman/listinfo/freebsd-arch

Mark Johnston

2017-08-11 20:00:31 UTC

Permalink

Post by Warner Losh

Post by Mark Johnston
There is also a "bootme" GPT flag that is currently ignored when booting
using boot1.efi. We use it to avoid in-place upgrades: there are two
root filesystems, only one of which is mounted during boot. Upgrades are
done by installing the to-version of the OS to the inactive root
filesystem, flipping the "bootme" flag off on the active partition,
flipping it on on the inactive partition, and rebooting.

Correct. EFI Boot Manager has a concept of profiles being active or not
active. That's how this functionality will be implemented. There's no
standardized notion of a partition being bootable or not. That's a FreeBSD
extension. same with bootnext and bootfail flags.
So upgrades would become flipping the boot order in UEFI:BootOrder and/or
toggling active on the two UEFI:BootXXXX variables. It just changes the
command you need to issue from gpart <mumble> to efibootmgr <mumble>.

Post by Mark Johnston
How should this interact with step 4 of your algorithm? Should we prefer
a GPT "bootme" entry over a filesystem on the same device as boot1.efi?

No. We shouldn't. That duplicates functionality, and the flags bits we need
to make these decisions are a pain to get to in boot1.efi. They are FreeBSD
specific, but won't be carried forward since better functionality already
exists in the BootOrder and BootXXX variables.
This is much more flexible as well, since it allows you to have multiple
profiles that point to the same partition, but have different parameters,
only one of which is active at any time, which is impossible with the old
GPT scheme.

Ok, that's fine with me. I just wanted to point out that that
functionality is both useful and currently missing, so I'm glad that
there will be a replacement with a simple interface.

Warner Losh

2017-08-16 04:15:18 UTC

Permalink

It turns out that the path in EFI_LOAD_OPTIONS is really a list of paths. I
believe it would be more robust to use that fact. The list will include
boot1.efi, loader.efi and kernel. However, it could just as easily include
only loader.efi and kernel. The exact interpretation of the extra path
names is OS specific.

If you want to boot ZFS, it can be encoded as File('ZFS') or
File('ZFS:pool'). It's all OS specific.

What to do about /? How do we find it? That's easy. It is the last path in
the list that has a HD specifier (eg, the File part of the path, if
present, is ignored). This also means you could pass in
HD(<mumble1>)/File('/efi/FreeBSD/boot1.efi'),
HD(<mumble2>)/File('/boot/loader.efi'),
HD(<mumble3>)/File('/boot/kernel/kernel'), HD(<mumble4>). In that case
there's 4 partitions involved, we load different bits off each one. Or the
last 3 could be the same (and the final one wouldn't be needed). It would
also allow each phase of the loader to know where it is and find the next
thing to load.

so we don't need FreeBSD:BootLoaderXXX.

Post by Warner Losh
1.
If a ZFS pool exists with a bootable environment, boot it.
2.
If a partition with a known filesystem exists on the same device as
boot1.efi was loaded from, and it contains /boot/loader.efi, use it.
3.
Set FreeBSD:BootFailedXXXX to the reason for the failure and return
Failure to UEFI (so UEFI goes to the next item in the list)
If FreeBSD:BootArgumentsXXXX exists, parse it like ASCII text and pass it
to the boot loader found in steps 1-4 above. Otherwise pass nothing.

It would be more in line with the standard, it seems, to pass this in with
the 'optional data'. If we had the thing being loaded interpret the data as
a list of strings. The current loader uses the first string and passes the
rest to the next loader. This would eliminate the need for
BootArgumentsXXXX.

Post by Warner Losh
If the FreeBSD:Update variable is set, the rc system will update the boot
order so that UEFI:BootCurrent is at the start of UEFI:BootOrder. This is
currently done in /etc/rc.d/gptupdate. I propose a /etc/rc.d/efiupdate to
do this, and to report all the BootFailedXXXX variables ala gptupdate.
FreeBSD currently implements a ‘boot once’ option that’s similar to
‘BootNext’ but happens only once. In UEFI, BootNext almost implements this.
The BIOS loads that option, then deletes the BootNext env variable. Unless
the OS does something the make it permanent, this is the same as FreeBSD’s
boot once (because the order then reverts to BootOrder). I think that
BootFailed needs to be implemented with the extra protocol outlined in step
5.
Allan Jude has raised some issues about ZFS. ZFS has it’s own way to deal
with all this and he’s curious how this fits into that. EFI doesn’t know
about ZFS, so at most we can only define what happens when boot1.efi takes
over. In that case, I proposed to him that we do what we do now, and
selection of what ZFS thing can be done either as I proposed, or via the
zfsbootcfg process if nothing further than ZFS is specified (or
automatically selected).
How does loader.efi fit for people that load it directly?
Do we want to enable / disable the automatic looking for boot locations
separately from what to boot being explicitly specified or not? I’m leaning
against.
Can nextboot -k foo be implemented with this? My first notion is that it
would FreeBSD:KernelXXXX to specify the kernel and FreeBSD:NextKernelXXXX
which would load that kernel and then unset FreeBSD:NextKernelXXXX. Ditto
for FreeBSD:KernelFlagsXXXX and FreeBSD:NextKernelFlagsXXXX. nextboot(8)
would set all these. Maybe FreeBSD:LoaderXXXX and FreeBSD:NextLoaderXXXX
too...

With the above interpretation we wouldn't need any of this. You'd just
create a new BootXXX and point BootNext at it.

I'll update the doc with these, and am open to comments on them.

Warner

Andriy Gapon

2017-08-16 07:44:24 UTC

Permalink

Post by Warner Losh
If you want to boot ZFS, it can be encoded as File('ZFS') or
File('ZFS:pool'). It's all OS specific.

Just a note that it would be nice to be able to select a specific
filesystem within a ZFS pool as well. That would provide a more robust
alternative to zfsbootcfg on EFI systems.

--
Andriy Gapon

Warner Losh

2017-08-16 14:55:39 UTC

Permalink

Post by Andriy Gapon

Post by Warner Losh
If you want to boot ZFS, it can be encoded as File('ZFS') or
File('ZFS:pool'). It's all OS specific.

Just a note that it would be nice to be able to select a specific
filesystem within a ZFS pool as well. That would provide a more robust
alternative to zfsbootcfg on EFI systems.

I concur. I'd like to be able to specify the whole path somehow. I'm
looking for the standard way to do that. I also need a way to just say
"Find a ZFS pool so we do what we do now" and "Find ZFS pool FRED and do
what we do now"

Someone (maybe you) on IRC suggested zfs:/pool/file/system:/path/to/file
since that's what's used now. It doesn't look like a typical URI, but
looking at RFC 3986 I see that it fits the ABNF at the end, as well as
being congruent with some of the examples. Since there's no 'authority'
section, it's fine. I'm not aware of needing any authority or credentials
to access the pool, so I think that's fine.

Warner