New behaviors for loader.efi

Post by Eric McCorkle
Following the earlier discussion on the fate of loader.efi, and in light
of my GELI work, I've begun working on modifications to loader.efi to
allow it to be installed to the ESP, while maintaining
back-compatibility with the legacy system.

Cool. I have some concerns, as this doesn't follow the doc I posted a while
ago. I've not yet worked through the boot1.efi removal in that document,
however.

Post by Eric McCorkle
There's a couple of points that probably warrant discussion before I go
any farther.
First, we'll need to figure out what to do with boot.conf, which
previously contained the arguments that would get passed to loader.
Personally, I think the logical place for this is /efi/boot/config or
/efi/boot.conf

I would vote neither. If it belongs on the ESP at all, it belongs in
\efi\freebsd\boot.conf. We now own \efi\freebsd, so that's where all our
boot files need to wind up in our install. But I'm not sure that we need it
at all, since boot.conf is only for boot1/boot2 in the current system
(though it does set serial by default, which might be helpful to preserve).
It would be even better, though, to use the command line that's passed in
as part of the UEFI:BootXXXX environment variable. Since we have to move it
anyway, we should do things in the EFI way, which is to store all the
nonvolatile info in UEFI environment variables (and in this case, the
command line / aux info in the BootXXXX variable). We really need to stop
polluting \efi\boot with anything, except on removable media.

Post by Eric McCorkle
Second, does loader.conf exist on the boot filesystem, or should it also
exist on the ESP? I'm inclined to think this should be left on the boot
filesystem, as there may be more than one, and it's potentially specific
to the kernel installed there.

I think it should be in /boot/loader.conf on /, not the ESP. We should only
have \efi\freebsd\loader.efi in the ESP.

Post by Eric McCorkle
Last, loader has typically relied on the fact that it's installed
directly on the boot filesystem, so it can obtain a good initial
currdev/loaddev. If it's installed on the ESP, it potentially has to go
hunting for the boot device.

I'm not sure that it should look very hard, and it should only look as a
last result.

UEFI:BootCurrent lists the current boot variable. It points to the
UEFI:BootXXXX that contains a LoadOption variable. This variable contains a
series of DevicePaths. The first one is what the UEFI BIOS uses to load
loader.efi (installed as ESP:\EFI\FREEBSD\LOADER.EFI). The second one in
the list will point to the kernel to load. That way we shouldn't have to go
looking at all. We assume that the root is the same partition we load the
kernel comes from. We should only go looking if we fail to find the second
path in the LoadOption.

I'm stealing (back) code from my boot1 refactor for this, which first

Post by Eric McCorkle
looks at partitions on the same disk, then looks at all devices. The
actual test, I'm thinking, is to look for /boot/loader.conf,
/boot/kernel/kernel, and possibly other files. Of course, we also need
to retain the legacy behavior so that existing systems don't break.

No, we don't. boot1.efi already does this, and legacy systems will already
have this installed. So we don't have to recreate this behavior if we don't
want to since updated systems will need to follow the efibootmgr protocol.
There's no way the loader.efi will fit on the old ESPs we created anyway...
loader.efi needs to cope with being loaded this way, but it doesn't have to
recreate boot1.efi's behavior.

All we need to do is to find / when we're booting of removable media. This
means we can use a super-simplified method to find it. If you want
something more complicated, you must use the EFI boot manager protocol. I'm
in the final stages, btw, at work of reviewing code we've written to
implement a mostly Linux CLI compatable efibootmgr.

Post by Eric McCorkle
1) Parse command-line args (this is to maintain back-compatibility)

UEFI:BootXXXX also provides a way to pass this in.

Post by Eric McCorkle
2) Load /efi/boot.conf or /efi/boot/config if they exist, parse the args
inside as if they were extra args
4) If loaddev/currdev are set by command-line arguments, use them as-is
and stop

I don't want to do this from the command line. We follow the EFI boot
manager protocol. There's no need to invent our own here, especially since
the loader's devices aren't familiar to people. It adds extra complication
for no benefit.

Post by Eric McCorkle
5) Otherwise, try the legacy currdev/loaddev detection scheme (this will
fail if we're installed to the ESP)

We don't need to do this...

Post by Eric McCorkle
6) If that fails, check the preferred devices

At most we need to check the same device we were booted from if the EFI
variables aren't set.

Post by Eric McCorkle
7) If that fails, check all devices

I really really want to never do this again. It causes more problems than
it solves and is part of the boot1.efi hack we should run away from. None
of our other systems do it, and boot1.efi did it only as a kludge because
it didn't implement the EFI Boot Manager protocol.

I think we should do it as I described above: find things directly from the
BootXXXX variable, per the EFI Boot Manager Protocol, and then fallback to
the first UFS filesystem with /boot/kernel/kernel on the same device we
were loaded from.

Post by Eric McCorkle
8) If that fails, continue without a currdev (which will drop us to the
loader prompt)

That's fine.

Post by Eric McCorkle
What do people think of this procedure?

I have some issues with it.

Warner

Julian Elischer

2017-10-26 04:52:53 UTC

Cool. I have some concerns, as this doesn't follow the doc I posted a while
ago. I've not yet worked through the boot1.efi removal in that document,
however.

boot.conf is also used to set kenv values so that they have knows
values when entering the kernel.
POLA suggests that it should continue to do that.

I think it should be in /boot/loader.conf on /, not the ESP. We should only
have \efi\freebsd\loader.efi in the ESP.

why not have both a place to set defaults for all Filesystems (in
ESP) and a filesystem specific one? (on the fs)

I'm not sure that it should look very hard, and it should only look as a
last result.
UEFI:BootCurrent lists the current boot variable. It points to the
UEFI:BootXXXX that contains a LoadOption variable. This variable contains a
series of DevicePaths. The first one is what the UEFI BIOS uses to load
loader.efi (installed as ESP:\EFI\FREEBSD\LOADER.EFI). The second one in
the list will point to the kernel to load. That way we shouldn't have to go
looking at all. We assume that the root is the same partition we load the
kernel comes from. We should only go looking if we fail to find the second
path in the LoadOption.
I'm stealing (back) code from my boot1 refactor for this, which first

depending on what behaviour you are talking about..
loader.conf setting kenv values is pretty embedded into out product
for example..

Post by Warner Losh
All we need to do is to find / when we're booting of removable media. This
means we can use a super-simplified method to find it. If you want
something more complicated, you must use the EFI boot manager protocol. I'm
in the final stages, btw, at work of reviewing code we've written to
implement a mostly Linux CLI compatable efibootmgr.

Post by Eric McCorkle
1) Parse command-line args (this is to maintain back-compatibility)

UEFI:BootXXXX also provides a way to pass this in.

Post by Eric McCorkle
5) Otherwise, try the legacy currdev/loaddev detection scheme (this will
fail if we're installed to the ESP)

We don't need to do this...

Post by Eric McCorkle
6) If that fails, check the preferred devices

At most we need to check the same device we were booted from if the EFI
variables aren't set.

Post by Eric McCorkle
7) If that fails, check all devices

I really really want to never do this again. It causes more problems than
it solves and is part of the boot1.efi hack we should run away from. None
of our other systems do it, and boot1.efi did it only as a kludge because
it didn't implement the EFI Boot Manager protocol.
I think we should do it as I described above: find things directly from the
BootXXXX variable, per the EFI Boot Manager Protocol, and then fallback to
the first UFS filesystem with /boot/kernel/kernel on the same device we
were loaded from.

and ZFS? I haven't creates a UFS system for some time

Post by Eric McCorkle
8) If that fails, continue without a currdev (which will drop us to the
loader prompt)

That's fine.

Post by Eric McCorkle
What do people think of this procedure?

I have some issues with it.
Warner
_______________________________________________
https://lists.freebsd.org/mailman/listinfo/freebsd-arch

Warner Losh

2017-10-26 15:31:42 UTC

Post by Eric McCorkle
Following the earlier discussion on the fate of loader.efi, and in light

Post by Eric McCorkle
of my GELI work, I've begun working on modifications to loader.efi to
allow it to be installed to the ESP, while maintaining
back-compatibility with the legacy system.
Cool. I have some concerns, as this doesn't follow the doc I posted a

while
ago. I've not yet worked through the boot1.efi removal in that document,
however.
There's a couple of points that probably warrant discussion before I go

Post by Eric McCorkle
any farther.
First, we'll need to figure out what to do with boot.conf, which
previously contained the arguments that would get passed to loader.
Personally, I think the logical place for this is /efi/boot/config or
/efi/boot.conf
I would vote neither. If it belongs on the ESP at all, it belongs in

\efi\freebsd\boot.conf. We now own \efi\freebsd, so that's where all our
boot files need to wind up in our install. But I'm not sure that we need it
at all, since boot.conf is only for boot1/boot2 in the current system
(though it does set serial by default, which might be helpful to preserve).
It would be even better, though, to use the command line that's passed in
as part of the UEFI:BootXXXX environment variable. Since we have to move it
anyway, we should do things in the EFI way, which is to store all the
nonvolatile info in UEFI environment variables (and in this case, the
command line / aux info in the BootXXXX variable). We really need to stop
polluting \efi\boot with anything, except on removable media.

boot.conf is also used to set kenv values so that they have knows values
when entering the kernel.
POLA suggests that it should continue to do that.

On some platforms it is, but usually that's done via loader.conf or
device.hints. The current boot1.efi doesn't do this, for example, nor do
many of the other loaders.

Second, does loader.conf exist on the boot filesystem, or should it also

Post by Eric McCorkle
exist on the ESP? I'm inclined to think this should be left on the boot
filesystem, as there may be more than one, and it's potentially specific
to the kernel installed there.
I think it should be in /boot/loader.conf on /, not the ESP. We should

only
have \efi\freebsd\loader.efi in the ESP.

why not have both a place to set defaults for all Filesystems (in ESP)
and a filesystem specific one? (on the fs)

I don't understand this comment at all. loader.conf is in the specific /
we'll be loading from, period. That's how it has to be, otherwise if you
have multiple systems on one disk that need different loader.conf, it
fails. There's no benefit to also reading an extra one from here, but extra
complexity.

Last, loader has typically relied on the fact that it's installed

Post by Eric McCorkle
directly on the boot filesystem, so it can obtain a good initial
currdev/loaddev. If it's installed on the ESP, it potentially has to go
hunting for the boot device.
I'm not sure that it should look very hard, and it should only look as a

last result.
UEFI:BootCurrent lists the current boot variable. It points to the
UEFI:BootXXXX that contains a LoadOption variable. This variable contains a
series of DevicePaths. The first one is what the UEFI BIOS uses to load
loader.efi (installed as ESP:\EFI\FREEBSD\LOADER.EFI). The second one in
the list will point to the kernel to load. That way we shouldn't have to go
looking at all. We assume that the root is the same partition we load the
kernel comes from. We should only go looking if we fail to find the second
path in the LoadOption.
I'm stealing (back) code from my boot1 refactor for this, which first

already
have this installed. So we don't have to recreate this behavior if we don't
want to since updated systems will need to follow the efibootmgr protocol.
There's no way the loader.efi will fit on the old ESPs we created anyway...
loader.efi needs to cope with being loaded this way, but it doesn't have to
recreate boot1.efi's behavior.

depending on what behaviour you are talking about..
loader.conf setting kenv values is pretty embedded into out product for
example..

loader.conf and device.hints isn't an issue, and that behavior won't
change. It's only the super-funky boot.conf that people today only put -h
or -S115200 into on x86 (though some embedded systems have some hooks here
for limited setting of loader env variables).

All we need to do is to find / when we're booting of removable media. This

Post by Eric McCorkle
means we can use a super-simplified method to find it. If you want
something more complicated, you must use the EFI boot manager protocol. I'm
in the final stages, btw, at work of reviewing code we've written to
implement a mostly Linux CLI compatable efibootmgr.

Post by Eric McCorkle
1) Parse command-line args (this is to maintain back-compatibility)
UEFI:BootXXXX also provides a way to pass this in.

2) Load /efi/boot.conf or /efi/boot/config if they exist, parse the args

Post by Eric McCorkle
inside as if they were extra args
4) If loaddev/currdev are set by command-line arguments, use them as-is
and stop
I don't want to do this from the command line. We follow the EFI boot

manager protocol. There's no need to invent our own here, especially since
the loader's devices aren't familiar to people. It adds extra complication
for no benefit.
5) Otherwise, try the legacy currdev/loaddev detection scheme (this will

Post by Eric McCorkle
fail if we're installed to the ESP)
We don't need to do this...

6) If that fails, check the preferred devices

Post by Eric McCorkle
At most we need to check the same device we were booted from if the EFI

variables aren't set.
7) If that fails, check all devices

Post by Eric McCorkle
I really really want to never do this again. It causes more problems than

it solves and is part of the boot1.efi hack we should run away from. None
of our other systems do it, and boot1.efi did it only as a kludge because
it didn't implement the EFI Boot Manager protocol.
I think we should do it as I described above: find things directly from the
BootXXXX variable, per the EFI Boot Manager Protocol, and then fallback to
the first UFS filesystem with /boot/kernel/kernel on the same device we
were loaded from.

and ZFS? I haven't creates a UFS system for some time

Fallback does include ZFS too. But I really want to have the fallback be
"almost nothing at all" and if that doesn't work for you, you have to set
the right EFI env vars. The reason is that the DWIM behavior seems nice,
until you want to do something complicated, then it just gets in the way.
And most of the time, the 'almost nothing' case covers what you need. Our
current 'search for it' approach is fundamentally broken and we must move
away from it to a more explicit do this or fail approach. Otherwise, I
can't have a system with 4 disks with 2 images each on them, listed in some
custom order that I know about that the guessing scheme can't possibly know
about. In my case, I'd have 8 BootXXXX variables listing, say, p2 on each
disk in order, then p3 on each disk which might change to p3 on each disk
then p2 on each disk, or it may even be p3 on this subset of disks only,
then p2 if I discover down the road that I can't use one of the disks, but
am unable to turn it off or reformat it (flash fails read-only).

And yes, that's a real-world example from our next generation of servers. :)

Warner

8) If that fails, continue without a currdev (which will drop us to the

Post by Eric McCorkle
loader prompt)
That's fine.

What do people think of this procedure?

Post by Eric McCorkle
I have some issues with it.

Warner
_______________________________________________
https://lists.freebsd.org/mailman/listinfo/freebsd-arch

Eric McCorkle

2017-10-26 12:32:58 UTC

Post by Warner Losh
I would vote neither. If it belongs on the ESP at all, it belongs in
\efi\freebsd\boot.conf. We now own \efi\freebsd, so that's where all our
boot files need to wind up in our install. But I'm not sure that we need
it at all, since boot.conf is only for boot1/boot2 in the current system
(though it does set serial by default, which might be helpful to
preserve). It would be even better, though, to use the command line
that's passed in as part of the UEFI:BootXXXX environment variable.
Since we have to move it anyway, we should do things in the EFI way,
which is to store all the nonvolatile info in UEFI environment variables
(and in this case, the command line / aux info in the BootXXXX
variable). We really need to stop polluting \efi\boot with anything,
except on removable media.

That seems reasonable, however it might be worth having a backup
mechanism (boot.conf) in case the EFI vars get clobbered. Also, a file
with the args can be more easily signed and verified than EFI vars.

Post by Warner Losh
I think it should be in /boot/loader.conf on /, not the ESP. We should
only have \efi\freebsd\loader.efi in the ESP.

Also my inclination.

Post by Warner Losh
I'm not sure that it should look very hard, and it should only look as a
last result.
UEFI:BootCurrent lists the current boot variable. It points to the
UEFI:BootXXXX that contains a LoadOption variable. This variable
contains a series of DevicePaths. The first one is what the UEFI BIOS
uses to load loader.efi (installed as ESP:\EFI\FREEBSD\LOADER.EFI). The
second one in the list will point to the kernel to load. That way we
shouldn't have to go looking at all. We assume that the root is the same
partition we load the kernel comes from. We should only go looking if we
fail to find the second path in the LoadOption.

Right. The search would be a fallback mechanism, if all else fails. It
doesn't even need to succeed; it just needs to make a reasonable guess.

Post by Warner Losh
No, we don't. boot1.efi already does this, and legacy systems will
already have this installed. So we don't have to recreate this behavior
if we don't want to since updated systems will need to follow the
efibootmgr protocol. There's no way the loader.efi will fit on the old
ESPs we created anyway... loader.efi needs to cope with being loaded
this way, but it doesn't have to recreate boot1.efi's behavior.

OK, I was assuming there'd be an interim period where loader.efi still
gets installed to /boot/, in which case the dual-purpose would be
necessary. It sounds like you want to quit installing it there altogether.

With regard to GELI, it sounds like it can actually be committed in
parallel with any of the changes you're planning here. This would give
loader the ability to attach and explore EFI partitions, but you
wouldn't be able to boot a pure-GELI system until the loader-only setup
is viable.

Warner Losh

2017-10-26 15:19:15 UTC

So long as it's a backup method, that's fine. It occurs to me that it might
be useful for removable media where you can't count on EFI env vars and you
are bootstrapping such that you need the args (serial console is the
canonical case).

Post by Eric McCorkle
I think it should be in /boot/loader.conf on /, not the ESP. We should

Post by Warner Losh
only have \efi\freebsd\loader.efi in the ESP.

Also my inclination.

Excellent.

If we do have a boot.conf to supply a backup way of specifying command line
args, then we should look for it in ESP:\efi\freebsd\boot.conf.

Post by Eric McCorkle
I'm not sure that it should look very hard, and it should only look as a

Post by Warner Losh
last result.
UEFI:BootCurrent lists the current boot variable. It points to the
UEFI:BootXXXX that contains a LoadOption variable. This variable
contains a series of DevicePaths. The first one is what the UEFI BIOS
uses to load loader.efi (installed as ESP:\EFI\FREEBSD\LOADER.EFI). The
second one in the list will point to the kernel to load. That way we
shouldn't have to go looking at all. We assume that the root is the same
partition we load the kernel comes from. We should only go looking if we
fail to find the second path in the LoadOption.

Right. The search would be a fallback mechanism, if all else fails. It
doesn't even need to succeed; it just needs to make a reasonable guess.

The fallback mechanism should try too hard though. Trying hard gets in the
way of many things. The biggest one being when I have multiple disks in a
system that have multiple copies of the OS. You want the fallback mechanism
to try as limited a number of things as possible then FAIL so we can go to
the next BootXXXX in the boot order. So long as the guesses are super
limited, and/or only done done when we don't specify something explicit,
then that's OK.

So the change from the present would be:

(1) If a second path is specified in the BootXXXX option, then boot it
(might want to look in the list to generalize this, but for the moment
that's an enhancement). If we fail to boot an explicit path, fail back to
the EFI boot manager. We were told to do something explicitly, and we
couldn't do that, so we shouldn't go guessing (this includes ZFS BE, UFS,
etc)
(2) If no path is specified and if we come from a UFS or ZFS partition (the
boot1.efi vector), then try to boot the kernel off of that partition. If
that fails, then fail the boot back to EFI boot manager.
(3) If no path is specified and there's ZFS BE we can boot from, boot that
(fail if we can't)
(4) if no path is specified and we can find a UFS partition on the current
disk, boot that (fail if we can't)

Otherwise fail back to EFI boot manager for the next one on the list.

Post by Eric McCorkle
No, we don't. boot1.efi already does this, and legacy systems will

Post by Warner Losh
already have this installed. So we don't have to recreate this behavior
if we don't want to since updated systems will need to follow the
efibootmgr protocol. There's no way the loader.efi will fit on the old
ESPs we created anyway... loader.efi needs to cope with being loaded
this way, but it doesn't have to recreate boot1.efi's behavior.

Yes. I want to get loader.efi in shape to be the primary loader, but won't
get rid of boot1.efi right away until the installer catches up. A small
case could be made for having boot1.efi for legacy users that keeps simple
UFS / ZFS features we have in 11.x, but removes any features we've added
since then. It's still unclear to me if that's useful or not.

Post by Eric McCorkle
With regard to GELI, it sounds like it can actually be committed in
parallel with any of the changes you're planning here. This would give
loader the ability to attach and explore EFI partitions, but you
wouldn't be able to boot a pure-GELI system until the loader-only setup
is viable.

Yes. I agree. I'd like to see it so you could install one by hand in the
base prior to the installer supporting that. I'll have to look at the
individual changes carefully, but at a high level when I saw them they
looked like the right direction.

I'll try to find some time today to update the boot document and to try to
draw in points you have in your chain of trust documents because that's
also important. It might also be worth bringing in the 'self contained load
env' work that I know is going on elsewhere (basically, you load a loader
with a MD burned into it and boot from that, with the entire loader.efi
signed such that the shim-loader that floating around can load and verify
it -- a lite version of the stuff you are working on).

Warner

Eric McCorkle

2017-10-27 01:22:28 UTC