eric_metricspace.net (Eric McCorkle)
User

Projects

User does not belong to any projects.

User Details

User Since
Nov 7 2015, 11:25 PM (72 w, 5 d)

Recent Activity

Mon, Mar 6

eric_metricspace.net updated the diff for D9575: Boot-time Key Intake Metadata.

Add changes by Allan Jude, adding support to boot2

Mon, Mar 6, 6:23 PM
eric_metricspace.net added inline comments to D9909: Add getdesc interface to struct devsw.
Mon, Mar 6, 6:18 PM
eric_metricspace.net retitled D9909: Add getdesc interface to struct devsw from to Add getdesc interface to struct devsw.
Mon, Mar 6, 6:16 PM

Wed, Mar 1

eric_metricspace.net added a comment to D9798: Add explicit_bzero() to libstand.

Good change for my pending keybuf work

Wed, Mar 1, 3:03 PM

Feb 20 2017

eric_metricspace.net updated the diff for D9575: Boot-time Key Intake Metadata.
Feb 20 2017, 11:03 PM
eric_metricspace.net added a comment to D9687: Add string16 API.

I like the interface in lib/libefi/libefi.c

Feb 20 2017, 7:51 PM
eric_metricspace.net added a comment to D9687: Add string16 API.
In D9687#200246, @imp wrote:

All the string functions suffer from the same problem we have with the original string functions you replaced: they don't properly implement character conversions. Please see the efivar stuff I did in -current to see what you need to do there. And you should put the functions I wrote there into libstand. While these functions work well enough for ASCIIish things, they fail for anything more complex. If we're going to rework things, let's do it correctly and reuse what was done correctly for efivar (who, to be fair, stole and reworked the code from Marcel's earlier efi on ia64 work).

Feb 20 2017, 4:59 PM
eric_metricspace.net retitled D9687: Add string16 API from to Add string16 API.
Feb 20 2017, 2:06 PM
eric_metricspace.net updated the diff for D9679: Add API functions and interfaces for EFI refactoring.

Add some more utility functions

Feb 20 2017, 2:00 PM
eric_metricspace.net retitled D9680: Increase EFI MSDOSFS image size to 512Kib from to Increase EFI MSDOSFS image size to 512Kib.
Feb 20 2017, 12:12 AM
eric_metricspace.net added reviewers for D9679: Add API functions and interfaces for EFI refactoring: manu, emaste, imp.
Feb 20 2017, 12:05 AM
eric_metricspace.net added reviewers for D9679: Add API functions and interfaces for EFI refactoring: allanjude, tsoome, cpm.
Feb 20 2017, 12:04 AM
eric_metricspace.net retitled D9679: Add API functions and interfaces for EFI refactoring from to Add API functions and interfaces for EFI refactoring.
Feb 20 2017, 12:04 AM

Feb 19 2017

eric_metricspace.net added inline comments to D9575: Boot-time Key Intake Metadata.
Feb 19 2017, 11:51 PM
eric_metricspace.net added inline comments to D9575: Boot-time Key Intake Metadata.
Feb 19 2017, 9:28 PM
eric_metricspace.net updated the diff for D9575: Boot-time Key Intake Metadata.
Feb 19 2017, 9:28 PM
eric_metricspace.net added a comment to D7589: EFI boot1 and loader Refactoring.

Update on this patch: I'm going to attempt to break it up into some smaller changesets to make it more manageable.

Feb 19 2017, 1:03 PM
eric_metricspace.net added inline comments to D9575: Boot-time Key Intake Metadata.
Feb 19 2017, 12:55 PM

Feb 18 2017

eric_metricspace.net added a comment to D9575: Boot-time Key Intake Metadata.
In D9575#199104, @cem wrote:

This diff lacks context. Please use arc or diff with -U999999.

Feb 18 2017, 5:08 PM
eric_metricspace.net added inline comments to D9575: Boot-time Key Intake Metadata.
Feb 18 2017, 4:57 PM
eric_metricspace.net updated the diff for D9575: Boot-time Key Intake Metadata.
Feb 18 2017, 4:54 PM

Feb 16 2017

eric_metricspace.net added a comment to D9575: Boot-time Key Intake Metadata.
In D9575#199104, @cem wrote:
Feb 16 2017, 8:21 PM

Feb 15 2017

eric_metricspace.net updated the diff for D9575: Boot-time Key Intake Metadata.
Feb 15 2017, 4:12 PM

Feb 14 2017

eric_metricspace.net updated the diff for D9575: Boot-time Key Intake Metadata.

Moved MODINFOMD_KEYBUF definition to sys/linker.h

Feb 14 2017, 6:11 PM
eric_metricspace.net added inline comments to D9575: Boot-time Key Intake Metadata.
Feb 14 2017, 6:00 PM
eric_metricspace.net updated the diff for D9575: Boot-time Key Intake Metadata.

Added hook to clear GELI keys out of the intake buffer at mountroot.

Feb 14 2017, 5:59 PM
eric_metricspace.net added a comment to D9575: Boot-time Key Intake Metadata.

An observation: as presently implemented, this would bypass the password check for any GELI volume that was detached and later attached again. This is probably not desirable, as it could break security models.

One way to deal with this is to disable the keys after they are used to decrypt a volume. However, it warrants discussion.

I think we should explicit_bzero the keys/password as soon as we are done with them. GELI should already be doing this everything before the patch.

Feb 14 2017, 5:04 PM
eric_metricspace.net added a comment to D9575: Boot-time Key Intake Metadata.

An observation: as presently implemented, this would bypass the password check for any GELI volume that was detached and later attached again. This is probably not desirable, as it could break security models.

Feb 14 2017, 3:48 PM
eric_metricspace.net added inline comments to D9575: Boot-time Key Intake Metadata.
Feb 14 2017, 3:03 PM
eric_metricspace.net updated the diff for D9575: Boot-time Key Intake Metadata.
Feb 14 2017, 3:03 PM
eric_metricspace.net added inline comments to D9575: Boot-time Key Intake Metadata.
Feb 14 2017, 2:42 PM
eric_metricspace.net updated the diff for D9575: Boot-time Key Intake Metadata.

Addressed style issues.

Feb 14 2017, 2:40 PM

Feb 13 2017

eric_metricspace.net retitled D9575: Boot-time Key Intake Metadata from to Boot-time Key Intake Metadata.
Feb 13 2017, 8:43 PM

Jan 15 2017

eric_metricspace.net added a comment to D7589: EFI boot1 and loader Refactoring.

btw, there is also D8581 - it does work on another dimension however, by reusing disk and part API, and providing already familiar output for user.

Jan 15 2017, 1:15 PM

Jan 14 2017

eric_metricspace.net added a comment to D7589: EFI boot1 and loader Refactoring.

With partition - filesystem mapping we can get the device handle without much issue - the handle stack has handles from FS down to partition and disk handle and the api to access the next handle from stack is there.

With zfs the situation is much more complicated as it is dual disk and file system and we need interface to access BE list and to switch the "current" filesystem on top of the "disk", and UEFI api itself does not help us there. It is possible to create specific protocol on top of zfs pool to implement an mechanism to get information about BE's and to switch, but it means that instead of reducing complexity, we are adding more.... note that the dataset list specifying BE's can be quite long, on my dev host I currently do have 73 entries.

Also, as I already wrote, for human operating the machine, it is really important to know exactly what I'm going to boot from.

Jan 14 2017, 3:31 PM

Jan 13 2017

eric_metricspace.net added a comment to D7589: EFI boot1 and loader Refactoring.

tsoome, could you try out the new changes on your complex disk setup and let me know what the output looks like?

Jan 13 2017, 5:46 PM

Jan 7 2017

eric_metricspace.net updated the diff for D7589: EFI boot1 and loader Refactoring.

Fixed issue with directory listing file types.

Jan 7 2017, 11:56 PM
eric_metricspace.net added a comment to D7589: EFI boot1 and loader Refactoring.

It seems that I forgot about the file type issue with ls. I'll address that next, but please test the current patch to make sure it addresses the naming issue adequately.

Jan 7 2017, 9:50 PM
eric_metricspace.net updated the diff for D7589: EFI boot1 and loader Refactoring.

Added functionality to print out information about the filesystems with lsdev.

Jan 7 2017, 9:49 PM

Dec 6 2016

eric_metricspace.net added a comment to D7589: EFI boot1 and loader Refactoring.

ok, got it built, and initial test. And there is this problem...

OK lsdev

EFI0:    EFI(SIMPLE_FILE_SYSTEM)
EFI1:    EFI(SIMPLE_FILE_SYSTEM)
EFI2:    EFI(SIMPLE_FILE_SYSTEM)
EFI3:    EFI(SIMPLE_FILE_SYSTEM)
EFI4:    EFI(SIMPLE_FILE_SYSTEM)
EFI5:    EFI(SIMPLE_FILE_SYSTEM)
EFI6:    EFI(SIMPLE_FILE_SYSTEM)
EFI7:    EFI(SIMPLE_FILE_SYSTEM)

net devices:

net0:

OK

Dec 6 2016, 4:26 AM

Dec 5 2016

eric_metricspace.net added a comment to D7589: EFI boot1 and loader Refactoring.

Yep, we can skip dosfs there, and for very simple reason - if the firmware is not able to read system partition, there is no boot anyhow.

Dec 5 2016, 3:40 PM
eric_metricspace.net added a comment to D7589: EFI boot1 and loader Refactoring.

OK, based on others' reproduction of tsoome's issue on -hackers, I have a credible diagnostic theory.

Dec 5 2016, 2:39 PM
eric_metricspace.net added a comment to D7589: EFI boot1 and loader Refactoring.

tsoome, can I get you to build and test with the extra_logging branch on my github repo: https://github.com/emc2/freebsd

Dec 5 2016, 1:33 PM

Dec 2 2016

eric_metricspace.net added a comment to D7589: EFI boot1 and loader Refactoring.

I've posted a CFT to -hackers, -current, and -amd64.

Dec 2 2016, 5:41 PM

Dec 1 2016

eric_metricspace.net added a comment to D7589: EFI boot1 and loader Refactoring.

I am getting rejects on patch, could you rebase on current head?:)

tsoome@freebsd-2:/usr/eric % find . -name \*.rej
./lib/libstand/stand.h.rej
./sys/boot/efi/boot1/zfs_module.c.rej
./sys/boot/efi/boot1/ufs_module.c.rej
./sys/boot/efi/boot1/boot_module.h.rej
tsoome@freebsd-2:/usr/eric %

Strange. I had rebased just before I sent this up. I am rebasing from the github master, if that changes things at all. The last commit in the history is an hour ago.

My github repo for this is here: https://github.com/emc2/freebsd/tree/efize_new

Be aware: I'm using rebase -i to squash everything down to one commit at this point, so there's a lot of forced pushes going into efize_new

that may explain, as my fbsd source instances are all against svn ... so could it be the git has been missing some updates?

There's probably some lag going on somewhere. I see Ed Maste did push a change to the EFI loaders.

It might make more sense to build off of my repo in order to diagnose the issue your setup found.

Note, i may have been hit also by my own bug... see D8644

Dec 1 2016, 6:59 PM
eric_metricspace.net added a comment to D7589: EFI boot1 and loader Refactoring.

I am getting rejects on patch, could you rebase on current head?:)

tsoome@freebsd-2:/usr/eric % find . -name \*.rej
./lib/libstand/stand.h.rej
./sys/boot/efi/boot1/zfs_module.c.rej
./sys/boot/efi/boot1/ufs_module.c.rej
./sys/boot/efi/boot1/boot_module.h.rej
tsoome@freebsd-2:/usr/eric %

Strange. I had rebased just before I sent this up. I am rebasing from the github master, if that changes things at all. The last commit in the history is an hour ago.

My github repo for this is here: https://github.com/emc2/freebsd/tree/efize_new

Be aware: I'm using rebase -i to squash everything down to one commit at this point, so there's a lot of forced pushes going into efize_new

that may explain, as my fbsd source instances are all against svn ... so could it be the git has been missing some updates?

Dec 1 2016, 5:07 PM
eric_metricspace.net added a comment to D7589: EFI boot1 and loader Refactoring.

I did build from current state, the patch did apply clean, build faile due to -lstand issue noted before, thats simple to fix.

Unfortunately the boot is not going well, last message is:

Initializing modules: FS Backend-

the '-' is spinner it seems. The VM has 4 disks, illumos mirror zfs on first 2, third has fbsd, 4th (for this test) has EFI system partition, and zfs and ufs partitions, zfs partition has bootfs with /boot directory.

I did build from current state, the patch did apply clean, build faile due to -lstand issue noted before, thats simple to fix.

Unfortunately the boot is not going well, last message is:

Initializing modules: FS Backend-

the '-' is spinner it seems. The VM has 4 disks, illumos mirror zfs on first 2, third has fbsd, 4th (for this test) has EFI system partition, and zfs and ufs partitions, zfs partition has bootfs with /boot directory.

Failing in FS Backend would indicate that something's going wrong with the filesystem attach phase. The FS Backend initialization scans all device handles and attaches the right filesystem drivers.

If have been testing on my own laptops with 2 disks, one with an EFI system partition and a ZFS zolume, one with a ZFS cache and log.

Therefore, the most likely culprit is UFS, though multiple ZFS volumes or the illumos mirror could be a problem.

What VM are you using? The easiest way to hunt this down would be to replicate your setup locally. The second easiest would be to get you a patch with extra logging.

vmware fusion. The illumos zfs mirror by itself should not be problem as loader zfs reader can read it both ways without problems (illumos loader can access fbsd pools and vice versa), also multiple pools should not be an issue as such - and definitely is scenario for tests. UFS may be issue, altho in this test setup, its just empty, I had that idea to create /boot on it as next step:) There is also another potential pain point - the memory allocations - as boot1 is now linked with libstand, could it be the malloc/free used all around are interfering somehow...

Replaced uses of AllocatePool with malloc. Tested with a UFS partition, no problem.

tsoome, could you see if this affects anything? If not, then email me directly to coordinate diagnosis of the bug your setup found.

I am getting rejects on patch, could you rebase on current head?:)

tsoome@freebsd-2:/usr/eric % find . -name \*.rej
./lib/libstand/stand.h.rej
./sys/boot/efi/boot1/zfs_module.c.rej
./sys/boot/efi/boot1/ufs_module.c.rej
./sys/boot/efi/boot1/boot_module.h.rej
tsoome@freebsd-2:/usr/eric %

Dec 1 2016, 4:47 PM
eric_metricspace.net updated the diff for D7589: EFI boot1 and loader Refactoring.

Replaced uses of AllocatePool with malloc. Tested with a UFS partition, no problem.

Dec 1 2016, 3:15 PM

Nov 22 2016

eric_metricspace.net added a comment to D7589: EFI boot1 and loader Refactoring.

I did build from current state, the patch did apply clean, build faile due to -lstand issue noted before, thats simple to fix.

Unfortunately the boot is not going well, last message is:

Initializing modules: FS Backend-

the '-' is spinner it seems. The VM has 4 disks, illumos mirror zfs on first 2, third has fbsd, 4th (for this test) has EFI system partition, and zfs and ufs partitions, zfs partition has bootfs with /boot directory.

I did build from current state, the patch did apply clean, build faile due to -lstand issue noted before, thats simple to fix.

Unfortunately the boot is not going well, last message is:

Initializing modules: FS Backend-

the '-' is spinner it seems. The VM has 4 disks, illumos mirror zfs on first 2, third has fbsd, 4th (for this test) has EFI system partition, and zfs and ufs partitions, zfs partition has bootfs with /boot directory.

Failing in FS Backend would indicate that something's going wrong with the filesystem attach phase. The FS Backend initialization scans all device handles and attaches the right filesystem drivers.

If have been testing on my own laptops with 2 disks, one with an EFI system partition and a ZFS zolume, one with a ZFS cache and log.

Therefore, the most likely culprit is UFS, though multiple ZFS volumes or the illumos mirror could be a problem.

What VM are you using? The easiest way to hunt this down would be to replicate your setup locally. The second easiest would be to get you a patch with extra logging.

vmware fusion. The illumos zfs mirror by itself should not be problem as loader zfs reader can read it both ways without problems (illumos loader can access fbsd pools and vice versa), also multiple pools should not be an issue as such - and definitely is scenario for tests. UFS may be issue, altho in this test setup, its just empty, I had that idea to create /boot on it as next step:) There is also another potential pain point - the memory allocations - as boot1 is now linked with libstand, could it be the malloc/free used all around are interfering somehow...

Nov 22 2016, 2:54 PM

Nov 21 2016

eric_metricspace.net added a comment to D7589: EFI boot1 and loader Refactoring.

I did build from current state, the patch did apply clean, build faile due to -lstand issue noted before, thats simple to fix.

Unfortunately the boot is not going well, last message is:

Initializing modules: FS Backend-

the '-' is spinner it seems. The VM has 4 disks, illumos mirror zfs on first 2, third has fbsd, 4th (for this test) has EFI system partition, and zfs and ufs partitions, zfs partition has bootfs with /boot directory.

Nov 21 2016, 2:51 PM
eric_metricspace.net updated the diff for D7589: EFI boot1 and loader Refactoring.

IFC, also properly restored preferred functionality

Nov 21 2016, 1:42 AM

Oct 12 2016

eric_metricspace.net added a comment to D7589: EFI boot1 and loader Refactoring.

The preferred partition functionality is restored. The boot1 program now obtains the device handle from the loaded image handle, and checks for the presence of an EFI_SIMPLE_FILE_SYSTEM interface after attaching all drivers before scanning all partitions. This will have the effect of preferring the boot device on which the boot1 program resides.

Oct 12 2016, 11:27 AM
eric_metricspace.net updated the diff for D7589: EFI boot1 and loader Refactoring.
Oct 12 2016, 11:24 AM

Sep 17 2016

eric_metricspace.net added a comment to D7589: EFI boot1 and loader Refactoring.

To proceed, I need guidance on what exactly should be done to restore the preferred functionality.

Sep 17 2016, 10:16 PM
eric_metricspace.net added a comment to D7589: EFI boot1 and loader Refactoring.
In D7589#162680, @manu wrote:
Sep 17 2016, 10:14 PM
eric_metricspace.net added a comment to D7589: EFI boot1 and loader Refactoring.
In D7589#162585, @imp wrote:
In D7589#162441, @manu wrote:
In D7589#157710, @imp wrote:

It has by far caused us no end of pain. What's part10:? Our systems have 4 drives with 9 partitions on each one (except the ones that have 2, surprise!). The ordering is absolutely random as well, it seems. Some systems get part10 being the boot disk, others get part 2. Totally nuts.

I would like it if loader.efi understood partitions. @manu looked into it at BSDCan, however from my understanding, he found it was easier to teach U-Boot to expose partitions to loader.efi than add the existing GPT/MBR code to loader.efi.

If someone was to get it working I would imagine we could rename the partitions to be something like disk0p0 or disk1s3. The device name may be different than the kernel, however the partition id should be the same.

I think that the problem I got during bsdcan was boot1 unable to work well with unaligned partition so I would need to find time to look at this again (because of course I don't have the patches anymore).

On the review itself I'm a bit concerned about what this would mean for U-Boot.
I admit that I haven't read all the code but it seems that with this patch the UEFI implementation would need to support EFI_SIMPLE_FILE_SYSTEM (which U-Boot doesn't).
This is true that arm/arm64 aren't Tier-1 arch like AMD64 and that one UEFI implementation is required to support this protocol (from my understanding of the spec) but all the effort I've put on running boot1.efi/loader.efi on arm/arm64 (because this is the right way to take) would be useless if this is the case.

Erik: Can you confirm or not that EFI_SIMPLE_FILE_SYSTEM is needed ?
If this is the case we would need a fallback on what boot1/loader is currently doing.

Since we're implementing a EFI_SIMPLE_FILE_SYSTEM for UFS and ZFS, why would u-boot not supporting it matter?

Warner

Sep 17 2016, 10:10 PM
eric_metricspace.net added a comment to D7589: EFI boot1 and loader Refactoring.
In D7589#162756, @cem wrote:

My biggest concern is that this is a giant patch that does a bunch of different things. I'd like to see smaller commits that do one thing at a time.

Sep 17 2016, 10:06 PM
eric_metricspace.net added inline comments to D7589: EFI boot1 and loader Refactoring.
Sep 17 2016, 10:01 PM
eric_metricspace.net updated the diff for D7589: EFI boot1 and loader Refactoring.

Fix a bunch of style issues. Still need to restore preferred partition support.

Sep 17 2016, 9:55 PM

Aug 21 2016

eric_metricspace.net added a comment to D7589: EFI boot1 and loader Refactoring.

The preferred stuff probably got squashed in a merge. A question worth asking: should I reinstate it as-is, or is there any minor improvement that could be done opportunistically?

Aug 21 2016, 8:49 PM
eric_metricspace.net added a comment to D7589: EFI boot1 and loader Refactoring.

RE: device names- There are options here. EFI has a protocol that can be used to attach an arbitrary string as a description. It wouldn't be too hard to print that name out if it's present on a handle. You could generate those names from a number of sources, in fact: filesystem labels, ZFS UUID's, or EFI device paths as a fallback mechanism.

Aug 21 2016, 8:47 PM
eric_metricspace.net added a comment to D7589: EFI boot1 and loader Refactoring.

There isn't a need to clean up memory after boot1, and in fact, that will probably break things. Once an EFI driver is initialized and attached to handles, it stays around until ExitBootServices or until the driver is unloaded (which currently isn't done ever).

Aug 21 2016, 8:44 PM
eric_metricspace.net retitled D7589: EFI boot1 and loader Refactoring from to EFI boot1 and loader Refactoring.
Aug 21 2016, 12:10 PM

Jan 8 2016

eric_metricspace.net added inline comments to D4515: ZFS EFI Boot Support #2.
Jan 8 2016, 1:28 PM

Jan 6 2016

eric_metricspace.net added a comment to D4515: ZFS EFI Boot Support #2.

I closed the other review. What exactly remains to be done on this?

Jan 6 2016, 2:52 AM
eric_metricspace.net abandoned D4104: ZFS EFI Boot Support.

Closing. Conduct all review on https://reviews.freebsd.org/D4515

Jan 6 2016, 2:51 AM

Dec 29 2015

eric_metricspace.net added a comment to D4104: ZFS EFI Boot Support.

Work on this has moved to https://reviews.freebsd.org/D4515. Please make any requests for revision there.

Dec 29 2015, 10:03 PM

Dec 19 2015

eric_metricspace.net added a comment to D4515: ZFS EFI Boot Support #2.

Thanks for taking care of the remaining cleanup.

Dec 19 2015, 4:57 PM

Dec 10 2015

eric_metricspace.net added a comment to D4104: ZFS EFI Boot Support.

Thanks for submitting the update. I am currently tied up dealing with year-end stuff, so not much time to work on it. I will have more time after mid next week.

Dec 10 2015, 4:18 AM

Dec 7 2015

eric_metricspace.net added inline comments to D4104: ZFS EFI Boot Support.
Dec 7 2015, 1:18 PM
eric_metricspace.net updated the diff for D4104: ZFS EFI Boot Support.

Fixed strlen bug

Dec 7 2015, 1:14 PM

Nov 29 2015

eric_metricspace.net added a comment to D4104: ZFS EFI Boot Support.

Also, it should cleanly apply to the current head and build in make world now.

Nov 29 2015, 12:25 AM
eric_metricspace.net added a comment to D4104: ZFS EFI Boot Support.

Updated diff to address many issues.

Nov 29 2015, 12:22 AM
eric_metricspace.net updated the diff for D4104: ZFS EFI Boot Support.

General cleanup, address style issues. Also slight edits to makefiles.

Nov 29 2015, 12:16 AM

Nov 7 2015

eric_metricspace.net retitled D4104: ZFS EFI Boot Support from to ZFS EFI Boot Support.
Nov 7 2015, 11:46 PM