ARM64: Fixing bug in memstick img creation script

[der_semihalf.com]

Summary

Script did not work because there was mistake in line preparing final image from efi and freebsd partition.
Without this fix it is not possible to create memstick img with this script as the gpart will refuse to create efi partition within mbr scheme.

The patched script succeeded in creating img, that has been uploaded to usb flash drive (and sata disk) and successfully booted on ThunderX.

[emaste]

MBR was originally done intentionally because it avoided a problem with gpt not auto-booting due to a UUID, however if other UEFI fw refuses to boot with MBR this will be the way we have to go.

Which firmware did you test this one with?

[der_semihalf.com]

The problem was not with the EFI. The gpart, that has been called by, mkimg refused to create such combination of partitions.
When I have invoked "make memstick (...)" I got memstick.img.part prepared but mkimg fails producing memstick.img.
When I have tested the mkimg command line it seemd that the "-s mbr" does not want to work with "-p efi:=(...)".
producing error: "mkimg: partition 1: Invalid argument" . Changing the "-s mbr" to "-s gpt" fixed the problem and created image did boot.
I did not know abot the problem with UUID you have mentioned. I have faced problem creating the img itself - but this is good to know. I wil give you the firmware info on Monday since I have no remote access to board right now.

[der_semihalf.com]

The oldest firmwre version, I have tested, is:
Firmware Version: 2016-02-12 12:59:55
BDK Version: thunder-release-v1.20-2-gd555028, Branch: remotes/origin/thunder-dev, Built: pią, 12 lut 2016, 11:58:38 UTC
("pią 12 lut" is "Fri Feb 12")

[der_semihalf.com]

ed, gjb, wma can you please proceed with review? Do you have any more comments?

[gjb]

To the best of my knowledge, this image requires MBR. andrew@ can confirm if this is still true.

[der_semihalf.com]

The script does not work at all because the syntax of mkimg is broken. This script generates no image when there is "-s mbr". You are not able to create memstick.img with this script. I have been able to create GPT type memstick and boot it of SATA and USB flash drive on ThunderX.

Here is EFI shell:

Shell> ver
UEFI Interactive Shell v2.1
EDK II
UEFI v2.40 (Cavium Thunder cn88xx EFI sdk-test-tag-xxx-46-g8f94dfc Feb 12 2016 13:01:59, 0x00000004)

Shell> map
Mapping table
      FS0: Alias(s):HD36b0c0b:;BLK1:
          PciRoot(0x0)/Pci(0x11,0x0)/USB(0x1,0x0)/USB(0x2,0x0)/HD(1,GPT,C5135B52-3307-11E6-91E9-650D143E1A5F,0x3,0x640)
      FS1: Alias(s):HD45a0a1:;BLK4:
          PciRoot(0x1)/Pci(0x8,0x0)/Sata(0x0,0x0,0x0)/HD(1,GPT,30B441DE-3308-11E6-A606-001517E77F34,0x28,0x640)
     BLK7: Alias(s):
          VenHw(25E45362-4074-46DC-88A0-79D6A23F3C9D)
     BLK0: Alias(s):
          PciRoot(0x0)/Pci(0x11,0x0)/USB(0x1,0x0)/USB(0x2,0x0)
     BLK2: Alias(s):
          PciRoot(0x0)/Pci(0x11,0x0)/USB(0x1,0x0)/USB(0x2,0x0)/HD(2,GPT,C5135BC3-3307-11E6-91E9-650D143E1A5F,0x643,0x122110)
     BLK3: Alias(s):
          PciRoot(0x1)/Pci(0x8,0x0)/Sata(0x0,0x0,0x0)
     BLK5: Alias(s):
          PciRoot(0x1)/Pci(0x8,0x0)/Sata(0x0,0x0,0x0)/HD(2,GPT,30BC2011-3308-11E6-A606-001517E77F34,0x668,0x8DFF9C0)
     BLK6: Alias(s):
          PciRoot(0x1)/Pci(0x8,0x0)/Sata(0x0,0x0,0x0)/HD(3,GPT,30C6BF3C-3308-11E6-A606-001517E77F34,0x8E00028,0x70F85F)

Shell> 
Shell> fs0:
FS0:\> cd efi
FS0:\efi\> cd boot
FS0:\efi\boot\> ls
Warning. UEFI function 'gRT->GetTime' returned: No Media.
Directory of: FS0:\efi\boot\
04/12/2016  20:56 <DIR>           512  .
04/12/2016  20:56 <DIR>           512  ..
04/12/2016  20:56             131,072  BOOTAA64.EFI
04/12/2016  20:56                  13  STARTUP.NSH
          2 File(s)     131,085 bytes
          2 Dir(s)
FS0:\efi\boot\> startup.nsh

And here it is booting:

>>FreeBSD EFI boot block
   Loader path: /boot/loader.efi

   Initializing modules: ZFS UFS
   Probing 8 block devices......*..++ done
    ZFS found no pools
    UFS found 3 partitions
Consoles: EFI console  
Command line arguments: lUS_BLK_LOCK not completed
Image base: 0x1ff7bb3000TUS_BLK_LOCK not completed
EFI version: 2.40_BR_STATUS_BLK_LOCK not completed
EFI Firmware: Cavium Thunder cn88xx EFI sdk-test-tag-xxx-46-g8f94dfc Feb 12 2016
 13:01:59 (rev 0.04)

FreeBSD/arm64 EFI loader, Revision 1.1
(root@arm64-prime, Tue Jun  7 16:44:13 CEST 2016)
Loading /boot/defaults/loader.conf
/boot/kernel/kernel text=0x67fb03 data=0x829e0+0x2fafa8 syms=[0x8+0xd9bc0+0x8+0x
d681d]
\
Hit [Enter] to boot immediately, or any other key for command prompt.
Booting [/boot/kernel/kernel]...               
Using DTB provided by EFI at 0x1fff656018.
KDB: debugger backends: ddb
KDB: current backend: ddb
Copyright (c) 1992-2016 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 11.0-ALPHA2 #1 c3a3813(fmaster2)-dirty: Wed Jun 15 15:06:01 CEST 2016
    root@arm64-prime:/usr/home/der/arm64.testbuildenv/arm64.aarch64/usr/home/der/repos/freebsd_10/sys/GENERIC arm64
FreeBSD clang version 3.8.0 (tags/RELEASE_380/final 262564) (based on LLVM 3.8.0)
WARNING: WITNESS option enabled, expect reduced performance.
Starting CPU 1 (1)
Starting CPU 2 (2)
Starting CPU 3 (3)
Starting CPU 4 (4)
Starting CPU 5 (5)
Starting CPU 6 (6)
Starting CPU 7 (7)

[gjb]

Adding marcel@, since this is the absolute first time I am hearing MBR does not work with mkimg(1).

[emaste]

The script does not work at all because the syntax of mkimg is broken. This script generates no image when there is "-s mbr".

mkimg definitely supports "-s mbr"; although I have not physically used a USB stick with a ThunderX I have booted one from a SATA disk. Image was generated with a script using this mkimg invocation:
mkimg -s MBR -p efi:=$ROOTDIR/boot/boot1.efifat -p freebsd:=$ROOTDIR.ufs -p freebsd::2G -o $ROOTDIR.image

Did mkimg report an error, or did it run and just produce no output? What build host are you running make-memstick.sh on? I wonder if there's been some fix to mkimg in a later version.

All of that said I believe we do want to switch to gpt partitioning for these images, but want @andrew to comment as I believe we have some backwards compatibility issues with the (older, but unsure which rev) OVMF firmware used by QEMU.

[emaste]

producing error: "mkimg: partition 1: Invalid argument" . Changing the "-s mbr" to "-s gpt" fixed the problem and created image did boot.

Oh, I missed this note at first, looking...

[der_semihalf.com]

Just try this:

~/tmp/mkimg]$ dd if=/dev/zero of=efi.img.part bs=1024 count=$((1024*4))
4096+0 records in
4096+0 records out
4194304 bytes transferred in 0.055327 secs (75809436 bytes/sec)
~/tmp/mkimg]$ dd if=/dev/zero of=other.img.part bs=1024 count=$((1024*4))
4096+0 records in
4096+0 records out
4194304 bytes transferred in 0.050146 secs (83641674 bytes/sec)
~/tmp/mkimg]$ mkimg -s mbr efi:=efi.img.part freebsd:=other.img.part  -o final.img
~/tmp/mkimg]$ mkimg -s mbr -p efi:=efi.img.part -p freebsd:=other.img.part  -o final.img
mkimg: partition 1: Invalid argument
~/tmp/mkimg]$ mkimg -s MBR -p efi:=efi.img.part -p freebsd:=other.img.part  -o final.img
mkimg: partition 1: Invalid argument
~/tmp/mkimg]$ mkimg -s gtp -p efi:=efi.img.part -p freebsd:=other.img.part  -o final.img
mkimg: scheme: Invalid argument
~/tmp/mkimg]$ mkimg -s gpt -p efi:=efi.img.part -p freebsd:=other.img.part  -o final.img
~/tmp/mkimg]$ echo $?
0
~/tmp/mkimg]$

[emaste]

It works:

feynman% mkimg -s mbr -p efi:=efi.img.part -p freebsd:=other.img.part -o final.img
feynman% echo $?
0

What host version are you running mkimg on? It seems like you don't have rS278968, which should be in 10.2 and 10.3.

[der_semihalf.com]

Oh.sorry True. This one is really old.

[marcel]

If mkimg still returns 0 when it doesn't like the command line options, then we should definitely fix that. You want scripts to be able to check for (or just fail) in that case and not create a broken image.

[emaste]

If mkimg still returns 0 when it doesn't like the command line options, then we should definitely fix that.

It returns non-0 when the command line args fail. The issue here is that this was run on an older host that did not support efi on mbr.

That said, we should make this change once we're sure the issues w/ QEMU OVMF firmware are sorted out.
We also might want to consider adding set -e to make-memstick.

[emaste]

This comment was removed by emaste.

[emaste]

Prebuilt OVMF AArch64 UEFI images are available from Linaro at https://wiki.linaro.org/LEG/UEFIforQEMU
I tested with the latest one available here: http://snapshots.linaro.org/components/kernel/leg-virt-tianocore-edk2-upstream/1036/QEMU-AARCH64/RELEASE_CLANG35/QEMU_EFI.fd

A FreeBSD image boots automatically with both GPT and MBR partitioning, tested with:
mkimg -s GPT -p efi:=$ROOTDIR/boot/boot1.efifat -p freebsd:=$ROOTDIR.ufs -p freebsd::2G -o $ROOTDIR.image

That is I think we should go ahead with this change now.

[marcel]

Probably good to get feedback from people on arm@ or re@.
I'll approve in case there has been silence so far...

[emaste]

Add @andrew who set this to mbr originally to work around buggy early QEMU firmware.

[wma]

Closed by
https://svnweb.freebsd.org/changeset/base/302387