[new port] devel/fb-adb: Better shell for Android devices

[jbeich]

Summary

It's my first port doing cross-build, so expect rough edges.

• @bapt to check whether hacks around Mk/* are justified.
• @marino to check gnatdroid toolchain is used correctly.
• @kevlo to test runtime.

fb-adb builds for native, android-armv7 (+pie) and android-x86 (+pie). The latter two are stubs for native client to communicate with remote target(s). Currently, native means freebsd-x86 and dragonfly-x86 as both linuxulator and ADA bootstrap aren't available on other architectures.

fb-adb is a tool for interacting with Android systems. It does much of
what adb does, but with better remote shell support and, hopefully,
fewer bugs. Differences between adb and fb-adb are that fb-adb:

• is binary clean (no LF -> CRLF mangling)
• transmits and updates window size
• distinguishes standard output and standard error
• properly muxes streams with independent flow control
• allows for ssh-like pty allocation control
• propagates program exit status instead of always exiting with status 0
• properly escapes program arguments
• kills remote program
• provides a generic facility to elevate to root without re-escaping

https://github.com/facebook/fb-adb

Test Plan

IMPORTANT: c6_64 (linux base) is completely untested as i686-linux-android-ar (elf64) crashes during stub-x86/configure

poudriere bulk -t logs: 93i386, 93amd64, 102amd64, 101i386
poudriere bulk -t logs with inverted options: 93i386, 93amd64, 102amd64, 101i386

$ portlint -AC
WARN: Makefile: [93]: the options to USES are not sorted.  Please consider sorting them.
WARN: Makefile: --build, --mandir, and --infodir are not needed in CONFIGURE_ARGS as they are already set in bsd.port.mk.
WARN: Makefile: CFLAGS/CXXFLAGS are not needed in CONFIGURE_ENV as they are already added there in bsd.port.mk.
FATAL: Makefile: CPPFLAGS is already passed in CONFIGURE_ENV via bsd.port.mk.  If you need to override the default value, alter CPPFLAGS in the Makefile instead with CPPFLAGS+=...
FATAL: Makefile: LDFLAGS is already passed in CONFIGURE_ENV via bsd.port.mk.  If you need to override the default value, alter LDFLAGS in the Makefile instead with LDFLAGS+=...
FATAL: Makefile: the last line of Makefile has to be .include <bsd.port(.post).mk>
WARN: Makefile: use of DISTFILES with single file discouraged. distribution filename should be set by DISTNAME and EXTRACT_SUFX.
WARN: Makefile: no port directory /usr/ports/shells/${BASH_CMD found, even though it is listed in BUILD_DEPENDS.
WARN: Makefile: wrong dependency value for BUILD_DEPENDS. BUILD_DEPENDS requires 2 or 3 colon-separated tuples.
WARN: Makefile: DISTFILES/DISTNAME affects WRKSRC. take caution when changing them.
3 fatal errors and 7 warnings found.

$ bash
[foo@bar ~]$ eval "$(fb-adb bash-completion)"
[foo@bar ~]$ fb-adb<TAB>
bash-completion  fput             ps-json          start-daemon
ctar             getprop          rcmd             stop-daemon
fcat             help             rdex             xcmd
fget             logcat-json      readlink
finfo            logw             shell

$ fb-adb getprop ro.build.version.release
{"ro.build.version.release":"5.1.1"}
$ fb-adb shell
foo@bar:/ $ ^D
$ fb-adb fget /data/data/com.android.providers.media/databases/external.db
fget: open("/data/data/com.android.providers.media/databases/external.db"): Permission denied
$ fb-adb fget --root /data/data/com.android.providers.media/databases/external.db
$ fb-adb fput /entropy /dev/random
$ dd if=/dev/random count=0x10 | fb-adb rcmd dd of=/dev/random
16+0 records in
16+0 records out
8192 bytes transferred in 0.000133 secs (61518128 bytes/sec)
16+0 records in
16+0 records out
8192 bytes transferred in 0.005 secs (1638400 bytes/sec)
$ fb-adb ctar /system/xbin | tar tf - | head
system/xbin/
system/xbin/[
system/xbin/[[
system/xbin/add-property-tag
system/xbin/adjtimex
system/xbin/arp
system/xbin/ash
system/xbin/awk
system/xbin/base64
system/xbin/basename

Beware of upstream bugs e.g., #25, #32.

[jbeich]

Changes since previous version:

• Added ASSERT and DEBUG options
• Group variable overrides for cross-build
• Pass plain -O and -g to stub flags
• Improve a few comments including upstream status
• Summary has updated poudriere logs

$ fgrep -c XXX Makefile
7

portlint (due to a bug) no longer shows the following warnings

WARN: Makefile: CFLAGS/CXXFLAGS are not needed in CONFIGURE_ENV as they are already added there in bsd.port.mk.
FATAL: Makefile: CPPFLAGS is already passed in CONFIGURE_ENV via bsd.port.mk.  If you need to override the default value, alter CPPFLAGS in the Makefile instead with CPPFLAGS+=...
FATAL: Makefile: LDFLAGS is already passed in CONFIGURE_ENV via bsd.port.mk.  If you need to override the default value, alter LDFLAGS in the Makefile instead with LDFLAGS+=...

[jbeich]

• DragonFly support via GNATDroid. The toolchain lacks non-armv7 (x86, mips, aarch64) and 20+ API level (lollipop, marshmallow) but it's not critical for fb-adb.
• Move .include <bsd.port.options.mk> near the end to underline what actually needs it.

[marino]

FYI it's very near the top of my to-do list to add the missing APIs (and bump default API to kitkat)

[marino]

Jan,
Are you sure it's necessary to download NDK?

When you build gnatdroid-armv7, it installs gnatdroid-sysroot which installs the headers of the API selected (they come from NDK).

With the recent commit, this means API 19 (Kitkat) headers are installed by default at /usr/local/android/android-19-arm. Couldn't this port use that?

I did some initial investigation and it should not be too hard to make a gnatdroid-x86 variant. Moreever, I can test that in virtualbox!

[marino]

also, gnatdroid-sysroot now has Lollipop (API 21) available. NDK doesn't have API 22 or 23 yet.

[jbeich]

• Drop NDK to check how much it simplifies Makefile

poudriere bulk -t logs: 93i386, 93amd64, 102amd64, 101i386

In D3930#81925, @marino wrote:

Jan,
Are you sure it's necessary to download NDK?

Only to build stub-x86 (one of defaults upstream). NDK build uses linux-x86 (host) toolchain, not just sysroot.

Would it bad idea to keep NDK support (with associated hacks) in initial commit then kill in the next? I'd like being able to rollback for debugging.

When you build gnatdroid-armv7, it installs gnatdroid-sysroot which installs the headers of the API selected (they come from NDK).

With the recent commit, this means API 19 (Kitkat) headers are installed by default at /usr/local/android/android-19-arm. Couldn't this port use that?

--sysroot headers are supposed to be used by default and they're not in /usr/local/android/android-19-arm:

$ env -i /usr/local/android/ARMv7/bin/arm-aux-linux-androideabi-gcc -v -xc -</dev/null
...
ignoring nonexistent directory "/usr/local/android/usr/local/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/local/android/ARMv7/lib/gcc/arm-aux-linux-androideabi/4.9.3/include
 /usr/local/android/ARMv7/lib/gcc/arm-aux-linux-androideabi/4.9.3/include-fixed
 /usr/local/android/ARMv7/lib/gcc/arm-aux-linux-androideabi/4.9.3/../../../../arm-aux-linux-androideabi/include
 /usr/local/android/usr/include
End of search list.
...

fb-adb works fine with API 19 (Kitkat) as used by default upstream and, if you've checked NDK=on logs, was successfully tested on FreeBSD. No issues with recent lang/gnatdroid* update.

I did some initial investigation and it should not be too hard to make a gnatdroid-x86 variant. Moreever, I can test that in virtualbox!

Nice, my hardware is too slow for virtualization. Restoring stub-x86 in fb-adb should be trivial.

In D3930#81926, @marino wrote:

also, gnatdroid-sysroot now has Lollipop (API 21) available.

fb-adb works fine with API 21 (Lollipop) as well after fixing a typo.

[jbeich]

Timeout. Landing without review and restoring NDK support.

More reasons to not like GNATDroid:

• Tied to GCC's build a copy for every target (waste of resources)
• API sysroot cannot be changed without rebuild (not package-friendly, aarch64/x86_64 are only in android-21)
• API sysroot only works for one Android architecture at a time (see its DISTNAME)

Ideally, ports wanting to cross-compile should try host compiler first but

• devel/binutils needs --enable-targets=all (maybe also kill trivial slaves)
• FreeBSD either fix its stddef.h et al. or not prohibit copies shipped with compiler
• libgcc.a implementation for every supported target triple (via compiler_rt?)
• Better multilib story to avoid sysroot for cross-arch on the same OS (base packaging?)

USE_LINUX cross-build issues are mostly irrelevant here:

• Lack of -devel packages to build stuff not in the repo like freshplayerplugin, anything CUDA
• Main-repo packages are often too old for upstream projects to support e.g., OpenCV
• No sysctl to change /compat/linux path fallthrough which makes it hard to use manually managed chroot together with ports that depend on USE_LINUX
• linux_dist-* idea died off but scaled better beyond single blessed distribution
• No linux_base-* built from source e.g., to patch stuff not supported by linuxulator

[marino]

FWIW, I have three ports waiting:
lang/gnatdroid-sysroot-x86
lang/gnatdroid-binutils-x86
lang/gnatdroid-x86

The last one needs more work because there is no signal trampoline for android-x86 in gcc yet, which is a gnat thing.

The criticisms list against gnatdroid aren't unique to gnatdroid.
Each API would need a separate package.
One could explode multiple sysroots but the one compiler per API requirement remains.