New ports for WebAssembly System Interface (WASI) libc, clang builtins (compiler-rt), libc++, libcxxabi

[val_packett.cool]

Summary

WASI is a "syscall" interface for WebAssembly, significantly inspired by CloudABI. It is supported in clang 8 and up, but to actually build C programs, it needs clang builtins and the libc/sysroot. And for C++, also libc++ and libcxxabi.

This is used like: clang11 -Os -target wasm32-wasi --sysroot /usr/local/share/wasi-sysroot wasi-demo.c -o wasi-demo.wasm

This is useful not only for development, but also Firefox can now use WASI with lucet / rlbox to sandbox the web font parser.

These ports work but maybe don't look that great yet, feedback welcome.

• our clang does not ship with its OS-independent stdarg.h and stddef.h :( For now I've added them (from llvm 9) as extra files and installed into the WASI sysroot.
  • There's probably a better way of doing this?
  • I didn't add them as download files because I couldn't figure out how to configure it to avoid trying to extract them as archives but still extract the github archives
  • Can they be included in the LLVM port? Why are we using system ones?
• the compiler-rt port shares distfiles and some variables with the LLVM port

[brooks]

The missing std*.h files are due to our inability to build FreeBSD when they are installed. We really need to land D17002 and MFC it so we can eventually start installing them.

I wonder if there's a way to include wasi-compiler-rt90 in the llvm90 port. I don't really understand the compiler-rt build, but I know there's some desire upstream to allow it to build for multiple targets so we might want to see if we can do that.

[swills]

devel/wasi-libc/Makefile

28 ↗

(On Diff #66113)

This appears to have a space instead of a tab after the =.

[swills]

portlint -C reports a number of errors with wasi-compiler-rt90/Makefile:

FATAL: Makefile: DISTVERSION appears out-of-order.
FATAL: Makefile: order must be PORTNAME/PORTVERSION/DISTVERSIONPREFIX/DISTVERSION/DISTVERSIONSUFFIX/PORTREVISION/PORTEPOCH/CATEGORIES/MASTER_SITES/MASTER_SITE_SUBDIR/PROJECTHOST/PKGNAMEPREFIX/PKGNAMESUFFIX/DISTNAME/EXTRACT_SUFX/DISTFILES/DIST_SUBDIR/EXTRACT_ONLY.
FATAL: Makefile: URL "https://prereleases.llvm.org/${LLVM_RELEASE}/${RCDIR}" should end with "/" or a group name (e.g. :something).
WARN: Makefile: using hyphen in PORTNAME. consider using PKGNAMEPREFIX and/or PKGNAMESUFFIX.
WARN: Makefile: use of DISTFILES with single file discouraged. distribution filename should be set by DISTNAME and EXTRACT_SUFX.
FATAL: Makefile: LICENSE_NAME appears out-of-order.
FATAL: Makefile: order must be LICENSE/LICENSE_COMB/LICENSE_GROUPS(_\w+)?/LICENSE_NAME(_\w+)?/LICENSE_TEXT(_\w+)?/LICENSE_FILE(_\w+)?/LICENSE_PERMS(_\w+)?/LICENSE_DISTFILES(_\w+)?.
WARN: Makefile: "BUILD_DEPENDS" has to appear earlier.
WARN: Makefile: "USES" has to appear earlier.
WARN: Makefile: do not use DISTFILES and DISTNAME to control WRKSRC. how about "WRKSRC=${WRKDIR}/${PORTNAME}-${DISTVERSION}.src"?
FATAL: breaks INDEX (sh: /nonexistentlocal/bin/clang90: not found make: "/usr/ports/Mk/Uses/compiler.mk" line 78: warning: "/nonexistentlocal/bin/clang90 --version" returned non-zero status make: "/usr/ports/Mk/Uses/compiler.mk" line 133: warning: "/nonexistentlocal/bin/clang++90 -### /dev/null 2>&1" returned non-zero status).
6 fatal errors and 5 warnings found.

[mikael]

devel/wasi-compiler-rt90/Makefile

6 ↗

(On Diff #66113)

Can you update to 9.0.1?

32 ↗

(On Diff #66113)

LLVM_VERSION is not defined, is it LLVM_SUFFIX or LLVM_DEFAULT?
clang90 is a build dep only or is it required to compile wasm stuff?

34 ↗

(On Diff #66113)

No need to put compiler:c++11-lib, you specify CXX/CC=clang90 below

36 ↗

(On Diff #66113)

There is a plist error: Error: Orphaned: @dir llvm90/lib/freebsd

maybe use this in post-install or create a pkg-plist file.

${ECHO} "llvm${LLVM_SUFFIX}/lib/clang/${LLVM_RELEASE}/lib/wasi/libclang_rt.builtins-wasm32.a" >> ${TMPPLIST}
${ECHO} "@dir llvm90/lib/freebsd" >> ${TMPPLIST}

devel/wasi-libc/Makefile

10 ↗

(On Diff #66113)

from https://github.com/CraneStation/wasi-libc/blob/master/LICENSE:
wasi-libc as a whole is multi-licensed under the Apache License v2.0 with LLVM Exceptions, the Apache License v2.0, and the MIT License
and other third-party licences
-> APACHE20 is missing, not sure we have something for "Apache License v2.0 with LLVM Exceptions"

[swills]

devel/wasi-compiler-rt90/Makefile

36 ↗

(On Diff #66113)

Better to use PLIST_FILES:

PLIST_FILES=    llvm${LLVM_SUFFIX}/lib/clang/${LLVM_RELEASE}/lib/wasi/libclang_rt.builtins-wasm32.a \
                "@dir llvm90/lib/freebsd"

[val_packett.cool]

devel/wasi-compiler-rt90/Makefile

36 ↗

(On Diff #66113)

better to rmdir :)

devel/wasi-libc/Makefile

10 ↗

(On Diff #66113)

That wasn't there until recently :) Now updated to newer commit, license *is* like this.

[mikael]

devel/wasi-compiler-rt90/pkg-descr

4 ↗

(On Diff #69326)

builtings? What does that mean?

[val_packett.cool]

devel/wasi-compiler-rt90/pkg-descr

4 ↗

(On Diff #69326)

builtins of course :D

[mikael]

I wonder if it's a good idea to have 'PKGNAMESUFFIX= ${LLVM_SUFFIX}', do you plan to support multiple version (wasi-compiler-rt11 or wasi-compiler-rt12)?

[val_packett.cool]

In D22960#552671, @mikael wrote:

I wonder if it's a good idea to have 'PKGNAMESUFFIX= ${LLVM_SUFFIX}', do you plan to support multiple version (wasi-compiler-rt11 or wasi-compiler-rt12)?

For compiler-rt, definitely. It's installed into llvm${LLVM_SUFFIX}/lib/clang/${LLVM_RELEASE}/lib/wasi/libclang_rt.builtins-wasm32.a and used by *that* clang.

For cxx(abi) I'm less sure. Different versions would actually be conflicting.

[ali_mashtizadeh.com]

The WASI folks seem fine upstreaming patches. Might be nice to make clean pull requests for these projects after we resolve Brook's comment.

Been a bit busy but I need to update my pull request with wasmtime. Probably can do that tonight or tomorrow so it builds on BSD.

https://github.com/bytecodealliance/wasmtime/pull/1675

[val_packett.cool]

In D22960#555148, @ali_mashtizadeh.com wrote:

The WASI folks seem fine upstreaming patches. Might be nice to make clean pull requests for these projects after we resolve Brook's comment.

The only patch here is disabling a diff check in wasi-libc that complains about __arraycount not being present in one of the lists, everything else is just built as-is.

https://github.com/bytecodealliance/wasmtime/pull/1675

Dropped a comment there :)

[val_packett.cool]

• Update to LLVM 11 and latest wasi-libc
• libc++ now only supports building from the llvm monorepo
• I've integrated the libc++abi build into the libc++ port to avoid extracting the monorepo a whole extra time for the tiny libc++abi build
• Removed version suffix from the libc++ port since it installs into an unversioned ${PREFIX}/share/wasi-sysroot

[val_packett.cool]

oops, also ranlib cxxabi

---

BTW, pkg issue about silly warnings on packaging wasi-libcxx https://github.com/freebsd/pkg/issues/1892