Page MenuHomeFreeBSD

Update jemalloc to version 5.3.0
Needs ReviewPublic

Authored by minsoochoo0122_proton.me on Aug 11 2023, 5:21 AM.
Referenced Files
Unknown Object (File)
Tue, Dec 17, 8:01 PM
Unknown Object (File)
Wed, Dec 11, 12:29 AM
Unknown Object (File)
Fri, Nov 29, 10:59 AM
Unknown Object (File)
Thu, Nov 28, 1:27 AM
Unknown Object (File)
Nov 25 2024, 9:56 PM
Unknown Object (File)
Nov 25 2024, 4:13 PM
Unknown Object (File)
Nov 25 2024, 9:23 AM
Unknown Object (File)
Nov 15 2024, 1:51 PM

Details

Reviewers
vangyzen
Group Reviewers
Contributor Reviews (src)
Summary

jemalloc 5.3.0 was released in 2022 but has not been imported into FreeBSD base system yet. The new version of jemalloc includes new features and bug fixes which are listed on release note.

Test Plan
make tinderbox

regression test

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Passed
Unit
No Test Coverage
Build Status
Buildable 53127
Build 50018: arc lint + arc unit

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Thanks for the work! One thing I would like to mention is that it seems having some issue building or running on 32-bit older CURRENT: https://github.com/jemalloc/jemalloc/pull/2228

I did some test on 32-bit FreeBSD and I found out that when clock_gettime is used on 64bit machine but compiled in 32bit, it sets tv_sec (time_t) as -1. On 32bit FreeBSD, clock_gettime works as expected.

Thanks for the work! One thing I would like to mention is that it seems having some issue building or running on 32-bit older CURRENT: https://github.com/jemalloc/jemalloc/pull/2228

I did some test on 32-bit FreeBSD and I found out that when clock_gettime is used on 64bit machine but compiled in 32bit, it sets tv_sec (time_t) as -1. On 32bit FreeBSD, clock_gettime works as expected.

that seems like it's a bug. On powerpc and amrv7 tv_sec should be the same. On i386, tv_sec is only32-bit, but we still have a dozen years left until it's inadequate. Can you finle a bug, if you haven't already, with a way to reproduce this?

In D41421#943650, @imp wrote:

Thanks for the work! One thing I would like to mention is that it seems having some issue building or running on 32-bit older CURRENT: https://github.com/jemalloc/jemalloc/pull/2228

I did some test on 32-bit FreeBSD and I found out that when clock_gettime is used on 64bit machine but compiled in 32bit, it sets tv_sec (time_t) as -1. On 32bit FreeBSD, clock_gettime works as expected.

that seems like it's a bug. On powerpc and amrv7 tv_sec should be the same. On i386, tv_sec is only32-bit, but we still have a dozen years left until it's inadequate. Can you finle a bug, if you haven't already, with a way to reproduce this?

Here is a bug report which includes an explanation and a way to reproduce the problem.

I still get error

cd /usr/src; _PARALLEL_SUBDIR_OK=1 time env MACHINE_ARCH=amd64  MACHINE=amd64  CPUTYPE= BUILD_TOOLS_META=.NOMETA CC="cc -target x86_64-unknown-freebsd14.0 --sysroot=/usr/obj/usr/src/amd64.amd64/tmp -B/usr/obj/usr/src/amd64.amd64/tmp/usr/bin" CXX="c++  -target x86_64-unknown-freebsd14.0 --sysroot=/usr/obj/usr/src/amd64.amd64/tmp -B/usr/obj/usr/src/amd64.amd64/tmp/usr/bin"  CPP="cpp -target x86_64-unknown-freebsd14.0 --sysroot=/usr/obj/usr/src/amd64.amd64/tmp -B/usr/obj/usr/src/amd64.amd64/tmp/usr/bin"  AS="as" AR="ar" ELFCTL="elfctl" LD="ld"  LLVM_LINK="" NM=nm OBJCOPY="objcopy"  RANLIB=ranlib STRINGS=  SIZE="size" STRIPBIN="strip"  INSTALL="install -U"  PATH=/usr/obj/usr/src/amd64.amd64/tmp/bin:/usr/obj/usr/src/amd64.amd64/tmp/usr/sbin:/usr/obj/usr/src/amd64.amd64/tmp/usr/bin:/usr/obj/usr/src/amd64.amd64/tmp/legacy/usr/sbin:/usr/obj/usr/src/amd64.amd64/tmp/legacy/usr/bin:/usr/obj/usr/src/amd64.amd64/tmp/legacy/bin:/usr/obj/usr/src/amd64.amd64/tmp/legacy/usr/libexec::/sbin:/bin:/usr/sbin:/usr/bin  SYSROOT=/usr/obj/usr/src/amd64.amd64/tmp make  -f Makefile.inc1  BWPHASE=everything  DESTDIR=/usr/obj/usr/src/amd64.amd64/tmp all
(cd /usr/src/lib/csu/tests/dynamic &&  DEPENDFILE=.depend.init_test  NO_SUBDIR=1 make -f /usr/src/lib/csu/tests/dynamic/Makefile _RECURSING_PROGS=t  PROG=init_test )
Building /usr/obj/usr/src/amd64.amd64/lib/csu/tests/dynamic/init_test.o
Building /usr/obj/usr/src/amd64.amd64/lib/csu/tests/dynamic/init_test.full
ld: error: undefined reference due to --no-allow-shlib-undefined: _pthread_getname_np
>>> referenced by /usr/obj/usr/src/amd64.amd64/tmp/lib/libc.so.7
cc: error: linker command failed with exit code 1 (use -v to see invocation)
*** Error code 1

Stop.

Why doesn't the build system accept _pthread_getname_np but accepts other _pthread_*s like _pthread_join? I already tried adding _pthread_getname_np to FBSDprivate_1.0 in lib/libthr/pthread.map and lib/libc/gen/Symbol.map but it doesn't work either.

It seems extremely late in the release to be doing a malloc update.

lib/libc/stdlib/jemalloc/Makefile.inc
17

This changes the C standard used for all of libc. It can't be part of this commit.

Fixed build error with missing _pthread_getname_np symbol reference

lib/libc/stdlib/jemalloc/Makefile.inc
17

jemalloc 5.3.0 uses gnu11 for build. Without CSTD= gnu11 flag, the build generates these errors:

In file included from jemalloc_cache_bin.c:2:
In file included from /usr/src/contrib/jemalloc/include/jemalloc/internal/jemalloc_internal_includes.h:52:
In file included from /usr/src/contrib/jemalloc/include/jemalloc/internal/arena_structs.h:4:
In file included from /usr/src/contrib/jemalloc/include/jemalloc/internal/arena_stats.h:8:
In file included from /usr/src/contrib/jemalloc/include/jemalloc/internal/pa.h:11:
In file included from /usr/src/contrib/jemalloc/include/jemalloc/internal/pac.h:6:
/usr/src/contrib/jemalloc/include/jemalloc/internal/san_bump.h:12:25: error: redefinition of typedef 'ehooks_t' is a C11 feature [-Werror,-Wtypedef-redefinition]
typedef struct ehooks_s ehooks_t;
                        ^
/usr/src/contrib/jemalloc/include/jemalloc/internal/ehooks.h:21:25: note: previous definition is here
typedef struct ehooks_s ehooks_t;
                        ^
In file included from jemalloc_cache_bin.c:2:
In file included from /usr/src/contrib/jemalloc/include/jemalloc/internal/jemalloc_internal_includes.h:52:
In file included from /usr/src/contrib/jemalloc/include/jemalloc/internal/arena_structs.h:4:
In file included from /usr/src/contrib/jemalloc/include/jemalloc/internal/arena_stats.h:8:
In file included from /usr/src/contrib/jemalloc/include/jemalloc/internal/pa.h:11:
/usr/src/contrib/jemalloc/include/jemalloc/internal/pac.h:77:22: error: redefinition of typedef 'pac_t' is a C11 feature [-Werror,-Wtypedef-redefinition]
typedef struct pac_s pac_t;
                     ^
/usr/src/contrib/jemalloc/include/jemalloc/internal/san_bump.h:13:22: note: previous definition is here
typedef struct pac_s pac_t;

Therefore, I think that bumping CSTD for libc to c11 is a necessary step to adopt jemalloc 5.3.0. I tried cleanworld, buildworld, installworld and rebooted, and luckily, the system boots and works fine without any crash or error messages. In this case, should I open a new revision regarding to changing CSTD to gnu11 for libc?

lib/libc/stdlib/jemalloc/Makefile.inc
17

If we're going to change the C standard for libc we can't just rely on a reboot smoke test. We need to examine any new warnings generated and solicit input about that change on its own. It's probably a good thing to do, but we can't sneak it in as a part of another change.

FWIW should be trivial to add ifdef/define/endif guards around the above typedefs to address the errors above as a workaround.

jrtc27 added inline comments.
lib/libc/stdlib/jemalloc/Makefile.inc
17
.for src in ${JEMALLOCSRCS}
CFLAGS.${src}+=-std=gnu11
.endfor

should work, FWIW, if needed

Created separate revision for pthread_getname_np and _pthread_getname_np. This revision depends on D41461

Removed gnu11 flag and fixed gnu99 incompatibility

Updating diff file to conform recent changes in lib/libc/stdlib/malloc

Parent revision D41461 is closed. Now we can test this on local machines on -CURRENT.

Parent revision D41461 is closed. Now we can test this on local machines on -CURRENT.

I've been testing this on main and stable/13 (with some tweaks). If there are any specific tests I can do that would be helpful let me know

Parent revision D41461 is closed. Now we can test this on local machines on -CURRENT.

I've been testing this on main and stable/13 (with some tweaks). If there are any specific tests I can do that would be helpful let me know

In the release notes, there are some new features added in jemalloc 5.3.0. For example,

  • Make the behavior of realloc(ptr, 0) configurable with opt.zero_realloc
  • Add the thread.idle mallctl which hints that the calling thread will be idle for a nontrivial period of time.
  • Add mallctl interfaces:
    • opt.zero_realloc (@davidtgoldblatt)
    • opt.cache_oblivious (@interwq)
    • opt.prof_leak_error (@yunxuo)
    • opt.stats_interval (@interwq)
    • opt.stats_interval_opts (@interwq)
    • opt.tcache_max (@interwq)
    • opt.trust_madvise (@azat)
    • prof.prefix (@zhxchen17)
    • stats.zero_reallocs (@davidtgoldblatt)
    • thread.idle (@davidtgoldblatt)
    • thread.peak.{read,reset} (@davidtgoldblatt)

and so on.

By the way, because this is not a major version bump (5.x.x), will this version bump be abled to be MFCed to stable/14?

Parent revision D41461 is closed. Now we can test this on local machines on -CURRENT.

I've been testing this on main and stable/13 (with some tweaks). If there are any specific tests I can do that would be helpful let me know

In the release notes, there are some new features added in jemalloc 5.3.0. For example,

  • Make the behavior of realloc(ptr, 0) configurable with opt.zero_realloc
  • Add the thread.idle mallctl which hints that the calling thread will be idle for a nontrivial period of time.
  • Add mallctl interfaces:
    • opt.zero_realloc (@davidtgoldblatt)
    • opt.cache_oblivious (@interwq)
    • opt.prof_leak_error (@yunxuo)
    • opt.stats_interval (@interwq)
    • opt.stats_interval_opts (@interwq)
    • opt.tcache_max (@interwq)
    • opt.trust_madvise (@azat)
    • prof.prefix (@zhxchen17)
    • stats.zero_reallocs (@davidtgoldblatt)
    • thread.idle (@davidtgoldblatt)
    • thread.peak.{read,reset} (@davidtgoldblatt)

and so on.

It would be really nice if we can test this against with the tests under /usr/tests . I know it is still very tricky to run it. If you are interested, please check the scripts at https://github.com/freebsd/freebsd-ci and let me know (and/or -testing mailing list) if you have any questions.

By the way, because this is not a major version bump (5.x.x), will this version bump be abled to be MFCed to stable/14?

Theoretically, yes, but it would be nice to check the compatibility thoroughly.

I think it might make sense to split the switch to the jemalloc memalign() into a separate commit.

lib/libc/gen/memalign.c
38

Does jemalloc's implementation handle this compatibly?..

lib/libc/gen/memalign.c
38

While glibc implementation passes to malloc when alignment is 0, jemalloc implementation and illumos implementaion sets errno EINVAL and return NULL.

However, from glibc man page:

The obsolete function memalign() allocates size bytes and returns a pointer to the allocated memory. The memory address will be a multiple of alignment, which must be a power of two.

glibc's implementation for align=0 was added through PR 269688, but it is not documented in glibc. Whereas, the glibc man page says it "must be a power of two". This function is also obsolete according to glibc man page and undocumented in FreeBSD man page. Therefore, I think that there isn't enough reason for maintaining compatability despite the fact that it is obsolete and undocumented, and it is okay to adopt jemalloc/illumos way to set errno to EINVAL and return NULL

Parent revision D41461 is closed. Now we can test this on local machines on -CURRENT.

I've been testing this on main and stable/13 (with some tweaks). If there are any specific tests I can do that would be helpful let me know

In the release notes, there are some new features added in jemalloc 5.3.0. For example,

  • Make the behavior of realloc(ptr, 0) configurable with opt.zero_realloc
  • Add the thread.idle mallctl which hints that the calling thread will be idle for a nontrivial period of time.
  • Add mallctl interfaces:
    • opt.zero_realloc (@davidtgoldblatt)
    • opt.cache_oblivious (@interwq)
    • opt.prof_leak_error (@yunxuo)
    • opt.stats_interval (@interwq)
    • opt.stats_interval_opts (@interwq)
    • opt.tcache_max (@interwq)
    • opt.trust_madvise (@azat)
    • prof.prefix (@zhxchen17)
    • stats.zero_reallocs (@davidtgoldblatt)
    • thread.idle (@davidtgoldblatt)
    • thread.peak.{read,reset} (@davidtgoldblatt)

and so on.

It would be really nice if we can test this against with the tests under /usr/tests . I know it is still very tricky to run it. If you are interested, please check the scripts at https://github.com/freebsd/freebsd-ci and let me know (and/or -testing mailing list) if you have any questions.

By the way, because this is not a major version bump (5.x.x), will this version bump be abled to be MFCed to stable/14?

Theoretically, yes, but it would be nice to check the compatibility thoroughly.

# cd /usr/src
# kyua test -k /usr/tests/Kyuafile lib/libc

passes with 0 failures.

lwhsu added a subscriber: vangyzen.

@vangyzen: are you interested in reviewing this? as you did the update jemalloc to 5.2.1.

Thank you for asking, but I didn't actually do the last update. @jasone did it, but had to revert it. I simply un-reverted his work.

How does jemalloc 5.3.0 handle when alignment is 0

jemalloc follows the illumos way of handling alignment when it is zero, whereas the current FreeBSD and glibc returns memory space using malloc(). If we are okay with this compatability issue, this can be landed to 15-CURRENT branch. I think 12-STABLE, 13-STABLE, 14-STABLE should stick with jemalloc 5.2.1 to keep the compatability with FreeBSD's memalign.

Any ideas?

How does jemalloc 5.3.0 handle when alignment is 0

jemalloc follows the illumos way of handling alignment when it is zero, whereas the current FreeBSD and glibc returns memory space using malloc(). If we are okay with this compatability issue, this can be landed to 15-CURRENT branch. I think 12-STABLE, 13-STABLE, 14-STABLE should stick with jemalloc 5.2.1 to keep the compatability with FreeBSD's memalign.

Any ideas?

IMO the only reason to support memalign is to be compatible. We should do what glibc does regardless of its official documentation.

How does jemalloc 5.3.0 handle when alignment is 0

jemalloc follows the illumos way of handling alignment when it is zero, whereas the current FreeBSD and glibc returns memory space using malloc(). If we are okay with this compatability issue, this can be landed to 15-CURRENT branch. I think 12-STABLE, 13-STABLE, 14-STABLE should stick with jemalloc 5.2.1 to keep the compatability with FreeBSD's memalign.

Any ideas?

IMO the only reason to support memalign is to be compatible. We should do what glibc does regardless of its official documentation.

I totally agree with that point, but I'm not sure how we can implement memalign exception and maintain it in future releases of jemalloc. This might cause confusion to some users who are already using jemalloc in other projects or systems because they know FreeBSD uses jemalloc, but we're making an exception here.

If we are going make this exception, what would be the best way to implement it? Should I patch je_memalign() thorugh FREEBSD-diffs?

How does jemalloc 5.3.0 handle when alignment is 0

jemalloc follows the illumos way of handling alignment when it is zero, whereas the current FreeBSD and glibc returns memory space using malloc(). If we are okay with this compatability issue, this can be landed to 15-CURRENT branch. I think 12-STABLE, 13-STABLE, 14-STABLE should stick with jemalloc 5.2.1 to keep the compatability with FreeBSD's memalign.

Any ideas?

IMO the only reason to support memalign is to be compatible. We should do what glibc does regardless of its official documentation.

I totally agree with that point, but I'm not sure how we can implement memalign exception and maintain it in future releases of jemalloc. This might cause confusion to some users who are already using jemalloc in other projects or systems because they know FreeBSD uses jemalloc, but we're making an exception here.

If we are going make this exception, what would be the best way to implement it? Should I patch je_memalign() thorugh FREEBSD-diffs?

Yes, I'd restore patching it out as was done in previous releases.

How does jemalloc 5.3.0 handle when alignment is 0

jemalloc follows the illumos way of handling alignment when it is zero, whereas the current FreeBSD and glibc returns memory space using malloc(). If we are okay with this compatability issue, this can be landed to 15-CURRENT branch. I think 12-STABLE, 13-STABLE, 14-STABLE should stick with jemalloc 5.2.1 to keep the compatability with FreeBSD's memalign.

Any ideas?

IMO the only reason to support memalign is to be compatible. We should do what glibc does regardless of its official documentation.

I totally agree with that point, but I'm not sure how we can implement memalign exception and maintain it in future releases of jemalloc. This might cause confusion to some users who are already using jemalloc in other projects or systems because they know FreeBSD uses jemalloc, but we're making an exception here.

If we are going make this exception, what would be the best way to implement it? Should I patch je_memalign() thorugh FREEBSD-diffs?

Yes, I'd restore patching it out as was done in previous releases.

Would you like me to keep memalign() by using the original version in memalign.c or by adding if branch in jemalloc.c 's je_memalign?

I'll comment on diff to show what I mean

contrib/jemalloc/include/jemalloc/jemalloc.h
32

Option 1: If this is set to #undef, we can keep using our own memalign() by restoring deleted memalign.c file and related makefiles and symbol maps.

contrib/jemalloc/src/jemalloc.c
3185

Option 2:

if (alignment == 0)
    return je_malloc(size);
contrib/jemalloc/include/jemalloc/jemalloc.h
32

Option 1: If this is set to #undef, we can keep using our own memalign() by restoring deleted memalign.c file and related makefiles and symbol maps.

I prefer this option. The trivial implementation is fine and reduces the burden on other malloc implementations.

@lwhsu The Github pull request about CI testing is merged now.

Thanks, I just saw it.

I think this patch generally looks good. I think we still need to do more building (for all supported arches) and testing (at least for tier 1 arches) to import.
I will try to squeeze time to work on it. If you are interested in running tests in FreeBSD's test suite with this patch, please let me know.

Build Success

minsoochoo in jemalloc at freebsd on  jemalloc took 34.8s
➜ make -j24 tinderbox
[Creating objdir /usr/obj/usr/src/freebsd/jemalloc/amd64.amd64...]
--- tinderbox ---
--- universe_prologue ---
--------------------------------------------------------------
>>> make universe started on Tue Apr  9 18:20:49 EDT 2024
--------------------------------------------------------------
--- universe_amd64_prologue ---
--- universe_arm_prologue ---
--- universe_arm64_prologue ---
--- universe_i386_prologue ---
--- universe_powerpc_prologue ---
--- universe_riscv_prologue ---
--- universe_amd64_prologue ---
>> amd64 started on Tue Apr  9 18:20:49 EDT 2024
--- universe_arm64_prologue ---
>> arm64 started on Tue Apr  9 18:20:49 EDT 2024
--- universe_amd64_amd64 ---
--- universe_arm_prologue ---
>> arm started on Tue Apr  9 18:20:49 EDT 2024
--- universe_arm64_aarch64 ---
--- universe_arm_armv7 ---
--- universe_powerpc_prologue ---
>> powerpc started on Tue Apr  9 18:20:49 EDT 2024
--- universe_riscv_prologue ---
>> riscv started on Tue Apr  9 18:20:49 EDT 2024
--- universe_i386_prologue ---
>> i386 started on Tue Apr  9 18:20:49 EDT 2024
--- universe_powerpc_powerpc ---
--- universe_powerpc_powerpc64 ---
--- universe_powerpc_powerpc64le ---
--- universe_riscv_riscv64 ---
--- universe_i386_i386 ---
>> arm64.aarch64 buildworld started on Tue Apr  9 18:20:49 EDT 2024
>> arm.armv7 buildworld started on Tue Apr  9 18:20:49 EDT 2024
>> amd64.amd64 buildworld started on Tue Apr  9 18:20:49 EDT 2024
>> powerpc.powerpc64 buildworld started on Tue Apr  9 18:20:49 EDT 2024
>> powerpc.powerpc buildworld started on Tue Apr  9 18:20:49 EDT 2024
>> powerpc.powerpc64le buildworld started on Tue Apr  9 18:20:49 EDT 2024
>> i386.i386 buildworld started on Tue Apr  9 18:20:49 EDT 2024
>> riscv.riscv64 buildworld started on Tue Apr  9 18:20:49 EDT 2024
--- universe_powerpc_powerpc ---
>> powerpc.powerpc buildworld completed on Tue Apr  9 18:34:13 EDT 2024
--- universe_arm_armv7 ---
>> arm.armv7 buildworld completed on Tue Apr  9 18:35:32 EDT 2024
--- universe_arm_kernels ---
--- universe_kernels_prologue ---
>> arm kernels started on Tue Apr  9 18:35:38 EDT 2024
--- universe_kernconf_arm_ARMADA38X ---
>> arm.armv7 ARMADA38X kernel started on Tue Apr  9 18:35:38 EDT 2024
--- universe_kernconf_arm_GENERIC ---
>> arm.armv7 GENERIC kernel started on Tue Apr  9 18:35:42 EDT 2024
--- universe_kernconf_arm_GENERIC-NODEBUG ---
>> arm.armv7 GENERIC-NODEBUG kernel started on Tue Apr  9 18:35:43 EDT 2024
--- universe_kernconf_arm_LINT ---
>> arm.armv7 LINT kernel started on Tue Apr  9 18:35:44 EDT 2024
--- universe_kernconf_arm_TEGRA124 ---
>> arm.armv7 TEGRA124 kernel started on Tue Apr  9 18:35:45 EDT 2024
--- universe_kernconf_arm_ZEDBOARD ---
>> arm.armv7 ZEDBOARD kernel started on Tue Apr  9 18:36:02 EDT 2024
--- universe_riscv_riscv64 ---
>> riscv.riscv64 buildworld completed on Tue Apr  9 18:37:54 EDT 2024
--- universe_riscv_kernels ---
--- universe_kernels_prologue ---
>> riscv kernels started on Tue Apr  9 18:38:06 EDT 2024
--- universe_kernconf_riscv_GENERIC ---
>> riscv.riscv64 GENERIC kernel started on Tue Apr  9 18:38:06 EDT 2024
--- universe_kernconf_riscv_GENERIC-NODEBUG ---
>> riscv.riscv64 GENERIC-NODEBUG kernel started on Tue Apr  9 18:38:08 EDT 2024
--- universe_kernconf_riscv_LINT ---
>> riscv.riscv64 LINT kernel started on Tue Apr  9 18:38:13 EDT 2024
--- universe_powerpc_powerpc64le ---
>> powerpc.powerpc64le buildworld completed on Tue Apr  9 18:38:35 EDT 2024
--- universe_i386_i386 ---
>> i386.i386 buildworld completed on Tue Apr  9 18:39:14 EDT 2024
--- universe_i386_kernels ---
--- universe_kernels_prologue ---
>> i386 kernels started on Tue Apr  9 18:39:36 EDT 2024
--- universe_kernconf_i386_GENERIC ---
>> i386.i386 GENERIC kernel started on Tue Apr  9 18:39:36 EDT 2024
--- universe_kernconf_i386_GENERIC-NODEBUG ---
>> i386.i386 GENERIC-NODEBUG kernel started on Tue Apr  9 18:39:36 EDT 2024
--- universe_kernconf_i386_LINT ---
>> i386.i386 LINT kernel started on Tue Apr  9 18:39:39 EDT 2024
--- universe_kernconf_i386_MINIMAL ---
>> i386.i386 MINIMAL kernel started on Tue Apr  9 18:39:40 EDT 2024
--- universe_kernconf_i386_PAE ---
>> i386.i386 PAE kernel started on Tue Apr  9 18:39:42 EDT 2024
--- universe_arm_kernels ---
>> arm.armv7 ZEDBOARD kernel completed on Tue Apr  9 18:45:38 EDT 2024
--- universe_kernconf_arm_TEGRA124 ---
>> arm.armv7 TEGRA124 kernel completed on Tue Apr  9 18:45:46 EDT 2024
--- universe_kernconf_arm_ARMADA38X ---
>> arm.armv7 ARMADA38X kernel completed on Tue Apr  9 18:46:23 EDT 2024
--- universe_kernconf_arm_GENERIC ---
>> arm.armv7 GENERIC kernel completed on Tue Apr  9 18:46:37 EDT 2024
--- universe_kernconf_arm_GENERIC-NODEBUG ---
>> arm.armv7 GENERIC-NODEBUG kernel completed on Tue Apr  9 18:46:53 EDT 2024
--- universe_kernconf_arm_LINT ---
>> arm.armv7 LINT kernel completed on Tue Apr  9 18:48:44 EDT 2024
--- universe_kernels ---
>> arm kernels completed on Tue Apr  9 18:48:44 EDT 2024
--- universe_arm_done ---
>> arm completed on Tue Apr  9 18:48:49 EDT 2024
--- universe_riscv_kernels ---
--- universe_kernconf_riscv_GENERIC ---
>> riscv.riscv64 GENERIC kernel completed on Tue Apr  9 18:49:18 EDT 2024
--- universe_kernconf_riscv_GENERIC-NODEBUG ---
>> riscv.riscv64 GENERIC-NODEBUG kernel completed on Tue Apr  9 18:49:27 EDT 2024
--- universe_powerpc_powerpc64 ---
>> powerpc.powerpc64 buildworld completed on Tue Apr  9 18:50:07 EDT 2024
--- universe_powerpc_kernels ---
--- universe_kernels_prologue ---
>> powerpc kernels started on Tue Apr  9 18:50:11 EDT 2024
--- universe_kernconf_powerpc_GENERIC ---
>> powerpc.powerpc GENERIC kernel started on Tue Apr  9 18:50:11 EDT 2024
--- universe_kernconf_powerpc_GENERIC-NODEBUG ---
>> powerpc.powerpc GENERIC-NODEBUG kernel started on Tue Apr  9 18:50:12 EDT 2024
--- universe_kernconf_powerpc_LINT ---
>> powerpc.powerpc LINT kernel started on Tue Apr  9 18:50:14 EDT 2024
--- universe_kernconf_powerpc_MPC85XX ---
>> powerpc.powerpc MPC85XX kernel started on Tue Apr  9 18:50:17 EDT 2024
--- universe_kernconf_powerpc_GENERIC64 ---
>> powerpc.powerpc64 GENERIC64 kernel started on Tue Apr  9 18:50:21 EDT 2024
--- universe_kernconf_powerpc_GENERIC64-NODEBUG ---
>> powerpc.powerpc64 GENERIC64-NODEBUG kernel started on Tue Apr  9 18:50:31 EDT 2024
--- universe_kernconf_powerpc_LINT64 ---
>> powerpc.powerpc64 LINT64 kernel started on Tue Apr  9 18:50:47 EDT 2024
--- universe_kernconf_powerpc_QORIQ64 ---
>> powerpc.powerpc64 QORIQ64 kernel started on Tue Apr  9 18:50:59 EDT 2024
--- universe_kernconf_powerpc_GENERIC64LE ---
>> powerpc.powerpc64le GENERIC64LE kernel started on Tue Apr  9 18:51:02 EDT 2024
--- universe_kernconf_powerpc_GENERIC64LE-NODEBUG ---
>> powerpc.powerpc64le GENERIC64LE-NODEBUG kernel started on Tue Apr  9 18:51:25 EDT 2024
--- universe_i386_kernels ---
--- universe_kernconf_i386_MINIMAL ---
>> i386.i386 MINIMAL kernel completed on Tue Apr  9 18:51:52 EDT 2024
--- universe_powerpc_kernels ---
--- universe_kernconf_powerpc_LINT64LE ---
>> powerpc.powerpc64le LINT64LE kernel started on Tue Apr  9 18:54:01 EDT 2024
--- universe_riscv_kernels ---
--- universe_kernconf_riscv_LINT ---
>> riscv.riscv64 LINT kernel completed on Tue Apr  9 18:54:19 EDT 2024
--- universe_kernels ---
>> riscv kernels completed on Tue Apr  9 18:54:19 EDT 2024
--- universe_riscv_done ---
>> riscv completed on Tue Apr  9 18:54:28 EDT 2024
--- universe_i386_kernels ---
--- universe_kernconf_i386_PAE ---
>> i386.i386 PAE kernel completed on Tue Apr  9 18:55:15 EDT 2024
--- universe_kernconf_i386_GENERIC ---
>> i386.i386 GENERIC kernel completed on Tue Apr  9 18:55:15 EDT 2024
--- universe_kernconf_i386_GENERIC-NODEBUG ---
>> i386.i386 GENERIC-NODEBUG kernel completed on Tue Apr  9 18:55:33 EDT 2024
--- universe_arm64_aarch64 ---
>> arm64.aarch64 buildworld completed on Tue Apr  9 18:56:03 EDT 2024
--- universe_arm64_kernels ---
--- universe_kernels_prologue ---
>> arm64 kernels started on Tue Apr  9 18:56:08 EDT 2024
--- universe_kernconf_arm64_GENERIC ---
>> arm64.aarch64 GENERIC kernel started on Tue Apr  9 18:56:08 EDT 2024
--- universe_kernconf_arm64_GENERIC-KASAN ---
>> arm64.aarch64 GENERIC-KASAN kernel started on Tue Apr  9 18:56:09 EDT 2024
--- universe_kernconf_arm64_GENERIC-KMSAN ---
>> arm64.aarch64 GENERIC-KMSAN kernel started on Tue Apr  9 18:56:15 EDT 2024
--- universe_kernconf_arm64_GENERIC-NODEBUG ---
>> arm64.aarch64 GENERIC-NODEBUG kernel started on Tue Apr  9 18:56:18 EDT 2024
--- universe_kernconf_arm64_GENERIC-UP ---
>> arm64.aarch64 GENERIC-UP kernel started on Tue Apr  9 18:56:21 EDT 2024
--- universe_kernconf_arm64_LINT ---
>> arm64.aarch64 LINT kernel started on Tue Apr  9 18:56:37 EDT 2024
--- universe_kernconf_arm64_LINT-ACPI ---
>> arm64.aarch64 LINT-ACPI kernel started on Tue Apr  9 18:56:44 EDT 2024
--- universe_kernconf_arm64_LINT-FDT ---
>> arm64.aarch64 LINT-FDT kernel started on Tue Apr  9 18:57:38 EDT 2024
--- universe_i386_kernels ---
--- universe_kernconf_i386_LINT ---
>> i386.i386 LINT kernel completed on Tue Apr  9 18:57:53 EDT 2024
--- universe_kernels ---
>> i386 kernels completed on Tue Apr  9 18:57:53 EDT 2024
--- universe_i386_done ---
>> i386 completed on Tue Apr  9 18:58:06 EDT 2024
--- universe_amd64_amd64 ---
>> amd64.amd64 buildworld completed on Tue Apr  9 18:58:49 EDT 2024
--- universe_amd64_kernels ---
--- universe_kernels_prologue ---
>> amd64 kernels started on Tue Apr  9 18:58:54 EDT 2024
--- universe_kernconf_amd64_FIRECRACKER ---
>> amd64.amd64 FIRECRACKER kernel started on Tue Apr  9 18:58:54 EDT 2024
--- universe_kernconf_amd64_GENERIC ---
>> amd64.amd64 GENERIC kernel started on Tue Apr  9 18:58:57 EDT 2024
--- universe_kernconf_amd64_GENERIC-KASAN ---
>> amd64.amd64 GENERIC-KASAN kernel started on Tue Apr  9 18:59:00 EDT 2024
--- universe_kernconf_amd64_GENERIC-KCSAN ---
>> amd64.amd64 GENERIC-KCSAN kernel started on Tue Apr  9 18:59:07 EDT 2024
--- universe_kernconf_amd64_GENERIC-KMSAN ---
>> amd64.amd64 GENERIC-KMSAN kernel started on Tue Apr  9 18:59:14 EDT 2024
--- universe_powerpc_kernels ---
--- universe_kernconf_powerpc_MPC85XX ---
>> powerpc.powerpc MPC85XX kernel completed on Tue Apr  9 19:02:05 EDT 2024
--- universe_kernconf_powerpc_GENERIC ---
>> powerpc.powerpc GENERIC kernel completed on Tue Apr  9 19:02:52 EDT 2024
--- universe_amd64_kernels ---
--- universe_kernconf_amd64_GENERIC-NODEBUG ---
>> amd64.amd64 GENERIC-NODEBUG kernel started on Tue Apr  9 19:02:56 EDT 2024
--- universe_powerpc_kernels ---
--- universe_kernconf_powerpc_GENERIC-NODEBUG ---
>> powerpc.powerpc GENERIC-NODEBUG kernel completed on Tue Apr  9 19:03:00 EDT 2024
--- universe_amd64_kernels ---
--- universe_kernconf_amd64_LINT ---
>> amd64.amd64 LINT kernel started on Tue Apr  9 19:03:52 EDT 2024
--- universe_powerpc_kernels ---
--- universe_kernconf_powerpc_QORIQ64 ---
>> powerpc.powerpc64 QORIQ64 kernel completed on Tue Apr  9 19:04:22 EDT 2024
--- universe_amd64_kernels ---
--- universe_kernconf_amd64_LINT-NOINET ---
>> amd64.amd64 LINT-NOINET kernel started on Tue Apr  9 19:04:23 EDT 2024
--- universe_kernconf_amd64_LINT-NOINET6 ---
>> amd64.amd64 LINT-NOINET6 kernel started on Tue Apr  9 19:04:54 EDT 2024
--- universe_powerpc_kernels ---
--- universe_kernconf_powerpc_LINT ---
>> powerpc.powerpc LINT kernel completed on Tue Apr  9 19:05:04 EDT 2024
--- universe_amd64_kernels ---
--- universe_kernconf_amd64_LINT-NOIP ---
>> amd64.amd64 LINT-NOIP kernel started on Tue Apr  9 19:06:11 EDT 2024
--- universe_powerpc_kernels ---
--- universe_kernconf_powerpc_GENERIC64 ---
>> powerpc.powerpc64 GENERIC64 kernel completed on Tue Apr  9 19:06:41 EDT 2024
--- universe_kernconf_powerpc_LINT64 ---
>> powerpc.powerpc64 LINT64 kernel completed on Tue Apr  9 19:07:08 EDT 2024
--- universe_amd64_kernels ---
--- universe_kernconf_amd64_MINIMAL ---
>> amd64.amd64 MINIMAL kernel started on Tue Apr  9 19:07:18 EDT 2024
--- universe_kernconf_amd64_MINIMALUP ---
>> amd64.amd64 MINIMALUP kernel started on Tue Apr  9 19:07:25 EDT 2024
--- universe_powerpc_kernels ---
--- universe_kernconf_powerpc_GENERIC64-NODEBUG ---
>> powerpc.powerpc64 GENERIC64-NODEBUG kernel completed on Tue Apr  9 19:07:37 EDT 2024
--- universe_kernconf_powerpc_GENERIC64LE ---
>> powerpc.powerpc64le GENERIC64LE kernel completed on Tue Apr  9 19:07:55 EDT 2024
--- universe_kernconf_powerpc_GENERIC64LE-NODEBUG ---
>> powerpc.powerpc64le GENERIC64LE-NODEBUG kernel completed on Tue Apr  9 19:08:17 EDT 2024
--- universe_kernconf_powerpc_LINT64LE ---
>> powerpc.powerpc64le LINT64LE kernel completed on Tue Apr  9 19:11:20 EDT 2024
--- universe_kernels ---
>> powerpc kernels completed on Tue Apr  9 19:11:20 EDT 2024
--- universe_powerpc_done ---
>> powerpc completed on Tue Apr  9 19:11:21 EDT 2024
--- universe_arm64_kernels ---
--- universe_kernconf_arm64_GENERIC-KASAN ---
>> arm64.aarch64 GENERIC-KASAN kernel completed on Tue Apr  9 19:16:01 EDT 2024
--- universe_kernconf_arm64_GENERIC-UP ---
>> arm64.aarch64 GENERIC-UP kernel completed on Tue Apr  9 19:16:13 EDT 2024
--- universe_kernconf_arm64_GENERIC ---
>> arm64.aarch64 GENERIC kernel completed on Tue Apr  9 19:16:16 EDT 2024
--- universe_kernconf_arm64_GENERIC-NODEBUG ---
>> arm64.aarch64 GENERIC-NODEBUG kernel completed on Tue Apr  9 19:17:06 EDT 2024
--- universe_amd64_kernels ---
--- universe_kernconf_amd64_FIRECRACKER ---
>> amd64.amd64 FIRECRACKER kernel completed on Tue Apr  9 19:18:53 EDT 2024
--- universe_kernconf_amd64_MINIMALUP ---
>> amd64.amd64 MINIMALUP kernel completed on Tue Apr  9 19:20:53 EDT 2024
--- universe_kernconf_amd64_MINIMAL ---
>> amd64.amd64 MINIMAL kernel completed on Tue Apr  9 19:21:02 EDT 2024
--- universe_arm64_kernels ---
--- universe_kernconf_arm64_LINT-ACPI ---
>> arm64.aarch64 LINT-ACPI kernel completed on Tue Apr  9 19:21:28 EDT 2024
--- universe_amd64_kernels ---
--- universe_kernconf_amd64_GENERIC ---
>> amd64.amd64 GENERIC kernel completed on Tue Apr  9 19:21:34 EDT 2024
--- universe_arm64_kernels ---
--- universe_kernconf_arm64_LINT ---
>> arm64.aarch64 LINT kernel completed on Tue Apr  9 19:21:52 EDT 2024
--- universe_amd64_kernels ---
--- universe_kernconf_amd64_GENERIC-KCSAN ---
>> amd64.amd64 GENERIC-KCSAN kernel completed on Tue Apr  9 19:22:10 EDT 2024
--- universe_kernconf_amd64_GENERIC-KASAN ---
>> amd64.amd64 GENERIC-KASAN kernel completed on Tue Apr  9 19:22:14 EDT 2024
--- universe_arm64_kernels ---
--- universe_kernconf_arm64_LINT-FDT ---
>> arm64.aarch64 LINT-FDT kernel completed on Tue Apr  9 19:22:15 EDT 2024
--- universe_amd64_kernels ---
--- universe_kernconf_amd64_GENERIC-NODEBUG ---
>> amd64.amd64 GENERIC-NODEBUG kernel completed on Tue Apr  9 19:22:38 EDT 2024
--- universe_arm64_kernels ---
--- universe_kernconf_arm64_GENERIC-KMSAN ---
>> arm64.aarch64 GENERIC-KMSAN kernel completed on Tue Apr  9 19:22:42 EDT 2024
--- universe_kernels ---
>> arm64 kernels completed on Tue Apr  9 19:22:42 EDT 2024
--- universe_arm64_done ---
>> arm64 completed on Tue Apr  9 19:22:42 EDT 2024
--- universe_amd64_kernels ---
--- universe_kernconf_amd64_LINT-NOINET ---
>> amd64.amd64 LINT-NOINET kernel completed on Tue Apr  9 19:23:07 EDT 2024
--- universe_kernconf_amd64_LINT-NOIP ---
>> amd64.amd64 LINT-NOIP kernel completed on Tue Apr  9 19:23:15 EDT 2024
--- universe_kernconf_amd64_LINT-NOINET6 ---
>> amd64.amd64 LINT-NOINET6 kernel completed on Tue Apr  9 19:23:19 EDT 2024
--- universe_kernconf_amd64_LINT ---
>> amd64.amd64 LINT kernel completed on Tue Apr  9 19:23:20 EDT 2024
--- universe_kernconf_amd64_GENERIC-KMSAN ---
>> amd64.amd64 GENERIC-KMSAN kernel completed on Tue Apr  9 19:23:32 EDT 2024
--- universe_kernels ---
>> amd64 kernels completed on Tue Apr  9 19:23:32 EDT 2024
--- universe_amd64_done ---
>> amd64 completed on Tue Apr  9 19:23:32 EDT 2024
--- universe_epilogue ---
--------------------------------------------------------------
>>> make universe completed on Tue Apr  9 19:23:32 EDT 2024
(started Tue Apr  9 18:20:49 EDT 2024)
--------------------------------------------------------------

I can test on AMD64 (currently using it) and ARM64 and i386. I might be able to do BE PPC64.

So I've been trying to recreate the steps here and finding the path kinda tricky.
I plan on landing the pull request, kinda. Since I want to do it as a merge commit from a vendor branch, I gotta create a vendor branch and that's being super tricky.

And all the other changes, I'm doing separately. I want a clean as possible merge. There's 4 conflicts.

But the hard part is that jemalloc has gone generate this file happy and so we have to do something approximately like:

  1. Do vendor branch import
    1. Copy files from upstream
    2. apply patches to the .in files
    3. autoconf.sh <lots of args>
    4. commit and tag
  2. Then do the merge
    1. and also you gotta generate the namespace.h files by hand here
    2. and then selectively trim
    3. and resolve the merge conflicts
    4. and also things like isthreaded and a few other I'm still sorting through

Before xz, I'd have just landed this since we didn't have a good vendor import, but post xz, I think we need to be more careful. Which also means, I think, some pre-import un-thinning of what we have in contrib/jemalloc. Plus I have much angst about this deviating so much from a typical vendor import, though the upstream release rate is slow enough I'm not super worried (though there's been a fair number of changes since the release).

Though all in all, I think once it's sorted in head, we can have it in 14.3...

All this to "quickly" verify that this is good code (I'm 99.99% sure it is from the diffs I've run, but I've not chased down all the diffs just yet).

lib/libc/stdlib/jemalloc/Makefile.inc
17

The above won't work, since we add the JEMALLOCSRCS to MISRCS after mangling the name

diff --git a/lib/libc/stdlib/malloc/jemalloc/Makefile.inc b/lib/libc/stdlib/malloc/jemalloc/Makefile.inc
index d916d69f471f..b804490835b7 100644
--- a/lib/libc/stdlib/malloc/jemalloc/Makefile.inc
+++ b/lib/libc/stdlib/malloc/jemalloc/Makefile.inc
@@ -12,6 +12,7 @@ CFLAGS+=-D_USE_LG_VADDR_WIDE
 .for src in ${JEMALLOCSRCS}
 MISRCS+=jemalloc_${src}
 CLEANFILES+=jemalloc_${src}
+CFLAGS.jemalloc_${src}=-std=gnu11
 jemalloc_${src}: ${SRCTOP}/contrib/jemalloc/src/${src} .NOMETA
        ln -sf ${.ALLSRC} ${.TARGET}
 .endfor

is what's needed.

And why gnu11? Does anybody know why upstream chose gnu11 instead of c11?

lib/libc/stdlib/jemalloc/Makefile.inc
17

I'm sorely tempted to just do this bit right here ASAP, btw. It's not all of libc, and I think it's constrained enough to be fine.

lib/libc/stdlib/jemalloc/Makefile.inc
17

Also, there's some other new c11 features, related to anonymous unions and/or flexible arrays, that also cause problems if you try to fix the re-decl with ugly CPP logic. Once you specify C11, life is almost good.

/usr/local/bin/x86_64-unknown-freebsd15.0-ld: _malloc_message: undefined version: FBSD_1.0
/usr/local/bin/x86_64-unknown-freebsd15.0-ld: _malloc_options: undefined version: FBSD_1.0
/usr/local/bin/x86_64-unknown-freebsd15.0-ld: _malloc_first_thread: undefined version: FBSDprivate_1.0
/usr/local/bin/x86_64-unknown-freebsd15.0-ld: failed to set dynamic section sizes: bad value

so more diffs against the pull request to see what I've missed. Maybe these are the old, ancient symbols that Brooks killed?

lib/libc/stdlib/jemalloc/Makefile.inc
17

Also, there's some other new c11 features, related to anonymous unions and/or flexible arrays, that also cause problems if you try to fix the re-decl with ugly CPP logic. Once you specify C11, life is almost good.

/usr/local/bin/x86_64-unknown-freebsd15.0-ld: _malloc_message: undefined version: FBSD_1.0
/usr/local/bin/x86_64-unknown-freebsd15.0-ld: _malloc_options:: FBSD_1.0
/usr/local/bin/x86_64-unknown-freebsd15.0-ld: _malloc_first_thread: undefined version: FBSDprivate_1.0
/usr/local/bin/x86_64-unknown-freebsd15.0-ld: failed to set dynamic section sizes: bad value

so more diffs against the pull request to see what I've missed. Maybe these are the old, ancient symbols that Brooks killed?

The undefined version bits are obsolete things we patch back in for symbol compatibility. We went way overboard exposing non-standard symbols in public namespaces and now we pay the price forever (or until libc.so.8).

These should certainly all be relegated being compat symbols as I did with some others in 5680cf6dc6e25cffa3930e9cb06f6982fcb80209.

I'm not sure what the cause of the last bit is.

contrib/jemalloc/include/jemalloc/internal/hash.h
202

Using JEMALLOC_FALLTHROUGH was introduced in 7bcf5319c1106b761bcb2a4facfeccf1b9db758d.

But the upstream jemalloc and 5.3.0 tag use break instead. (https://github.com/jemalloc/jemalloc/blob/54eaed1d8b56b1aa528be3bdd1877e59c56fa90c/include/jemalloc/internal/hash.h#L202). I'm sticking with break because I get compilation error fallthrough annotation does not directly precede switch label. So if there is no reason that we must use the macro, I would prefer to stick with the upstream.

lib/libc/stdlib/jemalloc/Makefile.inc
17

I'm reverting gnu11 part. After rebasing main and building again, build succeeds without having gnu11 option.

lib/libc/stdlib/jemalloc/Makefile.inc
17

I updated main to have the quoted change for you to remove gnu11 here. After the diff reduction changes, I've figured out what changes we need in contrib/jemalloc, and when in the import process we need to do the config, etc to avoid the FreeBSD-diff file and reduce what we have to check into our tree. There's also some diffs I need to sort out with out FreeBSD includes too. I may need to fix upstream detection of a few symbols as well, since I have to change config still...

Thanks for rebasing. I'm using this as a reference to make sure I can reproduce what you've done. When I push that work, I'll credit you as the author since I'm just verifying there's nothing unexpected and also trying to find FreeBSD changes we no longer need... Brooks' comments are next up for me....

Thanks for all the hard work to get to this point... It's certainly making the updating of the vendor branch to current project standards way easier.

contrib/jemalloc/include/jemalloc/internal/hash.h
202

I must have read upstream vs the change incorrectly. It sure looked like upstream used JEMALLOC_FALLTHROUGH in the diffs I ran, but I've reverted so there's no conflict here now.