User Details
- User Since
- Aug 6 2014, 5:32 AM (276 w, 2 d)
Tue, Nov 19
I ran tests with Diff 64526 for 16 hours without seeing any problems.
Mon, Nov 18
20191118 12:46:52 all (2/12): sort.sh panic: Lookup object not swappable cpuid = 5 time = 1574077630 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00e3f802a0 vpanic() at vpanic+0x17e/frame 0xfffffe00e3f80300 panic() at panic+0x43/frame 0xfffffe00e3f80360 swp_pager_meta_lookup() at swp_pager_meta_lookup+0x68/frame 0xfffffe00e3f80390 swp_pager_xfer_source() at swp_pager_xfer_source+0x66/frame 0xfffffe00e3f803d0 swp_pager_meta_transfer() at swp_pager_meta_transfer+0x1b1/frame 0xfffffe00e3f80450 swap_pager_copy() at swap_pager_copy+0x267/frame 0xfffffe00e3f80480 vm_object_collapse() at vm_object_collapse+0x178/frame 0xfffffe00e3f804f0 vm_object_deallocate() at vm_object_deallocate+0x4b1/frame 0xfffffe00e3f80540 vm_map_process_deferred() at vm_map_process_deferred+0xad/frame 0xfffffe00e3f80560 vm_map_remove() at vm_map_remove+0xeb/frame 0xfffffe00e3f80590 exec_new_vmspace() at exec_new_vmspace+0x186/frame 0xfffffe00e3f805f0 exec_elf64_imgact() at exec_elf64_imgact+0x82a/frame 0xfffffe00e3f806e0 kern_execve() at kern_execve+0x5e9/frame 0xfffffe00e3f80a40 sys_execve() at sys_execve+0x4c/frame 0xfffffe00e3f80ac0 amd64_syscall() at amd64_syscall+0x2d0/frame 0xfffffe00e3f80bf0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00e3f80bf0 --- syscall (59, FreeBSD ELF64, sys_execve), rip = 0x80086675a, rsp = 0x7fffffffe728, rbp = 0x7fffffffe730 ---
Sun, Nov 17
I tested D22405.64459.diff for 10 hours without seeing any problems.
Tue, Nov 12
I ran the full stress2 test using 64144. No problems seen.
Mon, Nov 11
I only saved the vmcore: https://people.freebsd.org/~pho/vmcore.775.xz
Just an update: I'm at test number 534 out of 677 with an uptime of 20 hours (tests are sorted by increasing run time).
No problems so far.
Sun, Nov 10
With Diff 64139 I got a "panic: vm_object_madvise: page 0xfffffe0000174cf8 is not managed", which to me seems unrelated.
https://people.freebsd.org/~pho/stress/log/dougm063.txt
I completed a full stress2 test with D22261.64008.diff.
The only problem I saw was this (unrelated) panic: https://people.freebsd.org/~pho/stress/log/mark107.txt
Thu, Nov 7
Wed, Nov 6
Sat, Nov 2
I ran all of the stress2 tests on Diff 63804 without seeing any problems.
Wed, Oct 30
Still doesn't compile.
Mon, Oct 28
Sun, Oct 27
(kgdb) l *trap_pfault+0x99
0xffffffff8104d129 is in trap_pfault (../../../amd64/amd64/trap.c:743).
738 * lock, then it is most likely a fatal kernel page fault.
739 * If WITNESS is enabled, then it's going to whine about
740 * bogus LORs with various VM locks, so just skip to the
741 * fatal trap handling directly.
742 */
743 if (td->td_critnest != 0 ||
744 WITNESS_CHECK(WARN_SLEEPOK | WARN_GIANTOK, NULL,
745 "Kernel page fault") != 0) {
746 trap_fatal(frame, eva);
747 return (-1);
(kgdb)This is a KPTI test. This looks unrelated to me, but ...
20191027 15:16:52 all (469/677): kpti.sh Fata lpanic: Assertion p->tp_row < t->t_winsize.tp_row failed at ../../../teken/teken.c:105 cpuid = 10 time = 1572185813 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffff807d872fd60 vpanic() at vpanic+0x19d/frame 0xfffff807d872fdb0 panic() at panic+0x43/frame 0xfffff807d872fe10 teken_subr_do_putchar() at teken_subr_do_putchar+0x138/frame 0xfffff807d872fe60 teken_subr_regular_character() at teken_subr_regular_character+0x27a/frame 0xfffff807d872fea0 teken_input_char() at teken_input_char+0xa7/frame 0xfffff807d872fec0 teken_input() at teken_input+0x111/frame 0xfffff807d872fef0 termcn_cnputc() at termcn_cnputc+0x6e/frame 0xfffff807d872ff20 cnputc() at cnputc+0x7d/frame 0xfffff807d872ff50 cnputs() at cnputs+0x7a/frame 0xfffff807d872ff80 putchar() at putchar+0x150/frame 0xfffff807d8730000 kvprintf() at kvprintf+0x145/frame 0xfffff807d8730120 ??() at 0xffffffff00013051/frame 0xfffff807d87301f0 printf() at printf+0x43/frame 0xfffff807d8730250 trap_fatal() at trap_fatal+0x9c/frame 0xfffff807d87302b0 trap_pfault() at trap_pfault+0x99/frame 0xfffff807d8730310 trap() at trap+0x46f/frame 0xfffff807d8730420 calltrap() at calltrap+0x8/frame 0xfffff807d8730420 --- trap 0x246, rip = 0x80020ac38, rsp = 0x7fffffffdc90, rbp = 0x7fffffffe760 --- KDB: enter: panic [ thread pid 256166277 tid 3224064 ] Stopped at kdb_enter+0x3b: movq $0,kdb_why db>
Oct 20 2019
Here's a
Oct 19 2019
I tested 63458 for 17 hours with a "-O0" build of uma_core.c. LGTM.
Oct 18 2019
This too, what not what I was hoping for:
20191018 08:58:59 all (284/676): pipe2.sh
Oct 17 2019
Oct 15 2019
With debug.vmmap_check=1 I saw this problem:
https://people.freebsd.org/~pho/stress/log/dougm056.txt
Oct 14 2019
Oct 12 2019
I ran all of the stress2 test on 63130. I found one problem, which to me looks unrelated.
https://people.freebsd.org/~pho/stress/log/quota10-2.txt
Oct 10 2019
startup_alloc from "UMA Zones", 4 boot pages left startup_alloc from "UMA Zones", 3 boot pages left startup_alloc from "UMA Hash", 2 boot pages left startup_alloc from "UMA Zones", 1 boot pages left Entering uma_startup1 with 0 boot pages left panic: map 0xfffff80003002000 max = 7d8df000, max_left = 1ff80000000, max_right = 7d8df000 cpuid = 0 time = 1 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xffffffff824ef830 vpanic() at vpanic+0x19d/frame 0xffffffff824ef880 panic() at panic+0x43/frame 0xffffffff824ef8e0 _vm_map_assert_consistent() at _vm_map_assert_consistent+0x3e1/frame 0xffffffff824ef940 vm_map_splay() at vm_map_splay+0x176/frame 0xffffffff824ef9a0 vm_map_lookup_entry() at vm_map_lookup_entry+0xc7/frame 0xffffffff824ef9f0 vm_map_insert() at vm_map_insert+0x1b5/frame 0xffffffff824efae0 kmem_init() at kmem_init+0xab/frame 0xffffffff824efb10 vm_mem_init() at vm_mem_init+0x51/frame 0xffffffff824efb20 mi_startup() at mi_startup+0x210/frame 0xffffffff824efb70 btext() at btext+0x2c KDB: enter: panic [ thread pid 0 tid 0 ] Stopped at kdb_enter+0x3b: movq $0,kdb_why db> x/s version version: FreeBSD 13.0-CURRENT #1 r353388M: Thu Oct 10 12:56:20 CEST 2019\012 pho@mercat1.netperf.freebsd.org:/usr/src/sys/amd64/compile/PHO\012 db>
Oct 5 2019
I ran tests on D21882.62898 for more than a day. No problems seen.
Oct 4 2019
Oct 3 2019
I ran tests on D21882.62858.diff for 5 1/2 hours without seeing any problems. I'll move on to 62874.
Sep 27 2019
Sep 6 2019
Tested D20664.61715.diff for 6 hours on amd64. No problems seen.
Sep 4 2019
I ran a brief 10 hour test on amd64 with D20664.61638.diff. No problems seen.
Sep 3 2019
Sep 2 2019
It is still an issue with HEAD:
: : if_delmulti_locked: detaching ifnet instance 0xfffff803d7598000 if_delmulti_locked: detaching ifnet instance 0xfffff803d7598000 if_delmulti_locked: detaching ifnet instance 0xfffff803bc652800 if_delmulti_locked: detaching ifnet instance 0xfffff803bc652800 if_delmulti_locked: detaching ifnet instance 0xfffff803bc652800 if_delmulti_locked: detaching ifnet instance 0xfffff803bc652800 if_delmulti_locked: detaching ifnet instance 0xfffff803bc652800 if_delmulti_locked: detaching ifnet instance 0xfffff803bc652800
Sep 1 2019
I ran the full stress2 test suite with D21471.61489.diff on amd64. No problems seen.
Aug 29 2019
Aug 28 2019
Aug 27 2019
Aug 25 2019
I ran the full stress2 test with D20814.61081.diff on amd64. LGTM.
I ran all of the stress2 tests with D21357.61095.diff on amd64. LGTM.
Aug 23 2019
I should be able to free up some test hardware later today.
Aug 21 2019
I ran a brief 2 hour test with D21320.61033.diff. No problems seen.
Aug 16 2019
D13671.60851.diff LGTM.
Aug 3 2019
I ran tests on D20863.60414.diff for 6 hours and got this ktr log.
Aug 1 2019
I ran tests for 15 hours on D20814.60344.diff without seeing any problems.
I completed a full stress2 test on amd64 without seeing any problems.
Jul 31 2019
With D20814.60322.diff I got:
20190731 19:27:23 all (1/1): mmap10.sh panic: vm_map_unwire: lookup failed cpuid = 12 time = 1564594061 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00c96c5790 vpanic() at vpanic+0x19d/frame 0xfffffe00c96c57e0 panic() at panic+0x43/frame 0xfffffe00c96c5840 vm_map_wire_locked() at vm_map_wire_locked+0x69d/frame 0xfffffe00c96c5900 vm_map_wire() at vm_map_wire+0x40/frame 0xfffffe00c96c5930 kern_mlock() at kern_mlock+0x179/frame 0xfffffe00c96c5980 amd64_syscall() at amd64_syscall+0x2d6/frame 0xfffffe00c96c5ab0 fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe00c96c5ab0
D20814.60315.diff
20190731 16:18:08 all (17/73): mmap10.sh panic: Lock vm map (user) not exclusively locked @ ../../../vm/vm_map.c:3102 cpuid = 12 time = 1564582693 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00c7c80800 vpanic() at vpanic+0x19d/frame 0xfffffe00c7c80850 panic() at panic+0x43/frame 0xfffffe00c7c808b0 _sx_xunlock() at _sx_xunlock+0xab/frame 0xfffffe00c7c808d0 _vm_map_unlock() at _vm_map_unlock+0x75/frame 0xfffffe00c7c80900 vm_map_wire() at vm_map_wire+0x58/frame 0xfffffe00c7c80930 kern_mlock() at kern_mlock+0x179/frame 0xfffffe00c7c80980 amd64_syscall() at amd64_syscall+0x2d6/frame 0xfffffe00c7c80ab0
Jul 30 2019
I ran all of the threaded tests from stress2 on amd64 with D21088.60283.diff.
No problems seen.
Just a follow up. I'm currently testing your patch on amd64 where I have not yet seen any problems.
I got this after a 5 hour test on i386, followed by a shutdown:
panic: vm_fault_hold: fault on nofault entry, addr: 0x83326000 cpuid = 0 time = 1564486041 KDB: stack backtrace: db_trace_self_wrapper(e6666d,1bdcda4,0,23a17534,bd2eb1,...) at db_trace_self_wrapper+0x2a/frame 0x23a17508 kdb_backtrace(e,0,0,2b06100,2db5000,...) at kdb_backtrace+0x2e/frame 0x23a17568 vpanic(163709e,23a175b0,23a175b0,23a17680,12c2044,...) at vpanic+0x121/frame 0x23a17590 panic(163709e,158dad9,83326000,4895800,7000004,...) at panic+0x14/frame 0x23a175a4 vm_fault_hold(2db5000,83326000,2,0,0) at vm_fault_hold+0x2474/frame 0x23a17680 vm_fault(2db5000,83326000,2,0) at vm_fault+0x4a/frame 0x23a176a8 trap_pfault(8332680c) at trap_pfault+0xf3/frame 0x23a176f4 trap(23a177bc,8,28,28,83df8640,...) at trap+0x3c0/frame 0x23a177b0 calltrap() at 0xffc0316d/frame 0x23a177b0 --- trap 0xc, eip = 0xf7f255, esp = 0x23a177fc, ebp = 0x23a17814 --- funsetownlst(27e5ea50) at funsetownlst+0xa5/frame 0x23a17814 pgdelete(27e5ea6c,23c00a80,1651842,36,23a178f8,...) at pgdelete+0x60/frame 0x23a17834 leavepgrp(298d46a8,0,1651842,396,44c,...) at leavepgrp+0xd5/frame 0x23a1784c proc_reap(23c00a80,298d46a8,23a17ba4,36,1edeb50,...) at proc_reap+0x714/frame 0x23a178f8 proc_to_reap(23c00a80,298d46a8,7,0,0,...) at proc_to_reap+0x718/frame 0x23a17998 kern_wait6(23c00a80,7,0,0,23a17ba4,...) at kern_wait6+0x37a/frame 0x23a17a78 kern_wait(23c00a80,ffffffff,23a17ba4,36,0,...) at kern_wait+0x119/frame 0x23a17b80 sys_wait4(23c00a80,23c00d0c) at sys_wait4+0x68/frame 0x23a17c08 syscall(23a17ce8,3b,3b,3b,0,...) at syscall+0x304/frame 0x23a17cdc Xint0x80_syscall() at 0xffc033b7/frame 0x23a17cdc
Ran tests for another 23 hours on D20863.60217.diff. No problems seen.
Jul 27 2019
I tested D20814.60178.diff for 13 hours on amd64 without seeing any problems.
Jul 25 2019
The i386 tests finished with one further problem and an incomplete report: https://people.freebsd.org/~pho/stress/log/alc011.txt
Jul 24 2019
I finished the full stress2 tests on amd64 without any issues.
Jul 22 2019
Sure, happy to.
I ran tests on a GENERICish kernel for 12 hours without any issue, however a kernel build with "options KTR" does not compile:
Jul 21 2019
I still see the panic with D20863.59987.diff:
I ran a brief 4 hour test with this patch.
I completed a full stress2 test with D20947.59944.diff on amd64. No problems seen.
With D20863.59975.diff I got:
Jul 18 2019
A have completed a full stress2 test of D20833.59786.diff without seeing any problems.
Jul 17 2019
I have completed a full stress2 test with D20863.59745.diff on amd64 without seeing any problems.
Jul 15 2019
Some KTR output for a short test of D20863.59745.diff:
https://people.freebsd.org/~pho/stress/log/dougm051.txt
I tested D20949.59742.diff with all of the threaded tests I have, on i386. No problems seen.
Jul 14 2019
With D20863.59728.diff I got:
Jul 13 2019
With D20863.59658.diff I got:
Jul 12 2019
I finished the full stress2 test of D20893.59605.diff without finding any problems.
Jul 11 2019
The test has now run for 12 hours without any problems. I'll let the test run for the full ~48 hours.
Jul 10 2019
As far as I can tell the "panic: freeing free block" was introduced by r349777 and this patch didn't fix that.
I have not observed any other problems while testing with D20893.59572.diff
I have tested D20893.59572.diff on amd64 and briefly on i386.
I still see the "panic: freeing free block" on both a MEMGUARD and a GENERICish build.
Jul 9 2019
Looks like this patch is responsible for the panics I see.
20190709 11:56:55 all (1/1): sort.sh swap_pager: out of swap space swp_pager_getswapspace(32): failed Jul 9 12:03:41 mercat1 kernel: pid 3581 (sort), jid 0, uid 0, was killed: out of swap space Jul 9 12:03:42 mercat1 kernel: pid 3583 (sort), jid 0, uid 0, was killed: out of swap space panic: freeing free block: ffffc0, size 16, mask 1 cpuid = 7 time = 1562666624 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe102d95b270 vpanic() at vpanic+0x19d/frame 0xfffffe102d95b2c0 panic() at panic+0x43/frame 0xfffffe102d95b320 blst_meta_free() at blst_meta_free+0x12b/frame 0xfffffe102d95b360 blst_meta_free() at blst_meta_free+0x102/frame 0xfffffe102d95b3a0 blst_meta_free() at blst_meta_free+0x102/frame 0xfffffe102d95b3e0 blst_meta_free() at blst_meta_free+0x102/frame 0xfffffe102d95b420 blst_meta_free() at blst_meta_free+0x102/frame 0xfffffe102d95b460 blist_free() at blist_free+0x2e/frame 0xfffffe102d95b480 swp_pager_freeswapspace() at swp_pager_freeswapspace+0x8a/frame 0xfffffe102d95b4a0 swp_pager_meta_free_all() at swp_pager_meta_free_all+0xbb/frame 0xfffffe102d95b4f0 swap_pager_dealloc() at swap_pager_dealloc+0x115/frame 0xfffffe102d95b510 vm_object_terminate() at vm_object_terminate+0x27b/frame 0xfffffe102d95b560 vm_object_deallocate() at vm_object_deallocate+0x412/frame 0xfffffe102d95b5c0 vm_map_process_deferred() at vm_map_process_deferred+0x7f/frame 0xfffffe102d95b5e0 vm_map_remove() at vm_map_remove+0xc6/frame 0xfffffe102d95b610 vmspace_exit() at vmspace_exit+0xd3/frame 0xfffffe102d95b650 exit1() at exit1+0x5ad/frame 0xfffffe102d95b6c0 sigexit() at sigexit+0xdaf/frame 0xfffffe102d95b9a0 postsig() at postsig+0x336/frame 0xfffffe102d95ba70 ast() at ast+0x4c7/frame 0xfffffe102d95bab0 doreti_ast() at doreti_ast+0x1f/frame 0x7fffffffded0 KDB: enter: panic [ thread pid 3581 tid 100238 ] Stopped at kdb_enter+0x3b: movq $0,kdb_why db> x/s version version: FreeBSD 13.0-CURRENT #5 r349777: Tue Jul 9 11:37:16 CEST 2019\012 pho@mercat1.netperf.freebsd.org:/usr/src/sys/amd64/compile/MEMGUARD\012 db>
The same test works fine on r349776.
I'll try again and avoid a panic, but in the meantime here's some trace: https://people.freebsd.org/~pho/stress/log/dougm047.txt
Jul 8 2019
So, this is unrelated to D20833.
Came across this panic:
Jul 7 2019
I ran tests for 5 hours on i386 with D20833.59496.diff. No problems seen.
I have started a brief test run on i386 with D20833.59496.diff.
Jul 6 2019
I tested D20833.59460.diff for 5 hours without seeing any problems.
Jul 5 2019
I have run tests on D20579.59404.diff (with the typo fixed) for 8 hours. This included a buildworld / installworld.
cc -c -O2 -pipe -fno-strict-aliasing -g -nostdinc -I. -I../../.. -I../../../contrib/ck/include -I../../../contrib/libfdt -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -MD -MF.depend.subr_blist.o -MTsubr_blist.o -fdebug-prefix-map=./machine=/usr/src/sys/amd64/include -fdebug-prefix-map=./x86=/usr/src/sys/x86/include -mcmodel=kernel -mno-red-zone -mno-mmx -mno-sse -msoft-float -fno-asynchronous-unwind-tables -ffreestanding -fwrapv -fstack-protector -gdwarf-2 -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wcast-qual -Wundef -Wno-pointer-sign -D__printf__=__freebsd_kprintf__ -Wmissing-include-dirs -fdiagnostics-show-option -Wno-unknown-pragmas -Wno-error-tautological-compare -Wno-error-empty-body -Wno-error-parentheses-equality -Wno-error-unused-function -Wno-error-pointer-sign -Wno-error-shift-negative-value -Wno-address-of-packed-member -mno-aes -mno-avx -std=iso9899:1999 -Werror ../../../kern/subr_blist.c
../../../kern/subr_blist.c:659:17: error: use of undeclared identifier 'BLIST_MAP_RADIX'
if (maxcount % BLIST_MAP_RADIX != 0)
^
1 error generated.
*** Error code 1Ran a brief test (5 hours) on D20833.59430.diff. No problems seen.
I have run tests on D20635.59379.diff for 24 hours without seeing any problems.
Jul 4 2019
Jul 3 2019
I ran tests on D20833.59317.diff for 14 hours. LGTM.