Page MenuHomeFreeBSD

Convert the msun tests to ATF
ClosedPublic

Authored by arichardson on Feb 19 2021, 5:32 PM.

Details

Summary

This provides better error messages that just an assertion failure and
also makes it easier to mark individual tests as XFAIL.
It was also helpful when coming up with D28786 and D28787.

Diff Detail

Repository
rG FreeBSD src repository
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

arichardson retitled this revision from Convert lib/msun/tests/logarithm_test to use ATF to Convert the msun tests to ATF.Feb 22 2021, 6:01 PM
arichardson edited the summary of this revision. (Show Details)

Most of this was just replacing assert( -> ATF_CHECK(, but let me know if you want me to change anything else.

ngie requested changes to this revision.Feb 23 2021, 4:51 AM

This conversion is on the right direction, but incomplete.

lib/msun/tests/cexp_test.c
127–128

printfs like this can be deleted since they exist purely for outputting debug information via TAP.

158–159

Delete printf.

199–200

Delete printf.

220–221

Delete printf.

251–252

Delete printf.

301

Please add an extra newline before this.

lib/msun/tests/conj_test.c
88–93

asserts should be converted into ATF_REQUIRE_MSG(..) in cases like this: not doing so could result in non-sensical cascading failures.

91–99

This needs to be changed into 2 asserts:

ATF_REQUIRE_MSG(
    cfpequal(libconjf(in), expected),
    "conjf(%#.2g + %#.2gI): wrong value", creal(in), cimag(in)
);
ATF_REQUIRE_MSG(
    !fetestexcept(FE_ALL_EXCEPT),
    "conjf(%#.2g + %#.2gI): threw an exception", creal(in), cimag(in)
);
100–106

Please use ATF_REQUIRE_MSG here.

107–113

Please use ATF_REQUIRE_MSG instead.

lib/msun/tests/csqrt_test.c
299–302

Each of the test cases should be split up using ATF_TP_ADD_TCS(tp).

369–370

Should add a newline.

Would be better to bring up the test cases, as right now a failure will result in the whole test failing.

lib/msun/tests/exponential_test.c
141–184

These tests should be split up, otherwise a single assertion failure will tank the rest of the test program.

lib/msun/tests/fmaxmin_test.c
104–185

Please split up the test cases.

135–137

These comments can be removed if they no longer apply.

lib/msun/tests/invctrig_test.c
348–349

add newline

lib/msun/tests/invtrig_test.c
453–454

newline

lib/msun/tests/lrint_test.c
146–147

newline

lib/msun/tests/lround_test.c
75
  1. The xfail should be applied to the affected areas; refactoring the tests will allow that to be possible.
  2. The message should be more helpful: what assert failed and why did it fail?
126–127

newline

lib/msun/tests/nan_test.c
123–124

newline

lib/msun/tests/nearbyint_test.c
111–112

TAP printf can be deleted.

152–153

TAP printf can be deleted.

171–172

newline

lib/msun/tests/next_test.c
124–125

TAP printf can be deleted. New test case after this.

139–140

TAP printf can be deleted. New test case after this.

165–166

TAP printf can be deleted. New test case after this.

213–222

TAP printf can be deleted. New test case after this.

240–241

Delete TAP printf

This revision now requires changes to proceed.Feb 23 2021, 4:51 AM
arichardson added inline comments.
lib/msun/tests/conj_test.c
107–113

Thanks, also looking at the this should have been conjl all along since it's just doing the same thing again.

lib/msun/tests/csqrt_test.c
299–302

I've split it into csqrt/csqrtf/csqrtl tests.

lib/msun/tests/fmaxmin_test.c
135–137

Seems like they no longer fail after -fno-builtin D28577

lib/msun/tests/lround_test.c
75

Basically all asserts fail because the exception flags INVALID/INEXACT are not being set. I can try to add the xfails to individual tests but I don't have enough time to do further debugging now. I can also leave the entire test disabled as is the current state.

arichardson marked 2 inline comments as done.
arichardson edited the summary of this revision. (Show Details)
  • Address review comments
lib/msun/tests/lround_test.c
75
Standard error:
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:74: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for lround(0.25)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:74: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for lroundf(0.25)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:74: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for llround(0.25)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:74: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for llroundf(0.25)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:75: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for lround(0.5)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:75: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for lroundf(0.5)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:75: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for llround(0.5)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:75: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for llroundf(0.5)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:76: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for lround(-0.5)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:76: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for lroundf(-0.5)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:76: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for llround(-0.5)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:76: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for llroundf(-0.5)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:79: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for lround(0x1234.fp0)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:79: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for lroundf(0x1234.fp0)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:79: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for llround(0x1234.fp0)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:79: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for llroundf(0x1234.fp0)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:80: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for lround(__builtin_inff())
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for lround(__builtin_inff())
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:80: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for lroundf(__builtin_inff())
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for lroundf(__builtin_inff())
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:80: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for llround(__builtin_inff())
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for llround(__builtin_inff())
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:80: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for llroundf(__builtin_inff())
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for llroundf(__builtin_inff())
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:81: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for lround(__builtin_nanf(""))
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for lround(__builtin_nanf(""))
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:81: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for lroundf(__builtin_nanf(""))
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for lroundf(__builtin_nanf(""))
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:81: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for llround(__builtin_nanf(""))
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for llround(__builtin_nanf(""))
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:81: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for llroundf(__builtin_nanf(""))
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for llroundf(__builtin_nanf(""))
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:92: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for lround(0x8000000000000000.0p0)
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for lround(0x8000000000000000.0p0)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:93: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for lroundf(0x8000000000000000.0p0f)
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for lroundf(0x8000000000000000.0p0f)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:96: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for lround(-0x8000000000000800.0p0)
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for lround(-0x8000000000000800.0p0)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:97: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for lroundf(-0x8000010000000000.0p0f)
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for lroundf(-0x8000010000000000.0p0f)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:105: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for llround(0x8000000000000000.0p0)
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for llround(0x8000000000000000.0p0)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:106: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for llroundf(0x8000000000000000.0p0f)
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for llroundf(0x8000000000000000.0p0f)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:109: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for llround(-0x8000000000000800.0p0)
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for llround(-0x8000000000000800.0p0)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:110: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for llroundf(-0x8000010000000000.0p0f)
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for llroundf(-0x8000010000000000.0p0f)
===> Expected failures
lround_test:main  ->  expected_failure: https://bugs.freebsd.org/205451: 48 checks failed as expected; see output for more details  [0.009s]

Does this look okay now?

I'll take a gander in a few :).

  • next_test: Replace abort() with atf_tc_fail_check

@ngie It would be great if you could have another look at this change. I now have quite a few follow-up msun fixes (getting close to having the test passing on AArch64) that depend on this.

@ngie Will you be able to look at this change this week? As this is blocking a lot of follow-up changes I'd really like to get this committed soon. If not I can fix any outstanding issues in post-commit review?

Drop incorrect comment in csqrt_test

I will commit this at the end of this week if I don't see any positive/negative reviews until this. Any remaining changes can be done post-commit IMO.

arichardson added inline comments.
lib/msun/tests/fmaxmin_test.c
135–137

This is fixed by the follow-up change D28884

lib/msun/tests/lround_test.c
75

I have some follow-up changes to fix this test. Will post those once I've committed the other reviews.

This revision was not accepted when it landed; it landed in state Needs Review.Mar 22 2021, 12:01 PM
This revision was automatically updated to reflect the committed changes.