Page MenuHomeFreeBSD
Differential D28798

Convert the msun tests to ATF
ClosedPublic
Actions

Authored by arichardson on Feb 19 2021, 5:32 PM.
Tags
None
Referenced Files
Unknown Object (File)
Thu, Apr 18, 12:47 PM
Unknown Object (File)
Thu, Apr 18, 12:21 PM
Unknown Object (File)
Thu, Apr 18, 12:20 PM
Unknown Object (File)
Thu, Apr 18, 12:18 PM
Unknown Object (File)
Thu, Apr 18, 12:17 PM
Unknown Object (File)
Wed, Apr 17, 10:32 PM
Unknown Object (File)
Wed, Apr 17, 8:07 PM
Unknown Object (File)
Wed, Apr 17, 7:50 PM
View All 89 Files
Subscribers
imp

Details

Reviewers
dim
lwhsu
ngie
Group Reviewers
tests
Commits
rGb69904b8916c: Convert the msun tests to ATF
rG133bc645077d: Convert the msun tests to ATF
Summary

This provides better error messages that just an assertion failure and
also makes it easier to mark individual tests as XFAIL.
It was also helpful when coming up with D28786 and D28787.

Diff Detail

Repository
rS FreeBSD src repository - subversion
Lint
Lint Passed
Unit
No Test Coverage
Build Status
Buildable 37281
Build 34170: arc lint + arc unit

Event Timeline

arichardson created this revision.Feb 19 2021, 5:32 PM
Herald added a subscriber: imp. · View Herald TranscriptFeb 19 2021, 5:32 PM
arichardson requested review of this revision.Feb 19 2021, 5:32 PM
Harbormaster completed remote builds in B37195: Diff 84282.Feb 19 2021, 5:32 PM
arichardson added a parent revision: D28786: lib/msun: Avoid FE_INEXACT for x86 log2l/log10l.Feb 22 2021, 6:00 PM
arichardson updated this revision to Diff 84489.Feb 22 2021, 6:01 PM

Update all tests to ATF

Harbormaster completed remote builds in B37281: Diff 84489.Feb 22 2021, 6:01 PM
arichardson retitled this revision from Convert lib/msun/tests/logarithm_test to use ATF to Convert the msun tests to ATF.Feb 22 2021, 6:01 PM
arichardson edited the summary of this revision. (Show Details)
arichardson added a comment.Feb 22 2021, 6:04 PM

Most of this was just replacing assert( -> ATF_CHECK(, but let me know if you want me to change anything else.

ngie requested changes to this revision.Feb 23 2021, 4:51 AM

This conversion is on the right direction, but incomplete.

lib/msun/tests/cexp_test.c
128

printfs like this can be deleted since they exist purely for outputting debug information via TAP.

160

Delete printf.

202

Delete printf.

224

Delete printf.

256

Delete printf.

307

Please add an extra newline before this.

lib/msun/tests/conj_test.c
88–93

asserts should be converted into ATF_REQUIRE_MSG(..) in cases like this: not doing so could result in non-sensical cascading failures.

93–104

This needs to be changed into 2 asserts:

ATF_REQUIRE_MSG(
    cfpequal(libconjf(in), expected),
    "conjf(%#.2g + %#.2gI): wrong value", creal(in), cimag(in)
);
ATF_REQUIRE_MSG(
    !fetestexcept(FE_ALL_EXCEPT),
    "conjf(%#.2g + %#.2gI): threw an exception", creal(in), cimag(in)
);
107–118

Please use ATF_REQUIRE_MSG here.

121–132

Please use ATF_REQUIRE_MSG instead.

lib/msun/tests/csqrt_test.c
299–301

Each of the test cases should be split up using ATF_TP_ADD_TCS(tp).

374–375

Should add a newline.

Would be better to bring up the test cases, as right now a failure will result in the whole test failing.

lib/msun/tests/exponential_test.c
150–159

These tests should be split up, otherwise a single assertion failure will tank the rest of the test program.

lib/msun/tests/fmaxmin_test.c
124–148

Please split up the test cases.

135–137

These comments can be removed if they no longer apply.

lib/msun/tests/invctrig_test.c
349–350

add newline

lib/msun/tests/invtrig_test.c
454–455

newline

lib/msun/tests/lrint_test.c
147–148

newline

lib/msun/tests/lround_test.c
72
  1. The xfail should be applied to the affected areas; refactoring the tests will allow that to be possible.
  2. The message should be more helpful: what assert failed and why did it fail?
120–121

newline

lib/msun/tests/nan_test.c
120–121

newline

lib/msun/tests/nearbyint_test.c
113

TAP printf can be deleted.

156

TAP printf can be deleted.

172–173

newline

lib/msun/tests/next_test.c
122

TAP printf can be deleted. New test case after this.

135

TAP printf can be deleted. New test case after this.

159

TAP printf can be deleted. New test case after this.

204

TAP printf can be deleted. New test case after this.

224

Delete TAP printf

This revision now requires changes to proceed.Feb 23 2021, 4:51 AM
arichardson marked 27 inline comments as done.Feb 23 2021, 11:07 AM
arichardson added inline comments.
lib/msun/tests/conj_test.c
121–132

Thanks, also looking at the this should have been conjl all along since it's just doing the same thing again.

lib/msun/tests/csqrt_test.c
299–301

I've split it into csqrt/csqrtf/csqrtl tests.

lib/msun/tests/fmaxmin_test.c
135–137

Seems like they no longer fail after -fno-builtin D28577

lib/msun/tests/lround_test.c
72

Basically all asserts fail because the exception flags INVALID/INEXACT are not being set. I can try to add the xfails to individual tests but I don't have enough time to do further debugging now. I can also leave the entire test disabled as is the current state.

arichardson updated this revision to Diff 84524.Feb 23 2021, 11:10 AM
arichardson marked 2 inline comments as done.
arichardson edited the summary of this revision. (Show Details)
  • Address review comments
Harbormaster completed remote builds in B37298: Diff 84524.Feb 23 2021, 11:10 AM
arichardson added inline comments.Feb 23 2021, 11:11 AM
lib/msun/tests/lround_test.c
72
Standard error:
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:74: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for lround(0.25)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:74: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for lroundf(0.25)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:74: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for llround(0.25)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:74: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for llroundf(0.25)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:75: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for lround(0.5)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:75: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for lroundf(0.5)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:75: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for llround(0.5)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:75: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for llroundf(0.5)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:76: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for lround(-0.5)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:76: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for lroundf(-0.5)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:76: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for llround(-0.5)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:76: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for llroundf(-0.5)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:79: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for lround(0x1234.fp0)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:79: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for lroundf(0x1234.fp0)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:79: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for llround(0x1234.fp0)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:79: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for llroundf(0x1234.fp0)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:80: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for lround(__builtin_inff())
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for lround(__builtin_inff())
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:80: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for lroundf(__builtin_inff())
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for lroundf(__builtin_inff())
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:80: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for llround(__builtin_inff())
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for llround(__builtin_inff())
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:80: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for llroundf(__builtin_inff())
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for llroundf(__builtin_inff())
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:81: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for lround(__builtin_nanf(""))
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for lround(__builtin_nanf(""))
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:81: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for lroundf(__builtin_nanf(""))
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for lroundf(__builtin_nanf(""))
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:81: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for llround(__builtin_nanf(""))
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for llround(__builtin_nanf(""))
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:81: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for llroundf(__builtin_nanf(""))
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for llroundf(__builtin_nanf(""))
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:92: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for lround(0x8000000000000000.0p0)
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for lround(0x8000000000000000.0p0)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:93: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for lroundf(0x8000000000000000.0p0f)
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for lroundf(0x8000000000000000.0p0f)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:96: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for lround(-0x8000000000000800.0p0)
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for lround(-0x8000000000000800.0p0)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:97: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for lroundf(-0x8000010000000000.0p0f)
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for lroundf(-0x8000010000000000.0p0f)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:105: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for llround(0x8000000000000000.0p0)
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for llround(0x8000000000000000.0p0)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:106: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for llroundf(0x8000000000000000.0p0f)
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for llroundf(0x8000000000000000.0p0f)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:109: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for llround(-0x8000000000000800.0p0)
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for llround(-0x8000000000000800.0p0)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:110: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for llroundf(-0x8000010000000000.0p0f)
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for llroundf(-0x8000010000000000.0p0f)
===> Expected failures
lround_test:main  ->  expected_failure: https://bugs.freebsd.org/205451: 48 checks failed as expected; see output for more details  [0.009s]
arichardson mentioned this in D28577: While building libm, turn off any math-related compiler builtins.Feb 23 2021, 11:13 AM
arichardson added a child revision: D28884: Remove XFAILs from fmaxmin test.Feb 23 2021, 11:27 AM
arichardson mentioned this in D28884: Remove XFAILs from fmaxmin test.Feb 23 2021, 7:01 PM
arichardson added a comment.Feb 25 2021, 9:24 PM

Does this look okay now?

ngie added a comment.Feb 26 2021, 3:08 AM
In D28798#647799, @arichardson wrote:

Does this look okay now?

I'll take a gander in a few :).

arichardson updated this revision to Diff 84747.Feb 26 2021, 1:04 PM
  • next_test: Replace abort() with atf_tc_fail_check
Harbormaster completed remote builds in B37407: Diff 84747.Feb 26 2021, 1:04 PM
arichardson added a child revision: D29039: Allow lib/msun/logarithm_test to pass on ld128 platforms.Mar 3 2021, 4:28 PM
arichardson added a comment.Mar 5 2021, 9:40 AM

@ngie It would be great if you could have another look at this change. I now have quite a few follow-up msun fixes (getting close to having the test passing on AArch64) that depend on this.

arichardson added a child revision: D29076: Fix lib/msun/tests/csqrt_test on platforms with 128-bit long double.Mar 5 2021, 11:17 AM
arichardson updated this revision to Diff 85215.Mar 5 2021, 4:30 PM

Make lround_test pass

Harbormaster completed remote builds in B37612: Diff 85215.Mar 5 2021, 4:30 PM
arichardson added a child revision: D29091: Improve test messages for msun tests.Mar 5 2021, 4:34 PM
arichardson added a child revision: D29095: lib/msun/tests: Skip fenv_test:masking if exceptions can't be trapped.Mar 5 2021, 4:48 PM
arichardson added a comment.Mar 10 2021, 12:47 PM

@ngie Will you be able to look at this change this week? As this is blocking a lot of follow-up changes I'd really like to get this committed soon. If not I can fix any outstanding issues in post-commit review?

arichardson updated this revision to Diff 85720.Mar 14 2021, 2:42 PM

Drop incorrect comment in csqrt_test

Harbormaster completed remote builds in B37823: Diff 85720.Mar 14 2021, 2:42 PM
arichardson added a reviewer: tests.Mar 14 2021, 2:49 PM
arichardson added a comment.Mar 16 2021, 11:40 AM

I will commit this at the end of this week if I don't see any positive/negative reviews until this. Any remaining changes can be done post-commit IMO.

arichardson marked an inline comment as done.Mar 16 2021, 11:42 AM
arichardson added inline comments.
lib/msun/tests/fmaxmin_test.c
135–137

This is fixed by the follow-up change D28884

lib/msun/tests/lround_test.c
72

I have some follow-up changes to fix this test. Will post those once I've committed the other reviews.

This revision was not accepted when it landed; it landed in state Needs Review.Mar 22 2021, 12:01 PM
Closed by commit rG133bc645077d: Convert the msun tests to ATF (authored by arichardson). · Explain Why
This revision was automatically updated to reflect the committed changes.
arichardson added a commit: rG133bc645077d: Convert the msun tests to ATF.
arichardson added a commit: rGb69904b8916c: Convert the msun tests to ATF.Apr 22 2021, 11:09 AM

Revision Contents
Changeset List

PathSize
lib/
msun/
tests/
36 lines
74 lines
25 lines
50 lines
2 lines
38 lines
297 lines
96 lines
13 lines
44 lines
67 lines
84 lines
85 lines
49 lines
45 lines
54 lines
71 lines
14 lines
109 lines
11 lines
8 lines

Diff 84489

View Options

lib/msun/tests/Makefile

Loading...
View Options

lib/msun/tests/cexp_test.c

Loading...
View Options

lib/msun/tests/conj_test.c

Loading...
View Options

lib/msun/tests/csqrt_test.c

Loading...
View Options

lib/msun/tests/ctrig_test.c

Loading...
View Options

lib/msun/tests/exponential_test.c

Loading...
View Options

lib/msun/tests/fenv_test.c

Loading...
View Options

lib/msun/tests/fma_test.c

Loading...
View Options

lib/msun/tests/fmaxmin_test.c

Loading...
View Options

lib/msun/tests/ilogb_test.c

Loading...
View Options

lib/msun/tests/invctrig_test.c

Loading...
View Options

lib/msun/tests/invtrig_test.c

Loading...
View Options

lib/msun/tests/logarithm_test.c

Loading...
View Options

lib/msun/tests/lrint_test.c

Loading...
View Options

lib/msun/tests/lround_test.c

Loading...
View Options

lib/msun/tests/lround_test.t

Loading...
View Options

lib/msun/tests/nan_test.c

Loading...
View Options

lib/msun/tests/nearbyint_test.c

Loading...
View Options

lib/msun/tests/next_test.c

Loading...
View Options

lib/msun/tests/rem_test.c

Loading...
View Options

lib/msun/tests/test-utils.h

Loading...
View Options

lib/msun/tests/trig_test.c

Loading...