Page MenuHomeFreeBSD
Differential D28798

Convert the msun tests to ATF
ClosedPublic
Actions

Authored by arichardson on Feb 19 2021, 5:32 PM.
Tags
None
Referenced Files
Unknown Object (File)
Mon, Dec 9, 7:55 AM
Unknown Object (File)
Sun, Dec 8, 1:50 AM
Unknown Object (File)
Fri, Dec 6, 9:49 PM
Unknown Object (File)
Thu, Dec 5, 1:15 AM
Unknown Object (File)
Tue, Dec 3, 11:13 AM
Unknown Object (File)
Nov 23 2024, 11:20 AM
Unknown Object (File)
Nov 15 2024, 7:14 PM
Unknown Object (File)
Nov 15 2024, 4:03 PM
View All Files
Subscribers
imp

Details

Reviewers
dim
lwhsu
ngie
Group Reviewers
tests
Commits
rGb69904b8916c: Convert the msun tests to ATF
rG133bc645077d: Convert the msun tests to ATF
Summary

This provides better error messages that just an assertion failure and
also makes it easier to mark individual tests as XFAIL.
It was also helpful when coming up with D28786 and D28787.

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

arichardson created this revision.Feb 19 2021, 5:32 PM
Herald added a subscriber: imp. · View Herald TranscriptFeb 19 2021, 5:32 PM
arichardson requested review of this revision.Feb 19 2021, 5:32 PM
Harbormaster completed remote builds in B37195: Diff 84282.Feb 19 2021, 5:32 PM
arichardson added a parent revision: D28786: lib/msun: Avoid FE_INEXACT for x86 log2l/log10l.Feb 22 2021, 6:00 PM
arichardson updated this revision to Diff 84489.Feb 22 2021, 6:01 PM

Update all tests to ATF

Harbormaster completed remote builds in B37281: Diff 84489.Feb 22 2021, 6:01 PM
arichardson retitled this revision from Convert lib/msun/tests/logarithm_test to use ATF to Convert the msun tests to ATF.Feb 22 2021, 6:01 PM
arichardson edited the summary of this revision. (Show Details)
arichardson added a comment.Feb 22 2021, 6:04 PM

Most of this was just replacing assert( -> ATF_CHECK(, but let me know if you want me to change anything else.

ngie requested changes to this revision.Feb 23 2021, 4:51 AM

This conversion is on the right direction, but incomplete.

lib/msun/tests/cexp_test.c
127–128

printfs like this can be deleted since they exist purely for outputting debug information via TAP.

158–159

Delete printf.

199–200

Delete printf.

220–221

Delete printf.

251–252

Delete printf.

302

Please add an extra newline before this.

lib/msun/tests/conj_test.c
88–93

asserts should be converted into ATF_REQUIRE_MSG(..) in cases like this: not doing so could result in non-sensical cascading failures.

91–99

This needs to be changed into 2 asserts:

ATF_REQUIRE_MSG(
    cfpequal(libconjf(in), expected),
    "conjf(%#.2g + %#.2gI): wrong value", creal(in), cimag(in)
);
ATF_REQUIRE_MSG(
    !fetestexcept(FE_ALL_EXCEPT),
    "conjf(%#.2g + %#.2gI): threw an exception", creal(in), cimag(in)
);
100–106

Please use ATF_REQUIRE_MSG here.

107–113

Please use ATF_REQUIRE_MSG instead.

lib/msun/tests/csqrt_test.c
299–302

Each of the test cases should be split up using ATF_TP_ADD_TCS(tp).

370–371

Should add a newline.

Would be better to bring up the test cases, as right now a failure will result in the whole test failing.

lib/msun/tests/exponential_test.c
141–184

These tests should be split up, otherwise a single assertion failure will tank the rest of the test program.

lib/msun/tests/fmaxmin_test.c
104–185

Please split up the test cases.

135–137

These comments can be removed if they no longer apply.

lib/msun/tests/invctrig_test.c
349–350

add newline

lib/msun/tests/invtrig_test.c
454–455

newline

lib/msun/tests/lrint_test.c
147–148

newline

lib/msun/tests/lround_test.c
75
  1. The xfail should be applied to the affected areas; refactoring the tests will allow that to be possible.
  2. The message should be more helpful: what assert failed and why did it fail?
127–128

newline

lib/msun/tests/nan_test.c
124–125

newline

lib/msun/tests/nearbyint_test.c
111–112

TAP printf can be deleted.

152–153

TAP printf can be deleted.

172–173

newline

lib/msun/tests/next_test.c
124–125

TAP printf can be deleted. New test case after this.

139–140

TAP printf can be deleted. New test case after this.

165–166

TAP printf can be deleted. New test case after this.

213–222

TAP printf can be deleted. New test case after this.

240–241

Delete TAP printf

This revision now requires changes to proceed.Feb 23 2021, 4:51 AM
arichardson marked 27 inline comments as done.Feb 23 2021, 11:07 AM
arichardson added inline comments.
lib/msun/tests/conj_test.c
107–113

Thanks, also looking at the this should have been conjl all along since it's just doing the same thing again.

lib/msun/tests/csqrt_test.c
299–302

I've split it into csqrt/csqrtf/csqrtl tests.

lib/msun/tests/fmaxmin_test.c
135–137

Seems like they no longer fail after -fno-builtin D28577

lib/msun/tests/lround_test.c
75

Basically all asserts fail because the exception flags INVALID/INEXACT are not being set. I can try to add the xfails to individual tests but I don't have enough time to do further debugging now. I can also leave the entire test disabled as is the current state.

arichardson updated this revision to Diff 84524.Feb 23 2021, 11:10 AM
arichardson marked 2 inline comments as done.
arichardson edited the summary of this revision. (Show Details)
  • Address review comments
Harbormaster completed remote builds in B37298: Diff 84524.Feb 23 2021, 11:10 AM
arichardson added inline comments.Feb 23 2021, 11:11 AM
lib/msun/tests/lround_test.c
75
Standard error:
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:74: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for lround(0.25)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:74: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for lroundf(0.25)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:74: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for llround(0.25)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:74: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for llroundf(0.25)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:75: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for lround(0.5)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:75: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for lroundf(0.5)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:75: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for llround(0.5)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:75: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for llroundf(0.5)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:76: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for lround(-0.5)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:76: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for lroundf(-0.5)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:76: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for llround(-0.5)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:76: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for llroundf(-0.5)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:79: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for lround(0x1234.fp0)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:79: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for lroundf(0x1234.fp0)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:79: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for llround(0x1234.fp0)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:79: (0x20) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x20 for llroundf(0x1234.fp0)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:80: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for lround(__builtin_inff())
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for lround(__builtin_inff())
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:80: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for lroundf(__builtin_inff())
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for lroundf(__builtin_inff())
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:80: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for llround(__builtin_inff())
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for llround(__builtin_inff())
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:80: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for llroundf(__builtin_inff())
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for llroundf(__builtin_inff())
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:81: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for lround(__builtin_nanf(""))
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for lround(__builtin_nanf(""))
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:81: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for lroundf(__builtin_nanf(""))
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for lroundf(__builtin_nanf(""))
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:81: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for llround(__builtin_nanf(""))
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for llround(__builtin_nanf(""))
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:81: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for llroundf(__builtin_nanf(""))
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for llroundf(__builtin_nanf(""))
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:92: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for lround(0x8000000000000000.0p0)
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for lround(0x8000000000000000.0p0)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:93: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for lroundf(0x8000000000000000.0p0f)
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for lroundf(0x8000000000000000.0p0f)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:96: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for lround(-0x8000000000000800.0p0)
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for lround(-0x8000000000000800.0p0)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:97: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for lroundf(-0x8000010000000000.0p0f)
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for lroundf(-0x8000010000000000.0p0f)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:105: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for llround(0x8000000000000000.0p0)
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for llround(0x8000000000000000.0p0)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:106: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for llroundf(0x8000000000000000.0p0f)
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for llroundf(0x8000000000000000.0p0f)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:109: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for llround(-0x8000000000000800.0p0)
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for llround(-0x8000000000000800.0p0)
*** Expected check failure: https://bugs.freebsd.org/205451: /Users/alex/cheri/freebsd/lib/msun/tests/lround_test.c:110: (0x01) != fetestexcept((0x04 | 0x02 | 0x20 | 0x01 | 0x08 | 0x10)): unexpected exception flags: got 0 not 0x1 for llroundf(-0x8000010000000000.0p0f)
*** Expected check failure: https://bugs.freebsd.org/205451: FE_INVALID not set correctly for llroundf(-0x8000010000000000.0p0f)
===> Expected failures
lround_test:main  ->  expected_failure: https://bugs.freebsd.org/205451: 48 checks failed as expected; see output for more details  [0.009s]
arichardson mentioned this in D28577: While building libm, turn off any math-related compiler builtins.Feb 23 2021, 11:13 AM
arichardson added a child revision: D28884: Remove XFAILs from fmaxmin test.Feb 23 2021, 11:27 AM
arichardson mentioned this in D28884: Remove XFAILs from fmaxmin test.Feb 23 2021, 7:01 PM
arichardson added a comment.Feb 25 2021, 9:24 PM

Does this look okay now?

ngie added a comment.Feb 26 2021, 3:08 AM
In D28798#647799, @arichardson wrote:

Does this look okay now?

I'll take a gander in a few :).

arichardson updated this revision to Diff 84747.Feb 26 2021, 1:04 PM
  • next_test: Replace abort() with atf_tc_fail_check
Harbormaster completed remote builds in B37407: Diff 84747.Feb 26 2021, 1:04 PM
arichardson added a child revision: D29039: Allow lib/msun/logarithm_test to pass on ld128 platforms.Mar 3 2021, 4:28 PM
arichardson added a comment.Mar 5 2021, 9:40 AM

@ngie It would be great if you could have another look at this change. I now have quite a few follow-up msun fixes (getting close to having the test passing on AArch64) that depend on this.

arichardson added a child revision: D29076: Fix lib/msun/tests/csqrt_test on platforms with 128-bit long double.Mar 5 2021, 11:17 AM
arichardson updated this revision to Diff 85215.Mar 5 2021, 4:30 PM

Make lround_test pass

Harbormaster completed remote builds in B37612: Diff 85215.Mar 5 2021, 4:30 PM
arichardson added a child revision: D29091: Improve test messages for msun tests.Mar 5 2021, 4:34 PM
arichardson added a child revision: D29095: lib/msun/tests: Skip fenv_test:masking if exceptions can't be trapped.Mar 5 2021, 4:48 PM
arichardson added a comment.Mar 10 2021, 12:47 PM

@ngie Will you be able to look at this change this week? As this is blocking a lot of follow-up changes I'd really like to get this committed soon. If not I can fix any outstanding issues in post-commit review?

arichardson updated this revision to Diff 85720.Mar 14 2021, 2:42 PM

Drop incorrect comment in csqrt_test

Harbormaster completed remote builds in B37823: Diff 85720.Mar 14 2021, 2:42 PM
arichardson added a reviewer: tests.Mar 14 2021, 2:49 PM
arichardson added a comment.Mar 16 2021, 11:40 AM

I will commit this at the end of this week if I don't see any positive/negative reviews until this. Any remaining changes can be done post-commit IMO.

arichardson marked an inline comment as done.Mar 16 2021, 11:42 AM
arichardson added inline comments.
lib/msun/tests/fmaxmin_test.c
135–137

This is fixed by the follow-up change D28884

lib/msun/tests/lround_test.c
75

I have some follow-up changes to fix this test. Will post those once I've committed the other reviews.

This revision was not accepted when it landed; it landed in state Needs Review.Mar 22 2021, 12:01 PM
Closed by commit rG133bc645077d: Convert the msun tests to ATF (authored by arichardson). · Explain Why
This revision was automatically updated to reflect the committed changes.
arichardson added a commit: rG133bc645077d: Convert the msun tests to ATF.
arichardson added a commit: rGb69904b8916c: Convert the msun tests to ATF.Apr 22 2021, 11:09 AM

Revision Contents
Changeset List

PathSize
lib/
msun/
tests/
36 lines
80 lines
85 lines
79 lines
2 lines
84 lines
298 lines
96 lines
176 lines
70 lines
68 lines
85 lines
85 lines
49 lines
49 lines
53 lines
72 lines
86 lines
109 lines
11 lines
8 lines

Diff 86136

View Options

lib/msun/tests/Makefile

Loading...
View Options

lib/msun/tests/cexp_test.c

Loading...
View Options

lib/msun/tests/conj_test.c

Loading...
View Options

lib/msun/tests/csqrt_test.c

Loading...
View Options

lib/msun/tests/ctrig_test.c

Loading...
View Options

lib/msun/tests/exponential_test.c

Loading...
View Options

lib/msun/tests/fenv_test.c

Loading...
View Options

lib/msun/tests/fma_test.c

Loading...
View Options

lib/msun/tests/fmaxmin_test.c

Loading...
View Options

lib/msun/tests/ilogb_test.c

Loading...
View Options

lib/msun/tests/invctrig_test.c

Loading...
View Options

lib/msun/tests/invtrig_test.c

Loading...
View Options

lib/msun/tests/logarithm_test.c

Loading...
View Options

lib/msun/tests/lrint_test.c

Loading...
View Options

lib/msun/tests/lround_test.c

Loading...
View Options

lib/msun/tests/lround_test.t

Loading...
View Options

lib/msun/tests/nan_test.c

Loading...
View Options

lib/msun/tests/nearbyint_test.c

Loading...
View Options

lib/msun/tests/next_test.c

Loading...
View Options

lib/msun/tests/rem_test.c

Loading...
View Options

lib/msun/tests/test-utils.h

Loading...
View Options

lib/msun/tests/trig_test.c

Loading...