Page MenuHomeFreeBSD

tests: Require allow_network_access for tests needing name resolution
ClosedPublic

Authored by olivier on Apr 2 2025, 9:49 AM.
Tags
None
Referenced Files
Unknown Object (File)
Fri, Dec 12, 12:23 AM
Unknown Object (File)
Fri, Dec 12, 12:23 AM
Unknown Object (File)
Fri, Dec 12, 12:23 AM
Unknown Object (File)
Wed, Dec 10, 10:26 PM
Unknown Object (File)
Wed, Nov 26, 1:17 PM
Unknown Object (File)
Mon, Nov 24, 6:19 PM
Unknown Object (File)
Mon, Nov 24, 6:19 PM
Unknown Object (File)
Mon, Nov 24, 6:19 PM
Subscribers

Details

Summary

Tests that mandate working name resolution or network now require the kuya variable allow_network_access set.

Test Plan

without the patch:

root@bigone:/usr/tests # ifconfig igb0 down
root@bigone:/usr/tests # kyua test lib/libc/net/getaddrinfo
lib/libc/net/getaddrinfo/getaddrinfo:timeout_specific  ->  passed  [0.028s]
lib/libc/net/getaddrinfo/getaddrinfo:timeout2  ->  passed  [0.031s]
lib/libc/net/getaddrinfo/getaddrinfo:timeout  ->  passed  [0.034s]
lib/libc/net/getaddrinfo/getaddrinfo:nofamily  ->  failed: /usr/src/lib/libc/tests/net/getaddrinfo/getaddrinfo.c:224: Expected 1 (EAI_ADDRFAMILY), got 2 (Name could not be resolved at this time)  [0.038s]
lib/libc/net/getaddrinfo/getaddrinfo:netdown  ->  passed  [0.041s]
lib/libc/net/getaddrinfo/getaddrinfo:basic  ->  failed: /usr/src/lib/libc/tests/net/getaddrinfo/getaddrinfo.c:94: Expected 0, got 2 (Name could not be resolved at this time)  [0.044s]
lib/libc/net/getaddrinfo/getaddrinfo_test:scopeaddr  ->  passed  [0.017s]
lib/libc/net/getaddrinfo/getaddrinfo_test:unsupported_family  ->  passed  [0.015s]
lib/libc/net/getaddrinfo/getaddrinfo_test:specific  ->  passed  [0.021s]
lib/libc/net/getaddrinfo/getaddrinfo_test:empty_servname  ->  passed  [0.028s]
lib/libc/net/getaddrinfo/getaddrinfo_test:sock_raw  ->  passed  [0.025s]
lib/libc/net/getaddrinfo/getaddrinfo_test:basic  ->  passed  [0.034s]
lib/libc/net/getaddrinfo/getaddrinfo_test:empty_hostname  ->  passed  [0.039s]

Results file id is usr_tests.20250402-093916-720189
Results saved to /root/.kyua/store/results.usr_tests.20250402-093916-720189.db

11/13 passed (0 broken, 2 failed, 0 skipped)

With the patch:

root@bigone:/usr/tests # ifconfig igb0 down
root@bigone:/usr/tests # kyua test lib/libc/net/getaddrinfo
lib/libc/net/getaddrinfo/getaddrinfo:timeout_specific  ->  passed  [0.030s]
lib/libc/net/getaddrinfo/getaddrinfo:timeout2  ->  passed  [0.033s]
lib/libc/net/getaddrinfo/getaddrinfo:timeout  ->  passed  [0.036s]
lib/libc/net/getaddrinfo/getaddrinfo:nofamily  ->  skipped: Required configuration property 'allow_network_access' not defined  [0.039s]
lib/libc/net/getaddrinfo/getaddrinfo:netdown  ->  passed  [0.043s]
lib/libc/net/getaddrinfo/getaddrinfo:basic  ->  skipped: Required configuration property 'allow_network_access' not defined  [0.045s]
lib/libc/net/getaddrinfo/getaddrinfo_test:scopeaddr  ->  passed  [0.017s]
lib/libc/net/getaddrinfo/getaddrinfo_test:unsupported_family  ->  passed  [0.015s]
lib/libc/net/getaddrinfo/getaddrinfo_test:empty_servname  ->  passed  [0.028s]
lib/libc/net/getaddrinfo/getaddrinfo_test:specific  ->  passed  [0.022s]
lib/libc/net/getaddrinfo/getaddrinfo_test:sock_raw  ->  passed  [0.025s]
lib/libc/net/getaddrinfo/getaddrinfo_test:basic  ->  passed  [0.034s]
lib/libc/net/getaddrinfo/getaddrinfo_test:empty_hostname  ->  passed  [0.039s]

Results file id is usr_tests.20250402-093726-785753
Results saved to /root/.kyua/store/results.usr_tests.20250402-093726-785753.db

11/13 passed (0 broken, 0 failed, 2 skipped)

Diff Detail

Lint
Lint Skipped
Unit
Tests Skipped

Event Timeline

olivier created this revision.

I'm good for letting these tests get skipped in CI for now, in the long term I still hope we can depend on local service instead of external dependency, as mentioned in D49594#1131294

This revision is now accepted and ready to land.Apr 2 2025, 12:06 PM

It seems that as a temporary solution it is good to have them explicitly "tagged" to easily find the ones which need revising to avoid external dependencies.

I'm in the same boat as @lwhsu : papering over this is fine, but it still doesn't fix the other 3 test cases I mentioned in that are failing though: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=285826 .

The metadata var should probably be added to other lib/libc testcases. I'll check and see whether or not they're still enabled...