In D20092#434855, @trasz wrote:

In D20092#433990, @ngie wrote:

This looks good to me (not a ports committer though, so just reviewing the change).

As a volunteer/co-maintainer on the LTP project, your suggestions make sense.

The only thing that I would do is install LTP to /compat/linux/opt, if at all possible. If not, we can come up with a better way to build/relocate LTP (I wrote the current build system about a decade ago for the project).

But it does install to /compat/linux/opt - except for the documentation (where it follows Linux conventions) and two binaries - not sure what's going on with those two.

The user space and test changes all look great. I’ll defer to someone else (like kib@) about the VM changes.

In D20005#430003, @cem wrote:

It's not a static number, AFAICT, depending on how the kernel maps libraries or waves hands does stuff with virtual memory. Hence the sloppy 66xxx match. I think @lwhsu's proposal is something like atf_check [ "$num" -gt 65535 ], which verifies that the number exceeds the lower 16 bit limit (which is the coredump feature the regression test is attempting to verify).

This looks good to me (not a ports committer though, so just reviewing the change).

In D20124#433272, @ngie wrote:

I’m a bit ambivalent about this change, because it’s up on the wiki in greater detail: https://wiki.freebsd.org/TestingFreeBSD . kyua(1) references atf(7) IIRC, but it’s a level of indirection.

My point is, I’d personally prefer not mentioning just ATF in tests(7), but I want it to be enough of a living document where multiple sources can contribute to it, but mentioning the wiki page doesn’t seem like something that’s generally done from a manpage because the location and content in a wiki page can change.

I’m a bit ambivalent about this change, because it’s up on the wiki in greater detail: https://wiki.freebsd.org/TestingFreeBSD . kyua(1) references atf(7) IIRC, but it’s a level of indirection.

I'm ok with the changes, but there are some kinks that should probably be worked out before committing this.

I don't think benchmarks is an appropriate category. LTP consists primarily of functional and crash tests.

This change is very reasonable.

Abandoning review based on in-depth analysis from @tuexen.

In D20017#430611, @lwhsu wrote:

ncat is from nmap port and has nothing to do with nc, their names are similar but they are different things.

It's fine to install tests requirements from ports, after all, kyua is installed from ports and it's an essential tool for running the tests.

For marking the tests, what I was saying is "skip the tests (and not report failure) when there is no nmap (ncat) installed, and have a message to users to install nmap."

In D20017#430585, @tuexen wrote:

In D20017#430241, @lwhsu wrote:

OK, I just check we have nmap in the test vm image and it does the work: https://github.com/freebsd/freebsd-ci/commit/689e19c8c71e1892f476c9ef9392f6b443d0ed15
The other failures are due to some perl stuff which I haven't had time on it: https://ci.freebsd.org/job/FreeBSD-head-amd64-dtrace_test/lastCompletedBuild/testReport/common.ip/t_dtrace_contrib/tst_ipv4localsctp_ksh/

I'm debugging this now. Seems to be a kernel issue. Will let you know once I have understood and fixed the issue.

Is using ncat and issue?

In D20017#430242, @lwhsu wrote:

I'd prefer let's fix these test cases.

Ok, let's proceed with this change. I'll work on fixing the style issues on my branch.

@olivier: Approved! Please be sure to update the commit message before you commit. Thank you so very much <3!

Committed as rS346574 and rS346576.

Move the SUBDIR+= profile outside the architecture dependent blocks

In D20005#429909, @lwhsu wrote:

Thanks for the better fix. BTW, I still think these checks would be better to be rewritten as getting the real number and compare with 65535.

I've proposed a different solution in D20005.

This moved SUBDIR+= profile inside another .if block :/...