tests/ci: Stop hiding output

[emaste]

Summary

These tests are executed via Cirrus-CI but all build output is redirected to a file, which is deleted from the ephemeral VM, precluding debugging.

See for example https://cirrus-ci.com/task/4928014397472768

[bofh]

These were actually added by @imp. I am not sure whether if we should remove them. But I think the best approach would be is to create different targets for CIRRUS-CI or other cloud based CI systems. But I am open to ideas.

So far I recall that the rational behind these were if someone is running them on their own terminal it explodes with too much unnecessary information.

[imp]

Yes. These are uselessly verbose for developers before my change. For local testing the errors eould drown in a sea of verbosity. For local use, "go check file X" matches things like make tinderbox better. It slso copes with make -j100 where the error can be well back in the scrollback or even lost. Developers need this mode. It needs to fit in a dozen or two lines to be useful to run locally as part of their normal process.

For cirrus ci, you want to fetch a single file because its interface is so limited. Build artifacts are possible, but tricky to obtain. It's a different use case that needs different defaults to be useful.

[emaste]

if someone is running them on their own terminal it explodes with too much unnecessary information

OK, but doing this makes them unusable in Cirrus-CI and we should take them out until it's addressed somehow.

I'm also a bit puzzled why we're running buildworld again here when it's already been run by an earlier Cirrus task.

[imp]

In D51177#1168191, @emaste wrote:

if someone is running them on their own terminal it explodes with too much unnecessary information

OK, but doing this makes them unusable in Cirrus-CI and we should take them out until it's addressed somehow.

I'm also a bit puzzled why we're running buildworld again here when it's already been run by an earlier Cirrus task.

As originally designed, they weren't for cirrus ci.
They were supposed to be targets developers ran to get the same data as the CI checks would give. That top lime goal has been lost along the way. I want sonething I can run locally that gives me a sumnary of whether or not if I submit this branch to CI it will work or not.. that was the original ask and that's why I moved the output. I never thought this would be used in something like cirrus ci, so any inconsistencies are due to serving two, incompatible goals.

[emaste]

For cirrus ci, you want to fetch a single file because its interface is so limited.

I don't understand this comment -- Cirrus makes it easy to fetch the log from an individual task, from the same example https://cirrus-ci.com/task/5631701839249408 you can expand build_world and see the details, and there's a link to grab the whole log https://api.cirrus-ci.com/v1/task/5631701839249408/logs/build_world.log

In contrast the second build world executed here produces only

make -C tests/ci TARGET=${TARGET} TARGET_ARCH=${TARGET_ARCH} CITYPE=smoke ci
[Creating objdir /usr/obj/tmp/cirrus-ci-build/amd64.amd64/tests/ci]
truncate -s 512M /tmp/meta.zEWhds
tar rvf /tmp/meta.zEWhds -C /tmp/meta.zv9CEB .
a .
Building world for amd64
make TARGET=amd64 TARGET_ARCH=amd64 -j8 -C /tmp/cirrus-ci-build/tests/ci/../.. -DWITH_META_MODE   __MAKE_CONF=/dev/null SRCCONF=/etc/src.conf   buildworld > /tmp/cirrus-ci-build/tests/ci/_.amd64.ci-buildworld 2>&1 ||  (echo "ci-buildworld failed, check _.amd64.ci-buildworld for details" ; false)
ci-buildworld failed, check _.amd64.ci-buildworld for details
*** Error code 1
Stop.
make: stopped in /tmp/cirrus-ci-build/tests/ci

and now that the job has completed _.amd64.ci-buildworld no longer exists.

[bofh]

In D51177#1168191, @emaste wrote:

if someone is running them on their own terminal it explodes with too much unnecessary information

OK, but doing this makes them unusable in Cirrus-CI and we should take them out until it's addressed somehow.

I'm also a bit puzzled why we're running buildworld again here when it's already been run by an earlier Cirrus task.

The goal was to move them gradually into the same code used by everyone rather than having different targets. These targets are supposed to run only when there is a PULL request. I can only see two other matrix for PULL branches. So I am not sure how it is run by an earlier task.

So here are the possibilities that I can do:

1. We can create different targets without redirecting the logs
2. We can grab those artifacts and add them to the CIRRUS task
3. We can merge the other PULL branch builds into these matrix after we complete 1 or 2.

[bofh]

See https://reviews.freebsd.org/D51178

[imp]

I'm agnostic about how we get there, so long as I have a local command that doesn't produce too much output that I can run before commit.

I kinda like what bofm proposed.

And my comments about CI may be due to a wonky Jenkins setup I had to deal with...

I was mostly providing historical context, not objecting, per se, to things.