Enable OFED build by default.

[kib]

Summary

This change enables both userspace and modules build for OFED by default. Also, it fixes some issues, namely:

• correct location for libmlx5 and libibverbs, required by libpcap when OFED is used;
• move mostly useless examples, as well as subnet manager, builds under the new option WITH_OFED_EXTRA, disabled by default.

Test Plan

Tested by tinderbox with default options and with WITH_OFED_EXTRA enabled.

[kib]

Remove libmlx5.so.1 and libibverbs.so.1 from /usr/lib.

[• hselasky]

You also need to create a WITHOUT_OFED_EXTRA file.

[kib]

In D16230#344354, @hselasky wrote:

You also need to create a WITHOUT_OFED_EXTRA file.

Why ? Each option has only one WITH/WITHOUT description file.

[• hselasky]

It is common to define both?? See for example:

./tools/build/options/WITHOUT_MLX5TOOL
./tools/build/options/WITH_MLX5TOOL

Among other.

[kib]

In D16230#344553, @hselasky wrote:

It is common to define both?? See for example:

./tools/build/options/WITHOUT_MLX5TOOL
./tools/build/options/WITH_MLX5TOOL

Among other.

I think it is excessive for MLX5TOOL, was it added by you ?

I see that both WITH_XXX and WITHOUT_XXX files are present when operation modes descriptions are different enough to warrant different text. When an option turns some feature on and off, without side effects worth describing, the text for one mode is enough.

[• hselasky]

Yes, I added those files and I see there are a couple of more pairs.

[bdrewery]

usr.bin/ofed/Makefile

11 ↗

(On Diff #45187)

SIBDIR should be spelled SUBDIR.
But you can also just make these 3 lines: SUBDIR.${MK_OFED_EXTRA}+= opensm

[kib]

Fix typo.